What to define before a humanoid pilot

A humanoid pilot should begin with a task that two people can describe in the same way. A broad goal such as improving productivity may explain why a team is interested, but it cannot tell an observer whether a run succeeded. Before selecting a demonstration, write down the starting condition, the action and the completion condition. The resulting sheet becomes a shared reference for the operator, vendor and people assessing the result.

This guide offers a planning framework for an initial evaluation. It is not a robot safety assessment or a claim about a particular product's performance. The participating teams still need appropriate technical and workplace review for the equipment and setting. The aim here is to make the proposed task, responsibilities and evidence clear enough for those reviews and for a useful commercial conversation.

Define the object and the movement

Suppose a proposed task is transferring a container from a staging surface to a destination. Describe the container's dimensions, mass range, material and permitted variation. Define the starting orientation and the acceptable destination position. Include the condition of the object: an open, damaged or partly filled container may behave differently from a closed empty one. The team should agree which variations belong in the first test.

Write the completion condition so an observer can record it consistently. Does the container need to be placed within a marked area, with a particular face forward and no contents lost? Does a separate system need to acknowledge receipt? A robot's motion can end before the task is complete for the surrounding operation. Include that final handoff in the task definition rather than relying on a video that stops at a convenient moment.

Mark the environment boundary

Record the layout used for the test, including working surfaces, access routes and nearby equipment. Note the conditions that may change during the evaluation. Lighting, floor condition, object placement and surrounding activity can all be part of the setup. The team should identify which conditions are fixed, which may vary within a range and which would stop or postpone a run.

A useful environmental record is specific enough to recreate the test. Photographs and a layout drawing may help, but label them with a date and revision. If the test is moved to another area, treat that as a change to the conditions rather than assuming it is equivalent. A result from one location may be informative for another, yet the transfer needs its own review and observations.

Separate the people and system responsibilities

List what happens before the robot begins. A person may stage objects, clear a destination or confirm a work area. Those actions are part of the operating process even if they are outside the robot's assigned task. Record who performs them and how long they take. Otherwise, a selected robot cycle can look efficient while the surrounding preparation remains invisible.

Assign responsibility for starting a run, observing it, intervening and recording the outcome. The same person may hold several roles in a small pilot, but the roles should still be written down. Clarify who can approve a change to the setup and who can pause the evaluation when the condition is unclear. Those decisions should be settled before a difficult event occurs.

Describe intervention before counting success

An intervention is any agreed human action that assists, resets or changes the run. Define categories that fit the task. They might include repositioning an object, clearing an obstruction, restarting a program or asking the robot to repeat a step. Keep the categories simple enough for observers to use, and allow a free-text note for events that do not fit the initial list.

Do not compress every assisted completion into an unqualified success. A run may reach the destination after substantial help. That result is still useful, but the assistance is part of the evidence. Record the reason, time and effect of the intervention. A pattern of repeated assistance often points directly to the next engineering or process question.

Agree on a test window and denominator

A report needs to say how many attempts were made and over what period. Define an attempt, a completed cycle and an excluded event. If a run is stopped because the test area is unavailable, record that separately from a robot fault. If a fault prevents any attempts for an hour, that downtime should remain visible rather than disappearing from a cycle-only success rate.

Use several measures that answer different questions. Completion count describes task outcomes. Cycle time describes pace under the recorded conditions. Intervention count describes support effort. Availability over the agreed window describes how often the setup was ready to work. None of these measures alone gives the entire commercial picture. The report should explain what each includes and excludes.

Set acceptance criteria before the demonstration

A team should choose its own acceptance criteria in light of the task and business need. Avoid borrowing a threshold from an unrelated robot or process simply because it looks credible. If the first pilot is exploratory, state which questions it must answer rather than inventing a product acceptance score. A learning objective can be clear and useful without pretending the system is already ready for routine deployment.

For a more structured evaluation, specify the minimum evidence needed for each criterion. A timing target may require recorded runs across a defined range of object placements. An intervention criterion may require an observer's log. A service criterion may require a response exercise and a documented escalation route. Match the evidence to the decision instead of collecting a large amount of material with no agreed purpose.

Keep a baseline for the surrounding process

Describe how the task is performed today, including preparation and exceptions. The baseline should cover the same boundary used for the pilot. Comparing a robot's selected motion time with a person's entire workflow will produce a misleading result. Use the same completion condition and note differences in staffing, setup and equipment.

The baseline can also reveal whether the proposed task is the right first candidate. A process with frequent unrecorded exceptions may need better documentation before a robot trial can teach much. A simple, stable task may offer clearer learning even if it is less dramatic. The best first pilot is the one whose result can inform a decision, not necessarily the one that makes the most striking clip.

Review the evidence and the next decision

At the end of the window, compare the report with the original task sheet. Identify where the conditions changed, which criteria were addressed and which questions remain open. Keep the run records and selected examples connected. If a result depends on a particular configuration or a person's assistance, say so in the conclusion.

The next decision could be another bounded trial, a revised task, a service discussion or a decision to stop. Write down why that step follows from the evidence. A pilot that finds an unsuitable task can still be valuable if it prevents a larger commitment based on an attractive demonstration. Before beginning, prepare a one-page acceptance sheet with the task boundary, environment, roles, intervention categories and decision criteria. Review it with the participating teams and use it throughout the evaluation.