Pi 0.5 / OpenPI access#

The project provides Pi 0.5 data adaptation, fine-tuning and remote inference examples based on OpenPI. This chapter explains how to align data and the files that need to be checked when accessing. It uses an independent OpenPI path and operates differently from LeRobot’s built-in policy of the same name.

Important

Existing examples still reference the old LeKiwiClient and lekiwi_host interfaces, while the current unified software exports AlohaMiniClient and alohamini_host. Additionally, the example README mentions scripts/serve_policy_http.py which is not in the example directory. Therefore, this chapter serves as advanced access instructions; before version matching and service startup entry are completed, the original deployment command will not be marked as an official process that can be directly executed by the current version.

1. What parts does the system consist of?#

part

function

Collected LeRobot dataset

Provide images, status, actions and task descriptions

OpenPI training environment

Load Pi 0.5 base weights and fine-tune

data adapter

Map the robot fields to the model interface used by the example

HTTP Policy Service

Receives the current observation and returns an action or action sequence

Execute client locally

Obtain real machine observations, call services, and then send actions

Robot Host

Driver actual hardware and return status

The “18-dimensional internal space” in this section refers to the space of the adaptation layer of this AlohaMini example and does not imply that all Pi 0.5 configurations assume the same dimensions.

2. Sample files#

The directory is located at AlohaMini/examples/pi0.5_openpi/:

File

function

alohamini_policy.py

Input, output, and 16/18-dimensional motion adaptation

data_loader.py

Data reading related adaptation

http_policy_server.py

HTTP service implementation

evaluate_bi.py

Local execution and remote policy invocation examples

README.md

Training configuration and historical deployment instructions

View all sample files. These files need to be accessed according to the actual version of OpenPI and robot software. Copying the README command alone is not enough to complete the deployment.

3. Align robot dimensions#

robot

Original interface dimensions

Example adaptation method

AlohaMini 1

16

Add virtual joint positions and align them to the 18-dimensional space of the adaptation layer

AlohaMini 2 / Pro

18

Pass directly in the order of matching fields

It is not enough that the dimensions are the same. The names, order, coordinate meanings and units of state and action must be checked. The fact that the second generation has more wrist joints than the first generation cannot be replaced by correct mapping by arbitrarily padding zeros to the end of the array.

robot_dof and dataset_action_dim in the original configuration should correspond to the actual data. The same adaptation must be used for training and execution so that the output can be correctly mapped back to the robot.

4. Mapping cameras, status and actions#

The original example uses three image slots:

OpenPI example key

Dataset source in the example

cam_high

observation.images.chest , another default example uses head_top

cam_left_wrist

observation.images.wrist_left

cam_right_wrist

observation.images.wrist_right

state

observation.state

actions

action

prompt

Task description in data

The current camera configuration of the whole machine uses names such as forward, and the chest and left wrist are not enabled by default. Before accessing, read the actual data set fields and then modify RepackTransform. The above table cannot be applied to any data set as it is.

The example highlights the need to map prompt; missing this field causes training to not include the task description as expected.

5. Absolute action and incremental action#

The second-generation Pro configuration example uses increments for the six motion joints of the left and right arms, and maintains absolute representation of the two grippers, three chassis components, and lift:

delta_action_mask = [
    True, True, True, True, True, True, False,
    True, True, True, True, True, True, False,
    False, False, False, False,
]

The 18 Boolean values here must be consistent with the field order of the adaptation layer. The input side DeltaActions and the output side AbsoluteActions are used in pairs; modifying only one side will result in execution semantic errors.

Images also come in BGR/RGB and optional flipping. It should be verified against the actual storage of data and training decoding results, rather than guessing the color order based on OpenCV or camera names.

6. Fine-tune the preparation sequence#

  1. Join the project’s AlohaMini policy adapter in a compatible OpenPI environment.

  2. Register data configuration and training configuration at src/openpi/training/config.py by example.

  3. Set the data path, robot dimensions, image mapping and task description.

  4. Check the incremental action mask and output recovery process.

  5. Calculate normalized statistics by scripts/compute_norm_stats.py according to the OpenPI version used.

  6. Point to the actual available Pi 0.5 base weight before starting fine-tuning.

Original 2nd generation Pro examples use action_horizon=10, num_train_steps=50000, batch_size=4, num_workers=2. These are sample configurations and are not recommended values for all data sizes and devices.

7. Address and data flow of remote inference#

The inference service and bot hosts are different endpoints: server_url refers to the policy server and remote_ip refers to the bot. When the service is deployed on another machine, the client’s localhost points to the client itself and cannot replace the server IP.

After accessing, you should first verify that the service can read the request and return limited numerical actions of the correct dimensions and order, and then check how the robot client handles units, preprocessing, expired feedback, and pause recovery.

The current service implementation returns fields such as action, actions and server_timing, which are used for single-step actions, action sequences and server time consumption respectively. Which field is used should be determined by the actual protocol of the executing client.

8. Complete the compatibility check before using the computer.#

Need to confirm one by one:

  • The OpenPI adapter import location and registration configuration are correct.

  • The HTTP startup script exists and loads the correct checkpoint.

  • The local client is compatible with the current AlohaMiniClient interface.

  • robot_model can correctly configure 16/18-dimensional hardware.

  • The three-way image mapping is consistent with the actual data.

  • Training and inference use the same color, normalization, and action transformations.

  • The execution phase retains control of the current Host and feedback checks.

For users who have completed standard teleoperation, ACT training and evaluation, entering this path will make it easier to distinguish between adaptation problems and basic hardware problems.

9. Complete configuration code and historical deployment process#

OpenPI configuration reference contains the complete definition of LeRobotAlohaMiniDataConfig, TrainConfig for Pro, remote service and 16/18-dimensional client examples. The configuration code is retained below for easy comparison and modification one by one.

Data configuration factory#

Put the project’s alohamini_policy.py into the policy directory that matches the OpenPI version, and complete the corresponding import and configuration registration in src/openpi/training/config.py. The following code relies on the existing dataclasses, tyro, Sequence and other contexts of the file and is not an independently executable script.

@dataclasses.dataclass(frozen=True)
class LeRobotAlohaMiniDataConfig(DataConfigFactory):
    default_prompt: str | None = None
    bgr_to_rgb: bool = False
    flip_images_hw: bool = False
    robot_dof: int = 16  # 16 or 18
    dataset_action_dim: int = 16
    use_delta_actions: bool = True

    delta_action_mask: tyro.conf.Suppress[Sequence[bool]] = dataclasses.field(
        default_factory=lambda: [True] * 18
    )

    repack_transforms: tyro.conf.Suppress[_transforms.Group] = dataclasses.field(
        default=_transforms.Group(
            inputs=[
                _transforms.RepackTransform(
                    {
                        "images": {
                            "cam_high": "observation.images.head_top",
                            "cam_left_wrist": "observation.images.wrist_left",
                            "cam_right_wrist": "observation.images.wrist_right",
                        },
                        "state": "observation.state",
                        "actions": "action",
                    }
                )
            ]
        )
    )

    action_sequence_keys: Sequence[str] = ("action",)

    @override
    def create(self, assets_dirs: pathlib.Path, model_config: _model.BaseModelConfig) -> DataConfig:
        if self.robot_dof not in (16, 18):
            raise ValueError(f"robot_dof must be 16 or 18, got {self.robot_dof}")

        expanded_dim = 18  # Pi0.5 internal action space
        data_transforms = _transforms.Group(
            inputs=[
                alohamini_policy.AlohaMiniInputs(bgr_to_rgb=self.bgr_to_rgb),
                alohamini_policy.AlignToPi05ActionSpace(robot_dof=self.robot_dof),
            ],
            outputs=[
                alohamini_policy.AlohaMiniOutputs(internal_dim=18, robot_dof=self.robot_dof)
            ],
        )
        if self.flip_images_hw:
            data_transforms = _transforms.Group(
                inputs=[
                    alohamini_policy.AlohaMiniInputs(bgr_to_rgb=self.bgr_to_rgb),
                    _transforms.FlipImages(flip_h=True, flip_w=True),
                    alohamini_policy.AlignToPi05ActionSpace(robot_dof=self.robot_dof),
                ],
                outputs=data_transforms.outputs,
            )

        if self.use_delta_actions:
            mask = list(self.delta_action_mask)
            if len(mask) != 18:
                raise ValueError(
                    f"delta_action_mask length must be 18, got {len(mask)}")
            data_transforms = data_transforms.push(
                inputs=[_transforms.DeltaActions(mask)],
                outputs=[_transforms.AbsoluteActions(mask)],
            )

        model_transforms = ModelTransformFactory(
            default_prompt=self.default_prompt)(model_config)

        return dataclasses.replace(
            self.create_base_config(assets_dirs, model_config),
            repack_transforms=self.repack_transforms,
            data_transforms=data_transforms,
            model_transforms=model_transforms,
            action_sequence_keys=self.action_sequence_keys,
        )

Pro training configuration#

The following configuration items need to be put into the corresponding configuration collection. name="alohamini2pro" is the OpenPI training configuration name; it and robot_model of the whole host belong to different program parameters.

TrainConfig(
        name="alohamini2pro",
        model=pi0_config.Pi0Config(pi05=True, action_horizon=10),
        data=LeRobotAlohaMiniDataConfig(
            repo_id="/path/to/your/dataset_root",
            base_config=DataConfig(prompt_from_task=True),
            default_prompt="your job description",
            robot_dof=18,
            dataset_action_dim=18,
            use_delta_actions=True,
            repack_transforms=_transforms.Group(
                inputs=[
                    _transforms.RepackTransform(
                        {
                            "images": {
                                "cam_high": "observation.images.chest",
                                "cam_left_wrist": "observation.images.wrist_left",
                                "cam_right_wrist": "observation.images.wrist_right",
                            },
                            "state": "observation.state",
                            "actions": "action",
                            "prompt": "prompt", # This parameter must be added, otherwise the model training will not include the task description
                        }
                    )
                ]
            ),
            # AlohaMini2Pro is native 18D: arm joints are trained as deltas,
            # grippers/base/lift stay absolute.
            delta_action_mask=[
                True, True, True, True, True, True, False,
                True, True, True, True, True, True, False,
                False, False, False, False,
            ],
            assets=AssetsConfig(
                assets_dir="/path/to/your/asset/path",
                asset_id="/dataset/file/name", # assets_dir and assed_id form your dataset path, the path is equivalent to repo_id
            ),
        ),
        weight_loader=weight_loaders.CheckpointWeightLoader(
            "gs://openpi-assets/checkpoints/pi05_base/params",
        ),
        num_train_steps=50000,
        batch_size=4,
        num_workers=2,
        wandb_enabled=False,
    ),

Complete fine-tuning and deployment in sequence#

  1. Replace repo_id, assets_dir, asset_id and the task description to ensure the combined path points to the real data.

  2. Check the prompt mapping and three-way image name; the second generation/Pro uses 18-dimensional configuration, and the first generation uses the corresponding 16→18 adaptation.

  3. Compute statistics using the installed version of OpenPI scripts/compute_norm_stats.py and check the asset output location.

  4. Select the registered configuration according to the training entrance of this version and load the Pi0.5 basic weight. You need to confirm the dependency version and training entry parameters before running; the current example does not provide a complete environment lock file.

  5. Select the actually generated checkpoint, prepare the HTTP service startup entry on the remote end, and then verify the request and return actions.

  6. After completing the interface adaptation of the current AlohaMiniClient, first conduct offline dimension and unit verification, and then connect to the real machine.

scripts/serve_policy_http.py is not provided in the sample directory, and lekiwi_host and LeKiwiClient are also different from the current interface. Before running, you need to complete the service entrance, adapt the client interface, and complete verification on the corresponding device.