NVIDIA IsaacLab Arena & LeRobot#

本节介绍 LeRobot 的通用功能。请按目标机器人、数据集和运行环境配置示例；用于 AlohaMini 2 / 2 Pro 前，需要确认硬件与接口兼容性。

LeRobot EnvHub now supports GPU-accelerated simulation with IsaacLab Arena for policy evaluation at scale. Train and evaluate imitation learning policies with high-fidelity simulation — all integrated into the LeRobot ecosystem.

IsaacLab Arena integrates with NVIDIA IsaacLab to provide:

🤖 Humanoid embodiments: GR1, G1, Galileo with various configurations
🎯 Manipulation & loco-manipulation tasks: Door opening, pick-and-place, button pressing, and more
⚡ GPU-accelerated rollouts: Parallel environment execution on NVIDIA GPUs
🖼️ RTX Rendering: Evaluate vision-based policies with realistic rendering, reflections and refractions
📦 LeRobot-compatible datasets: Ready for training with GR00T N1x, PI0, SmolVLA, ACT, and Diffusion policies
🔄 EnvHub integration: Load environments from HuggingFace EnvHub with one line

Installation#

Prerequisites#

Hardware requirements are shared with Isaac Sim, and are detailed in Isaac Sim Requirements.

NVIDIA GPU with CUDA support
NVIDIA driver compatible with IsaacSim 5.1.0
Linux (Ubuntu 22.04 / 24.04)

Setup#

# 1. Create conda environment
conda create -y -n lerobot-arena python=3.11
conda activate lerobot-arena
conda install -y -c conda-forge ffmpeg=7.1.1

# 2. Install Isaac Sim 5.1.0
pip install "isaacsim[all,extscache]==5.1.0" --extra-index-url https://pypi.nvidia.com

# Accept NVIDIA EULA (required)
export ACCEPT_EULA=Y
export PRIVACY_CONSENT=Y

# 3. Install IsaacLab 2.3.0
git clone https://github.com/isaac-sim/IsaacLab.git
cd IsaacLab
git checkout v2.3.0
./isaaclab.sh -i
cd ..

# 4. Install IsaacLab Arena
git clone https://github.com/isaac-sim/IsaacLab-Arena.git
cd IsaacLab-Arena
git checkout release/0.1.1
pip install -e .
cd ..


# 5. Install LeRobot (evaluation extra for env/policy evaluation)
git clone https://github.com/huggingface/lerobot.git
cd lerobot
pip install -e ".[evaluation]"
cd ..


# 6. Install additional dependencies
pip install onnxruntime==1.23.2 lightwheel-sdk==1.0.1 vuer[all]==0.0.70 qpsolvers==4.8.1
pip install numpy==1.26.0 # Isaac Sim 5.1 depends on numpy==1.26.0, this will be fixed in next release

Evaluating Policies#

Pre-trained Policies#

The following trained policies are available:

Policy	Architecture	Task	Link
pi05-arena-gr1-microwave	PI0.5	GR1 Microwave	HuggingFace
smolvla-arena-gr1-microwave	SmolVLA	GR1 Microwave	HuggingFace

Evaluate SmolVLA#

pip install -e ".[smolvla]"
pip install numpy==1.26.0 # revert numpy to version 1.26

lerobot-eval \
    --policy.path=nvidia/smolvla-arena-gr1-microwave \
    --env.type=isaaclab_arena \
    --env.hub_path=nvidia/isaaclab-arena-envs \
    --rename_map='{"observation.images.robot_pov_cam_rgb": "observation.images.robot_pov_cam"}' \
    --policy.device=cuda \
    --env.environment=gr1_microwave \
    --env.embodiment=gr1_pink \
    --env.object=mustard_bottle \
    --env.headless=false \
    --env.enable_cameras=true \
    --env.video=true \
    --env.video_length=10 \
    --env.video_interval=15 \
    --env.state_keys=robot_joint_pos \
    --env.camera_keys=robot_pov_cam_rgb \
    --trust_remote_code=True \
    --eval.batch_size=1

Evaluate PI0.5#

pip install -e ".[pi]"
pip install numpy==1.26.0 # revert numpy to version 1.26

提示

PI0.5 requires disabling torch compile for evaluation:

TORCH_COMPILE_DISABLE=1 TORCHINDUCTOR_DISABLE=1 lerobot-eval \
    --policy.path=nvidia/pi05-arena-gr1-microwave \
    --env.type=isaaclab_arena \
    --env.hub_path=nvidia/isaaclab-arena-envs \
    --rename_map='{"observation.images.robot_pov_cam_rgb": "observation.images.robot_pov_cam"}' \
    --policy.device=cuda \
    --env.environment=gr1_microwave \
    --env.embodiment=gr1_pink \
    --env.object=mustard_bottle \
    --env.headless=false \
    --env.enable_cameras=true \
    --env.video=true \
    --env.video_length=15 \
    --env.video_interval=15 \
    --env.state_keys=robot_joint_pos \
    --env.camera_keys=robot_pov_cam_rgb \
    --trust_remote_code=True \
    --eval.batch_size=1

提示

To change the number of parallel environments, use the --eval.batch_size flag.

What to Expect#

During evaluation, you will see a progress bar showing the running success rate:

Stepping through eval batches:   8%|██████▍    | 4/50 [00:45<08:06, 10.58s/it, running_success_rate=25.0%]

Video Recording#

To enable video recording during evaluation, add the following flags to your command:

--env.video=true \
--env.video_length=15 \
--env.video_interval=15

For more details on video recording, see the IsaacLab Recording Documentation.

提示

When running headless with --env.headless=true, you must also enable cameras explicitly for camera enabled environments:

--env.headless=true --env.enable_cameras=true

Output Directory#

Evaluation videos are saved to the output directory with the following structure:

outputs/eval/<date>/<timestamp>_<env>_<policy>/videos/<task>_<env_id>/eval_episode_<n>.mp4

For example:

outputs/eval/2026-01-02/14-38-01_isaaclab_arena_smolvla/videos/gr1_microwave_0/eval_episode_0.mp4

Training Policies#

To learn more about training policies with LeRobot, please refer to the training documentation:

Sample IsaacLab Arena datasets are available on HuggingFace Hub for experimentation:

Dataset	Description	Frames
Arena-GR1-Manipulation-Task	GR1 microwave manipulation	~4K
Arena-G1-Loco-Manipulation-Task	G1 loco-manipulation	~4K

Environment Configuration#

Full Configuration Options#

from lerobot.envs.configs import IsaaclabArenaEnv

config = IsaaclabArenaEnv(
    # Environment selection
    environment="gr1_microwave",      # Task environment
    embodiment="gr1_pink",            # Robot embodiment
    object="power_drill",             # Object to manipulate

    # Simulation settings
    episode_length=300,               # Max steps per episode
    headless=True,                    # Run without GUI
    device="cuda:0",                  # GPU device
    seed=42,                          # Random seed

    # Observation configuration
    state_keys="robot_joint_pos",     # State observation keys (comma-separated)
    camera_keys="robot_pov_cam_rgb",  # Camera observation keys (comma-separated)
    state_dim=54,                     # Expected state dimension
    action_dim=36,                    # Expected action dimension
    camera_height=512,                # Camera image height
    camera_width=512,                 # Camera image width
    enable_cameras=True,              # Enable camera observations

    # Video recording
    video=False,                      # Enable video recording
    video_length=100,                 # Frames per video
    video_interval=200,               # Steps between recordings

    # Advanced
    mimic=False,                      # Enable mimic mode
    teleop_device=None,               # Teleoperation device
    disable_fabric=False,             # Disable fabric optimization
    enable_pinocchio=True,            # Enable Pinocchio for IK
)

Using Environment Hub directly for advanced usage#

Create a file called test_env_load_arena.py or download from the EnvHub:

import logging
from dataclasses import asdict
from pprint import pformat
import torch
import tqdm
from lerobot.configs import parser
from lerobot.configs.eval import EvalPipelineConfig


@parser.wrap()
def main(cfg: EvalPipelineConfig):
    """Run random action rollout for IsaacLab Arena environment."""
    logging.info(pformat(asdict(cfg)))

    from lerobot.envs import make_env

    env_dict = make_env(
        cfg.env,
        n_envs=cfg.env.num_envs,
        trust_remote_code=True,
    )
    env = next(iter(env_dict.values()))[0]
    env.reset()
    for _ in tqdm.tqdm(range(cfg.env.episode_length)):
        with torch.inference_mode():
            actions = env.action_space.sample()
            obs, rewards, terminated, truncated, info = env.step(actions)
            if terminated.any() or truncated.any():
                obs, info = env.reset()
    env.close()


if __name__ == "__main__":
    main()

Run with:

python test_env_load_arena.py \
    --env.environment=g1_locomanip_pnp \
    --env.embodiment=gr1_pink \
    --env.object=cracker_box \
    --env.num_envs=4 \
    --env.enable_cameras=true \
    --env.seed=1000 \
    --env.video=true \
    --env.video_length=10 \
    --env.video_interval=15 \
    --env.headless=false \
    --env.hub_path=nvidia/isaaclab-arena-envs \
    --env.type=isaaclab_arena

Creating New Environments#

First create a new IsaacLab Arena environment by following the IsaacLab Arena Documentation.

Clone our EnvHub repo:

git clone https://huggingface.co/nvidia/isaaclab-arena-envs

Modify the example_envs.yaml file based on your new environment. Upload your modified repo to HuggingFace EnvHub.

提示

Your IsaacLab Arena environment code must be locally available during evaluation. Users can clone your environment repository separately, or you can bundle the environment code and assets directly in your EnvHub repo.

Then, when evaluating, use your new environment:

lerobot-eval \
    --env.hub_path=<your-env-hub-path>/isaaclab-arena-envs \
    --env.environment=<your new environment> \
    ...other flags...

We look forward to your contributions!

Troubleshooting#

CUDA out of memory#

Reduce batch_size or use a GPU with more VRAM:

--eval.batch_size=1

EULA not accepted#

Set environment variables before running:

export ACCEPT_EULA=Y
export PRIVACY_CONSENT=Y

Video recording not working#

Enable cameras when running headless:

--env.video=true --env.enable_cameras=true --env.headless=true

Policy output dimension mismatch#

Ensure action_dim matches your policy:

--env.action_dim=36

libGLU.so.1 Errors during Isaac Sim initialization#

Ensure you have the following dependencies installed, this is likely to happen on headless machines.

sudo apt update && sudo apt install -y libglu1-mesa libxt6

Lightwheel LW-BenchHub#

Lightwheel is bringing Lightwheel-Libero-Tasks and Lightwheel-RoboCasa-Tasks with 268 tasks to the LeRobot ecosystem. LW-BenchHub collects and generates large-scale datasets via teleoperation that comply with the LeRobot specification, enabling out-of-the-box training and evaluation workflows. With the unified interface provided by EnvHub, developers can quickly build end-to-end experimental pipelines.

Install#

Assuming you followed the Installation steps, you can install LW-BenchHub with:

conda install pinocchio -c conda-forge -y
pip install numpy==1.26.0 # revert numpy to version 1.26

sudo apt-get install git-lfs && git lfs install

git clone https://github.com/LightwheelAI/lw_benchhub
git lfs pull # Ensure LFS files (e.g., .usd assets) are downloaded

cd lw_benchhub
pip install -e .

For more detailed instructions, please refer to the LW-BenchHub Documentation.

Lightwheel Tasks Dataset#

LW-BenchHub datasets are available on HuggingFace Hub:

Dataset	Description	Tasks	Frames
Lightwheel-Tasks-X7S	X7S LIBERO and RoboCasa	117	~10.3M
Lightwheel-Tasks-Double-Piper	Double-Piper LIBERO	130	~6.0M
Lightwheel-Tasks-G1-Controller	G1-Controller LIBERO	62	~2.7M
Lightwheel-Tasks-G1-WBC	G1-WBC RoboCasa	32	~1.5M

For training policies, refer to the Training Policies section.

Evaluating Policies#

Pre-trained Policies#

The following trained policies are available:

Policy	Architecture	Task	Layout	Robot	Link
smolvla-double-piper-pnp	SmolVLA	L90K1PutTheBlackBowlOnThePlate	libero-1-1	DoublePiper-Abs	HuggingFace

Evaluate SmolVLA#

lerobot-eval \
  --policy.path=LightwheelAI/smolvla-double-piper-pnp \
  --env.type=isaaclab_arena \
  --rename_map='{"observation.images.left_hand_camera_rgb": "observation.images.left_hand", "observation.images.right_hand_camera_rgb": "observation.images.right_hand", "observation.images.first_person_camera_rgb": "observation.images.first_person"}' \
  --env.hub_path=LightwheelAI/lw_benchhub_env \
  --env.kwargs='{"config_path": "configs/envhub/example.yml"}' \
  --trust_remote_code=true \
  --env.state_keys=joint_pos \
  --env.action_dim=12 \
  --env.camera_keys=left_hand_camera_rgb,right_hand_camera_rgb,first_person_camera_rgb \
  --policy.device=cuda \
  --eval.batch_size=10 \
  --eval.n_episodes=100

Environment Configuration#

Evaluation can be quickly launched by modifying the robot, task, and layout settings in the configuration file.

Full Configuration Options#

# =========================
# Basic Settings
# =========================
disable_fabric: false
device: cuda:0
sensitivity: 1.0
step_hz: 50
enable_cameras: true
execute_mode: eval
episode_length_s: 20.0 # Episode length in seconds, increase if episodes timeout during eval

# =========================
# Robot Settings
# =========================
robot: DoublePiper-Abs # Robot type, DoublePiper-Abs, X7S-Abs, G1-Controller or G1-Controller-DecoupledWBC
robot_scale: 1.0

# =========================
# Task & Scene Settings
# =========================
task: L90K1PutTheBlackBowlOnThePlate # Task name
scene_backend: robocasa
task_backend: robocasa
debug_assets: null
layout: libero-1-1 # Layout and style ID
sources:
  - objaverse
  - lightwheel
  - aigen_objs
object_projects: []
usd_simplify: false
seed: 42

# =========================
# Object Placement Retry Settings
# =========================
max_scene_retry: 4
max_object_placement_retry: 3

resample_objects_placement_on_reset: true
resample_robot_placement_on_reset: true

# =========================
# Replay Configuration Settings
# =========================
replay_cfgs:
  add_camera_to_observation: true
  render_resolution: [640, 480]

NVIDIA IsaacLab Arena & LeRobot#

Installation#

Prerequisites#

Setup#

Evaluating Policies#

Pre-trained Policies#

Evaluate SmolVLA#

Evaluate PI0.5#

What to Expect#

Video Recording#

Output Directory#

Training Policies#

Environment Configuration#

Full Configuration Options#

Using Environment Hub directly for advanced usage#

Creating New Environments#

Troubleshooting#

CUDA out of memory#

EULA not accepted#

Video recording not working#

Policy output dimension mismatch#

libGLU.so.1 Errors during Isaac Sim initialization#

See Also#

Lightwheel LW-BenchHub#

Install#

Lightwheel Tasks Dataset#

Evaluating Policies#

Pre-trained Policies#

Evaluate SmolVLA#

Environment Configuration#

Full Configuration Options#

See Also#