realhandinc/realhand-teleop

RealHand Teleop

RealHand Teleop is an Isaac Sim, Isaac Lab, and Isaac Teleop integration for teleoperating a RealHand P7 bimanual robot equipped with two RealHand L6 dexterous hands from a PICO 4 Ultra headset.

This repository is a robotics software and asset bundle. It does not contain pretrained model weights.

Release Contents

Version 0.1.0 includes:

• RealHand P7 bimanual robot and dual L6 hand URDF and mesh assets.
• PICO 4 Ultra OpenXR controller teleoperation.
• PICO 4 Ultra OpenXR hand-tracking teleoperation.
• Pink-based dual-arm inverse kinematics.
• L6 controller-trigger and hand-joint retargeting.
• A self-contained inspection and kitting workcell.
• Offline retargeting and scene validation scripts.

Controller mode maps each tracked controller pose to the corresponding robot end effector. Each trigger linearly controls the associated L6 hand from open to closed.

Hand-tracking mode maps tracked wrist poses to the robot end effectors and retargets the tracked hand joints to the corresponding L6 joints.

Tested Compatibility

Component	Tested version
Operating system	Ubuntu 24.04.4 LTS
Python	3.12.3
Isaac Sim	6.0.0.1
Isaac Lab source	release/3.0.0-beta2
Isaac Lab commit	1a88953ca1cd246feebb9e33b8d381c2c0b4736d
Isaac Lab package metadata	6.1.5
Isaac Teleop	1.3.105rc1
XR device	PICO 4 Ultra
XR interface	OpenXR

Other versions have not yet been validated. See COMPATIBILITY.md for the exact release contract.

Prerequisites

Install these separately before using this bundle:

1. A supported NVIDIA GPU driver.
2. Isaac Sim and Isaac Lab at the tested versions.
3. Isaac Teleop at the tested version.
4. PICO 4 Ultra OpenXR tracking.
5. CloudXR only when using the current wireless XR streaming path.

CloudXR is not included in this repository. Its runtime and EULA must be obtained and accepted separately from NVIDIA.

Download

Install or invoke the Hugging Face CLI, then download the tagged release:

cd ~/Work/Projects
uvx hf download realhandinc/realhand-teleop \
  --revision v0.1.0 \
  --local-dir realhand-teleop
cd realhand-teleop

The project can live in any directory. Point the launcher to the Isaac Lab checkout and Python environment when they are not in the default locations:

export ISAACLAB_ROOT="$HOME/Work/IsaacLab"
export ISAACLAB_ENV="$HOME/Work/env_isaaclab"

Install And Validate

source "$ISAACLAB_ENV/bin/activate"

python -m pip install -e \
  examples/isaac_lab/p7_realhand_bimanual/source

python scripts/teleop/check_install.py
python scripts/teleop/test_retargeters.py

The editable install is convenient but not required by the launcher because it also adds the extension and shared retargeter paths to PYTHONPATH.

First-Time CloudXR Setup

Accept the NVIDIA CloudXR EULA once before the first XR session:

source "$ISAACLAB_ENV/bin/activate"
python -m isaacteleop.cloudxr --accept-eula

After the runtime reports that it is running, press Ctrl+C. The regular launcher starts CloudXR automatically, so a separate CloudXR terminal is not required for normal use.

Activate the environment and enter the downloaded repository:

source "$ISAACLAB_ENV/bin/activate"
cd ~/Work/Projects/realhand-teleop

To keep CloudXR running separately for debugging, start it in another terminal and source $HOME/.cloudxr/run/cloudxr.env before launching teleoperation.

Controller Teleoperation

P7L6_INPUT_MODE=controller \
PICO4U_DEVICE=cpu \
bash scripts/teleop/run_pico4u_teleop.sh

The left and right controllers independently drive the left and right arms. The corresponding trigger controls only that side's L6 hand.

Hand-Tracking Teleoperation

P7L6_INPUT_MODE=handtracking \
PICO4U_DEVICE=cpu \
bash scripts/teleop/run_pico4u_teleop.sh

The release defaults are the calibrated P7 and L6 controller and hand-tracking parameters used for version 0.1.0.

Default Scene

The bundled inspection_kitting scene contains:

• A compact inspection table.
• Two reachable sorting baskets.
• A thin cube-frame grasp object.
• Collision and rigid-body settings tuned for the P7 and L6 setup.

The supported release scene is self-contained and does not require external robot assets or an NVIDIA Nucleus asset server.

Project Layout

src/retargeters/realhand/
    Shared RealHand L6 retargeting code.

examples/isaac_lab/p7_realhand_bimanual/
    P7 and L6 assets, Isaac Lab task, scene, scripts, and documentation.

scripts/teleop/
    Stable root launch and validation entry points.

Useful Runtime Settings

P7L6_SELF_COLLISION=1
P7L6_ROBOT_YAW_DEG=90
P7L6_SCENE_MODE=inspection_kitting
P7L6_CONTROLLER_POSITION_SMOOTHING_ALPHA=1.0
P7L6_CONTROLLER_ORIENTATION_SMOOTHING_ALPHA=0.85
P7L6_CONTROLLER_MAX_ORIENTATION_STEP_DEG=90.0
P7L6_HAND_CURL_SMOOTHING_ALPHA=0.65
P7L6_HAND_THUMB_FLEX_SCALE=1.15
P7L6_HAND_THUMB_CLOSE_GAIN=1.80
P7L6_HAND_THUMB_OPPOSITION_GAIN=1.80

More PICO and OpenXR notes are available in examples/isaac_lab/p7_realhand_bimanual/docs/OPENXR_PICO4U.md.

Scope And Safety

This release controls the simulated P7 and L6 system. It is not a certified real-robot safety controller. Validate workspace limits, collision behavior, emergency stops, and command transport before adapting it to physical hardware.

License

RealHand-owned code, P7 and L6 assets, and the inspection scene are released under Apache-2.0. Files derived from Isaac Lab retain their BSD-3-Clause headers and terms. See LICENSE, NOTICE, and LICENSES/ for details.