YAML Metadata Warning:empty or missing yaml metadata in repo card

Check out the documentation for more information.

Cyclo-Lab → LeRobot Sim2Real Workflow

For FFW_SG2 (L-Table). HF account: JSHNSL.

Key point: each stage runs a different Python.

command	what it is	used for
`lerobot-python`	lerobot venv (`/root/lerobot_env`, py3.12 / lerobot 0.5.2 / data v3.0)	convert · upload · train
`python`	Isaac Sim (`_isaac_sim/python.sh`, py3.11 / lerobot 0.3.3 / data v2.1)	sim rollout (isaaclab)
`$ISAACLAB_PATH/_isaac_sim/python.sh`	full path of the `python` above	when pip-installing into the isaac python

⚠️ Version split: the training venv uses lerobot 0.5.2 (v3.0). The rollout isaac python is 3.11, which cannot install 0.5.2 (needs ≥3.12), so it stays on 0.3.3. The two are independent and do not affect each other.

Repository layout (`JSHNSL/humanoid-imitation-learning-en`)

Code · models · datasets live in this single repo. Base is ROBOTIS cyclo_lab + lerobot-cyclo 0.5.2.

humanoid-imitation-learning/
├── README.md                        this document (= WORKFLOW_en.md)
├── LEROBOT_UPGRADE.md               lerobot 0.3.3 → 0.5.2 upgrade
├── DASHBOARD_CHANGES.md             dashboard session-naming changes
├── HDF5_HUB_README.md               README for the HDF5 hub repo (uploaded by sync_docs.py)
├── push_dataset.py                  LeRobot dataset HF upload (training repo)
├── push_hdf5.py                     raw HDF5 HF upload (HDF5 hub repo)
├── merge_hdf5_demos.py              HDF5 merge (per session / stage)
├── hpo_optuna.py                    Optuna hyperparameter tuning + train from tuned params
├── hpo_train_shim.py                └ train launcher it uses (works around a diffusion sampler bug)
├── sg2_ltable_dashboard.py          recording · pipeline dashboard (session naming)
├── scripts/sim2real/imitation_learning/
│   ├── data_converter/isaaclab2lerobot.py      HDF5 → LeRobot(v3.0) conversion
│   └── inference/
│       ├── eval_policy_sim.py                   sim rollout evaluation
│       └── convert_policy_v05_to_v03.py         0.5.2 policy → 0.3.3 (for rollout)
├── models/                          trained policies (0.5.2 originals + _v03 for rollout)
└── datasets/ffw_sg2_ltable_merge/   v3.0 LeRobot dataset

Reproduction order: ① prepare cyclo_lab + lerobot-cyclo → ② place the scripts above → ③ run the pipeline below.

Setup — environment (one-time)

Prerequisites: NVIDIA Docker (Container Toolkit) · Isaac Sim container requirements satisfied · NVIDIA GPU + drivers

1. Clone the EKAIWORKER overlay + set up — cyclo_lab / ai_worker / robotis_applications at once:

git clone https://github.com/Disniekie01/EKAIWORKER.git AIWORKER
cd AIWORKER
./setup.sh ~/AIWORKER

setup.sh clones the three repos at pinned commits + initializes submodules + applies overlays automatically (it does not vendor the full upstream code). Components:

cyclo_lab — Isaac Sim environment · recording · Mimic pipeline (+ this workflow's scripts)

ai_worker — motion control

robotis_applications — VR teleop publisher

2. Start the Docker containers:

cd ~/AIWORKER/cyclo_lab/docker && ./container.sh start              # training · sim (required)
cd ~/AIWORKER/robotis_applications/docker && ./container.sh start   # VR teleop (for recording)
cd ~/AIWORKER/ai_worker/docker && ./container.sh start              # motion control (for recording)

Enter the container (all work happens inside):

cd ~/AIWORKER/cyclo_lab && ./docker/container.sh enter

Included: Isaac Sim 5.1.0 · Isaac Lab v2.3.0 · CycloneDDS 0.10.2 · LeRobot venv

3. Install LeRobot 0.5.2 (v3.0) (training venv /root/lerobot_env; details in LEROBOT_UPGRADE.md):

(a) If the venv already exists (lerobot-python alias works) — just upgrade:

lerobot-pip install "lerobot[training,av-dep] @ git+https://github.com/ROBOTIS-GIT/lerobot-cyclo.git"

(b) If the venv is missing — create it. ⚠️ python3 is the isaac 3.11 alias, so use the system python3.12:

/usr/bin/python3.12 -m venv /root/lerobot_env
/root/lerobot_env/bin/pip install --upgrade pip
/root/lerobot_env/bin/pip install "lerobot[training,av-dep] @ git+https://github.com/ROBOTIS-GIT/lerobot-cyclo.git" h5py
# register aliases
cat >> ~/.bashrc <<'EOF'
export LEROBOT_VENV=/root/lerobot_env
alias lerobot-python='${LEROBOT_VENV}/bin/python3'
alias lerobot-pip='${LEROBOT_VENV}/bin/pip'
alias lerobot-hf='${LEROBOT_VENV}/bin/hf'
alias lerobot-activate='source ${LEROBOT_VENV}/bin/activate'
EOF
source ~/.bashrc
lerobot-python -c "import lerobot; print(lerobot.__version__)"   # verify 0.5.2

⚠️ (REQUIRED) pin torch/torchvision — training fails with a version mismatch otherwise:

lerobot-python -c "import torch,torchvision; print(torch.__version__, torchvision.__version__)"
# must be  2.7.1+cu128 / 0.22.1+cu128
# if different (e.g. 2.7.0/0.22.0) force-reinstall:
lerobot-pip install --index-url https://download.pytorch.org/whl/cu128 \
    --force-reinstall "torch==2.7.1" "torchvision==0.22.1"

The rollout isaac python (3.11) stays on 0.3.3 (see 4-a).

4. (For recording) VR teleop dashboard:

cd ~/AIWORKER/cyclo_lab && python3 sg2_ltable_dashboard.py     # http://localhost:8765

5. (Optional) Datasets · models: from datasets/ · models/ of this repo.

→ From then on, just ./docker/container.sh enter and run stages 1–4 below.

Path convention (when following this repo)

This repo contains scripts/ · datasets/ffw_sg2_ltable_merge/ · models/. Commands below run from the cyclo_lab root (/workspace/cyclo_lab) and assume you placed the repo folders into cyclo_lab:

repo folder	put it in cyclo_lab at	path used in commands
`scripts/`	as-is	`scripts/sim2real/...`
`datasets/ffw_sg2_ltable_merge/`	under `./datasets/`	`--dataset.root=./datasets/ffw_sg2_ltable_merge`
`models/<name>/`	under `./models/`	rollout `--policy ./models/<name>`

If producing your own data: use the conversion output (./datasets/lerobot/<timestamp>) / training output (./outputs/train/<NAME>) instead.

1. HDF5 → LeRobot conversion (lerobot-python) — only when producing your own data

lerobot-python scripts/sim2real/imitation_learning/data_converter/isaaclab2lerobot.py \
    --task=Cyclo-Real-Pick-Place-LTable-FFW-SG2-v0 \
    --robot_type FFW_SG2 \
    --dataset_file ./datasets/ffw_sg2_l_table_raw.hdf5

Output: ./datasets/lerobot/<timestamp>/ (data/ videos/ meta/)

1.5 Merge multiple HDF5 files (optional) (lerobot-python)

When you want to merge several session/stage HDF5 files into one before converting. Script: merge_hdf5_demos.py (in the repo root → run inside the container at /workspace/cyclo_lab).

lerobot-python merge_hdf5_demos.py \
    --inputs <file1.hdf5> <file2.hdf5> ... \
    --output <merged.hdf5>

--inputs : HDF5 files to merge (in your chosen order, required)
--output : result file (required)
--prefix : demo-label prefix in the output (default demo)

Example — glob all joint sessions of a task:

lerobot-python merge_hdf5_demos.py \
    --inputs datasets/ffw_sg2_l_table/*/*_joint.hdf5 \
    --output datasets/ffw_sg2_l_table_merged_joint.hdf5

Demo labels (demo_0..demo_N) are naturally sorted + re-indexed sequentially
Files being merged must have the same robot · task · dims · cameras (usually merge joint files)
After merging, pass it to isaaclab2lerobot (stage 1) via --dataset_file
To mix raw files, first unify them with action_data_converter --action_type joint

2. HuggingFace upload — everything into the single `humanoid-imitation-learning` repo

Instead of separate dataset/model repos, collect everything under this repo's subfolders (datasets/, models/).

export HF_TOKEN=hf_write_token       # or: lerobot-python -c "from huggingface_hub import login; login()"
unset HF_HUB_OFFLINE                 # unset the offline var if it lingers

Dataset → a standalone dataset repo (must be a dataset-type repo so --dataset.repo_id can pull it in training):

lerobot-python push_dataset.py \
    --repo_id JSHNSL/ffw_sg2_ltable_merge \
    --root ./datasets/lerobot/<timestamp>       # the stage-1 conversion output

→ https://huggingface.co/datasets/JSHNSL/ffw_sg2_ltable_merge (used by training in stage 3)

Model (training result) → models/<name>/:

lerobot-python -c "
from huggingface_hub import upload_folder
upload_folder(folder_path='./outputs/train/act_merge/checkpoints/last/pretrained_model',
  repo_id='JSHNSL/humanoid-imitation-learning', repo_type='model',
  path_in_repo='models/act_merge')
"

So training is run with --policy.push_to_hub=false (to avoid creating separate repos), and you upload manually as above.

2.5 Raw HDF5 → HDF5 hub repo (`push_hdf5.py`)

push_dataset.py only uploads a converted LeRobot dataset (meta/ data/ videos/). The original HDF5 demos out of Isaac Sim (raw / annotate / ik / generate / joint) go to a separate "hub" dataset repo.

Why separate?

The training repo (ffw_sg2_ltable_merge) is pulled in full on every training run, so it must stay lean (HDF5 files are multi-GB)
The hub repo can accumulate HDF5 from other tasks over time

Hub repo: JSHNSL/ffw_sg2_hdf5 (dataset type, public)

export HF_TOKEN=hf_write_token
unset HF_HUB_OFFLINE

# 1) See what would be uploaded (uploads nothing)
lerobot-python push_hdf5.py \
    --repo_id JSHNSL/ffw_sg2_hdf5 \
    --paths datasets/ffw_sg2_l_table \
    --dry_run

# 2) Actually upload (repo is created if missing)
lerobot-python push_hdf5.py \
    --repo_id JSHNSL/ffw_sg2_hdf5 \
    --paths datasets/ffw_sg2_l_table

The folder structure is preserved inside the repo — tasks stay naturally separated:

ffw_sg2_l_table/20260710/1421_58_ffw_sg2_l_table_raw.hdf5
ffw_sg2_l_table/20260710/1425_58_ffw_sg2_l_table_ik.hdf5
...

Key options:

Option	Description
`--paths`	One or more files/directories. Directories are walked recursively (`*.hdf5`); the directory name becomes the top-level folder in the repo
`--dry_run`	Print the file list + total size and exit (always run this first)
`--path_in_repo`	Prefix inside the repo (e.g. `--path_in_repo hdf5`). Default: repo root
`--private`	Create the repo as private

Single file / several paths at once:

lerobot-python push_hdf5.py --repo_id JSHNSL/ffw_sg2_hdf5 \
    --paths datasets/ffw_sg2_l_table_joint.hdf5 datasets/other_dataset --dry_run

Download it back (lerobot-hf = the venv's hf CLI; --repo-type dataset is required):

# a single file
lerobot-hf download JSHNSL/ffw_sg2_hdf5 --repo-type dataset \
    0713_ffw_sg2_l_table_annotate_merge_num96.hdf5 \
    --local-dir ./datasets

# by pattern (e.g. only the joint stage)
lerobot-hf download JSHNSL/ffw_sg2_hdf5 --repo-type dataset \
    --include "*_joint.hdf5" --local-dir ./datasets

⚠️ HDF5 files are large (some are 3–5 GB each). Always check the total with --dry_run before uploading.

2.6 Deleting files already on HF

For when you uploaded something by mistake. Use lerobot-hf repos delete-files (needs a write token).

lerobot-hf auth login          # if you haven't

dataset repo (e.g. the HDF5 hub) — --repo-type dataset is required:

# one file
lerobot-hf repos delete-files JSHNSL/ffw_sg2_hdf5 \
    1425_58_ffw_sg2_l_table_ik.hdf5 --repo-type dataset

# several (space separated)
lerobot-hf repos delete-files JSHNSL/ffw_sg2_hdf5 \
    a.hdf5 b.hdf5 --repo-type dataset

# by pattern (fnmatch; '*' matches recursively)
lerobot-hf repos delete-files JSHNSL/ffw_sg2_hdf5 \
    "*_ik.hdf5" --repo-type dataset

# a whole folder
lerobot-hf repos delete-files JSHNSL/ffw_sg2_hdf5 \
    "ffw_sg2_l_table/*" --repo-type dataset

model repo (humanoid-imitation-learning) — omit --repo-type (model is the default):

lerobot-hf repos delete-files JSHNSL/humanoid-imitation-learning "models/act_8ep/*"

Delete the repo itself:

lerobot-hf repos delete JSHNSL/some-repo --repo-type dataset

Watch out:

⚠️ Forgetting --repo-type dataset makes it look in the model repo and fail oddly (same trap as downloading)
If you are unsure what a pattern will match, check the repo's Files and versions tab first
Add --create-pr to stage the deletion as a pull request instead of deleting immediately (safety net)

⚠️ Deleting does NOT free space → squash to reclaim it

HF is git-backed, so delete-files leaves the blob in history and repo storage does not shrink. To reclaim the space without deleting the repo, squash the history (collapse all commits into one, dropping old blobs).

# ① actually delete the file first (still present in history at this point)
lerobot-hf repos delete-files JSHNSL/ffw_sg2_hdf5 "file-to-remove.hdf5" --repo-type dataset

# ② squash the history -> this is where the space is actually reclaimed
lerobot-python -c "
from huggingface_hub import HfApi
HfApi().super_squash_history(repo_id='JSHNSL/ffw_sg2_hdf5', repo_type='dataset')
print('squashed')
"

Order matters: delete with ①, then squash with ②. The squash keeps only the current HEAD state and throws away every earlier commit, so the deleted file's blob goes with them.

The cost:

⚠️ Commit history is gone for good — this cannot be undone
⚠️ A squashed branch diverges from other branches and can no longer be merged into them
Only do it when you will never need an older revision. (For a raw-HDF5 archive repo the history carries little value, so the cost is low.)
There is no CLI command for it → Python only. Needs a write token.

Approach	Frees space	History	Repo
`delete-files` alone	❌	kept	kept
`delete-files` + `super_squash_history`	✅	gone	kept ← usually what you want
Delete + recreate the repo	✅	gone	must be rebuilt (README, settings)

3. Training — ACT / VQ-BeT / Diffusion (lerobot-python)

Data is pulled automatically from the dataset repo JSHNSL/ffw_sg2_ltable_merge (create it first in stage 2 if missing).

Common command (only change <POLICY>, <NAME>):

unset HF_HUB_OFFLINE
lerobot-python -m lerobot.scripts.lerobot_train \
    --dataset.repo_id=JSHNSL/ffw_sg2_ltable_merge \
    --policy.type=<POLICY> \
    --output_dir=./outputs/train/<NAME> \
    --policy.device=cuda \
    --policy.push_to_hub=false \
    --num_workers=8 --dataset.video_backend=pyav \
    --steps=100000 --save_freq=20000

policy	`<POLICY>`	`<NAME>`	notes
ACT	`act`	`act_merge`	default · fast · stable
VQ-BeT	`vqbet`	`vqbet_merge`	multimodal
Diffusion	`diffusion`	`diffusion_merge`	multimodal · slow · large model, needs `diffusers`

Module name (0.5.x): lerobot.scripts.lerobot_train (not the old ...train)
--dataset.repo_id : the dataset repo JSHNSL/ffw_sg2_ltable_merge → auto-downloaded/cached from HF (needs online + login)
⚠️ Do NOT use the model repo (JSHNSL/humanoid-imitation-learning) — it 404s as a dataset lookup. Use a dataset-type repo only.
To use a local folder → add --dataset.root=<folder with meta/info.json> (e.g. your own ./datasets/lerobot/<timestamp>)
--num_workers=8 --dataset.video_backend=pyav : avoids the worker segfault. (Safest is --num_workers=0, but slower.)
--policy.push_to_hub=false : upload manually to the repo in stage 2
Diffusion ModuleNotFoundError: diffusers → lerobot-pip install diffusers
Run them one at a time (concurrent runs OOM the GPU). Dataset must be v3.0 (if v2.1, convert via 3.5).

Checking training results

During training (terminal): step:200 ... loss:0.12 grdn:0.30 updt_s:0.01 ... → if loss keeps decreasing, it's fine
Checkpoints: outputs/train/<NAME>/checkpoints/ (020000/, last/ ...)
Final policy files: outputs/train/<NAME>/checkpoints/last/pretrained_model/ (config.json + model.safetensors)
Real performance (success rate): measured by the stage-4 sim rollout (this is the true evaluation)
Once verified → upload models/<NAME> in stage 2

3.2 Hyperparameter tuning — Optuna (`hpo_optuna.py`)

Give it a policy name + a dataset → it searches, saves the best config, trains with it, and uploads.

lerobot-pip install optuna        # once

Two files required: hpo_optuna.py + hpo_train_shim.py (same folder). The shim is the launcher that runs each trial's training.

One command: search → train → deploy

lerobot-hf auth login          # once, if you want to upload

lerobot-python hpo_optuna.py \
    --policy diffusion \
    --repo_id JSHNSL/ffw_sg2_ltable_merge \
    --n_trials 12 \
    --steps 5000 \
    --num_workers 8 \
    --then_train \
    --full_steps 100000 \
    --push_to_hub \
    --hub_name diffusion_tuned \
    --fresh

--policy: act | vqbet | diffusion

Outputs

Best config → outputs/hpo/<policy>/best_params.json
Trained policy → outputs/train/<policy>_tuned/checkpoints/last/pretrained_model/
Upload → models/<hub_name>/ in JSHNSL/humanoid-imitation-learning (weights + hpo_best_params.json)

Running the stages separately

What you want	Command
Search only	`hpo_optuna.py --policy act --n_trials 12 --steps 5000`
Train from the saved best	`hpo_optuna.py --policy act --train_best`
Train from the saved best + deploy	`hpo_optuna.py --policy act --train_best --push_to_hub`
Show results	`hpo_optuna.py --policy act --report_only`
Start over	add `--fresh` to any of the above

--then_train = "also train", --push_to_hub = "also deploy". With neither, it only searches. Re-running without --fresh accumulates trials in the existing study.

Key options

Flag	Default	Description
`--n_trials`	`20`	configs to try. With the core space (4–5 dims), 10–15 is enough
`--steps`	`8000`	steps per trial. Do not cut this too low (see below)
`--num_workers`	`8`	dataloader workers. At 8 the GPU is already saturated — more does not help
`--val_ratio`	`0.2`	hold out 20% of the episodes as val
`--fresh`	—	discard the existing study and start over
`--space`	`core`	see below
Full run
`--full_steps`	`100000`	steps for the full training run
`--save_freq`	`full_steps/5`	checkpoint interval
`--full_batch_size`	the tuned value	override when it OOMs
`--full_num_workers`	`--num_workers`	workers for the full run
`--hub_name` / `--hub_repo_id`	`<policy>_tuned` / humanoid repo	upload target
`--train_extra`	—	pass the rest straight to lerobot_train (must come last)

What gets tuned — `--space core` (default)

Policy	Searched
ACT	`optimizer_lr`, `optimizer_lr_backbone`, `optimizer_weight_decay`, `dropout`, `batch_size`
VQ-BeT	`optimizer_lr`, `optimizer_vqvae_lr`, `optimizer_weight_decay`, `dropout`, `batch_size`
Diffusion	`optimizer_lr`, `optimizer_weight_decay`, `scheduler_warmup_steps`, `batch_size`

chunk_size · horizon · n_action_steps · kl_weight · offset_loss_weight and friends are fixed at the lerobot defaults, not searched. Why:

The objective is val loss, and those hyperparameters change the definition of the loss itself.

Prediction length (horizon / chunk_size): predicting 8 steps ahead is inherently easier than 32, so it scores a lower loss → the search would always pick the shortest chunk, even though long action chunks are usually what makes these policies work.

Loss weights (kl_weight, offset_loss_weight): they multiply terms of the loss, so smaller simply means a smaller number — the minimum wins regardless of quality.

In other words, including them lets the search game the objective. What remains (learning rates, weight decay, dropout, batch size) only changes how well the same objective is optimised, so it can be ranked fairly.

--space full searches everything anyway, but prints the bias warning above.

Objective = held-out validation loss

LeRobot has no validation split and its built-in eval_freq only works for gym envs, so we split the episodes ourselves (--dataset.episodes). Each trial trains briefly on the train episodes, then that checkpoint is scored on the val episodes using the same preprocessor lerobot trains with; Optuna minimises that.

⚠️ Val loss is a ranking proxy — it does not prove task success. Make the final call with the stage-4 sim rollout success rate.

Budget

Measured at 5.7 step/s for diffusion (RTX PRO 5000, batch 16):

Setting	Total compute	Search time
`--n_trials 12 --steps 5000` (recommended)	60k steps	~3 h
`--n_trials 20 --steps 8000`	160k steps	~8 h

Plus ~5 h for the --full_steps 100000 run. (Pruning kills hopeless trials early, so it is faster in practice.)

⚠️ To save time, lower --n_trials — do NOT lower --steps. Short trials make an aggressively high learning rate look good (it drops fast early, then stalls), so the search picks a value that is bad for the real run.

⚠️ Parallel trials (--n_jobs) do not pay off on this GPU: a single run already uses 100% GPU / 10.5 GB.

Run long jobs inside tmux (especially on EC2)

If SSH drops, the training dies with it (connection reset by peer).

tmux new -s hpo
# ... enter the container and run the command above ...
# leave it: press Ctrl+B then D   (training keeps running)

tmux attach -t hpo      # reattach after reconnecting

3.5 Convert an old v2.1 dataset → v3.0 (lerobot-python)

Only if you have an older standalone v2.1 dataset repo. (This repo's data is already v3.0, so not needed.)

lerobot-python -m lerobot.scripts.convert_dataset_v21_to_v30 \
    --repo-id <your-old-v2.1-dataset-repo> \
    --push-to-hub true

Requires HF login · online (unset HF_HUB_OFFLINE if set)
If already v3.0, add --force-conversion
Newly produced data is already v3.0 (from the 0.5.2 converter) → no conversion needed

4. Sim rollout evaluation (python = isaac)

Why conversion is needed: the rollout loads the policy inside Isaac Sim (py3.11 / lerobot 0.3.3). Isaac Sim is 3.11 so it can't take 0.5.2 (≥3.12) → a 0.5.2-trained policy must be converted for 0.3.3 before rollout. (An older 0.3.3 policy can be used directly by repo_id, no conversion.)

4-a. isaac python setup (one-time)

$ISAACLAB_PATH/_isaac_sim/python.sh -m pip install --no-deps lerobot==0.3.3
$ISAACLAB_PATH/_isaac_sim/python.sh -m pip install \
    draccus deepdiff jsonlines termcolor packaging imageio datasets \
    einops safetensors huggingface_hub
# only when using diffusion:  diffusers

4-b. Convert 0.5.2 policy → 0.3.3 (only for policies you trained yourself, lerobot-python)

This repo's models/*_v03 are already converted → skip this step and go to 4-c. (Only needed if you trained your own.)

# --policy: your own training output folder (or a repo_id / downloaded folder)
lerobot-python scripts/sim2real/imitation_learning/inference/convert_policy_v05_to_v03.py \
    --policy ./outputs/train/vqbet_merge/checkpoints/last/pretrained_model \
    --out ./models/vqbet_merge_v03

Cleans up the config + injects normalization stats → makes it loadable by 0.3.3 (the architecture weights are unchanged, so it's fine)
If a new config field errors, add that field to the script's _STRIP_KEYS (as done for diffusion)
If there's a shape mismatch after conversion, the architecture changed → you'd need DDS (ACT/VQ-BeT/Diffusion are confirmed OK)
To keep the converted policy in the repo → upload to models/<name>_v03 (stage-2 style)

4-c. Run the rollout (python = isaac)

HF_HUB_OFFLINE=1 python scripts/sim2real/imitation_learning/inference/eval_policy_sim.py \
    --task Cyclo-Real-Pick-Place-LTable-FFW-SG2-v0 \
    --robot_type FFW_SG2 \
    --policy ./models/vqbet_merge_v03 --policy_type vqbet \
    --num_rollouts 10 --enable_cameras \
    --scripted_l_motion

--policy : this repo's ./models/<name>_v03 (or your own converted folder) / a repo_id (old 0.3.3 policy)
Switch policy: --policy_type act | vqbet | diffusion + point --policy to that policy (e.g. ./models/diffusion_merge_v03 --policy_type diffusion)
--scripted_l_motion: required for L-Table (base transport is scripted; the policy only learns grasp/place)
Force kill: from another terminal pkill -9 -f eval_policy_sim.py

4-d. (Alternative) Use 0.5.2 as-is — DDS 2-process

To use a 0.5.2 policy without conversion, serve the policy in a separate 3.12 process + talk to Isaac Sim (3.11) over DDS (the cyclo_intelligence approach). No conversion needed but heavier setup → for real-robot / production. For a quick sim check, the 4-b (conversion) route is more practical.

One-time environment fixes (reference)

issue	fix (all in the lerobot venv)
Upgrade to lerobot 0.5.2 (v3.0)	`lerobot-pip install "lerobot @ git+https://github.com/ROBOTIS-GIT/lerobot-cyclo.git"`
0.5.2 av conflict (`av.option`)	`lerobot-pip install "av>=15.0.0,<16.0.0"`
0.5.2 training needs accelerate	`lerobot-pip install "lerobot[training] @ git+https://github.com/ROBOTIS-GIT/lerobot-cyclo.git"`
Blackwell GPU (sm_120) torch	0.5.2 defaults to cu128, auto-resolved. (old 0.3.3: `lerobot-pip install --index-url https://download.pytorch.org/whl/cu128 --force-reinstall "torch==2.7.1" "torchvision==0.22.1"`)
Video-decode segfault during training	`--num_workers=0` (or `apt install ffmpeg`)
lerobot deps missing in isaac python	see 4-a (0.3.3 for rollout)

Policy types

act (default, solid) · vqbet (multimodal) · diffusion (multimodal, slow inference) At any stage, just change --policy.type / --policy_type and everything else stays the same.

One-line summary: convert (lerobot-python) → upload (lerobot-python) → train (lerobot-python) → rollout (python + HF_HUB_OFFLINE=1)

Downloads last month: -; Downloads are not tracked for this model. How to track

Inference Providers NEW

This model isn't deployed by any Inference Provider. 🙋 Ask for provider support