cosmos_nano_policy_so101

A 6-DOF SO-101 bin-placement action policy, post-trained from nvidia/Cosmos3-Nano on one RTX PRO 6000 (96 GB). This is checkpoint iteration 3750 of the run, the only checkpoint that has completed the task in evaluation. The LoRA adapters are merged into the base weights, and the checkpoint is exported as consolidated safetensors.

It is a research checkpoint with a low success rate. Read Benchmark before using it.

Training

Setting	Value
Base model	nvidia/Cosmos3-Nano
Experiment	action_policy_so101_nano_focus5 (cosmos-framework 5e67049 plus local SO-101 support)
Data	so101_bench_sim_6, restricted to 5 single-object instructions: green shoes, cardboard box, altoids container, flower pot, cooking spoon ("Place the X in the plastic bin"). 97 episodes, 87 used for training.
Action space	Absolute joint_pos, 6-D (5 arm joints plus gripper, LeRobot .pos units). Row 0 is the current state. Chunk of 32 steps at 30 fps.
Normalization	minmax against the calibration bounds: joints [-100, 100], gripper [0, 100] (so101_lerobot_stats.json)
Video	concat_view: front wrist camera stacked on top of the overhead camera, 480p
Method	LoRA rank 16 / alpha 32 on q/k/v/o_proj_moe_gen, plus the action heads (action2llm, llm2action, action_modality_embed) trained from random init
Schedule	Global batch 32, learning rate 1e-4 (action heads 5x), 200 warm-up steps, linear decay over 4000 iterations. This checkpoint is iteration 3750 (4.17 epochs).
Embodiment domain id	22 (so101)

Benchmark

Evaluated in the so101_bench Isaac Lab digital twin, So101Bench-Bin-v0: 100 single-object episodes (tasks/focus5.jsonl), 25 s per episode, 32 actions executed per inference call.

	Successes / episodes	Rate
Single-object, all runs to 2026-09-13	17 / 542	3.1%
Object on the same side of the table as the bin (runs on the fixed layout file)	13 / 252	5.2%
Object on the far side of the table (runs on the fixed layout file)	0 / 187	0%
4-object cluttered scenes	0 / 48	0%

• Every success had the object on the same side of the table as the bin. The far-side half of the task is unsolved.
• Nearly all failures are time-outs in which the target object never leaves the table. The policy either grasps and places the object, or never lifts it.
• Checkpoints 3500 and 4000 each scored 0 / 20. Iteration 4000 has the lowest training loss, so training loss did not predict success.
• The results depend on the action horizon: horizon 32 scored 4 / 70 while horizon 16 scored 0 / 63 on the same server and scenes.

Serving

The checkpoint is served with cosmos_framework.scripts.action_policy_server_robolab, an openpi websocket server. Every SO-101 flag below is required, and a missing one fails silently: the server starts but returns wrong actions.

huggingface-cli download kabilanKB/cosmos_nano_policy_so101 --local-dir cosmos_nano_policy_so101

python -m cosmos_framework.scripts.action_policy_server_robolab \
    --checkpoint-path cosmos_nano_policy_so101 \
    --port 8000 \
    --domain-name so101 \
    --action-dim 6 \
    --arm-joint-dim 5 \
    --action-space joint_pos \
    --conditioning-fps 30 \
    --no-flip-gripper \
    --action-normalization minmax \
    --normalizer-stats-path cosmos_nano_policy_so101/so101_lerobot_stats.json \
    --view-description 'The top half is from the front-facing wrist camera. The bottom half is from the fixed overhead camera.' \
    --no-guardrails

These flags depend on SO-101 support in the policy server (--arm-joint-dim, --no-flip-gripper, --action-normalization, --view-description), which is not in upstream cosmos-framework 5e67049. The pipeline that runs training, merge, export, serving and evaluation is in kabilankb/so101-cosmos-nano-policy.

Request format: prompt (the instruction), observation/image (the 960×640 concatenated view), observation/joint_position (5 values), observation/gripper_position (1 value).

Response: action, 32 × 6 absolute joint targets in raw .pos units.

Limitations

• Trained and evaluated only in simulation. It has not been tested on a physical SO-101.
• Covers five objects and one task family, with 18–25 demonstrations per instruction.
• Sampling is stochastic: the server draws a new seed for every request, and no scene has succeeded reliably.

License and attribution

This model is a derivative of NVIDIA Cosmos3-Nano, released under the OpenMDW License 1.1, and is distributed under the same license. Cosmos3-Nano's language backbone derives from Qwen/Qwen3-VL-8B-Instruct (Apache 2.0), whose tokenizer and processor files are bundled here.

Built with NVIDIA Cosmos. Post-trained by Kabilan KB.