LabVLA
Collection
5 items • Updated • 2
LabVLA-5B-LabUtopia-TransportBeaker
Task-specific LabVLA-5B fine-tuned model for the LabUtopia Level3 TransportBeaker robotic laboratory task.
Draft release: Private pending review of redistribution rights of the historical warm-start checkpoint and its contributing RoboInter and Open X-Embodiment datasets. No unverified model license is claimed.
Model
- Architecture: Qwen3-VL vision-language backbone with Diffusion Transformer (DiT) flow-matching action expert.
- Task: TransportBeaker, single-arm Franka manipulation, three RGB cameras.
- Actual initialization: mixed RoboInter + Open X-Embodiment Knowledge Isolation checkpoint at 20,000 steps. This is not direct fine-tuning from the published LabVLA-5B-Base weights.
- Task fine-tuning: 40,000 optimizer steps, effective global batch 192 (4 GPUs × per-GPU batch 48 × Gradient Accumulation 1), Learning Rate 5e-5, Warmup 2,000 steps.
- Trainable Vision Encoder, Language Model Gradient Checkpointing.
- Precision: bfloat16; Action Chunk Size: 50; inference denoising steps: 10.
- Raw action and state: 8 dimensions. The first seven arm dimensions are Delta Action; the last dimension is absolute gripper width.
- Output: self-contained inference-ready Safetensors checkpoint, not a fully resumable optimizer checkpoint.
The base_model metadata links the upstream LabVLA architecture and family; the actual mixed Knowledge Isolation initialization differs and is documented above.
Evaluation
Full 120-episode runs in the official LabUtopia Level3 TransportBeaker simulation with the Franka asset:
| Material Split | Success | Success Rate |
|---|---|---|
| Test Material (Out-of-Distribution) | 114 / 120 | 95.00% |
| Training Material (In-Distribution) | 118 / 120 | 98.33% |
These are simulation success rates, not physical-robot success rates.
Files and Loading
Keep this exact structure:
- pretrained_model/model.safetensors — 997 tensors, approximately 9.44 GB
- pretrained_model/config.json and vlm_config.json
- pretrained_model/tokenizer.json, tokenizer_config.json, merges.txt and vocab.json
- pretrained_model/chat_template.json and preprocessor files
- labvla_schema.json — three-camera mapping and Action Schema
- norm_stats.json — original training-time normalization
Download with the Hugging Face CLI and use the repository root as the custom LabVLA inference server's pretrained_path. Use the associated pretrained_model directory for local tokenizer/processor assets. A generic Transformers AutoModel loader is not sufficient to load this LabVLA checkpoint.
The bundled norm_stats.json matches the associated TransportBeaker dataset's meta/stats_canonical_grip.json byte-for-byte (SHA256 9e9da9599949fe24ad7c241cc63af930962e02faa4cd02d8a4c8096b645a4230). Do not replace it with generic dataset statistics.
Limitations and Licensing
Evaluated in simulation for this task only. The original LabUtopia data assets are CC BY-NC 4.0; source-code licensing is separate. The additional RoboInter/Open X-Embodiment initialization sources require a redistribution-license review before public release.
References
- LabVLA code and deployment: https://github.com/zjunlp/LabVLA
- Upstream model: https://huggingface.co/zjunlp/LabVLA-5B-Base
- LabUtopia: https://github.com/Rui-li023/LabUtopia
- LabVLA paper: https://arxiv.org/abs/2606.13578
Please cite the original LabVLA and LabUtopia papers when using this checkpoint.
Deployment Reproducibility
See DEPLOYMENT.md for the complete tested Policy Server command, Action Normalization settings, three-camera message protocol, and a real-observation inference smoke test. The checkpoint requires a compatible LabVLA loader supporting self-contained Qwen3-VL assets; an older model loader was found to be incompatible.
Model tree for zjunlp/LabVLA-5B-LabUtopia-TransportBeaker
Base model
zjunlp/LabVLA-5B-Base