jsiburian/vla0-3b-ur5e-pick-3obj-90

VLA-0 (3B) - UR5e pick-3obj-90

Vision-Language-Action policy fine-tuned from Qwen/Qwen2.5-VL-3B-Instruct on jsiburian/ur5e-pick-3obj-90-lerobot-v2 (90 teleoperated UR5e demonstrations spanning 12 pick tasks across 3 distinct target objects) using the VLA-0 recipe.

Model details

Base model	Qwen/Qwen2.5-VL-3B-Instruct
Method	VLA-0 (full fine-tune, 1000 discrete action bins, paired with language instruction)
Robot	UR5e (7-DoF)
Cameras	extra (3rd-person) + wrist (tiled side-by-side)
Action horizon	8
History	1
Image size	224 x 224 (tiled), random crop 0.9, brightness/contrast/saturation/hue aug

Training

Hardware	4 x NVIDIA H200
Per-GPU batch	16 (effective batch 64)
Optimizer	AdamW, effective lr=4e-5 (config lr 1e-5 x num_gpus), weight decay=1e-10
Schedule	Constant, AMP, FlashAttention-2
Iterations	12,500 (~14 outer epochs over 90 episodes)
Final train loss (per-rank epoch mean)	~0.031 at iter 12,500
Action mask aug prob	0.4

Loss is the standard VLA-0 next-token cross-entropy over discretized action tokens (vocab of 1000 bins per dim, 56 action tokens per sample). Random-guess baseline is ln(1000) ~ 6.91.

Files

Path	Purpose
model-0000{1,2}-of-00002.safetensors	Sharded Qwen2.5-VL weights
config.json, generation_config.json	HF model config
preprocessor_config.json, chat_template.json	Image + text processors
tokenizer*.json, vocab.json, merges.txt, added_tokens.json, special_tokens_map.json	Tokenizer
dataset_stats.pkl	Action de-normalization stats - required at inference
training_config.yaml	Full training-config snapshot from the run

dataset_stats.pkl is a Python pickle of {"out_ori_act": {"min": np.ndarray(7,), "max": np.ndarray(7,), "count": np.ndarray(1,)}}, which carries the per-dimension action min / max used to normalize / decode the 1000-bin action tokens.

Usage

Use the VLA-0 repo (or our RoboVerse-flavored fork) for inference; load weights directly with Qwen2_5_VLForConditionalGeneration.from_pretrained(...) and pair with dataset_stats.pkl for bin <-> continuous action conversion.

The dataset's task strings (12 of them) are short imperatives like "pick up the carrot", "pick up the cube", etc. Match the phrasing of your task prompt to one of those instructions when sampling actions.

Caveats

• Single-task imitation: no LR schedule, no test split, no held-out object. The model is fit hard to the 90-episode demonstration distribution.
• Two-camera tiled input - if you only have a third-person view at inference, results will be worse than what training metrics suggest.
• The companion PI-0.5 LoRA on the same dataset lives at jsiburian/pi05-ur5e-pick-3obj-90-lora.