StarVLA/VLAct_Qwen3_Pretrain

VLAct · Qwen3-VL-4B Continued-Pretraining Backbone

This repository contains the 100K-step VLAct continued-pretraining checkpoint introduced in Beyond Data Scaling: Representation-Centric Continued Pre-training for Vision-Language-Action Models. It is the recommended VLAct starting point for adapting Qwen3-VL-4B to a new robot, dataset, benchmark, or continuous action head.

This is a training checkpoint and reusable VLM/action backbone, not a directly deployable robot policy and not a standard transformers.AutoModel package. A downstream policy must define its camera, state, action, normalization, and control contracts, then be fine-tuned for the target embodiment. For a new or incompatible action space, initialize the downstream action head from scratch and transfer the shared VLAct backbone.

What is VLAct?

VLAct asks how a fixed amount of robot data can teach a backbone more transferable visual-action knowledge. Its continued-pretraining recipe combines three ideas:

1. Preserve the VLM prior. Freeze the vision encoder and lower half of the language model while mixing image-caption supervision with robot trajectories.
2. Diversify action supervision. Co-train OFT, PI, and GR00T continuous action heads on a shared latent representation, reducing specialization to one decoder.
3. Share action semantics across embodiments. Use a partially unified, padded 20-D action layout, mask inactive dimensions, and apply a wrap-aware objective to periodic joints.

During downstream fine-tuning, the pretraining heads can be discarded. The complete method, ablations, and evaluation protocols are documented in the paper and code repository.

Checkpoint details

Item	Value
Framework	StarVLA QwenHybrid_xrobot_padding
Base VLM	StarVLA/Qwen3-VL-4B-Instruct-Action
Pretraining heads	OFT + GR00T + PI, equal loss weights (1:1:1)
Action representation	Continuous, partially unified cross-embodiment padded layout
Action / state dimensions	20-D padded action / 14-D state
Action horizon	50 steps (future_action_window_size: 49)
Frozen modules	Vision encoder and language-model layers 0–17
Training step	100,000
Training hardware	16 GPUs
Per-GPU VLA batch size	16
Optimizer	AdamW, 5K-step warmup, cosine schedule
Learning rates	Qwen-VL interface 1e-5; action/base modules 1e-4
Seed	42

The checkpoint packages all three pretraining heads because they formed the co-supervision objective. Their presence does not mean a downstream policy must use any particular head.

Pretraining data

The VLA mixture combines heterogeneous AgileX/ALOHA and Franka data from five open dataset groups:

• InternData-A1
• RoboCOIN
• DROID
• DROID-100
• MolmoAct

The run also mixes image-text supervision from ShareGPT4V-COCO and LLaVA-ReCap CC3M. Dataset sampling is balanced across embodiments, inactive action dimensions are masked, and single-arm and dual-arm loss weights are 8.0 and 1.0, respectively.

See the data-preparation and continued-pretraining guide for the expected LeRobot layout, preprocessing commands, and launch configuration.

Recommended use: downstream fine-tuning

1. Install StarVLA

git clone https://github.com/starVLA/VLAct.git
cd VLAct

conda create -n vlact python=3.10 -y
conda activate vlact

# Install a CUDA-compatible PyTorch build first.
python -m pip install -r requirements.txt
python -m pip install flash-attn==2.7.4.post1 --no-build-isolation
python -m pip install -e .

2. Download the checkpoint

Run from the VLAct repository root:

hf download StarVLA/VLAct_Qwen3_Pretrain \
  --local-dir playground/Pretrained_models/VLAct-Qwen3VL4B-Pretrained

The checkpoint path is then:

playground/Pretrained_models/VLAct-Qwen3VL4B-Pretrained/checkpoints/steps_100000_pytorch_model.pt

Keep the downloaded directory structure unchanged. StarVLA resolves config.yaml and dataset_statistics.json from the run directory two levels above the checkpoint file.

3. Configure a downstream launcher

Set the launcher's pretrained_ckpt to the path above and retain these transfer settings:

--trainer.pretrained_checkpoint ${pretrained_ckpt} \
--trainer.random_init_action_model True \
--trainer.reset_steps True

random_init_action_model=True skips the checkpoint's pretraining action module while loading the shared backbone into the downstream framework. This is the recommended setting when the target head, action dimension, or embodiment differs from pretraining.

The repository includes launchers for RoboTwin 2.0, LIBERO-Plus, VLA-Arena, and DOMINO. For example:

bash scripts/run_scripts/RoboTwin/train_robotwin_qwen3oft.sh
bash scripts/run_scripts/LIBERO/train_libero_qwen3pi.sh
bash scripts/run_scripts/VLA-Arena/train_vla_arena_qwen3pi.sh
bash scripts/run_scripts/DOMINO/train_domino_qwen3oft.sh

Review each launcher's marked configuration block before training, especially base_vlm, dataset paths, run_root_dir, pretrained_ckpt, action horizon, and image-size settings.

Loading the complete pretraining model

The full checkpoint can be reconstructed only with the matching StarVLA framework and the packaged configuration. This mode is useful for inspecting the three-head model or continuing model training with the same architecture. It is generally not the right path for a new embodiment.

The .pt file contains model parameters only; it does not package optimizer or scheduler state. Loading it therefore does not reproduce a bit-for-bit training resume.

Files

VLAct-Qwen3VL4B-Pretrained/
├── README.md
├── config.yaml
├── training_config.original.yaml
├── dataset_statistics.json
├── summary.jsonl
└── checkpoints/
    └── steps_100000_pytorch_model.pt

File	Purpose
checkpoints/steps_100000_pytorch_model.pt	Full PyTorch state dict for the three-head model
config.yaml	Portable resolved configuration using the public base-model ID
training_config.original.yaml	Original resolved run configuration and local base-model path
dataset_statistics.json	Dataset statistics used by StarVLA normalization utilities
summary.jsonl	Saved-checkpoint step history

Checkpoint SHA-256:

5da4b96f9c923eabf35b7ac4c8928bef847266a044dbb8a239e3696aa305095d

Paper results

The following numbers come from downstream policies fine-tuned from VLAct; they are not zero-shot scores of this raw checkpoint. Controlled comparisons keep the downstream action head, initialization, data, optimizer, and fine-tuning budget fixed while changing the backbone weights.

Benchmark	VLAct	Matched Qwen3-VL-OFT baseline
LIBERO-Plus	82.6%	75.0%
VLA-Arena	54.8%	33.4%
RoboTwin 2.0 Base, Clean	80.5%	61.7%
RoboTwin 2.0 Data Scaling, Clean / Random	92.5% / 90.8%	88.2% / 88.3%
DOMINO, SR / MS	18.50 / 34.20	10.86 / 30.49

VLAct also reaches 49.5% on RoboCasa-GR1 with only 20% of the downstream trajectories and ranks 6th of 35 policies by success rate in the paper's August 24, 2026 RoboDojo leaderboard snapshot.

Configuration notes

The packaged files preserve several source-run details worth noting:

• training_config.original.yaml records a VLM auxiliary-loss scale of 0.2; the paper defines the reported caption objective with weight 0.5.
• The packaged config keeps the source run's legacy mixture key, agilex_franka_5data_manualvel_balance33_66_50; the public launcher uses agilex_franka_5data_manualvel_balance_50.

Use the packaged configuration to inspect or reproduce this specific artifact, and use the paper for the reported experimental protocol and results.

Intended use and limitations

This checkpoint is intended for research on VLA representation learning, action-head transfer, multi-embodiment pretraining, and downstream robot-policy adaptation.

• It has not been validated as a zero-shot or directly deployable policy.
• Safe deployment requires embodiment-specific action mapping, normalization, camera calibration, control-rate handling, workspace constraints, and independent safety systems.
• The pretraining action heads use a specific padded 20-D layout and are not automatically compatible with another robot or framework.
• Performance depends on downstream data, head design, optimization, observation configuration, and evaluation protocol.
• The model may inherit limitations and biases from its base VLM and pretraining datasets.

Citation

@misc{yang2026vlact,
  title   = {Beyond Data Scaling: Representation-Centric Continued Pre-training
             for Vision-Language-Action Models},
  author  = {Yang, Senqiao and Wang, Chengyao and Chen, Yuxin and Wang, Zixuan and
             Tang, Longxiang and Gui, Haokun and Ye, Jinhui and Lu, Changsheng and
             Wu, Xiaoyang and Zhu, Mingkang and Chen, Pengguang and Liu, Shu and
             Tian, Zhuotao and Zhao, Hengshuang and Yu, Bei and Jia, Jiaya},
  year    = {2026},
  month   = aug,
  note    = {Preprint},
  url     = {https://starvla.github.io/VLAct/}
}

License and acknowledgements

The checkpoint is released under the Apache License 2.0. The VLAct code repository is released separately under the MIT License. Users must also comply with the licenses and terms of the base model and training datasets.

VLAct builds on StarVLA, LeRobot, GR00T, and Qwen3-VL.

For questions, email yangsenqiao.ai@gmail.com or open an issue in the VLAct repository.