RLDX-1.1-PT-IMG

RLDX-1.1 overview

RLDX-1.1 is a Vision-Language-Action model (VLA) for human-like dexterous manipulation. Beyond the versatile intelligence inherited from pre-trained VLM backbones, RLDX-1.1 adds three functional capabilities — motion awareness, long-term memory, and physical sensing — through a unified Multi-Stream Action Transformer (MSAT) architecture, a synthetic-augmented training pipeline, and a real-time inference stack.

This repository hosts RLDX-1.1-PT-IMG, the pre-trained base checkpoint with single-image input. Every RLDX-1.1-FT-*-IMG checkpoint is fine-tuned from it. Use it as the starting point for new embodiments and tasks.

What's New in RLDX-1.1

Better performance. A refined RLDX-1 corpus plus a broad collection of open-source robot datasets gives a stronger pre-training foundation. Video-input fine-tunes enable the motion module, which RLDX-1 video checkpoints did not use.
Smoother actions in low DoF. Padded action dimensions are masked from pre-training on, so flow-matching noise no longer reaches unused joints. Keep mask_padded_action_dims=True when fine-tuning from this checkpoint.
Faster training and inference. Training is 2.27× faster than RLDX-1. Inference is up to 9.4× faster on H100 and 3.7× on RTX 5090, with fp8 on every supported GPU and nvfp4 on Jetson Thor, GB10 and RTX 5090. See Optimization.

Details are in the README.

RLDX-1.1 architecture

Model Family

Video input

Checkpoint	Type	Description
`RLDX-1.1-PT`	Base	Pre-trained
`RLDX-1.1-FT-ROBOCASA`	Fine-tuned	RoboCasa Kitchen
`RLDX-1.1-FT-GR1`	Fine-tuned	GR-1 Tabletop
`RLDX-1.1-FT-LIBERO`	Fine-tuned	LIBERO
`RLDX-1.1-FT-RC365`	Fine-tuned	RoboCasa365

Image input

Checkpoint	Type	Description
`RLDX-1.1-PT-IMG` (this repo)	Base	Pre-trained
`RLDX-1.1-FT-ROBOCASA-IMG`	Fine-tuned	RoboCasa Kitchen
`RLDX-1.1-FT-GR1-IMG`	Fine-tuned	GR-1 Tabletop
`RLDX-1.1-FT-LIBERO-IMG`	Fine-tuned	LIBERO
`RLDX-1.1-FT-RC365-IMG`	Fine-tuned	RoboCasa365

RLDX-1 checkpoints also load with this codebase; see Model Checkpoints.

Performance

Success rate (%) of the RLDX-1.1 checkpoint fine-tuned from this base on each benchmark's training set.

Benchmark	Success Rate	Checkpoint
RoboCasa Kitchen	70.4	`RLDX-1.1-FT-ROBOCASA-IMG`
GR-1 Tabletop	60.6	`RLDX-1.1-FT-GR1-IMG`
LIBERO	97.2	`RLDX-1.1-FT-LIBERO-IMG`
RoboCasa365	33.2	`RLDX-1.1-FT-RC365-IMG`

The comparison with other VLAs is in the README.

Quick Start

git clone --branch v1.1.0 https://github.com/RLWRLD/RLDX-1.git
cd RLDX-1
uv sync --python 3.10

Load a checkpoint and predict an action chunk in-process:

from rldx.serving.policy import load_policy
from rldx.data.embodiment_tags import EmbodimentTag

policy = load_policy(
    model_path="RLWRLD/RLDX-1.1-FT-ROBOCASA-IMG",
    embodiment_tag=EmbodimentTag.GENERAL_EMBODIMENT,
    device="cuda:0",
)

# observation: {"video": {...}, "state": {...}, "language": {...}}
actions, info = policy.get_action(observation)

The example loads a fine-tuned checkpoint. The base checkpoint keeps normalization statistics only for OXE_FRACTAL, OXE_BRIDGE_ORIG and OXE_DROID; for any other robot, fine-tune first.

Serve the same policy over ZeroMQ for simulator rollouts or a real robot:

uv run python rldx/eval/run_rldx_server.py \
    --model-path RLWRLD/RLDX-1.1-FT-ROBOCASA-IMG \
    --embodiment-tag GENERAL_EMBODIMENT \
    --host 0.0.0.0 --port 20000

Runtime profiles (fp8 / nvfp4, CUDA graphs) and Real-Time Chunking are described in docs/inference_server.md.

Fine-tune from `RLDX-1.1-PT-IMG`

uv run torchrun --nproc_per_node=<N_GPUS> rldx/experiment/launch_train.py \
    --base-model-path RLWRLD/RLDX-1.1-PT-IMG \
    data:paths --data.paths /path/to/your/dataset \
    --data.embodiment-tag GENERAL_EMBODIMENT \
    --modality-config-path /path/to/your_modality_config.py \
    --video-length 1 \
    --n-cog-tokens 64 \
    --num-gpus <N_GPUS> \
    --global-batch-size 64 \
    --learning-rate 1e-4 \
    --max-steps 60000 \
    --save-steps 5000 \
    --output-dir ./outputs/my_finetune

--video-length 1 matches this image-input checkpoint. Memory, physics, LoRA and the full flag list are in docs/training.md.

Model Details

Architecture: Multi-Stream Action Transformer (MSAT) with a Qwen3-VL vision-language backbone and cognition tokens, trained with flow matching.
Inputs: a single RGB image per camera, proprioceptive state, and a language instruction.
Outputs: action chunks of 16 steps.
Modules: motion module off; memory and physics off. Padded action dimensions are masked (mask_padded_action_dims=True).
Backbone: initialized from Qwen/Qwen3-VL-8B-Instruct and modified through further training (see Third-Party Notice).
Parameters: 6.6B.

A walkthrough of the architecture is in docs/architecture.md.

Intended Use and Limitations

Intended use. Research on robotic manipulation, fine-tuning on custom embodiments, simulation benchmarking, and non-commercial real-robot deployment under the RLWRLD Model License v2.0. Commercial use requires prior written permission from RLWRLD (§3.1).

Out of scope. Surveillance or monitoring of individuals without their consent, and any use that violates applicable laws or regulations (§3.5 of LICENSE.md).

Limitations. Performance depends heavily on embodiment match and data distribution. The base checkpoint is not expected to work zero-shot on embodiments outside its pre-training data; fine-tune it first. Memory and physics modules are not part of this checkpoint and are trained from scratch when enabled during fine-tuning.

Third-Party Notice

The vision-language backbone of this checkpoint was initialized from Qwen3-VL-8B-Instruct by the Qwen Team, Alibaba Cloud, licensed under the Apache License 2.0, and was modified by RLWRLD through further training. See NOTICE.

Citation

@article{rldx2026,
  title={RLDX-1 Technical Report},
  author={Dongyoung Kim and Huiwon Jang and Myungkyu Koo and Suhyeok Jang and Taeyoung Kim and others},
  year={2026},
  journal={arXiv preprint arXiv:2605.03269},
  eprint={2605.03269},
  archivePrefix={arXiv}
}

Please also cite the Qwen3-VL backbone and RoboCOIN, whose data was used in pre-training.

@misc{qwen3technicalreport,
      title={Qwen3 Technical Report},
      author={Qwen Team},
      year={2025},
      eprint={2505.09388},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2505.09388},
}

@article{robocoin,
    title={RoboCOIN: An Open-Sourced Bimanual Robotic Data Collection for Integrated Manipulation},
    author={Shihan Wu, Xuecheng Liu, Shaoxuan Xie, Pengwei Wang, Xinghang Li, Bowen Yang, Zhe Li, Kai Zhu, Hongyu Wu, Yiheng Liu, Zhaoye Long, Yue Wang, Chong Liu, Dihan Wang, Ziqiang Ni, Xiang Yang, You Liu, Ruoxuan Feng, Runtian Xu, Lei Zhang, Denghang Huang, Chenghao Jin, Anlan Yin, Xinlong Wang, Zhenguo Sun, Junkai Zhao, Mengfei Du, Mingyu Cao, Xiansheng Chen, Hongyang Cheng, Xiaojie Zhang, Yankai Fu, Ning Chen, Cheng Chi, Sixiang Chen, Huaihai Lyu, Xiaoshuai Hao, Yequan Wang, Bo Lei, Dong Liu, Xi Yang, Yance Jiao, Tengfei Pan, Yunyan Zhang, Songjing Wang, Ziqian Zhang, Xu Liu, Ji Zhang, Caowei Meng, Zhizheng Zhang, Jiyang Gao, Song Wang, Xiaokun Leng, Zhiqiang Xie, Zhenzhen Zhou, Peng Huang, Wu Yang, Yandong Guo, Yichao Zhu, Suibing Zheng, Hao Cheng, Xinmin Ding, Yang Yue, Huanqian Wang, Chi Chen, Jingrui Pang, YuXi Qian, Haoran Geng, Lianli Gao, Haiyuan Li, Bin Fang, Gao Huang, Yaodong Yang, Hao Dong, He Wang, Hang Zhao, Yadong Mu, Di Hu, Hao Zhao, Tiejun Huang, Shanghang Zhang, Yonghua Lin, Zhongyuan Wang and Guocai Yao},
    journal={arXiv preprint arXiv:2511.17441},
    url = {https://arxiv.org/abs/2511.17441},
    year={2025}
}

License

Released under the RLWRLD Model License v2.0, a non-commercial license with attribution and share-alike terms. See LICENSE.md for the full text. By using this model you agree to those terms, including the use restrictions in §3.5. The Qwen3-VL component remains subject to the Apache License 2.0 as described in NOTICE.