hyeonsjung/so101-world-model-v1

SO-101 World Model v1

Visionary is an action-conditioned video world model for SO-100/SO-101 and WidowX robot arms. It predicts future video in a learned latent space.

Source code: github.com/james0248/visionary

Compatible code revision: 64a2395c

This is a world model. It is not a robot control policy.

Components

The release contains two coupled components:

Component	Parameters	Purpose
Tokenizer	about 137M	Convert 240 x 320 video frames to and from latent tokens
Dynamics	about 312M	Predict latent video from prior video and robot actions

Both components use the step-100,000 inference exports. The dynamics weights are the exponential moving average parameters used by the evaluation code.

The model runs at 5 Hz. It has a 48-frame dynamics context window. This is 9.6 seconds at the training rate.

Install

git clone https://github.com/james0248/visionary.git
cd visionary
git checkout 64a2395c86298b0c0a35997e5696d510a6d78e94
uv sync
uv pip install "safetensors>=0.5.3"

Download and load

import json
from pathlib import Path

from flax.core import freeze
from flax.traverse_util import unflatten_dict
from huggingface_hub import snapshot_download
from hydra.utils import instantiate
from omegaconf import OmegaConf
from safetensors.flax import load_file

from visionary.models.dreamer4.tokenizer_preprocessor import TokenizerPreprocessor


def load_component(directory):
    config = OmegaConf.create(json.loads((directory / "config.json").read_text()))
    flat = load_file(str(directory / "model.safetensors"))
    variables = freeze(unflatten_dict({tuple(name.split("/")): value for name, value in flat.items()}))
    return config, variables


root = Path(snapshot_download("hyeonsjung/so101-world-model-v1"))

tokenizer_config, tokenizer_variables = load_component(root / "tokenizer")
dynamics_config, dynamics_variables = load_component(root / "dynamics")

tokenizer = instantiate(tokenizer_config)
dynamics = instantiate(dynamics_config)
preprocessor = TokenizerPreprocessor.from_config(
    json.loads((root / "tokenizer" / "preprocessor_config.json").read_text())
)

Actions must be normalized before they enter the dynamics model:

import json
import numpy as np

stats = json.loads((root / "normalizers" / "so101_action_stats.json").read_text())
q01 = np.asarray(stats["q01"], dtype=np.float32)
q99 = np.asarray(stats["q99"], dtype=np.float32)
normalized_actions = np.clip(2.0 * (actions - q01) / (q99 - q01) - 1.0, -1.0, 1.0)

Use embodiment ID 0 for SO-101 and 1 for WidowX. See embodiments.json for the action schema.

Training data

The tokenizer was trained on fixed-camera video from 733 public SO-100/SO-101 LeRobot repositories. The source corpus contains 36,320 episodes and 174.8 hours before final stream filtering.

The dynamics model was co-trained with equal source weights:

• SO-101 absolute joint pose actions: 29,687 episodes and 2,243,913 frames.
• WidowX end-effector delta actions from BridgeData V2 and SOAR-Data: 131,413 episodes and 6,741,197 frames.

The original SO-101 downloads did not record Hugging Face commit IDs. Five SO-101 source repositories were no longer public when this release was prepared. These are reproducibility and provenance limits. BridgeData V2 uses OpenMDW-1.1. SOAR-Data uses MIT. The source code has the data preparation details.

License

The Visionary code and model files are available under Apache-2.0. Third-party dataset terms still apply to the source data. This license grants only rights that the Visionary copyright holder can grant. See NOTICE.

Citation

@software{jung2026visionary,
  author = {Hyeonseok Jung},
  title = {Visionary: Action-Conditioned Robot World Models},
  year = {2026},
  url = {https://github.com/james0248/visionary}
}