Instructions to use easylearning/FutureWorlds with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Diffusers
How to use easylearning/FutureWorlds with Diffusers:
pip install -U diffusers transformers accelerate
import torch from diffusers import DiffusionPipeline # switch to "mps" for apple devices pipe = DiffusionPipeline.from_pretrained("easylearning/FutureWorlds", dtype=torch.bfloat16, device_map="cuda") prompt = "Astronaut in a jungle, cold color palette, muted colors, detailed, 8k" image = pipe(prompt).images[0] - Notebooks
- Google Colab
- Kaggle
FutureWorlds research checkpoints
FutureWorlds learns action-conditioned robotic world models from alternative predicted futures. This repository contains the paired research checkpoints for RT-1, BridgeV2, and RoboCasa. The implementation, protocols, and reproduction instructions are in the FutureWorlds GitHub repository.
Contents
Each dataset has a main predictor, an SFT predictor, and memspo200, grpo200, and ordinary200 comparison predictors. The dataset directory also contains its paired visual codec, action-range metadata, and configuration. RT-1 and RoboCasa predictors include trained text adapters. The original FP32 safetensors are kept without quantization or variant merging.
| Dataset | Main checkpoint | Text conditioning |
|---|---|---|
| RT-1 | MemSPO, 500 updates, R0 | T5-base plus trained adapters |
| BridgeV2 | MemSPO, 200 updates, E4/R1 | None in this recipe |
| RoboCasa | MemSPO, 400 updates, R0 | T5-base plus trained adapters |
The external T5-base model is not copied into this repository. Use revision a9723ea7f1b39c1eae772870f3b547bf6ef7e6c1 of google-t5/t5-base for the text-conditioned datasets.
Loading
These are custom action-conditioned predictors, not generic pipeline("text-to-video") models. Use the dataset-specific codec, action ranges, and text adapter. The included code/ directory contains the paired low-level checkpoint loader; see the GitHub loading guide for the full RGB pipeline and the model cards for variant details. main and memspo200 are different checkpoints.
Validation and limitations
The exact uploaded-file hashes are recorded in manifest.json; weights_manifest.json records component provenance and marks whether each listed hash belongs to a source file or a release file. The code repository reports first-frame agreement with the original entry point on one fixed case per dataset. Full-horizon GPU reproduction of the consolidated release pipeline remains pending. Results in the paper and GitHub README are existing experiment results, not a fresh evaluation of this upload. The repository does not contain raw RT-1, BridgeV2, or RoboCasa datasets, optimizer states, or third-party baseline weights.
Rights and attribution
No license has been selected for the original FutureWorlds additions or these checkpoints. Public visibility does not grant permission to use, modify, or redistribute them. Upstream components and the external T5 model retain their own licenses; see licenses/ and third-party notices. Contact the project owner for permissions.
- Downloads last month
- -