Luffuly/memorizon

Memorizon: Training World Models Beyond Their Context Window

Tingting Liao  ·  Xuezhi Liang  ·  Hao Li  ·  Guangyi Liu
_{Institute of Foundation Models (IFM), MBZUAI}

     

---

✨ This checkpoint

Model	Memorizon, 4-step distilled (Self-Forcing + DMD, CFG distilled in)
Base	Wan2.2-TI2V-5B
Input	one image + keyboard actions or a camera trajectory
Output	864×480, 16 fps, generated one second at a time
Sampling	4 steps, no CFG (read from the config)

🚀 Usage

git clone https://github.com/TingtingLiao/memorizon.git && cd memorizon && pip install -e .

python scripts/generate.py --checkpoint Luffuly/memorizon --image photo.jpg \
    --actions "w*16 l*24 w*16 l*24" --output out.mp4

from memorizon import MemorizonPipeline, actions_to_c2w, save_video

pipe = MemorizonPipeline.from_pretrained("Luffuly/memorizon")
video = pipe("photo.jpg", actions_to_c2w("w*16 l*24 w*16 l*24"))
save_video(video, "out.mp4")

🎮 Keyboard actions — one per 0.25 s

Key	Action	Key	Action
w s	forward / backward · 0.25 m	j l	turn left / right · 7.5°
a d	left / right · 0.25 m	i k	look up / down · 7.5°
.	stay	*N	repeat N times

Prompts — captioned automatically

Without a prompt, the image is captioned by Qwen3-VL-8B in the training format (memorizon/caption.py). Custom prompts should follow it — a perspective prefix, then one paragraph:

First-person perspective — character not visible. A winding paved path curves beneath a canopy of vibrant pink cherry blossoms, …

Camera trajectories

Instead of actions, pass --trajectory a [T, 4, 4] array of camera-to-world poses, T = 1 + 4 × seconds, in the first frame's coordinates (OpenCV axes), translations in metres / 4.

📖 Citation

@article{memorizon2026,
  title   = {Memorizon: Training World Models Beyond Their Context Window},
  author  = {Tingting Liao, Xuezhi Liang, Hao Li, Guangyi Liu},
  year    = {2026},
  note    = {arXiv preprint}
}