starry0929/Temporal-Context-Routing

The Missing Temporal Link: Temporal Context Routing
for Script-Driven Audio-Video Generation

Yichen Liu¹, Quanwei Zhang², Haozhe Wang³, Donghao Zhou⁴, Xiaojie Li
Yang Shi², Jiaming Liu², Ruihua Huang², Yingtian Zou⁵, Daquan Zhou¹

¹ Peking University  ·  ² Qwen Applications
³ HKUST  ·  ⁴ CUHK  ·  ⁵ Shanghai Jiao Tong University

Temporal Context Routing (TCR) aligns script-specified shot and dialogue timing with the shared video-audio timeline. Timing bypasses the text encoder and enters cross-attention logits as a routing score, separating what to generate from when it should appear. Built on LTX-2.3 22B, TCR reduces shot-boundary error from 1.11 s to 0.042 s—about one frame at 24 fps—and raises dialogue timing accuracy from 28.3% to 84.1%.

This repository hosts the released LoRA (temporal-context-routing.safetensors, ~2.3 GB) and the 200 held-out test prompts (test_prompts_200.json) used in the paper. Training and inference code is at DAGroup-PKU/Temporal-Context-Routing. Demos are available on the project page.

Files

File	What it is
temporal-context-routing.safetensors	TCR LoRA, rank 128, trained on LTX-2.3 22B (paper 7k-step recipe)
test_prompts_200.json	200 held-out script JSON files with per-shot / per-line time_range

You still need the LTX-2.3 22B checkpoint and the Gemma 3 12B text encoder locally. One 80 GB GPU is the working setup.

huggingface-cli download starry0929/Temporal-Context-Routing \
  temporal-context-routing.safetensors test_prompts_200.json \
  --local-dir ./weights

Then follow the code README:

bash scripts/infer.sh \
  --checkpoint /path/to/ltx-2.3-22b-dev.safetensors \
  --text-encoder-path /path/to/gemma-3-12b-it \
  --lora-path ./weights/temporal-context-routing.safetensors \
  --output outputs/tcr_infer.mp4

Citation

@inproceedings{liu2027missingtemporallink,
  title     = {The Missing Temporal Link: Temporal Context Routing
               for Script-Driven Audio-Video Generation},
  author    = {Liu, Yichen and Zhang, Quanwei and Wang, Haozhe and
               Zhou, Donghao and Li, Xiaojie and Shi, Yang and Liu, Jiaming and
               Huang, Ruihua and Zou, Yingtian and Zhou, Daquan},
  booktitle = {Under review},
  year      = {2027}
}

License

The LoRA is trained on LTX-2.3 and is intended to be used with that backbone under the LTX-2 Community License.