TRACE

Typed Reference-Aware Conversational Editing for Humanoid Motion Diffusion

arXiv · Code · Training guide · ModelScope mirror

TRACE generates and edits Unitree G1 humanoid motions from natural language. A language model predicts duration and sparse motion events. A diffusion Transformer combines these time-aligned events with the original text to generate continuous motion. Follow-up instructions edit the current motion while preserving local joint rotations outside the requested scope.

Released models

This repository contains the project's inference weights and normalization files. The two Qwen base models are downloaded separately from their official repositories.

Artifact	Role
generator.pt	Sparse-event motion generator; use with --boundary none
generator_boundary.pt	Generator with optional start/end pose references; default generation model
planner_adapter/	Qwen LoRA adapter predicting duration and motion events from text
edit_scope_adapter/	Separate Qwen LoRA adapter interpreting follow-up requests
editor.pt	Dedicated source-relative hand-height motion editor
generation_stats.npz	Generator normalization
editing_stats.npz	Dedicated editor normalization

The checkpoints are compatible with the TRACE inference code, not a generic Transformers text-generation pipeline. artifact_manifest.json records SHA-256 checksums, file sizes, and the associated code revision. No optimizer states or training motions are included.

Installation and download

Linux, Python 3.10–3.12, and an NVIDIA GPU are recommended. See the installation guide for driver and rendering requirements.

git clone https://github.com/Multimodal-Perception-Action/TRACE.git
cd TRACE
TRACE_PYTHON=python3.10 bash install.sh

# Download this repository's helper, then fetch and verify the model files.
.venv/bin/hf download kangzejian/TRACE download_trace.py --local-dir .
.venv/bin/python download_trace.py --output-dir models
.venv/bin/python run.py --check-models

The helper downloads TRACE artifacts, verifies their sizes and SHA-256 checksums, and obtains the official Qwen/Qwen3-4B-Instruct-2507 and Qwen/Qwen3-Embedding-4B snapshots. Allow approximately 1 GB for TRACE artifacts and 16 GB for the Qwen bases, plus cache and output space. Use --skip-upstream when the base models are already installed.

Individual files can also be obtained with huggingface_hub.hf_hub_download. No Hugging Face login is required to download this public release.

Generate motion

Walking without endpoint constraints

bash run.sh \
  --text "A person walks forward at a steady pace." \
  --boundary free \
  --output outputs/walk

Walking with start and end poses

bash run.sh \
  --text "A person walks forward at a steady pace." \
  --generator-checkpoint models/generator_boundary.pt \
  --start-ref examples/poses/idle_start.sps --start-frame 0 \
  --end-ref examples/poses/idle_end.sps --end-frame 0 \
  --output outputs/walk_idle

The current single-frame idle references are included in the code repository and under examples/poses/ here. The two endpoint references can be supplied independently. --boundary free uses the same boundary-capable model without pose constraints; --boundary none selects the original generator.

The default workflow is:

Text → predicted sparse events + text embedding → motion diffusion
     → smoothing + heel/toe contact refinement → 20 FPS SPS + video

motion_raw.sps is the motion before postprocessing; motion_final.sps is the final output. The workflow also produces array exports and visualization. Use a new output directory for each run and --skip-render when visualization is not required. Postprocessing is separate from the learned generator.

Edit through a follow-up instruction

bash run.sh \
  --text "Briefly bow forward, then return to standing." \
  --contact-refine none \
  --output outputs/bow

.venv/bin/python converse.py \
  --source outputs/bow \
  --session outputs/bow_chat \
  --instruction "Bow more deeply, keeping the arm movements unchanged." \
  --render-device 0

The session retains the source description and latest completed motion. A later turn only needs the session and the requested change:

.venv/bin/python converse.py \
  --session outputs/bow_chat \
  --instruction "Make the bow a little shallower."

This generator-based interface uses reference-guided diffusion and in-sampling kinematic optimization. The separate edit.py interface uses editor.pt for trained relative hand-height editing. These are distinct editing paths. See the editing guide for supported instructions, output checks, and before/after overlays.

Real-robot demonstrations

Download video. The video contains jumping-jack editing, in-the-wild instructions, and dance demonstrations. Physical execution uses a downstream controller, which is not included in this model repository.

Training and evaluation

The generator is trained on text–motion pairs from BONES-SEED in the G1 representation. Motion-derived support, root, and hand events supervise the language planner; inference uses events predicted from the input text. The boundary model is fine-tuned using actual training-motion endpoints. The dedicated editor additionally uses retargeted MotionFix pairs and constructed local-edit supervision. See the training recipe and configuration.

The paper reports semantic retrieval and distribution metrics for generation, together with editing comparisons and ablations. The code release includes the training recipe and inference implementation; standalone generator/editor training entry points are not yet included.

Filename-only training, validation, and test lists are available in the code repository. Obtain the corresponding motions and annotations directly from their providers under their terms. This repository does not redistribute those datasets or captions.

Intended use and limitations

TRACE supports research and prototyping of language-conditioned humanoid motion and reference-aware editing. Outputs are flat-ground kinematic references. The release does not provide scene perception, obstacle avoidance, or a guarantee of dynamic balance. Robot execution requires a separately validated controller and safety checks. Compositional action ordering and large edits remain challenging. Preserving local joint rotations does not imply that their world-space positions remain fixed when the root moves.

License and citation

Project-owned released weights and inference assets use Apache-2.0. Qwen bases and other third-party components retain their own licenses. The model license does not grant access to or redistribution rights in the training datasets; see licensing and third-party notices.

@article{kang2026trace,
  title={TRACE: Typed Reference-Aware Conversational Editing for Humanoid Motion Diffusion},
  author={Kang, Zejian and Luo, Yifu and Shen, Boyang and Li, Jingxuan and Chen, Zekai and Shi, Zekun and Xu, Yongxue and Hou, Zhenyu and Wu, Bingqian and Zhang, Shaolei and others},
  journal={arXiv preprint arXiv:2610.21515},
  year={2026}
}