SeMoCo: A Semantic-First Motion Codec for Motion Language Modeling

Introduction

SeMoCo is a semantic-first motion codec for motion language modeling. Instead of organizing codebooks purely by reconstruction residuals, each motion token contains one semantic token and a residual sequence of kinematic tokens, so that action-level meaning and fine-grained kinematic detail are encoded explicitly. A dual-axis generator built on these tokens models semantic progression across time and autoregressively refines the residual entries.

This repository contains the pretrained tokenizer, its ablation variants, and the Lite / Base text-to-motion generators trained on SeMoCo tokens.

Model Files

File	Model	Params
tokenizer/split_branch_sem.pt	SeMoCo tokenizer	22.9M
tokenizer/ablation/	Tokenizer ablation variants (split-branch / single-chain, ± semantic)	~22M
generator/lite.pt	Ours-Lite T2M generator	188M
generator/base.pt	Ours-Base T2M generator	391M

Tokenizer checkpoints are self-contained (codec_config + net + norm); generator checkpoints contain model + model_config + data_meta. Optimizer states are stripped. The generator code will be released separately.

Evaluation

Model	Headline results
SeMoCo tokenizer	MPJPE 12.83 mm / PA-MPJPE 11.07 mm (SMPL-22, symmetric-FK protocol); codebook usage 100%, perplexity 998, 198.4 tokens/s
Ours-Lite (T2M)	FID .186, R@1 .484, MedR 1.70
Ours-Base (T2M)	FID .181, R@1 .494, MedR 1.50

Usage

hf download poisonousID/SeMoCo --local-dir checkpoints/

Evaluation requires the code repository:

python -m tools.eval_recon_smpl22 \
    --checkpoint checkpoints/tokenizer/split_branch_sem.pt \
    --manifest <test.txt> --out recon_smpl22.json

License

Apache-2.0. Evaluation gates involving SMPL-X require the SMPL-X model, distributed separately under its own license.

Citation

@article{semoco,
  title   = {SeMoCo: A Semantic-First Motion Codec for Motion Language Modeling},
  author  = {OMEGA-team},
  journal = {arXiv preprint},
  year    = {2026}
}