pose2rot — Joint Positions → 6D Rotations for Arbitrary Skeletons

Pretrained checkpoints for the pose2rot model (Pose2RotMemoryRestModel): given a sequence of 3D joint positions, predict per-joint 6D rotations (forward kinematics then recovers the full skeletal animation). One model handles arbitrary skeletons across 72 animal species — quadrupeds, bipeds, birds, reptiles, dinosaurs, arthropods, limbless snakes — via T5 joint-name embeddings, skeleton graph attention, and rest-pose FiLM conditioning.

Code, training recipe, eval & QA scripts: https://github.com/CHDTevior/pose2rot

This is a derivative work of MocapAnything (MIT, © 2026 Dao Thien Phong; arXiv:2604.28130 MoCapAnything V2). ~29.7M params.

Checkpoints

file	training data	use case
pose2rot_v9_alldata_epoch60.pt	all 72 species	best for demos / inference (the species is seen)
pose2rot_v10_heldout_epoch60.pt	seen/rare/unseen held-out split (test motions excluded)	the decisive paper model for honest cross-topology eval
pose2rot_v8b_best_epoch40.pt	all species (earlier converged best)	reference

Each .pt holds {model_state, optimizer_state, epoch}. Configs: config_v9_alldata.yaml, config_v10_split_heldout.yaml (model section instantiates Pose2RotMemoryRestModel).

Results (geodesic angle error, degrees; MoCapAnything V2 reports 6.54° unseen / V1 ~17°)

model	seen	rare	unseen	overall
v9 all-data (oracle)	7.2°	5.9°	6.4°	6.53° ≈ MoCapAnything 6.54°
v10 true held-out	9.8°	12.7°	40.9°	28.0°

When the species is seen, the model matches SOTA (6.5°). On a true held-out test, cross-topology generalization is a ceiling: unseen species with close training relatives generalize partially (Goat 17°, Coyote 19°), topologically distinctive ones do not (Pigeon ~67°, Spider ~73°).

Usage

import torch
from utils.config_utils import load_yaml_config, instantiate_from_config  # from the GitHub repo

cfg = load_yaml_config("config_v9_alldata.yaml")
model = instantiate_from_config(cfg["model"]).eval().cuda()
model.load_state_dict(torch.load("pose2rot_v9_alldata_epoch60.pt", map_location="cpu")["model_state"])
# batch dict: position[B,T,J,3] + rest pose + T5 joint embeddings + skeleton graph + reference (see GitHub data/loader_v2.py)
pred_rot6d = model(batch)["pred_rot6d"]  # [B,T,J,6]

See https://github.com/CHDTevior/pose2rot for the full data pipeline, training, and evaluation.

License & Citation

MIT. Built on MocapAnything (Dao Thien Phong, MIT). Please also cite MoCapAnything (arXiv:2604.28130).