MocapAnything-MLX

MoCapAnything V2 (video2pose2rot) repacked for 3Dify Studio's native MLX path on Apple Silicon. It turns a monocular video plus a target skeleton into per-joint rotations on that skeleton. No Python is needed at runtime.

Folder	Contents	Size
mocap/model.safetensors	video2pose + pose2rot, F32 (38.5M params)	154 MB
dinov2-large-224/model.safetensors	DINOv2 ViT-L/14 image encoder, F16, position table pre-interpolated to 224 px (257 tokens)	607 MB
t5-base-encoder/model.safetensors	T5-base encoder only, F16, for joint-name embeddings	219 MB
t5-base-encoder/vocab.json	SentencePiece unigram vocab as [piece, score] rows	1 MB

Conversion

• mocap/ is the video2pos2rot_epoch60.pt state dict saved as safetensors. The key names are unchanged.
• dinov2-large-224/ is facebook/dinov2-large with embeddings.position_embeddings replaced by HF's interpolate_pos_encoding(·, 224, 224) output, and mask_token dropped.
• t5-base-encoder/ holds the encoder.* and shared.weight tensors of t5-base, plus tokenizer.json → model.vocab.

Verified (MLX Swift port vs reference)

• T5 joint-name embeddings match the shipped species_info_dict vectors (min cosine 1.0000).
• DINOv2 features match HF transformers on the same pixels (min token cosine 0.9999).
• Jaguar-Run (y0 reference, y60 input video): MPJPE 0.058 in normalised units, against 0.50 for holding the reference pose.

Licences

• MoCapAnything V2 code and weights: MIT, from phongdaot/MocapAnything and kehong/MoCapAnythingV2-weights.
• DINOv2: Apache-2.0, from facebook/dinov2-large.
• T5: Apache-2.0, from google-t5/t5-base.