ProAct motion checkpoints

Matched audio Flow Matching backbone (epoch 600) and text ControlNet (epoch 800) for ProAct.

Code: Social_robot_baseline, opensource branch.

File	Role
checkpoints/audio_backbone_model_600.pt	31 motion dimensions, 181 MFCC features per audio channel
checkpoints/controlnet_model_800.pt	Text ControlNet, 512-dimensional CLIP conditions; trained with the paired epoch-600 backbone
configs/*_train.yaml	Original training configurations; relative paths describe the original training layout
manifest.json, SHA256SUMS	Original artifact provenance, sizes, and SHA-256 hashes

These are original PyTorch training checkpoints containing epoch, model_state_dict, optimizer_state_dict, and scheduler_state_dict; they are not weights-only or SafeTensors exports. NumPy scalar metadata requires the loader compatibility used by the source project. These are the original trained checkpoint bytes, not upstream Semantic Gesticulator checkpoints. Their hashes match the pair used in the project's 2026-10-11 RTX 4090 validation at source commit 42093e050a2fa165ec9992058ee151736ac33854. That validation exercised audio and text-conditioned inference, 50 Euler steps, client inverse transforms, interruption and session reset. It did not evaluate physical robot safety or exact paper metrics.

Download

pip install huggingface_hub
hf download UsanoCoCr/ProAct --local-dir checkpoints/ProAct
export MOTION_BASE_CHECKPOINT="$PWD/checkpoints/ProAct/checkpoints/audio_backbone_model_600.pt"
export MOTION_CHECKPOINT="$PWD/checkpoints/ProAct/checkpoints/controlnet_model_800.pt"

Follow the source repository's runtime instructions and motion instructions. The data release provides the processed SeG data and its matching preprocessing exports (input_scaler.sav, output_scaler.sav, data_pipe_30fps.sav), together with a separate retargeted Photoreal subset. Obtain authorized matching audio-domain preprocessing exports separately and configure MOTION_AUDIO_DATA_ROOT, MOTION_CONTROLNET_DATA_ROOT and the client inverse-transform paths. Checkpoints alone do not supply these exports or the training dataset.

Data provenance and limitations

The audio training directory combines retargeted audio/motion sources, including audio2photoreal and ZEGGS; historical renamed pxb_fixed_scene* filenames do not prove Photoreal-only provenance. The ControlNet uses processed MOCCA Semantic Gesture data. Training mixtures and preprocessing should be considered when assessing generalization.

Source datasets retain their own terms. In particular, audio2photoreal uses CC BY-NC 4.0, ZEGGS uses CC BY-NC-ND 4.0, and SeG requires prior written consent for third-party redistribution. This repository does not include or relicense those datasets. Processed SeG files are shared in the data repository with explicit project-owner authorization; original SeG files remain at the official source. No additional authorization to redistribute ZEGGS-derived exports has been obtained, so the complete audio mixture and its fitted exports are withheld. ProActBench is not included.

License and attribution

The ProAct checkpoint files are shared under CC BY-NC 4.0. Use is limited to non-commercial purposes and requires attribution. This grant covers rights held by the releasing authors in the checkpoint files; it grants no additional rights in third-party source datasets, code, or assets. The source code's MIT license is separate.

Please cite ProAct, From Audio to Photoreal Embodiment, ZeroEGGS, and Semantic Gesticulator.