HandRSI checkpoints

Network weights and harnesses of the five HandRSI rounds. Code, evaluator and result tables are in the HandRSI code repository.

path	what it is
theta1/	θ1, the network after round 1: a LoRA fine-tune of the released ACE-Ego-Hand K-given checkpoint (ace_ego_hand_k.pt + ace_ego_hand_k.yml)
theta2/	θ2, the network after round 2. No Loop B checkpoint of rounds 3-5 beat it on val, so it was kept: θ3 = θ4 = θ5 = θ2, the final network
harness/H1 ... harness/H5	the harness of each round (chain.json: the modules in order with their parameters; modules/*.py). H5 is H5*, the final harness (39 modules)

Final system: θ2 + harness/H5.

Use

1. Set up the code repository and ACE-Ego-Hand as the repository's README says. The Wan2.2-Fun-5B-Control backbone, VideoX-Fun and the caption embedding come from ACE-Ego-Hand's own instructions; they are not part of this repository. ace_ego_hand_k.yml names them relative to the ACE-Ego-Hand checkout (ckpt/, third_party/, cache/), as ACE-Ego-Hand's own options file does.
2. Run a video:

python infer_video.py --video clip.mp4 --intrinsics fx,fy,cx,cy \
    --ckpt <this repository>/theta2 --harness <this repository>/harness/H5 --out results/clip

The video is processed in 81-frame segments, as in the evaluation. Harness modules that use image evidence need WiLoR (see the code repository's README).

Results (sealed test splits, evaluator of the code repository)

HOT3D (test, 27 recordings, 1162 segments)

	FAcc ↑	Recall ↑	F1 ↑	MPJPE-p ↓	PA-p ↓	MPJPE+OOS ↓	EPE2D-p ↓	GO-p ↓	CT-p ↓	Jitter ↓
ACE-Ego-Hand (released K-given checkpoint)	0.970	0.986	0.990	12.97	5.90	12.84	9.66	6.90	0.024	2.82
HandRSI network θ2 alone (θ3 = θ4 = θ5 = θ2)	0.982	0.994	0.994	9.37	4.71	10.35	5.31	5.80	0.013	2.51
HandRSI: θ2 + harness H5* (final, 39 modules)	0.982	0.996	0.994	8.37	4.37	9.70	4.07	5.16	0.013	1.34

ARCTIC (test = subject s05, 291 segments)

	FAcc ↑	Recall ↑	F1 ↑	MPJPE-p ↓	PA-p ↓	MPJPE+OOS ↓	EPE2D-p ↓	GO-p ↓	CT-p ↓	Jitter ↓
ACE-Ego-Hand (released K-given checkpoint)	0.986	0.995	0.996	15.58	7.43	16.54	9.36	12.45	0.024	2.58
HandRSI network θ2 alone (θ3 = θ4 = θ5 = θ2)	0.987	0.995	0.997	15.15	7.28	16.11	9.12	12.34	0.021	2.36
HandRSI: θ2 + harness H5* (final, 39 modules)	0.990	1.000	0.997	13.85	6.34	15.18	7.61	11.61	0.019	1.45

HOI4D (zero-shot: never used for training, gating or selection; 498 segments)

	FAcc ↑	Recall ↑	F1 ↑	MPJPE-p ↓	PA-p ↓	MPJPE+OOS ↓	EPE2D-p ↓	GO-p ↓	CT-p ↓	Jitter ↓
ACE-Ego-Hand (released K-given checkpoint)	0.994	0.994	0.994	23.45	11.02	22.93	15.60	19.39	0.058	2.10
HandRSI network θ2 alone (θ3 = θ4 = θ5 = θ2)	0.992	0.992	0.993	24.11	11.13	23.42	17.39	19.57	0.064	1.87
HandRSI: θ2 + harness H5* (final, 39 modules)	0.991	0.991	0.992	22.77	10.21	21.93	18.61	19.44	0.064	1.01

HOI4D is zero-shot (never used for training, gating or selection). Round-by-round tables with their controls: results/README.md in the code repository.

What a checkpoint holds

projectors, backbone_trainable (LoRA weights of the backbone) and raymap_heads are the trained tensors. step, rsi_history and rsi_distill record the training lineage (paths relative to the code repository's rsi/).

Licence

Weights: CC BY-NC 4.0, inherited from the ACE-Ego-Hand checkpoint they are fine-tuned from. Harness code: MIT, as in the code repository. MANO is not included and keeps its own licence.

Checksums (sha256)

08b2ff0c20e1c5008bb72d720965c5395122c7c7e667b073897678f5ede4919d  theta1/ace_ego_hand_k.pt
6f9202e16aca3e745ffa55e7651ce89d9f1fb924fd1c7407394282e4d7db56d3  theta2/ace_ego_hand_k.pt