StereoHand

Code

Joint Pi3 inference checkpoint combining temporal MANO 110,000 steps, temporal Mask 40,000 steps, and the Pi3 matching pi32 decoder/head. Temporal step counts are counted from the start of temporal fine-tuning. The hand Pi3 register token is used for shared inference.

Checkpoint

stereohand_pi3_mano110k_mask40k_matching_pi32.pt contains full reconstructed model states, including the frozen Pi3 and stereo parameters. Each hand branch is restored from its initial model plus its latest complete training delta. Independently trained Mask and MANO fusion layers are preserved. Identical tensors share storage; values and dtypes are unchanged. Optimizer state is omitted.

The file uses the stereohand-joint-v1 format:

import torch
checkpoint = torch.load(
    "stereohand_pi3_mano110k_mask40k_matching_pi32.pt",
    map_location="cpu", mmap=True, weights_only=True,
)
mask_state = checkpoint["states"]["mask"]
mano_state = checkpoint["states"]["mano"]
matching_state = checkpoint["states"]["matching"]

Use with StereoHand

Set up the code repository and its runtime assets using the repository instructions. Download the checkpoint and the compatibility utility:

from huggingface_hub import hf_hub_download
for name in ["stereohand_pi3_mano110k_mask40k_matching_pi32.pt", "unpack_checkpoint.py"]:
    hf_hub_download("cskrren/StereoHand", name, local_dir="weights")

The current SharedHeads loader expects separate Mask, MANO, and matching paths. Export them from this single-file distribution and update an existing reconstruction config:

python weights/unpack_checkpoint.py \
  weights/stereohand_pi3_mano110k_mask40k_matching_pi32.pt \
  --output weights/unpacked \
  --config /path/to/your/reconstruct.json

Use the resulting weights/unpacked/reconstruct.json with the repository's reconstruction command. Its runtime/source/asset paths come from your supplied config. The complete branch file is used for both initial and checkpoint inputs, which is compatible with the existing strict loader. The output directory must be new.

Validation

The merge checks shared Pi3/stereo equality, matching-backbone compatibility (retaining the hand register token), finite tensor values, and exact tensor equality after saving and reloading. VALIDATION.json records the result. This validates checkpoint assembly; it is not a new accuracy benchmark for the 110k snapshot.