YAML Metadata Warning:empty or missing yaml metadata in repo card
Check out the documentation for more information.
hand-pipeline (mirror of ggxxii/hand)
A local, documented mirror of the Hugging Face model archive
ggxxii/hand → hand_pipeline.zip.
What this is
An end-to-end monocular hand-motion recovery stack. Given a video, it detects, tracks, and reconstructs 3D hand motion (MANO parameters) using a temporal optimization built on Dyn-HaMR, fed by several off-the-shelf perception models.
Components
| Area | Component | Role |
|---|---|---|
| Optimizer | Dyn-HaMR (dyn-hamr/) |
Temporal MANO optimization; entry points run_opt.py, run_human_opt.py, run_batch_opt.py, run_vis.py |
| Hand recon | HaMeR (third-party/hamer/) |
Per-frame hand mesh/MANO regressor |
| 2D pose | ViTPose+ huge (third-party/hamer/third-party/ViTPose/) |
Whole-body keypoints |
| Segmentation | SAM3 (third-party/sam3/) |
Hand/subject masks |
| Camera / SLAM | ViPE + DROID-SLAM (third-party/vipe/, _DATA/droid.pth) |
Camera pose / video pose estimation |
| Motion prior | HMP / NeMF (_DATA/hmp_model/) |
Neural hand-motion prior |
| Body prior | human_body_prior (third-party/human_body_prior/) |
VPoser-style prior utilities |
| Constraints | BMC (_DATA/BMC/) |
Biomechanical bone-length / angle limits |
| Model | MANO (_DATA/data/mano/MANO_RIGHT.pkl) |
Hand parametric model |
Bundled weights (_DATA/, ~10.9 GB uncompressed)
| File | Size | Model |
|---|---|---|
_DATA/vitpose_ckpts/vitpose+_huge/wholebody.pth |
3.81 GB | ViTPose+ huge |
_DATA/sam3_ckpt/sam3.pt |
3.45 GB | SAM3 |
_DATA/hamer_ckpts/checkpoints/hamer.ckpt |
2.69 GB | HaMeR |
_DATA/hmp_model/results/model/optimizer.pth |
542 MB | HMP optimizer state |
_DATA/hmp_model/results/model/local_encoder.pth |
161 MB | HMP |
_DATA/hmp_model/results/model/nemf.pth |
93 MB | NeMF prior |
_DATA/BMC/joint_angles.npy |
90 MB | BMC stats |
_DATA/hamer_ckpts/hamer_demo_data.tar.gz |
17 MB | HaMeR demo data |
_DATA/hmp_model/results/model/global_encoder.pth |
17 MB | HMP |
_DATA/droid.pth |
16 MB | DROID-SLAM |
_DATA/data/mano/MANO_RIGHT.pkl |
3.8 MB | MANO |
| … + BMC/config/mean-std small files | <1 MB each |
Total archive: 10.04 GB on disk, 11.06 GB uncompressed, 2501 files.
Layout
hand_pipeline/
├── dyn-hamr/ # optimization pipeline + run_*.py entry points
├── third-party/ # hamer, vipe, sam3, human_body_prior, actionclip
└── _DATA/ # all checkpoints & model files (see table above)
Sample inputs — companion repo ggxxii/data_occ
The matching input videos live in a separate HF repo:
ggxxii/data_occ → data_occ.zip
(557 MB, 188 files, 184 .mp4). It is occluded-hand test footage, not
weights or code. Internal path: data/liuyufei/world2hand/data_occ/ (liuyufei
= co-author Yufei Liu; world2hand = internal project name). Sources:
- DexYCB — 1 clip
- HoloAssist — 2 large egocentric clips (~95–137 MB)
- Ropedia —
180 short stereo clips (1–5 MB each) ← ideal smoke-test inputs
Fetch it with:
curl -L "https://huggingface.co/ggxxii/data_occ/resolve/main/data_occ.zip" -o data_occ.zip
unzip data_occ.zip # → data/liuyufei/world2hand/data_occ/{DexYCB,holoassist,ropedia}/
Setup / run (unverified — see caveat)
The archive ships no top-level README, requirements.txt, or documented run
command of its own. Based on the structure, the intended flow is a Dyn-HaMR
run pointed at the bundled _DATA:
cd hand_pipeline/dyn-hamr
# create env from Dyn-HaMR / HaMeR requirements (see third-party/hamer/setup.py)
python run_opt.py # inspect argparse for --video / --data-root / --out flags
Caveat: these commands were not executed — no sample input or usage doc is included, and the exact CLI must be read from each
run_*.py. Verifyargparseargs before running. Target env that matches the bundled weights: Python 3.11, CUDA 12.4 / PyTorch 2.4.x, ~11 GB VRAM headroom (fits an H100 80 GB comfortably; likely runs on ≥16 GB cards).