neural_contact — Standalone Contact Predictor

Standalone Contact Predictor extracted from neural-dynamics (Contact-Aware Neural Dynamics, CVPR 2026).

In the original repo, contact prediction is not a separate class; it lives inside JointPolicyNetwork as ContactMLPDecoder. This package keeps only the contact branch (pose decoder removed) and trains it on EgoForce egocentric per-finger contact labels.

Layout

neural_contact/
├── contact_predictor/          # model package (from neural-dynamics)
│   ├── models/
│   │   ├── contact_predictor.py   # ContactPredictor (standalone)
│   │   └── encoders.py            # MLPEncoder / ObservationEncoder
│   ├── decoders/
│   │   └── contact_decoder.py     # ContactMLPDecoder
│   └── utils/
│       ├── nn_utils.py
│       └── focal_loss.py
├── training/
│   ├── dataloader/
│   │   └── egoforce_contact.py    # EgoForceContactDataset
│   ├── configs/
│   │   └── egoforce_contact.yaml
│   ├── trainer.py
│   └── train.py
├── inference/
│   ├── infer_contact.py
│   └── visualize_contact_video.py
├── outputs/viz_contact/        # example contact-overlay videos
└── scripts/
    ├── train_egoforce.sh
    ├── infer_egoforce.sh
    └── upload_to_hf.sh

Model

finger_features [B, obs_h, 80]     # 10 fingers × 8 scalars
contact_history [B, obs_h, 10]     # past binary contacts
future_actions  [B, pred_h, 80]    # future feature deltas (motion cue)
        │
        ▼
  ObservationEncoder + FutureActionEncoder
        │
        ▼
     Fusion MLP → condition [B, 1024]
        │
        ▼
  ContactMLPDecoder → contact_logits [B, pred_h, 10]

• contact_dim = 10: left/right × {thumb, index, middle, ring, pinky}
• Default horizons: obs_h=4, pred_h=8
• Loss: BCEWithLogits (optional Focal)

Data (EgoForce)

Default data root:

/path/to/egoforce/data/packages/*/training_ready/contact/contact_force_long.parquet

Packages with contact labels available locally:

• hand-cream-application
• paper-towel-hand-wiping
• sink-hand-washing
• tidy-living-room
• washing-machine-laundry

Label column is selected automatically: dynamic_contact_pass → final_contact_pass → contact_pass.

Per-finger features: tip_xy, depth, dist_2d/3d, hand_activation, flexion, pose_confidence.

Install

pip install -r requirements.txt
export PYTHONPATH=/path/to/neural_contact:$PYTHONPATH

Or use an existing conda env that already has torch, pandas, pyarrow, pyyaml, opencv-python.

Train

cd /path/to/neural_contact
bash scripts/train_egoforce.sh
# or
python -m training.train --config training/configs/egoforce_contact.yaml

Default output dir: outputs/egoforce_contact/ (best.pt, last.pt, history.json).

Inference / Eval

# Evaluate on val split (requires a trained checkpoint)
bash scripts/infer_egoforce.sh --split val

# Autoregressive rollout on one package
bash scripts/infer_egoforce.sh \
  --rollout \
  --package hand-cream-application \
  --output outputs/egoforce_contact/rollout_hand_cream.npz

Contact Visualization

Overlay EgoForce GT contact labels onto RGB clips:

python -m inference.visualize_contact_video \
  --package tidy-living-room --subtask subtask_000001

python -m inference.visualize_contact_video \
  --package hand-cream-application --full-package-video

Example outputs are under outputs/viz_contact/.

Mapping to neural-dynamics

neural-dynamics	neural_contact
ContactMLPDecoder	kept (same name)
ObservationConditioner + contact branch	ContactPredictor
JointPolicyNetwork pose head	removed
HDF5 hand_object_contact	EgoForce parquet per-finger contact
contact_dim=1	contact_dim=10

Source

Extracted from neural-dynamics/contact_aware_neural_dynamics/:

• decoders/contact_decoder.py
• models/joint_policy_network.py (contact path only)
• models/encoders.py
• utils/nn_utils.py, utils/focal_loss.py

Citation

If you use this contact predictor design, please cite the original paper:

@InProceedings{Jing_2026_CVPR,
    author    = {Jing, Changwei and Bandi, Jai Krishna and Ye, Jianglong and Duan, Yan and Abbeel, Pieter and Wang, Xiaolong and Yi, Sha},
    title     = {Contact-Aware Neural Dynamics},
    booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
    month     = {June},
    year      = {2026},
    pages     = {13442-13452}
}