TontonTremblay/doper2-handal-obj000021

DOPER2 — HANDal obj_000021

Pose estimation model for HANDal object 000021 trained with the DOPER2 pipeline.

Model details

• Backbone: convnext_tiny.dinov3_lvd1689m
• Keypoints: 64 (locations in keypoints_3d.json, units: metres)
• Pipeline stage: V5 (DR synth 10k + BOP PBR + onboarding pseudo-labels)
• Input size: 224 px (detector), 256 px (keypoint crop)

Files

File	Description
best.pth	Best checkpoint (lowest val kp_err_px)
keypoints_3d.json	3-D keypoint positions in metres
config.yaml	V5 training configuration
training_provenance.json	Full training args, data sources, git commit

Usage

import cv2, json, numpy as np
from doper2.infer import load_model, infer_image
from doper2.config import Doper2Config, DataConfig, ModelConfig

kp3d_json = "keypoints_3d.json"
cfg = Doper2Config(
    data=DataConfig(root=".", keypoints_3d_json=kp3d_json),
    model=ModelConfig(
        num_keypoints=64, det_input_size=224, crop_size=256,
        keypoint_head="heatmap",
        backbone="convnext_tiny.dinov3_lvd1689m",
    ),
)
model   = load_model("best.pth", cfg, device="cuda:0")
kp3d_mm = np.array(json.load(open(kp3d_json))["keypoints_3d"]) * 1000  # metres -> mm

img  = cv2.imread("your_image.jpg")
K    = np.array([[fx, 0, cx], [0, fy, cy], [0, 0, 1]], dtype=np.float64)
dets = infer_image(model, img, cfg, device="cuda:0", score_thr=0.3)
if dets:
    d   = max(dets, key=lambda x: x.score)
    ok, rvec, tvec = cv2.solvePnP(
        kp3d_mm, np.array(d.keypoints), K, None,
        flags=cv2.SOLVEPNP_SQPNP,
    )
    # tvec is in mm, R from cv2.Rodrigues(rvec)

BOP val results (obj_000021)

See TontonTremblay/doper2-handal-results for full evaluation tables and inference grids.