ruisv/bcdl-vitpose-wholebody

ViTPose-S whole-body for RDK S100/S100P — 133 keypoints

Compiled BPU models (.hbm) for the D-Robotics RDK S100 / S100P, ready to load — no ONNX export, no calibration, no hb_compile. Built and measured with BCDL, a C++17 inference and media library for the RDK S-series with Python bindings.

Upstream: easy_ViTPose ViTPose-S wholebody, COCO-WholeBody

Non-commercial. These weights carry a restriction the upstream code licence never mentions — see Licence at the bottom. Redistribution is permitted; commercial use is not. If you need a commercial build, the recipe is public and you can retrain or substitute the backbone.

Files

file	what it is
vitpose_s_wholebody_nashm_256x192.hbm	256x192, 133-keypoint heatmaps — 27.7 MB

Measured on an S100P

stage	latency	throughput
pose, per person	1.60 ms	623 FPS

hrt_model_exec perf, one thread, minimum of three runs, on a board first gated against its own previously recorded numbers. BPU time only — CPU pre/post-processing is on top and is listed per task in BCDL's benchmark results.

Use it

conda install -c https://mirrors.ruis.ai/conda -c conda-forge bcdl

import bcdl
engine = bcdl.Engine("vitpose_s_wholebody_nashm_256x192.hbm")
print(engine.input_shape(0), engine.output_shape(0))

Each task has a decoder in BCDL that turns those raw outputs into boxes, keypoints, masks, disparity or text — see the Python API (中文).

What to know before deploying

This is a top-down model: it runs once per person, on a crop, so it needs a detector in front of it and its cost grows with the head count. What you get for that is feet, a 68-point face and both hands — 133 keypoints against the 17 a bottom-up pose head gives.

A plain ViT's cost tracks its token count, which is why a ViT runs at 1.6 ms here: 256x192 is 192 tokens. The int8 build needed no LayerNorm intervention.

Two conventions must be copied exactly or the keypoints land off-target: the model takes RGB, and the crop geometry is not the mmpose affine — widen the box by 10 px, zero-pad to 3:4, resize.

Non-commercial. See the licence note above.

Licence

Apache-2.0 on the code, but the checkpoint is fine-tuned from MAE, which is CC-BY-NC-4.0, and trained on COCO-WholeBody. A fine-tune is a direct derivative of the weights it started from, so the non-commercial term reaches this build.

BCDL itself is Apache-2.0 and is unrelated to these terms — it is a general-purpose runtime that loads any .hbm. The licence above constrains these weights and this compiled artefact.

The conversion recipe — ONNX export, calibration, hb_compile config and the acceptance numbers — is public in bcdl-model-zoo, so this build can be reproduced or retargeted rather than taken on trust.