peasanttide/sam-3d-body-decoder-fp32

SAM 3D Body — decoder, widened to float32

One file, one change: the PromptableDecoder of SAM 3D Body with its bfloat16 initialisers widened to float32, so that it runs on ONNX Runtime's CPU execution provider.

Why

The decoder as distributed carries 130 bfloat16 initialisers. ORT's CPU provider has no MatMul kernel for bfloat16, so the graph loads and then refuses at the first matrix multiply:

Could not find an implementation for MatMul(13) node with name '/init_to_token/MatMul'

Every bfloat16 is a float32 with the low sixteen mantissa bits cleared, so widening is exact. There is no retraining, no quantisation and no change to any number — the weights are bit-identical to the originals once widened. What moved: 130 initialisers, 1 tensor inside a node attribute, and 63 Cast nodes whose target was bfloat16. Stale value_info was dropped and re-inferred.

The script that does it is widen_decoder.py, sixty lines of onnx and numpy, and is the whole of what was done here. Run it yourself rather than trusting this file if you would rather.

Provenance, and what this is not

This is not a conversion of SAM 3D Body from PyTorch. That conversion is AmmarkoV/SAM3DBody-cpp-onnx-models, whose accompanying C++ implementation SAM3DBody-cpp is MIT-licensed and is what taught us the pipeline's shapes and its parameter layout; the acknowledgement is theirs. This repository holds one graph out of that kit with one dtype changed, and nothing else — no backbone, no detector, no body model.

Licence

The SAM License, not MIT. These are derivative weights of Meta's SAM 3D Body, and section 1(b)(i) of that licence requires derivative works to be distributed under it with a copy attached; the copy is in LICENSE beside this card. Upstream's card says MIT, which covers their code and cannot cover Meta's weights. Using this file means accepting Meta's terms, including the prohibited uses in section 1(b)(v).

Used by

PeasantTide reads a pose off a period engraving and puts it on a peasant. crates/peasanttide_portrait runs this graph under ort.