stefanj0/docling-layout-heron-fp16-onnx

Docling Heron Layout — FP16 (ONNX)

FP16 ONNX export of docling-project/docling-layout-heron (Docling Heron, RT-DETRv2, 17-class document-layout detection at 640×640), re-exported fresh from the PyTorch/HF Transformers checkpoint. Produced for RailReaderCore's native WebGPU execution provider support (RailReader.Core.Analysis.WebGpu).

What this is

A drop-in FP16 replacement for docling-project/docling-layout-heron-onnx (same I/O contract, same postprocessing) intended for GPU inference. The backbone/encoder/decoder run in FP16; the sigmoid/top-k box-decode postprocessing is kept in FP32 for numerical stability.

Why re-export instead of converting the existing FP32 ONNX

Post-hoc FP16 conversion of the already-exported graph (via onnxconverter_common) does not work for this model — its postprocessing arithmetic hits a stale-type-annotation bug in that tool plus genuine mixed-precision sensitivity, and an iterative auto-fixer never converged. Exporting fresh from the PyTorch source avoids this: the tracer keeps every op's dtype consistent throughout.

Validation

Compared against the FP32 base model, on the real RailReaderCore pipeline (not just a synthetic tensor diff — a synthetic square-image test can hide an axis-order bug that only shows on a real, non-square page):

metric	value
block-centroid match, page 1 (real document)	13/14 blocks exact, one borderline near-duplicate detection
block-centroid match, additional test pages	exact or off-by-one on a near-duplicate
inference speedup (WebGPU EP vs CPU EP, laptop-class GPU)	~9.5×

The one-block difference pattern (an extra or missing near-duplicate detection at a borderline confidence threshold) is the same class of FP16 rounding noise seen when running the INT8 backbone-quantized variant of this model on WebGPU — not a functional regression.

I/O contract (identical to the FP32 base export)

• Inputs: images uint8 [1,3,640,640] NCHW; orig_target_sizes int64 [1,2] = [W, H] (note: [W, H], not the HF-standard [H, W] — this export deliberately matches the existing FP32 ONNX's convention).
• Outputs: labels [1,300] int64, boxes [1,300,4] float (xyxy in pixel space), scores [1,300] float.

17 classes: caption, footnote, formula, list_item, page_footer, page_header, picture, section_header, table, text, title, document_index, code, checkbox_selected, checkbox_unselected, form, key_value_region.

Recipe

Exported with torch.onnx.export from a wrapper around RTDetrV2ForObjectDetection.from_pretrained("docling-project/docling-layout-heron").half(), replicating RTDetrImageProcessor.post_process_object_detection's box-decode in FP32. Full reproducible tooling (including the export script and the axis-order fix): tools/onnx-fp16-export/export_heron_fp16.py in RailReaderCore.

License

Inherits the license of the base model docling-project/docling-layout-heron.