swiftail/BiRefNet_lite-onnx-coreml

BiRefNet_lite — ONNX re-export for the CoreML execution provider

This is not a new model. It is BiRefNet_lite by Peng Zheng et al. (BiRefNet, MIT), re-exported to ONNX so that ONNX Runtime's CoreML execution provider (Apple Silicon GPU) runs the whole graph in a single partition. All credit for the model goes to the BiRefNet authors.

Why a re-export

The published ONNX exports don't work well with the CoreML EP: the deformable convolution exports as huge GatherND intermediates (>4 GB on CPU), and CoreML either fails to compile it or splits the graph into ~100 partitions. This export (scripts in scripts/) changes only how operations are expressed, not the weights:

1. deform_conv2d → one bilinear GridSample per kernel tap, accumulated with 1×1 convolutions (numerically equivalent, max logit difference ~5e-5).
2. Swin window partition/reverse and image2patches reshaped to rank ≤ 5 (CoreML's limit).
3. qkv[0..2] indexing → unbind.
4. Shapes fixed to 1024×1024, ORT basic constant folding, and Gemm weights rewritten to transB=1 (fix_gemm.py) so the CoreML compile stays small and fast.

Result: 6924/6924 nodes on CoreML, ~0.4 s per image on an M3 Pro (CPU+GPU, MLProgram), output matching the CPU EP to within 2e-4 in alpha.

Usage

• File: model.onnx (fp32)
• Input input_image: [1, 3, 1024, 1024] float32 — RGB resized to 1024×1024 (bilinear, no crop), (v/255 − mean) / std with mean [0.485, 0.456, 0.406], std [0.229, 0.224, 0.225].
• Output output_image: [1, 1, 1024, 1024] logits — apply a sigmoid, resize to the image size.
• CoreML EP: MLProgram format, static shapes, compute units CPU+GPU.

Used by Mixer, a personal D&D session app, for in-app background removal of character art.

License

MIT, same as the original BiRefNet. Please cite the original work:

@article{zheng2024birefnet,
  title={Bilateral Reference for High-Resolution Dichotomous Image Segmentation},
  author={Zheng, Peng and Gao, Dehong and Fan, Deng-Ping and Liu, Li and Laaksonen, Jorma and Ouyang, Wanli and Sebe, Nicu},
  journal={CAAI Artificial Intelligence Research},
  year={2024}
}