Davide531/ben2-int8

BEN2 compact — weight-only int8

A weight-only int8 quantisation of PramaLLC/BEN2 base, built for Cutout, a background remover that runs entirely in the browser with onnxruntime-web.

What was done

160 Conv weight tensors were quantised to per-channel symmetric int8 — one scale per output channel, which is what keeps the output faithful. (An earlier per-tensor build produced corrupted mattes on some images and has been replaced.) Every activation stays in floating point, so this is the original BEN2 graph with quantised weights rather than a fully quantised model.

	Original BEN2 base	This
File size	223 MB	124 MB
Conv weights	float16	int8 (160 tensors)
Activations	float16	float16

Why

The full precision model fails in onnxruntime-web with std::bad_alloc in some environments, including on phones. This build completes the same inference where the 223 MB model cannot, on both the WASM and WebGPU backends.

Usage

const session = await ort.InferenceSession.create(bytes, {
  executionProviders: ['webgpu', 'wasm']
});

• Input input.1 — 1x3x1024x1024 float32, RGB, resized square, divided by 255, no mean/std normalisation.
• Output 17728 — 1x1x1024x1024 float32. Min-max normalise it across the whole map to get the alpha matte, then resize to the original image size.

Verified against the site's pinned onnxruntime-web 1.22.0 and against 1.30.0; both produce identical output.

Licence

MIT, following the original PramaLLC/BEN2. Weights by the BEN2 authors; quantised by the Cutout project.