tariqkhalif/slimsam-77-crack

SlimSAM 77% uniform, crack-tuned (SnapClaim)

The SlimSAM model SnapClaim runs in the browser to outline damage, with the mask decoder fine-tuned on concrete cracks. The vision encoder is unchanged, so the encoder ONNX files are the same as Xenova/slimsam-77-uniform and this repo drops in for it with transformers.js:

const model = await SamModel.from_pretrained("tariqkhalif/slimsam-77-crack");

Why

Cracks are thin. With one tap on a crack, stock SlimSAM's three candidate masks are usually the wall around it, or a blob. SnapClaim keeps the smallest candidate that still scores 0.5 or more. This model trains the decoder so that candidate actually traces the crack.

Results

Held-out split: 92 images (20%, seed 7) that training never saw. Five random taps on the crack per image, 460 taps in all. mIoU against the ground-truth mask, picking the candidate the same way the app does.

model	mIoU (app's pick)	mIoU (best of 3)
stock SlimSAM (PyTorch)	0.131	0.234
tuned, ONNX fp32 (WebGPU path)	0.459	0.476
tuned, ONNX quantized (wasm path)	0.457	0.474

The ONNX decoder matches PyTorch to within 0.0008 on the raw logits.

In the browser (transformers.js 3.0.2, quantized), with the embeddings from the stock encoder and SnapClaim's current rules (score 0.3 or more, 20 px or more, and no mask covering over half the frame), 276 taps on the same held-out split: stock 0.174, tuned 0.697 mIoU.

How SnapClaim uses it

Only for damage taps. The tuned decoder grows coin masks into the tape around the coin (1.2-3.3x wider on our samples), so SnapClaim keeps the stock decoder for the coin that sets the scale and computes one embedding per photo with the stock encoder. Code and training scripts: SnapClaim training/slimsam-crack/.

Training

• Data: Concrete Crack Segmentation Dataset, Özgenel (2019), Mendeley Data V1, doi:10.17632/jwsn7tfbrp.1, CC BY 4.0. 458 image and mask pairs, 80/20 split.
• Frozen encoder, embeddings cached once (plus a horizontal flip). Only the prompt encoder's mask decoder is trained.
• 12 epochs, AdamW lr 1e-4, cosine schedule, one random positive tap per image per step. Loss: SAM's focal x20 plus dice on the best candidate, plus MSE on the IoU head. The checkpoint was chosen on a validation slice of the training split.
• Quantized with the same settings as Xenova's export (dynamic, per-channel, QInt8 decoder weights).

Limits

The training photos are phone shots (mostly 4032x3024) of concrete surfaces with cracks. Results on other materials (plaster, brick, tile), on wide shots, and on non-crack damage such as dents or water stains have not been measured.

License

Apache-2.0, like the base model. The training data is CC BY 4.0. Credit: Çağlar Fırat Özgenel, Concrete Crack Segmentation Dataset.