agriggio/smart-onnx-models

ONNX models for SMART

ONNX models used by SMART, an AI helper for ART (AI masking, healing and raw denoising). They run with onnxruntime alone: no PyTorch needed.

SAM 2.1 (AI masking)

SAM 2.1 (Meta, Apache-2.0) image models, converted with tools/export_sam2_onnx.py (derived from samexporter). For each size (tiny, small, base_plus, large) there are two files:

• sam2.1_hiera_<size>_encoder.onnx: image [1,3,1024,1024] (RGB, 0..1, ImageNet mean/std normalized, stretched to 1024x1024) -> high_res_feats_0 [1,32,256,256], high_res_feats_1 [1,64,128,128], image_embed [1,256,64,64]. Run once per image.
• sam2.1_hiera_<size>_decoder.onnx: prompt encoder + mask decoder with the single-mask output of SAM 2 (multimask_output=False, with the stability-based fallback). Inputs: the three encoder outputs, point_coords [1,N,2] (pixels in 1024x1024 space), point_labels [1,N] (1 foreground, 0 background, 2/3 box corners), mask_input [1,1,256,256] and has_mask_input [1]. Outputs: low_res_masks [1,1,256,256] logits (upsample bilinearly to the image size and threshold at 0) and iou_predictions [1,1]. Feed low_res_masks back as mask_input to refine.

Checked against the PyTorch SAM2ImagePredictor: mask IoU >= 0.99998 over random click sequences (with refinement), for all four sizes.

LaMa (AI healing)

big-lama-dynamic-coreml.onnx is the LaMa big-lama model (Apache-2.0), from greycard's ONNX export, with a few ops rewritten (Gather -> Slice+Squeeze, Neg -> Mul, ConvTranspose with output_padding -> ConvTranspose+Slice) so that the onnxruntime CoreML provider can run it on the GPU (tools/make_lama_coreml.py). Same results as the original (max difference ~1e-5) and it runs on all providers. Inputs image [1,3,H,W] and mask [1,1,H,W] (H and W multiples of 8); for CoreML, give each session fixed sizes with free dimension overrides of 8*hk and 8*wk.

Checksums (sha256)

b301c76a5f18ea409a93b0ce0de3954ecbe910fa6b328a6f25450d4c41dc6318  sam2.1_hiera_base_plus_encoder.onnx
c4de4a5e5c0dfbf3434a53a23b6cb7edc2172e472becb0754f1d2840cea35960  sam2.1_hiera_large_encoder.onnx
d1eee90b60dd814dd16db38f4adcb7b7bf39686235b9afe5f00404bdde9188e5  sam2.1_hiera_small_encoder.onnx
769f6278eda2fdcb53c3e247e42259ce3f4002ef8e885d838ca4f2d6345c5d0c  sam2.1_hiera_tiny_encoder.onnx
89a0854ccf41ea96b75a1ae60cc8582936bf52dc602d4df8072f750413945c01  sam2.1_hiera_base_plus_decoder.onnx
76ba67bd3f608ba9c5ef1ab3fa2f312f7754afbe96fcd1cc36d691072d56e555  sam2.1_hiera_large_decoder.onnx
a5e2fdb10ed20763df5e2d5937bdbcb6ee108e63ea8bdb87582489360e847c76  sam2.1_hiera_small_decoder.onnx
cccf25b0b49819fb25c389841046dfbf5ac169be58d82e5cf6d6894675a43b36  sam2.1_hiera_tiny_decoder.onnx
f7707ac0db751c9cf35ec3eee149032068862c96e3f7ffa6109eef97ec36a414  big-lama-dynamic-coreml.onnx