glowskeleton/goblin-models

Goblin models

ONNX artifacts for Goblin, a GPU-accelerated node-based compositor. Everything Goblin's model installer downloads lives here, so a given Goblin version pins exactly the artifacts it was tested against rather than tracking upstream main.

Three groups.

sam2-video/ — Goblin re-exports for SAM2 video propagation

These are not upstream files. facebook/sam2.1-hiera-tiny has no published ONNX export of the video path, so these were produced for Goblin:

file	what it is
memory_attention.onnx	needed a real-valued RoPE rewrite — SAM2's complex tensors have no ONNX representation
memory_encoder.onnx	memory encoder export
prompt_encoder_mask_decoder.onnx	re-export carrying the obj_ptr output propagation needs, all four mask tokens, and the input_masks dense prompt
*.f32	constant tensors dumped from the PyTorch checkpoint — not models

The decoder here replaces the upstream one. The upstream build has no obj_ptr output, so video propagation cannot run against it.

vitmatte/ — ViTMatte with Einsum rewritten as MatMul

model.onnx is Xenova/vitmatte-small-composition-1k with its 24 Einsum nodes — 12 bhwc,hkc->bhwk and 12 bhwc,wkc->bhwk, ViT's decomposed relative position bias — rewritten as broadcasting MatMul.

That single op was the reason the model could only run on CPU:

• CoreML cannot execute Einsum, so it partitioned around all 24 (measured at ~51 partitions, no speedup over CPU).
• DirectML has a documented Einsum wrong-results bug (onnxruntime#19837) that silently corrupts the matte.

The rewrite is bit-identical on the CPU execution provider — max absolute difference 0.000e+00 over random inputs — and introduces no new operator and no opset change. Produced and verified by tools/ml/rewrite_vitmatte_einsum.py, which refuses to emit a model whose output drifts by more than 1e-4.

Goblin selects the execution provider from the graph rather than from a flag: a model still containing Einsum runs on CPU regardless of settings, because a wrong matte looks like a slightly different matte rather than an error.

sam2-upstream/ — mirror of the SAM2 image-mode files

Byte-for-byte copies of the ONNX from onnx-community/sam2.1-hiera-tiny-ONNX, mirrored so an install is self-contained and cannot change under a force-push upstream. prompt_encoder_mask_decoder.onnx here is the original community build; sam2-video/ holds the re-export that supersedes it.

Licensing

SAM 2 is Apache 2.0 (Meta). ViTMatte weights derive from hustvl/vitmatte-small-composition-1k. Derived artifacts here carry their upstream licenses; the export and rewrite tooling is part of Goblin.