EdgeTAM — ONNX exports for on-device Unity inference
ONNX exports of EdgeTAM used by the OnPrem Vision mobile app for promptable segmentation and mask tracking, running entirely on-device through ONNX Runtime.
This repo is public so the shipped app can fetch weights without embedding any credentials. It holds only the runtime graphs and constants.
Layout
fp32/ fp16/ npu/ five graphs each (see below)
constants/ precomputed tensors the tracking path needs
model_info.json tensor shapes, normalisation constants, memory layout
unity_manifest.json size + sha256 of every runtime file (integrity check)
Graphs
| File | Role |
|---|---|
image_encoder.onnx |
1024×1024 RGB → vision_feat (1,256,64,64) + two hi-res feature maps + positional encoding |
prompt_encoder.onnx |
point/label prompts → sparse + dense embeddings (auto-pad token added in-graph) |
mask_decoder.onnx |
image + prompt embeddings → low-res mask, IoU, object pointer, multimasks |
memory_encoder.onnx |
features + mask → mask-memory features and positions |
memory_attention.onnx |
conditions current features on the memory bank (the hot per-frame graph) |
Exported at opset 18, fixed 1024×1024 input. ImageNet normalisation
(mean 0.485/0.456/0.406, std 0.229/0.224/0.225). Full shape and
memory-layout detail is in model_info.json.
Variants
| Dir | Intended execution provider |
|---|---|
fp32/ |
CPU / XNNPACK. The reference path; what parity checks run against. |
npu/ |
CoreML / NNAPI. Same fp32 graphs, except a static-M=516 fp16 memory_attention that an NPU can compile (verified against the dynamic fp32 graph, max abs diff ≈1.5e-3). |
fp16/ |
Currently byte-identical to npu/. |
Only memory_attention.onnx differs between variants — fp32/ is 64 MB,
fp16/ and npu/ are 32 MB. The other four graphs are byte-identical across
all three, so the ~265 MB of files dedupe to ~149 MB of storage.
fp16 graphs must not run on the plain CPU EP — ORT has no fp16 CPU kernels. The consuming code enforces this invariant before session creation.
Integrity
unity_manifest.json lists the size and SHA-256 of all 19 runtime files:
{ "revision": "<hub commit>",
"files": [ { "path": "fp32/image_encoder.onnx", "size": 19775207, "sha256": "c41c05e3..." } ] }
The client streams each file to a .part temp and only commits it after both
match, so a truncated download never reaches the inference runtime. If you
republish a weight file, regenerate this manifest or clients will reject the
new file as corrupt.
Use
From the Unity app, weights are fetched on first launch into
Application.persistentDataPath/edgetam/, or ahead of time into
Assets/StreamingAssets/edgetam/. Direct URL pattern:
https://huggingface.co/onpremvision/edgetam-weights/resolve/main/<path>
hf download onpremvision/edgetam-weights --local-dir ./edgetam
Pin a commit instead of main to freeze a build to the weights it was tested with.
Related
onpremvision/vision_models(private) — parity-test fixtures, TensorRT engines, other exports.
EdgeTAM is from EdgeTAM: On-Device Track Anything Model (Meta Reality Labs); these are re-exports for on-device ONNX Runtime inference, redistributed under the license above.