BigStitcher interest point embedding: translation invariant, stacked extents (stack64)
A small 3-D convolutional network that turns a detected interest point (a DoG detection in a light-sheet volume) into a 64-D unit vector describing its surroundings at five scales. Two detections of the same physical structure in two overlapping views get similar vectors, so BigStitcher can match points by cosine similarity instead of by the geometry of their neighbours. Exported from pcmatch; evaluated inside BigStitcher by its own Java ONNX evaluator, no runtime needed.
Files
| file | content |
|---|---|
<name>.onnx |
the graph, opset 18, 22 operator types (Conv, Relu, MaxPool, Gemm, ReduceL2 and the elementwise ops of the standardisation) |
<name>.onnx.json |
the sidecar BigStitcher reads: tensor contract, sampling configuration, training record, fingerprint |
Both files must stay next to each other under these paired names.
Fingerprint: 6844e0e3f80c8eb2217a2021ab0c3215cddc49e11cd78269e47c47fc2ff0cb75 (sha256 of the configuration and all weights). Every array of vectors written with this model carries it as model.fingerprint; BigStitcher refuses to match vectors of different fingerprints.
Input and output
- Input
patches, float32[N, 5, 16, 16, 16], axes N, channel, z, y, x. Channel k is a cube of 16³ samples centred on the point with an edge of k × sigma for k = 5, 8, 13, 20, 30, where sigma is the detection scale of the point in full-resolution pixels. The cube is isotropic in physical units (the voxel size corrects the anisotropy), sampled trilinearly from the pyramid level whose resolution matches the sample spacing, never finer than the level the detection ran at. Raw intensities with any gain or offset; the standardisation is part of the graph. Samples outside the image are NaN and are masked inside the graph. - Output
descriptor, float32[N, 64], L2-normalised rows.
Matching
Cosine similarity between the vectors of two views. A pair is a candidate when the best partner's similarity is above 0.3 and passes a ratio test (0.85 in similarity space) against the second best; RANSAC on the candidates gives the view transformation as for geometric descriptors.
Training
Positive pairs are corresponding detections of registered view pairs (tolerance 2 px) from seven light-sheet datasets: bead-based registrations of a stitching dataset, of a mouse cortex sample, of a mouse head, of two further collaborator samples, and scale-space nuclei detections of a mouse embryo and of a fly brain; 27,330 positives in 36 pair groups, at most 2,000 per pair. Loss HardNet (margin 0.15) with hard negatives drawn within 500 px of the anchor (3 per anchor, 2 of them the nearest) and batches of 1,024 drawn locally; augmentation by rotations up to 8 degrees and 6 % affine jitter; per-point sigma; 3,000 steps, learning rate 1e-3, seed 0, final loss 0.0027, 12 minutes on one GPU. pcmatch commit 2c42a45, 2026-10-27.
Scope and limits
- Translation invariant only: the views have to be in roughly the same orientation (training augmentation ±8 degrees). A rotation-invariant model is a separate release.
- Trained on beads and nuclei-like blobs detected by a DoG; other structures may embed poorly.
- Points closer than about 15 sigma to the image border lose part of their largest cubes (masked, not zero-filled); far fewer such points in large volumes.
Export and verification
PyTorch 2.11 (dynamo exporter), opset 18. Checked against onnxruntime 1.31.1 on 64 points: maximal absolute difference 3.9e-7, minimal cosine 0.9999998. BigStitcher's Java evaluator reproduces pcmatch's vectors to 3e-5.
How to use
BigStitcher (Embed Interest Points), BigStitcher-Spark (embed-interestpoints) and pcmatch (pcmatch embed) sample the cubes identically; the stored vectors agree to float rounding.