LANTERN — Aβ plaque segmentation in 3D light-sheet microscopy (all cohorts)

5-fold ensemble of 3D U-Nets that segment amyloid-β plaques in cleared whole mouse brain imaged by light-sheet microscopy. Unlike the cohort-specific siblings (lantern-ki3-abeta, lantern-mapt_a/b/c-abeta) this one is trained on every hand-reviewed Aβ patch LANTERN has: 178 patches from 68 mice across the lecanemab (ki3, batches 1-3), vaccine, ApoE, fibrinogen and MAPT/APOE cohorts, imaged on LifeCanvas and Blaze systems. Each fold is the same frozen self-supervised ResEncL encoder with a decoder trained from scratch, plus a small bottleneck conditioner that feeds the network the two numbers per-tile normalisation removes: the log mean and log standard deviation of the raw tile intensities.

LANTERN = Light-sheet Automask Network for Transferable Embedding Representations.

What this is


Architecture	`ResEncLUNet` — ResEncL encoder (6 stages, features `[32,64,128,256,320,320]`) + `UNetDecoder` + `BottleneckConditioner`
Parameters	102.5 M total, 12.2 M trainable (encoder frozen)
Encoder init	nnBYOL3D self-supervised pretraining on ~50k near-iso-4 light-sheet patches (unchanged since)
Decoder init	random
Input	1 channel, `128³` voxels, near-iso grid (z 2.75–4.4 µm by cohort, 3.25 × 3.25 µm in-plane) + 2 scalars (raw tile mean and std)
Output	2 classes (background, plaque)
Weights	best-validation epoch of each fold
Folds	5, patch-level 5-fold CV — use all 5 and vote

Recipe

Binary target (the annotated-false-positive label 2 used in the MAPT GT is folded into background); ground-truth components under 4 voxels removed before training and validation (--gt-min-cc 4); plain DiceCE (CE + batch-Dice on plaque); AdamW lr 1e-3, 10-epoch warm-up then cosine, 250 epochs, batch 2 × 4 accumulation, bf16-mixed; best-validation checkpoint. The conditioner takes (log1p(raw_mean), log1p(raw_std)), standardised with constants measured on the fold's training split (stored as buffers in the checkpoint), passes them through Linear(2→32), broadcasts over the bottleneck feature map and mixes with a 1×1×1 conv (identity-initialised). During training each of the two numbers was dropped with probability 0.3.

Usage

Preprocessing must match exactly, and the two raw statistics are required (the forward raises without them).

import numpy as np, torch
from lantern.models.unet_seg import ResEncLUNet     # https://github.com/Arshya-Guru/LANTERN

def normalize(vol):                      # per-tile, exactly as in training
    lo, hi = np.percentile(vol, [0.5, 99.5])
    v = np.clip(vol, lo, hi).astype(np.float32)
    s = float(v.std())
    return (v - float(v.mean())) / (s if s > 1e-8 else 1.0)

nets = []
for k in range(5):
    ck = torch.load(f"fold{k}/seg_model.pt", map_location="cpu", weights_only=False)
    n = ResEncLUNet(num_classes=2, num_input_channels=1, conditioner=True)
    n.load_state_dict(ck["state_dict"])           # includes the conditioner's standardisation buffers
    nets.append(n.eval().cuda())

tile = tile.astype(np.float32)                                  # raw (128,128,128), NOT normalised yet
raw_mean = torch.tensor([[float(tile.mean())]]).cuda()         # statistics of the RAW tile
raw_std  = torch.tensor([[float(tile.std())]]).cuda()
x = torch.from_numpy(normalize(tile))[None, None].cuda()
prob = torch.zeros((128, 128, 128), device="cuda")
with torch.no_grad(), torch.autocast("cuda", dtype=torch.bfloat16):
    votes = sum((torch.softmax(n(x, raw_mean=raw_mean, raw_std=raw_std).float(), 1)[0, 1] >= 0.5).to(torch.uint8) for n in nets)
mask = (votes >= 3)

For volumes larger than 128³, tile at stride 64 (50% overlap), compute the two statistics per tile, and combine overlapping tiles by averaging each fold's softmax before thresholding (not a max over votes). Recommended post-processing: drop predicted plaque components under 8 near-iso voxels.

Training data

178 hand-reviewed 128³ patches from 68 mice (v7/total_set/v1): 74 ki3, 53 mc4, 40 mapt, 4 o22b, 4 prune, 3 vac. 19 patches contain no plaque (wild-type / negative-control animals and low-load MAPT brains); 25 patches carried an annotated-false-positive label that was folded into background. Labels come from several conventions: N4-threshold-plus-review for the ki3 and SSL-pool patches, hand drawing then mechanical extent harmonisation for MAPT. 142 train / 36 val per fold, random patch-level split, seed 1234.

Augmentation: axis flips, one 90° rotation, mild gamma and Gaussian noise.

Evaluation

Per-fold validation split (36 patches), object F1 at IoU ≥ 0.5 on 26-connected components, reported at each fold's best epoch and averaged over the 5 folds.

	as predicted	+ drop components < 8 voxels
object F1	0.459 ± 0.060	0.504
precision / recall	0.426 / 0.501
false positives per patch	100.7	70.0
voxel Dice	0.605

Per fold (best epoch): fold 0: 0.409 @ ep 99, fold 1: 0.446 @ ep 64, fold 2: 0.564 @ ep 119, fold 3: 0.398 @ ep 109, fold 4: 0.478 @ ep 24. Learning rates 3e-4 and 1e-4 landed within 0.02 of this configuration; a GT floor of 8 voxels scored ~0.02 higher raw / ~0.05 higher after the 8-voxel prediction filter (which favours it) at identical voxel Dice (0.63), so the 4-voxel floor was kept as the more faithful target for small plaques.

Limitations

Scale and normalisation sensitivity. Resample to the near-iso grid above and normalise per tile on raw intensities; pass the raw mean/std of the same tile.
Optimistic validation. Patch-level split, not subject-grouped, and the metric is read at the selected epoch; expect worse on an unseen animal.
Mixed label conventions. Plaque extent differs between cohorts (dense N4-threshold masks vs harmonised MAPT masks); the model learns a compromise.
No FP class. Bright non-plaque structures (vessels, tissue edges, ventricle walls) must be handled downstream; false positives per patch are high at the 0.5 threshold.
Not loadable by inference code that calls net(x) alone — the conditioner needs raw_mean and raw_std.
Not validated for absolute burden. Detection metrics on 128³ patches only.

Files

fold0..fold4/seg_model.pt   torch checkpoint: state_dict (best-val weights incl. conditioner buffers), config, provenance keys
config.json                 architecture + preprocessing + conditioner contract

Citation

Code: https://github.com/Arshya-Guru/LANTERN

Released for non-commercial research use (CC BY-NC 4.0). The underlying imaging data is not included.

Downloads last month: 39