EBC-JEPA per-event LED denoising: carrier "k frames (histogram)"

Frozen V-JEPA 2.1 tokens on LED, tokenised as k frames (histogram) (16 time bins of 1.25 ms paired into 8 tubelet tokens per cell), scored per event: each event looks up the token at its (time bin / 2, patch) and is judged with its own E15 neighbourhood patch. Routes in this repo:

  • V12 AC: token + E15 patch CNN, no history (v12_ac/).
  • Integrated: History Reader (kv) conditions the token on earlier chunks, a TemporalViT reads the patch, and Pixel-AC fuses the per-event scores into one decision per pixel and chunk (integrated/<run>/; result.json carries the Pixel-AC metrics with the AC-only metrics under ac). Run-name suffixes: _nokv = reader bypassed, _lb = scalar Local branch on, _vit_base = ViT-B/384 tokens.

All units: 6,000 training chunks, 4 epochs, official 713 chunks at a fixed threshold of 0.5, no selection on test.

unit route backbone kv LB SR NR DA
integrated/vit_base History Reader + TemporalViT AC + Pixel-AC ViT-B on off 0.8368 0.9058 0.8713
integrated/vit_large History Reader + TemporalViT AC + Pixel-AC ViT-L on off 0.8432 0.8992 0.8712
integrated/vit_large_lb TemporalViT AC only (no Pixel-AC) ViT-L on on 0.8380 0.8961 0.8670
integrated/vit_base_lb TemporalViT AC only (no Pixel-AC) ViT-B on on 0.8326 0.9004 0.8665
integrated/vit_large_nokv_lb TemporalViT AC only (no Pixel-AC) ViT-L off on 0.8249 0.9013 0.8631
integrated/vit_large_nokv TemporalViT AC only (no Pixel-AC) ViT-L off off 0.8289 0.8961 0.8625
integrated/vit_base_nokv TemporalViT AC only (no Pixel-AC) ViT-B off off 0.8208 0.9005 0.8606
v12_ac V12 AC: event's V-JEPA token + E15 patch CNN, no history ViT-L off off 0.8907 0.8272 0.8589

Reference on the same protocol: the collaborator's ViT-L histogram integrated run with Pixel-AC, DA .8714. Code: EBC-JEPA-share (denoising-local-run) and its share-contrib-vitb-downstream worktree.

Downloads last month

-

Downloads are not tracked for this model. How to track
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Collection including h3lloworld/ebc-jepa-led-time-bins

Evaluation results

  • Denoising accuracy (SR + NR) / 2, threshold 0.5 on LED official test (66 scenes, 713 chunks)
    self-reported
    0.871
  • Signal retention on LED official test (66 scenes, 713 chunks)
    self-reported
    0.837
  • Noise removal on LED official test (66 scenes, 713 chunks)
    self-reported
    0.906