LibreSEARAFTl-flow

SEA-RAFT L (ResNet34 encoders, 12 refinement iterations), an optical flow model, repackaged for LibreYOLO. Given two frames it predicts a dense displacement field from the first frame to the second, plus a per-pixel confidence derived from its mixture-of-Laplace uncertainty.

Upstream defines SEA-RAFT L as the M checkpoint run with 12 refinement iterations instead of 4 (config/eval/*-L.json), so this file carries the same learned parameters as LibreSEARAFTm-flow; the size: l metadata selects the 12-iteration forward.

Usage

from libreyolo import LibreYOLO

model = LibreYOLO("LibreSEARAFTl-flow.pt")      # auto-downloads from this repo
r = model(["frame_000.png", "frame_001.png"])[0]
r.flow.data          # (H, W, 2) float32, (dx, dy) in pixels, original canvas
r.flow.confidence    # (H, W) in [0, 1], 1 = most certain
r.flow.to_rgb()      # Middlebury colour wheel

for r in model("clip.mp4", stream=True):  # flow t to t+1 for every frame
    ...

Frames run at native resolution (padded to a multiple of 8); frames whose long side exceeds 960 px are downscaled and the flow is resized back with its vectors rescaled.

Source

Derived from princeton-vl/SEA-RAFT at commit 9137517ba24e628442aec097d3afe71d03503b75, checkpoint MemorySlices/Tartan-C-T-TSKH-spring540x960-M (revision eb97ef34ba5d856c3fa2cdcd073150c057ac8b69). Copyright (c) 2024, Princeton Vision & Learning Lab. Licensed under the BSD 3-Clause License.

Wang, Lipson and Deng, "SEA-RAFT: Simple, Efficient, Accurate RAFT for Optical Flow", ECCV 2024 (arXiv:2405.14793).

Modifications

LibreYOLO checkpoint-schema metadata wrap only; the upstream safetensors file stores the BatchNorm shared by bn3 and downsample.1 once, and the converter writes it under both names. Learned parameters are unchanged. See weights/convert_searaft_weights.py in the LibreYOLO source repository.

License

BSD 3-Clause License. See the LICENSE and NOTICE files in this repository.