LibreRAFTl-flow

RAFT (large, 5.3M parameters) optical flow, upstream checkpoint raft-things.pth, repackaged for LibreYOLO. Training schedule: FlyingChairs then FlyingThings3D (C+T); the upstream default for generalisation.

from libreyolo import LibreYOLO

model = LibreYOLO("LibreRAFTl-flow.pt")
r = model(["frame_000.png", "frame_001.png"])[0]
r.flow.data        # (H, W, 2) float32, (dx, dy) in pixels, frame 0 to frame 1
r.flow.to_rgb()    # Middlebury colour wheel

Inference runs at native resolution, padded to a multiple of 8 as upstream InputPadder, with 20 refinement iterations (upstream demo.py).

Source

Derived from princeton-vl/RAFT at commit 2888e15, file raft-things.pth from the release models.zip (sha256 fcfa4125d6418f4de95d84aec20a3c5f4e205101715a79f193243c186ac9a7e1). Copyright (c) 2020, princeton-vl. Licensed under the BSD 3-Clause License.

Modifications

The DataParallel module. key prefix is stripped and LibreYOLO checkpoint metadata is added (task: flow). Learned parameters are unchanged. See weights/convert_raft_weights.py in the LibreYOLO source repository. On 9 Sintel demo pairs (1024x436) the LibreYOLO output matches upstream RAFT bit for bit in fp32 (max abs difference 0.0 px).

License

BSD 3-Clause License. See the LICENSE and NOTICE files in this repository.