LayeredFlow pre-trained checkpoints

Pre-trained RAFT and MultiRAFT checkpoints for LayeredFlow: A Real-World Benchmark for Non-Lambertian Multi-Layer Optical Flow (ECCV 2024), by Hongyu Wen, Erich Liang, and Jia Deng.

These files are migrated from the checkpoint folder linked by the official LayeredFlow repository: original Google Drive release. The original checkpoint filenames are preserved.

Models

  • RAFT: single-layer optical-flow models evaluated on the first-layer and last-layer benchmark subsets.
  • MultiRAFT: the repository's multi-layer RAFT model, which predicts four optical-flow layers and is evaluated on the full benchmark.

The checkpoints use the original PyTorch .pth format and are loaded by the LayeredFlow evaluation scripts.

Files

Checkpoint Use Size (bytes)
raft-sintel.pth RAFT Sintel checkpoint; also used for MultiRAFT initialization 21,108,000
last_layer_L.pth RAFT last-layer evaluation 21,103,087
first_layer_S+L.pth RAFT first-layer evaluation 21,103,087
first_layer_S.pth RAFT first-layer evaluation 21,103,087
first_layer_L.pth RAFT first-layer evaluation 21,103,087
MultiRAFT_S+L.pth MultiRAFT multi-layer evaluation 34,091,437
MultiRAFT_L.pth MultiRAFT multi-layer evaluation 34,091,437

The seven checkpoints are unchanged copies of the original release. checkpoint-manifest.json records each source URL, byte size, and SHA-256 digest; SHA256SUMS contains the checksums.

Download and evaluate

Clone the LayeredFlow source repository and follow its installation and benchmark-data setup instructions. From the source repository root, download the checkpoints with the Hugging Face CLI:

hf download Hermera/LayeredFlow --include 'first_layer_*.pth' 'last_layer_*.pth' 'raft-sintel.pth' --local-dir RAFT/checkpoints
hf download Hermera/LayeredFlow --include 'MultiRAFT_*.pth' 'raft-sintel.pth' --local-dir MultiRAFT/checkpoints

This places the files under RAFT/checkpoints/ and MultiRAFT/checkpoints/, where the evaluation scripts expect them.

Run the following validation examples from the source repository root:

# First-layer optical flow
(cd RAFT && python3 evaluate_firstlayer.py --checkpoint first_layer_S+L.pth --dataset layeredflow)

# Last-layer optical flow
(cd RAFT && python3 evaluate_lastlayer.py --checkpoint last_layer_L.pth --dataset layeredflow)

# Multi-layer optical flow
(cd MultiRAFT && python3 evaluate.py --checkpoint MultiRAFT_S+L.pth --dataset layeredflow --mixed_precision)

The repository's evaluation scripts require a CUDA-capable environment. See the benchmark website for test-set submission instructions.

Related resources

Attribution and license

The original LayeredFlow code is distributed under the BSD 3-Clause license. This checkpoint migration does not introduce new license terms. Refer to the original release and its authors for checkpoint licensing.

Citation

@article{wen2024layeredflow,
    title={LayeredFlow: A Real-World Benchmark for Non-Lambertian Multi-Layer Optical Flow},
    author={Hongyu Wen and Erich Liang and Jia Deng},
    journal={arXiv preprint arXiv:2409.05688},
    year={2024},
}
Downloads last month

-

Downloads are not tracked for this model. How to track
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Dataset used to train Hermera/LayeredFlow

Paper for Hermera/LayeredFlow