antv311/propainter-mlx-weights

ProPainter weights for MLX

ProPainter (ICCV 2023) video inpainting, converted to safetensors for MLX on Apple Silicon, together with the two optical-flow models it runs with. These are the original weights, re-laid out for MLX: no retraining, no quantisation, fp32 as released.

Used by propainter-mlx, a native MLX port of ProPainter that downloads these files on first use, pinned by commit and SHA-256. No PyTorch is needed at any point.

file	model	params	licence	SHA-256
ProPainter.safetensors	ProPainter InpaintGenerator	39.4M	S-Lab 1.0 (non-commercial)	21033ebd1b6d135a…
recurrent_flow_completion.safetensors	RecurrentFlowCompleteNet (flow completion)	5.1M	S-Lab 1.0 (non-commercial)	5917fd0f85ed16ee…
raft-things.safetensors	RAFT (things)	5.3M	BSD-3-Clause	71d0acc39661bf08…
sea-raft.safetensors	SEA-RAFT-M (spring 540x960)	19.7M	BSD-3-Clause	5d1ba133243e864a…

manifest.json has every file's full SHA-256, its source checkpoint (URL and SHA-256), the upstream keys dropped as training-only, and the name and shape of every tensor.

Conversion

ProPainter, flow completion and RAFT come from the upstream v0.1.0 release; SEA-RAFT is Tartan-C-T-TSKH-spring540x960-M from MemorySlices. The conversion only moves values, it never changes them:

• module. prefixes stripped (RAFT was saved from DataParallel);
• conv weights (O, I, kH, kW) → (O, kH, kW, I), 3D conv (O, I, kT, kH, kW) → (O, kT, kH, kW, I) (MLX is NHWC);
• ProPainter's SoftSplit/SoftComp embeddings and FusionFeedForward fc1/fc2 reordered from torch's channel-major unfold order to (taps, C);
• training-only tensors dropped: the flow-completion edge head, BatchNorm num_batches_tracked, and a buffer the model recomputes.

Every upstream key maps by an explicit rule, and every tensor is checked against the MLX models.

Parity

Against upstream PyTorch at commit e870e79 (fp32): per-module relative error 1e-6 to 2.5e-4; end to end, PSNR of the port's compat pipeline against upstream's output frames: bmx-trees 58.5 dB, tennis 86.0 dB, running_car square / logo mask 62.0 / 62.4 dB, outpainting 54.0 dB. SEA-RAFT: 3.7e-5 relative error against the official PyTorch model.

Licence

ProPainter and flow completion: S-Lab License 1.0, non-commercial use only. Commercial use requires permission from the ProPainter authors (Dr. Shangchen Zhou, shangchenzhou@gmail.com; Prof. Chen Change Loy, ccloy@ntu.edu.sg). The converted files carry the same terms as the originals. RAFT (LICENSE-RAFT) and SEA-RAFT (LICENSE-SEA-RAFT) are BSD 3-Clause. See NOTICE.

Citation

@inproceedings{zhou2023propainter,
  title={{ProPainter}: Improving Propagation and Transformer for Video Inpainting},
  author={Zhou, Shangchen and Li, Chongyi and Chan, Kelvin C.K and Loy, Chen Change},
  booktitle={Proceedings of IEEE International Conference on Computer Vision (ICCV)},
  year={2023}
}
@inproceedings{teed2020raft,
  title={RAFT: Recurrent All-Pairs Field Transforms for Optical Flow},
  author={Teed, Zachary and Deng, Jia},
  booktitle={European Conference on Computer Vision (ECCV)},
  year={2020}
}
@article{wang2024sea,
  title={SEA-RAFT: Simple, Efficient, Accurate RAFT for Optical Flow},
  author={Wang, Yihan and Lipson, Lahav and Deng, Jia},
  journal={arXiv preprint arXiv:2405.14793},
  year={2024}
}