halfmoon-mind/still-seedvr2

SeedVR2-3B for Core ML (one 512 px frame, int4 DiT)

A Core ML conversion of SeedVR2-3B (SeedVR2: One-Step Video Restoration via Diffusion Adversarial Post-Training; code, weights), a one-step diffusion transformer for video and image restoration. The iOS app Still (한컷) downloads it on demand as a developer-menu experiment for maximum quality, and uses it on single frames.

SeedVR2 synthesises detail. Restored detail is a prediction, not evidence of captured detail.

Files

Path	Purpose
SeedVR2Encoder.mlpackage/	VAE encoder, ML Program (iOS 18+), FP16 weights
SeedVR2DiT0.mlpackage/ … SeedVR2DiT6.mlpackage/	The 32 DiT blocks in seven chunks (0–2, 3–5, 6–8, 9–14, 15–20, 21–26, 27–31), ML Program (iOS 18+), 4-bit weights in 32-weight blocks
SeedVR2Decoder.mlpackage/	VAE decoder, ML Program (iOS 18+), FP16 weights
manifest.json	Size and SHA-256 of every file in the packages
LICENSE	Apache License 2.0

Interface

All inputs and outputs are float16. Every package is fixed at one 512 × 512 output frame.

Package	Inputs	Outputs
SeedVR2Encoder	image [1, 3, 512, 512], RGB in [-1, 1]	latent [4096, 16]
SeedVR2DiT0	latent [4096, 16]	vid_out [1024, 2560], txt_out [58, 2560]
SeedVR2DiT1 … SeedVR2DiT5	vid [1024, 2560], txt [58, 2560]	vid_out, txt_out
SeedVR2DiT6	vid, txt	x0 [4096, 16]
SeedVR2Decoder	x0 [4096, 16]	image [1, 3, 512, 512], RGB in about [-1, 1]

• Encoder input. The low-quality frame enlarged to the output size. The reference CLI uses bilinear interpolation on Apple GPUs.
• Encoder output. The posterior mode × 0.9152, channels last. It is the DiT's conditioning latent.
• Decoder output. Taken before the CLI's clamp and color correction. The app keeps the decoded detail and takes color and light below about 16 px from the input.
• Larger images. The app cuts larger areas into 512 px tiles that overlap by 64 px. Neighbouring tiles cross in a 32 px seam in the middle of their overlap.

Measured on iPhone 14 Pro (A16, iOS 18.4), second run, one model resident at a time:

	Load	Run per frame
Encoder (Neural Engine)	0.3 s	1.7 s
DiT, seven chunks (GPU)	2.8 s	5.0 s
Decoder (Neural Engine)	15.1 s	3.5 s

• Memory. The footprint peaked at 1.6 GiB while the decoder loaded.
• Run the DiT on the GPU (.cpuAndGPU). The A16 Neural Engine computed these iOS 18 compressed weights wrongly. FP16 and the VAE were correct there, and the same packages were correct on an M4 Mac's Neural Engine.

Changes from the original weights

Notice of modifications under section 4(b) of the Apache License 2.0. The source was numz/SeedVR2_comfyUI at revision 09ced71023636e9bc8cdf9cdecfb2625d1e691e8, an FP16 repackaging of ByteDance-Seed/SeedVR2-3B under Apache 2.0:

• seedvr2_ema_3b_fp16.safetensors, SHA-256 2fd0e03a3dad24e07086750360727ca437de4ecd456f769856e960ae93e2b304
• ema_vae_fp16.safetensors, SHA-256 20678548f420d98d26f11442d3528f8b8c94e57ee046ef93dbb7633da8612ca1

The following were changed:

1. Conversion. Converted from PyTorch to Core ML ML Programs for one 512 × 512 output frame. Window indices and rotary positions became constants of that size.
2. Split. Split the DiT into the seven chunks listed above. The last chunk also takes the single Euler step (x0 = noise − v).
3. Precomputed constants. Some inputs are fixed in one-step restoration, so they were computed once and stored:
   • the timestep embedding for t = 1000;
   • the text conditioning: the published positive prompt embedding (pos_emb.pt) after txt_in;
   • the noise, from seed 42.
4. FP16 and size changes.
   • RMSNorm divides each token by its largest magnitude before squaring. The text stream reaches 28,640, and x² overflows FP16.
   • The timestep modulation is broadcast instead of repeated per token.
   • Attention is written as matrix products and a softmax.
   • In FP32 these chunks equal the original forward pass: the largest difference was 0.
5. Quantization. Quantized the DiT weights to 4-bit linear with 32-weight blocks (coremltools 9.0). The VAE stays FP16. On one test frame the decoded image is 34.5 dB from the FP32 model's, and the iPhone 14 Pro output is 56 dB from the Mac's.

The conversion scripts are kept in the app's repository (research/ReferenceSRStudy).

License

SeedVR2's code and weights are © their authors (ByteDance Seed) under the Apache License 2.0 (LICENSE). This conversion is distributed under the same license. This repository is not affiliated with or endorsed by ByteDance or the SeedVR authors.

Training data. The SeedVR authors describe their training data in the paper. It was not reviewed for this conversion, and no warranty is given that any particular use of the weights is free of claims arising from it.

Citation

@article{wang2025seedvr2,
  title={SeedVR2: One-Step Video Restoration via Diffusion Adversarial Post-Training},
  author={Wang, Jianyi and Lin, Shanchuan and Lin, Zhijie and Ren, Yuxi and Wei, Meng and Yue, Zongsheng and Zhou, Shangchen and Chen, Hao and Zhao, Yang and Yang, Ceyuan and Xiao, Xuefeng and Loy, Chen Change and Jiang, Lu},
  journal={arXiv preprint arXiv:2506.05301},
  year={2025}
}