Any plans to release the SemVAE weights for fine-tuning the 5B model?
Thanks for the great work on SeFi-Image! We'd love to fine-tune the 5B model, but hit the one missing piece: per the paper (Eqs. 1–6), training supervises the semantic stream with s1 = E_s(Φ(x)), and the released 5B checkpoint ships the DiT, Flux.2 Texture VAE, Qwen3-VL and scheduler — so z1 is reproducible — but not the SemVAE encoder E_s, without which we can't produce s1. The original SFD repo (https://github.com/yuemingPAN/SFD) does publish a SemVAE, but only for the smaller ImageNet model — built on DINOv2 ViT-B/14-reg (768-dim). For the 5B model, which uses DINOv2-Large (1024-dim) and was trained on ~400M+ images, the SemVAE weights aren't available anywhere.
Could you share whether you plan to release the SemVAE encoder weights (plus the exact DINOv2-Large variant used) for the 5B model? Encoder-only would already be enough to precompute s1 and enable faithful SFT. We'd really appreciate it, and we think the community would too — it would open up a genuinely nice research direction around the flexibility of fine-tuning semantic-first / dual-latent models, which is hard to explore without the original semantic encoder.
Hi,
Thanks a lot for your interest in SeFi-Image and for the note.
We have now uploaded the SemVAE assets for the models here:
https://huggingface.co/SeFi-Image/SeFi-Image-SemVAE
The release includes the original SemVAE checkpoint weights as well as the corresponding latent normalization statistics (latent_stats.pt), which should allow you to reproduce the
semantic latent supervision pipeline for fine-tuning.
Please note that these are the original training checkpoint weights, so the format may differ from the inference weights we released earlier. You may need to do a small conversion or
adaptation step depending on your fine-tuning pipeline.
If you have any further questions, please fill free to comment.
Thank you so much for uploading the SemVAE weights and the latent_stats.pt — that was exactly the missing piece. The format difference from the inference weights was easy to adapt (DINOv2-Large + the transformer encoder → μ, normalized with the latent stats), and reconstructions came out clean.
Happy to report that our test fine-tuning workflow for the 5B model just ran end-to-end successfully — precomputed s1/z1 supervision, dual-stream flow-matching, sampling and checkpointing all working. Really appreciate the quick turnaround; this opens up the semantic-first fine-tuning direction we were hoping to explore.