Semantic VAE

Purpose: Extremely fast convergence on illustraton data for a downstream DiT model

Trained from DINOv2 in 12 hours as follows:

Frozen DINOv2 -> LayerNorm -> 32 channels

Frozen DINOv2 with patch embed unfrozen -> LayerNorm -> 32 channels

64 channel bottleneck with Sigreg loss (1e-3 weight) from LeJEPA

VA-VAE style convolution decoder

Loss is all layers of DINOv2 and DINOv3 equally weighted MSE on input vs reconstruction.

There are no pixel losses/LPIPS/GAN used, this is DINO space loss only.

Downloads last month: -; Downloads are not tracked for this model. How to track

Inference Providers NEW

This model isn't deployed by any Inference Provider. 🙋 Ask for provider support