modelarchive/Nanosaur2-670M

Nanosaur2

ComfyUI support for Nanosaur2, a 670M parameter illustration text-to-image DiT. This model is for research purposes. Total training cost: $600

I am looking for compute funding! This model can be trained much further and I would also like to make a 2B version of it.

The PNG contains the ComfyUI workflow: drag it onto ComfyUI to load it.

Inference

Install

1. Copy nanosaur2_support to ComfyUI/custom_nodes.

2. Put the three model files in ComfyUI's model folders:

   File	Folder
   nanosaur2_diffusion_model.safetensors	models/diffusion_models
   nanosaur2_text_encoder.safetensors	models/text_encoders
   nanosaur2_vae.safetensors	models/vae

3. Start ComfyUI and drag nanosaur2_workflow.json onto it.

Prompts

Euler simple, 50 steps, CFG 4, 1024x aspect buckets, shift 3 built into the model inference

Either tags or natural language may be used. Prefix with "newest, masterpiece" quality tags on positive prompt and "oldest, low quality" on negative prompt.

Some artist tags work, you may need to upweight them like (artist:4). The prompt upweighting support is custom and affects the cross attention bias in the model, not the embedding magnitude.

Sampling uses alternating CFG and PDG (path-drop guidance from SPRINT [1]).

Model and training

Component	Details
Diffusion model	670M parameter DiT with adaLN-single [3], 2D RoPE [4], SwiGLU [5], QK-norm [6], SPRINT sparse middle blocks [1] and x-prediction [7]
Text encoder	gemma-3-270m, penultimate layer with the final norm, frozen
VAE	Semantic DINOv2 VAE (129M parameters)

Diffusion model. .

Training objective. Rectified flow: x-prediction with v-loss, logit-normal timesteps, 10% caption dropout. For CFG, text-dropped samples skip the middle blocks 50% of the time and the other samples 5% of the time, so the model learns the path-drop unconditional pass used by alternate and path_drop guidance.

Text encoder. Frozen Gemma3-270M [8], second to last layer.

VAE. Representation autoencoder [10, 11] trained from DINOv2-B [9] with: frozen DINOv2 (32ch), DINOv2 with unfrozen patch embed as in UniSpace [12], last 6 layers channel concat (32ch), conv decoder, losses are VISReg [13] 1e-3 on the 64-dim latent bottleneck, and DINO embedding MSE loss on each layer equal weighted of frozen DINOv2-B and DINOv3-B [14]. There are no pixel losses, no LPIPS and no GAN. The roundtrip DINO losses create a VAE latent that converges faster than current VAEs.

Training data and schedule.

1. VAE training: 50k steps (6 hours on 1xH100) from DINOv2-B.
2. Base training: 11 H100-days on a 4M-image illustration dataset. Dion3 optimizer [15] (lr 1e-3, scalar lr 1e-4, weight decay 0.01) with cosine schedule. 3 days at 256x256, 3 days at 512x512, 5 days at 1024x1024 aspect buckets.
3. Aesthetic tuning: 4 hours without SPRINT token dropping (all tokens run through the middle blocks): Dion3 1e-4, scalar 5e-5 with cosine decay.

LoRA training

uv run python custom_nodes/nanosaur2_support/train_lora.py /path/to/images

Trains on a folder of images with same-name .txt captions and writes models/loras/<folder name>.safetensors. Load it with LoraLoaderModelOnly. It uses the ComfyUI model files, without caching for simplicity. Images are resized and center-cropped to the closest 1024x aspect bucket.

References

1. Park et al. Sprint: Sparse-Dense Residual Fusion for Efficient Diffusion Transformers. ICLR 2026.
2. Peebles and Xie. Scalable Diffusion Models with Transformers. ICCV 2023.
3. Chen et al. PixArt-α: Fast Training of Diffusion Transformer for Photorealistic Text-to-Image Synthesis. ICLR 2024.
4. Su et al. RoFormer: Enhanced Transformer with Rotary Position Embedding. 2021.
5. Shazeer. GLU Variants Improve Transformer. 2020.
6. Dehghani et al. Scaling Vision Transformers to 22 Billion Parameters. ICML 2023.
7. Li and He. Back to Basics: Let Denoising Generative Models Denoise. CVPR 2026.
8. Gemma Team. Gemma 3 Technical Report. 2025.
9. Oquab et al. DINOv2: Learning Robust Visual Features without Supervision. TMLR 2024.
10. Zheng et al. Diffusion Transformers with Representation Autoencoders. 2025.
11. Singh et al. Improved Baselines with Representation Autoencoders. 2026.
12. Yan et al. UniSpace: Unified Visual Representation and Scalable Multimodal Modeling. 2026.
13. Wu, Balestriero and Levine. VISReg: Variance-Invariance-Sketching Regularization for JEPA training. 2026.
14. Siméoni et al. DINOv3. 2025.
15. Amsel et al. Dion3: Full-Stack Orthogonal Updates. 2026.