BE-ImgGen-Portrait-187M

BE = Byrne / Escarda — a text-to-image latent-diffusion DiT built entirely from the SpikeWhale / Escarda / Byrne family architecture, taken through six training campaigns to its current form: a painterly text-to-face generator with reliable coarse attribute conditioning (hair colour, beard, age, skin tone, freckles).

Final checkpoint: 294,236 steps · 162M DiT + 24M VAE · 55,030 5-point-aligned faces.

prompt → Escarda-86M-Base-JL (frozen, 640-d) ─┐
                                              ├→ cross-attn
noise → VAE latent 32×32×16 → patch4 → 64 tokens → Family DiT ─→ ε   (DDIM sampling)
        REPA aux: block-6 features aligned to frozen Byrne-VE (DINOv2-distilled)

Family traits used

From Byrne-VLM/vision/model.py: 2D-axial RoPE on the patch grid, RMSNorm, QK-Norm, SwiGLU, HRM refinement. From model_v2.py (SpikeWhale LM): MLA attention (q/o LoRA rank 128, MQA), hyper-connections. Added: adaLN-Zero timestep modulation (replacing weak additive timestep conditioning) and a REPA head (Yu et al. 2024) aligned to frozen Byrne-VE — the family's DINOv2 distillation, which made the family's JEPA trait load-bearing. Deliberately not ported: Jet-Long (64 patches — no long context), MTP (no sequential axis), Engram (no n-grams).

The journey (all runs)

#	run	data	steps	outcome
1	open-domain 37M	20K Midjourney imgs / 4,969 prompts	37.5K	texture only, no objects
2	+steps, +data	100K imgs / 24,637 prompts	→79.5K	better texture; unseen eps-loss 0.156→0.141
3	real captions	same imgs, gemini→qwen3→llava recaption (99,981 unique)	→129K	colour/tone prompt response; 0.136 unseen; still no objects
4	portraits 37M	20K YuNet-detected portraits	74K (fresh)	portrait composition, no faces
5	portraits 162M	50K portraits, 128-tok captions	186K (fresh)	face-like masses, smeared features
6a	eye alignment	50K aligned (2-point)	→210.8K (fine-tune)	first readable faces; val 0.195→0.163
6b	full-dataset faces	66,139 (source exhausted)	→260.1K	seed consistency 5/12→8-9/12
6c	5-point align + filter	55,030 (eyes+nose+mouth pinned; profiles/tiny dropped)	→294.2K	~10-11/12 readable; mouths/teeth fixed

The single most important finding: training-data geometry beat every compute lever. Eye alignment (a 4-hour data fix) did more than 2.5× data, 4.4× model size, and 100K+ extra steps combined; 5-point alignment then fixed exactly the region (mouths/lower faces) its predecessor left floating.

Datasets

• wanng/midjourney-v5-202304-clean was evaluated first and is unusable: it stores dead Discord CDN URLs (6/6 sampled → HTTP 404), not image bytes.
• Used instead: Photoroom/midjourney-v6-recap (1.23M rows, real bytes). The prompt column is ~half-broken fragment-welds; the gemini/qwen3/llava recaption columns are coherent per-image captions (cascade: 75.5% gemini / 3% qwen3 / 21% llava / 0.14% fallback).
• Faces were selected by YuNet detection on pixels, not caption keywords (a keyword filter's "portraits" were half food close-ups). The source is fully mined: 66,139 faces total, 55,030 after 5-point quality filtering.

Six silent bugs found en route

Each looked healthy in logs: (1) KL weight calibrated for sum-reduction applied to mean-reduction → NaN; (2) posterior collapse at kl=1e-6 (latent SNR 0.41; fixed 1e-8 → SNR 207+, guarded by an SNR gate); (3) strided DDPM sampling is invalid → replaced with DDIM; (4) a val split keyed to caption strings silently changed meaning when captions went per-image — only 9/2024 images were held out by both runs (fixed with a row-identity test set); (5) caption-keyword face filter selected food; (6) judging checkpoints on 30-step undersampled previews nearly killed the best intervention.

Usage

from inference import BEPipeline
pipe = BEPipeline("model.pt")          # cfg + latent_scale live in the ckpt
img = pipe.generate(
    ["A photograph, close-up portrait of an old man with a white beard"],
    guidance=2.5, steps=150, seed=5)[0]          # best settings found
img.save("out.png")

Text encoder (Quazim0t0/Escarda-86M-Base-JL) downloads at first use; trust_remote_code=True required. Best settings: guidance 2.5, DDIM eta 0, 100–150 steps, EMA weights (default). Guidance >4 degrades; coarse attributes are reliable, fine ones (glasses, expression) are not.

Honest limitations

Painterly, not photorealistic; residual eye asymmetry on weaker seeds; 256×256 only. The Midjourney face supply is exhausted — further gains need a purpose-built face dataset (e.g. FFHQ) or a larger model. Trained end-to-end on a single RTX 3060 (12GB).

Citation

If you use this model, please cite:

@misc{escarda86mbase,
  title        = {BE-ImgGen-Portrait-187M: A ~187M Portrait Image Gen},
  author       = {Dean Byrne (Quazim0t0)},
  year         = {2026},
  howpublished = {HuggingFace, \url{https://huggingface.co/Quazim0t0/BE-ImgGen-Portrait-187M}},
  note         = {Quazim0t0/BE-ImgGen-Portrait-187M}
}