GRT-30M-Image (MatterWaveAI/grt-30m-image)

GRT-30M-Image is a 100% Native Rotational Text-to-Image & Visual Patch Synthesis Language Model (29,371,239 total parameters, zero nn.Linear layers, zero nn.Conv2d layers) built on top of MatterWaveAI/grt-30m. Completing the bidirectional Vision symmetry alongside MatterWaveAI/grt-30m-vl (Image → Text), GRT-30M-Image synthesizes (B, 3, 224, 224) RGB images and discrete 4-codebook visual patch tokens directly from causal text representations through a purely rotational inverse Givens butterfly patch decoder (GRTPolarImageDecoderHead).

Architecture Summary

Component	Module	Parameters	Geometry
Language Backbone	GRTModel + GRTCosineHead	29,305,498	12-layer causal Givens Rotational Transformer (MatterWaveAI/grt-30m, frozen to preserve 100% text fluency)
Polar Image Decoder Head	GRTPolarImageDecoderHead (model.image_head)	65,741	15-stage Givens butterfly expand (d_model=256 → 1024) → GRTRadialGELU → 15-stage butterfly contract (1024 → 4×256 sub-patches) → 11-stage inverse patch cascade (256 → 1024 → 588 = 3×14×14) → spatial fold (B, 3, 224, 224)
Total (GRT-30M-Image)	GRTImageForConditionalGeneration (GRTForTextToImage)	29,371,239	0 nn.Linear, 0 nn.Conv2d
4-Codebook Visual RVQ Bank	GRTVisualRVQCodecBank (benchmark evaluation head)	32,772	4 planar-pair polar cosine codebooks (K = 64 entries × 128 pairs per codebook) over RGB spatial-spectral sub-bands

Visual Synthesis Alignment Metrics (HuygensAdam / Circular Order Parameter $R_p$ Splay Torque)

• Held-Out Visual RVQ Perplexity: 81.17 (unaligned) → 16.98 (aligned, 4.78× reduction)
• Visual RVQ Mean Top-1 Accuracy (4 Codebooks): 1.48% → 25.19% ([20.86%, 25.63%, 26.04%, 28.23%], 16.1× over random chance)
• Text-Conditioned Prompt Margin ($\Delta\mathcal{L}$ vs. Shuffled Prompts): -0.0074 → +0.6440 nats
• Patch Polar Cosine Similarity: 0.1004 → 0.6737
• 2D Spectral Magnitude Cosine Similarity: 0.1782 → 0.7857
• Reconstruction PSNR / Spectral SNR: 11.38 dB / -8.20 dB → 19.14 dB PSNR / +2.31 dB SNR (+10.51 dB gain)

---

8-Way Classical Geometric Optimizer Ablation (Intel Arc 140T XPU)

We benchmarked all 8 classical geometric, symplectic, and gyroscopic optimizers from transformers.models.grt.optimization_grt on MatterWaveAI/grt-30m-image (GRTPolarImageDecoderHead + 4-codebook visual RVQ polar cosine codec bank GRTVisualRVQCodecBank, K = 64 per codebook, 200 steps per optimizer, batch_size = 36, seed = 42, across a 144-concept procedural text-to-image benchmark of 6 colors × 6 geometric shapes × 4 spatial layouts):

Rank	Optimizer (optimizer_type)	Final Train Loss	Held-Out RVQ Val Loss	Held-Out RVQ Val PPL	RVQ Mean Top-1 Acc (4 Codebooks)	Codebook-0 Primary Acc	Text-Conditioned Margin ($\Delta\mathcal{L}$)	Patch Polar Cosine Sim	Spectral SNR / PSNR (dB)	Time (s)
1	HuygensAdam (huygens_adam)	2.8188	2.8317	16.9750	25.19%	20.86%	+0.6440	0.6737	2.31 / 19.14 dB	55.59s
2	PolarNaturalAdam (polar_natural_adam)	2.8244	2.8368	17.0605	25.08%	20.85%	+0.6333	0.6717	2.29 / 19.13 dB	185.71s
3	HessianPairAdam (hessian_pair_adam)	2.8260	2.8382	17.0850	25.04%	20.67%	+0.6341	0.6723	2.30 / 19.13 dB	172.73s
4	PairAdam (pair_adam)	2.8306	2.8417	17.1447	25.04%	20.48%	+0.6356	0.6731	2.31 / 19.14 dB	196.00s
5	CyclideAdam (cyclide_adam)	2.8337	2.8454	17.2093	25.02%	20.87%	+0.6292	0.6701	2.28 / 19.12 dB	85.62s
6	GyricAdam (gyric_adam)	2.8520	2.8651	17.5505	24.64%	20.04%	+0.6298	0.6608	2.21 / 19.04 dB	47.46s
7	HuygensSymplecticAdam (huygens_symplectic_adam)	2.8962	2.9087	18.3322	24.01%	19.65%	+0.6061	0.6508	2.14 / 18.97 dB	55.41s
8	SymplecticAdam (symplectic_adam)	2.8965	2.9089	18.3370	24.01%	19.65%	+0.6056	0.6508	2.14 / 18.97 dB	65.04s

Key Geometric Findings on Rotational Text-to-Image (GRTImage)

• HuygensAdam Sweeps #1 Across Visual RVQ Perplexity, Accuracy, Text-Conditioning Margin & Polar Cosine Similarity: On 2D spatial patch synthesis (256 patches of 14×14×3 unfolded across 41 Givens butterfly stages), Kuramoto–Huygens circular order-parameter splay torque ($R_p = |\frac{1}{P}\sum_p e^{i\theta_p}|$) prevents angular phase collapse across spatial sub-patches, achieving #1 held-out RVQ validation loss (2.8317, val_ppl = 16.9750), #1 4-codebook visual RVQ accuracy (25.19%), #1 text-conditioning margin (+0.6440 nats), and #1 patch polar cosine similarity (0.6737, 19.14 dB PSNR).
• PolarNaturalAdam (#2) & HessianPairAdam (#3) Excel on Multi-Stage Butterfly Curvature: Scaling angular updates by the Riemannian polar metric $1/(r^2 + \epsilon)$ and Cauchy–Riemann toroidal curvature outperforms first-order PairAdam (2.8368 and 2.8382 vs. 2.8417 validation loss).
• CyclideAdam Achieves #1 Primary Codebook-0 Accuracy (20.87%): Dupin cyclide conic metric preconditioning with deviatoric twist attains the highest primary RGB chromatic codebook accuracy (20.87%).

---

Quickstart (transformers)

import torch
from transformers import AutoTokenizer, GRTImageForConditionalGeneration

model_id = "MatterWaveAI/grt-30m-image"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = GRTImageForConditionalGeneration.from_pretrained(model_id, dtype=torch.float32)

prompt = "<|im_start|>user\nGenerate image (color=gold, shape=ring, layout=center): render a gold ring in the center region<|im_end|>\n<|im_start|>assistant\n"
inputs = tokenizer(prompt, return_tensors="pt", add_special_tokens=False)
outputs = model(**inputs)
print("Synthesized RGB image shape:", outputs.pixel_values.shape)  # (1, 3, 224, 224)
print("Synthesized polar patches shape:", outputs.image_patches.shape)   # (1, 256, 588)

---

Complete MatterWaveAI/grt-30m* Research Suite on Hugging Face

1. Base Causal LM: MatterWaveAI/grt-30m
2. Chat Fine-Tune: MatterWaveAI/grt-30m-chat
3. 4-Expert Mixture-of-Rotations (MoR): MatterWaveAI/grt-30m-mor
4. Circular-Pooled Sentence Embedding: MatterWaveAI/grt-30m-embedding
5. Cross-Encoder Reranker: MatterWaveAI/grt-30m-reranker
6. Vision-Language (GRTVL): MatterWaveAI/grt-30m-vl
7. Rotational Text-to-Image (GRTImage): MatterWaveAI/grt-30m-image
8. Canonical Tokenizer & Polar Embedding Bank: MatterWaveAI/grt-30m-tokenizer
9. Rotational Text-to-Speech (GRTTTS): MatterWaveAI/grt-30m-tts
10. Rotational Speech Recognition (GRTASR): MatterWaveAI/grt-30m-asr
11. Multimodal Vision-Audio-Language-Speech-Image (GRTOmni): MatterWaveAI/grt-30m-omni

• Whitepaper: The Geometric Rotation Transformer (GRT): A Purely Rotational Foundation Architecture (GRT_Whitepaper.pdf)

---

License

Apache 2.0

---

Citation

@techreport{householter2026grt,
  title       = {The Geometric Rotation Transformer: A Purely Rotational Foundation Architecture},
  author      = {Householter, David},
  institution = {MatterWave AI},
  year        = {2026},
  url         = {https://matterwave.ai}
}