GRT-30M-Omni (MatterWaveAI/grt-30m-omni)

GRT-30M-Omni is a 100% Native Rotational Vision-Audio-Language-Speech Omni Model (37,596,089 total parameters, zero nn.Linear dense matrices, zero nn.Conv1d layers) uniting image understanding, audio/speech perception, causal language generation, and 16 kHz harmonic speech synthesis in a single shared 256-dim (128 planar pairs) toroidal phase space.

Architecture Summary

Component	Module	Parameters	Geometry
Vision Tower	GRTVisionModel (model.visual)	4,954,258	14x14 polar patch stem + 2D RoPE + 6 bidirectional rotational blocks + 2x2 spatial merger → 64 visual tokens
Audio Tower	GRTAudioModel (model.audio_tower)	3,288,972	Polar STFT stem (n_fft=254, hop=160) + 1D RoPE + 4 bidirectional rotational blocks + 2x temporal merger → 64 audio tokens
Language Backbone	GRTModel + GRTCosineHead	29,305,498	12-layer causal Givens Rotational Transformer (MatterWaveAI/grt-30m, tied polar embeddings & cosine head)
Speech Vocoder Head	GRTPolarVocoderHead (model.speech_head)	47,361	15-stage Givens butterfly expand/RadialGELU/contract cascade (frames_per_token=4) + torch.istft @ 16 kHz
Total (GRT-30M-Omni)	GRTOmniForConditionalGeneration	37,596,089	100% Rotational Multimodal Transformer (0 nn.Linear, 0 nn.Conv1d)

Held-Out Multimodal Evaluation Summary

• Visual Captioning: Perplexity 1.18 | Color 95.8% | Shape 58.3% | Position 95.8%
• Audio Recognition: Perplexity 29312.46 → 1.10 | Voice 100.0% | Prosody 70.8% | Phrase 100.0%
• Joint Audio-Visual QA: Perplexity 884.35 → 1.06 | Exact Match 83.3%
• Spoken Response Vocoder (speech_head): Polar Cosine Similarity 0.9733 | Spectral SNR 10.49 dB

---

8-Way Classical Geometric Optimizer Ablation (Spark ROCm gfx1151)

We benchmarked all 8 classical geometric, symplectic, and gyroscopic optimizers from transformers.models.grt.optimization_grt on MatterWaveAI/grt-30m-omni across a controlled 200-step tri-modal alignment budget (80 Stage-1 GRTAudioModel rotational alignment steps + 70 Stage-2 Vision/Audio/AV multimodal GRTModel top-block alignment steps + 50 Stage-3 GRTPolarVocoderHead + 4-codebook RVQ speech codec steps, seed = 42, on AMD Radeon 8060S Graphics):

Rank	Optimizer (optimizer_type)	Final Train Loss	Held-Out Tri-Modal Val Loss	Combined Val PPL	Multimodal Text Val Loss	Text Token Top-1 Acc	RVQ Speech Val Loss	RVQ Speech Token Top-1 Acc	RVQ CB-0 Acc	Vocoder SNR (dB)	Time (s)
1	HuygensAdam (huygens_adam)	1.2299	1.2674	3.5516	0.1698	95.97%	2.3650	34.05%	33.50%	4.55 dB	526.81s
2	HessianPairAdam (hessian_pair_adam)	1.2588	1.2779	3.5891	0.1672	96.77%	2.3886	35.89%	34.70%	4.20 dB	493.02s
3	PairAdam (pair_adam)	1.2816	1.2920	3.6401	0.1456	96.60%	2.4385	33.47%	31.30%	4.42 dB	518.75s
4	PolarNaturalAdam (polar_natural_adam)	1.2888	1.2945	3.6492	0.1548	96.20%	2.4342	32.62%	32.18%	4.37 dB	370.28s
5	CyclideAdam (cyclide_adam)	1.2745	1.2957	3.6536	0.1460	96.18%	2.4454	34.21%	34.81%	4.35 dB	434.03s
6	GyricAdam (gyric_adam)	1.3115	1.3383	3.8126	0.1784	95.65%	2.4982	29.05%	29.91%	4.36 dB	450.13s
7	SymplecticAdam (symplectic_adam)	1.3448	1.3702	3.9361	0.1886	96.37%	2.5518	31.30%	30.02%	4.00 dB	444.90s
8	HuygensSymplecticAdam (huygens_symplectic_adam)	1.3553	1.3738	3.9503	0.2008	96.32%	2.5468	32.76%	31.44%	4.14 dB	517.53s

Key Geometric Findings on Unified Tri-Modal Alignment (GRTOmni)

• HuygensAdam Wins Combined Tri-Modal Loss (1.2674 / 3.5516 PPL): Across joint Vision + Audio + Text + Speech alignment, HuygensAdam prevents cross-modal phase collapse via its circular order parameter $R_p$ splay restoring torque, achieving #1 combined tri-modal validation loss (1.2674), #1 Visual Captioning loss (0.1574), #1 RVQ speech codec loss (2.3650), and #1 Omni vocoder SNR (4.55 dB).
• HessianPairAdam (Sophia-Torus) Wins Multimodal Text & RVQ Token Accuracies: HessianPairAdam achieves #1 multimodal text token top-1 accuracy (96.77%) and #1 4-codebook RVQ speech token top-1 accuracy (35.89%) (#2 combined tri-modal loss at 1.2779).
• PairAdam and CyclideAdam Lead Joint Audio-Visual QA Loss: On simultaneous image + audio reasoning (joint_audio_visual_qa), CyclideAdam (0.1066 loss, 98.58% token accuracy, #1 Codebook-0 speech accuracy 34.81%) and PairAdam (0.1079 loss, 98.58% token accuracy, #1 overall multimodal text loss 0.1456) lead all optimizers.

---

Quickstart (transformers)

import torch
from transformers import AutoTokenizer, GRTOmniForConditionalGeneration

model_id = "MatterWaveAI/grt-30m-omni"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = GRTOmniForConditionalGeneration.from_pretrained(model_id, dtype=torch.float32)

# Synthesize speech and decode text simultaneously
inputs = tokenizer("<|im_start|>user\nSpeak (alto, declarative): hello matter wave<|im_end|>\n<|im_start|>assistant\n", return_tensors="pt")
wav, spec = model.synthesize_speech(**inputs)
print("Waveform:", wav.shape, "Polar STFT:", spec.shape)

---

Complete MatterWaveAI/grt-30m* Research Suite on Hugging Face

1. Base Causal LM: MatterWaveAI/grt-30m
2. Chat Fine-Tune: MatterWaveAI/grt-30m-chat
3. 4-Expert Mixture-of-Rotations (MoR): MatterWaveAI/grt-30m-mor
4. Circular-Pooled Sentence Embedding: MatterWaveAI/grt-30m-embedding
5. Cross-Encoder Reranker: MatterWaveAI/grt-30m-reranker
6. Vision-Language (GRTVL): MatterWaveAI/grt-30m-vl
7. Canonical Tokenizer & Polar Embedding Bank: MatterWaveAI/grt-30m-tokenizer
8. Rotational Text-to-Speech (GRTTTS): MatterWaveAI/grt-30m-tts
9. Multimodal Vision-Audio-Language-Speech (GRTOmni): MatterWaveAI/grt-30m-omni

• Whitepaper: The Geometric Rotation Transformer (GRT): A Purely Rotational Foundation Architecture (GRT_Whitepaper.pdf)

---

License

Apache 2.0

---

Citation

@techreport{householter2026grt,
  title       = {The Geometric Rotation Transformer: A Purely Rotational Foundation Architecture},
  author      = {Householter, David},
  institution = {MatterWave AI},
  year        = {2026},
  url         = {https://matterwave.ai}
}