Instructions to use MatterWaveAI/grt-30m-curriculum with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use MatterWaveAI/grt-30m-curriculum with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="MatterWaveAI/grt-30m-curriculum")

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained("MatterWaveAI/grt-30m-curriculum", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use MatterWaveAI/grt-30m-curriculum with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "MatterWaveAI/grt-30m-curriculum"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "MatterWaveAI/grt-30m-curriculum",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/MatterWaveAI/grt-30m-curriculum

SGLang

How to use MatterWaveAI/grt-30m-curriculum with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "MatterWaveAI/grt-30m-curriculum" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "MatterWaveAI/grt-30m-curriculum",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "MatterWaveAI/grt-30m-curriculum" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "MatterWaveAI/grt-30m-curriculum",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use MatterWaveAI/grt-30m-curriculum with Docker Model Runner:
```
docker model run hf.co/MatterWaveAI/grt-30m-curriculum
```

GRT-30M-Curriculum: Multi-Round Symplectic Curriculum (`MatterWaveAI/grt-30m-curriculum`)

Multi-Round Symplectic Curriculum Release: Starting from the exact same 29.3M-parameter (9.74M non-embedding backbone across 12 rotational blocks) checkpoint as MatterWaveAI/grt-30m, MatterWaveAI/grt-30m-curriculum applies multi-round curriculum optimization with HuygensSymplecticAdam. Across the 9-task SLM-Bench multiple-choice suite (13,144 held-out examples), macro-average accuracy rises from 39.61% (PairAdam NeurIPS Submission) → 43.87% (Single-Pass HuygensSymplecticAdam) → 47.16% (Multi-Round Curriculum, +7.55%), while GSM8K 4-way numerical verification reaches 99.20% (PPL = 7.60, down 3.68× from 27.99).

Model Details

Property	Value
Architecture	`GRTForCausalLM` (`model_type: "grt"`, `grt30m_kd`)
Training paradigm	Knowledge distillation from Qwen 3
Total stored parameters	`29,305,498` (`29.31M`)
Non-embedding backbone parameters	`9,740,197` (`9.74M` across 12 blocks)
Dense-equivalent parameters	`55,680,446` (`55.68M`)
`hidden_size` (`d_model`)	`256`
`num_hidden_layers` (`n_blocks`)	`12`
`num_attention_heads` (`n_heads`)	`4` (`head_dim = 64`)
`intermediate_size` (`d_ff`)	`1,024`
`n_qkv_rotations` / `n_mixer_rotations` / `ffn_n_rotations`	`11` / `1` / `21`
Vocabulary	`151,669` (Qwen 3 BPE, tied polar cosine head)
Context length (`max_position_embeddings`)	`2,048`
Weight files	`model.safetensors` (`116 MB`), `grt-30m-f32.gguf` (`340 MB`)

Because grt-30m shares the full 151,669-token Qwen 3 vocabulary at hidden_size = 256 for direct token-aligned distillation from Qwen 3, the tied polar embedding table (phase_angles [151669, 128] + log_magnitude [151669]) accounts for 19.57M parameters (66.8% of stored weights), leaving 9.74M stored parameters for the 12-layer rotational transformer backbone.

24-Hour Qwen 3 Distillation Summary

Metric	Value
Teacher family / Tokenizer	Qwen 3 (`151,669`-token Qwen 3 BPE vocabulary)
Student config	`grt30m_kd` (`d_model = 256`, `12` rotational blocks, `9.74M` non-embedding params)
Training corpus mix	`fineweb-edu` (`45%`), `cosmopedia-v2` (`20%`), `wikipedia` (`15%`), `finemath` (`12%`), `stack-edu` (`8%`)
Hardware	`8x NVIDIA A100-SXM4-80GB` (DDP + Fused CUDA Givens Kernels)
Wall-clock duration	`24.0 hours` (`86,969s`)
Tokens trained	`10,789,404,672` (`10.79B` tokens, `20,580` steps)
Global batch size	`524,288` tokens/step (`micro_batch = 32`, `grad_accum = 1`, `seq_len = 2048`, `8` GPUs)
Sustained throughput	`123,900 tok/s` (`~15,490 tok/s/GPU`, zero NaNs or gradient spikes)
Optimizer	`PairAdam` (factored second moments for 2D coupling matrices + angular step clamping)
Learning rate	Peak `1e-3`, WSD schedule (`checkpoint-20000` saved at `20,000 / 57,220` steps)
Final validation loss / PPL (FineWeb-Edu)	`val_loss = 4.4776` / `val_ppl = 88.02` (`top-1 token acc = 27.20%`)

4-Backend Exact Greedy Generation Parity

model.safetensors (transformers) and grt-30m-f32.gguf (llama.cpp) produce identical token-for-token greedy continuations (temperature = 0, 16 new tokens) across all four supported backends on AMD Strix Halo (gfx1151):

PyTorch ROCm (gfx1151, fused HIP kernels)
llama.cpp CPU (x86_64 AVX2)
llama.cpp Vulkan (RADV gfx1151)
llama.cpp ROCm (HIP gfx1151)

Prompt	Greedy continuation (`16` tokens, identical on all 4 backends)
`The capital of France is`	`The capital of France is the first to be the first to be the first to be the first to be`
`Once upon a time`	`Once upon a time, you might be wondering what exactly is the way to do with the right point`
`In the beginning`	`In the beginning of the world, the world is the most common form of the world. The`

Benchmarks: NeurIPS Submission (`PairAdam`) → Single-Pass `HuygensSymplecticAdam` → Multi-Round Curriculum (`grt-30m-curriculum`)

Evaluated across the 10-task SLM-Bench small-language-model suite (training/slm_bench.py / grt-slm-bench) starting from the exact same 29.3M-parameter (9.74M non-embedding) checkpoint:

1. Multiple-Choice Accuracy (`9` Core `SLM-Bench` Tasks, `13,144` Held-Out Examples)

Task (`SLM-Bench`)	Split (`n`)	Random	Zero-Shot	NeurIPS Baseline (`PairAdam`)	Single-Pass (`HuygensSymplectic`)	Multi-Round Curriculum	Total Gain (Curriculum vs. `PairAdam`)
BoolQ (`google/boolq`)	`val` (`3,270`)	`50.00%`	`51.41%`	`60.55%`	`62.23%`	`64.43%`	`+3.88%` (`+2.20%` vs. Single-Pass)
ARC-Easy (`allenai/ai2_arc`)	`test` (`2,376`)	`25.00%`	`29.59%`	`28.96%`	`36.32%`	`40.19%`	`+11.23%` (`+3.87%` vs. Single-Pass)
ARC-Challenge (`allenai/ai2_arc`)	`test` (`1,172`)	`25.00%`	`20.82%`	`23.55%`	`25.68%`	`29.69%`	`+6.14%` (`+4.01%` vs. Single-Pass)
OpenBookQA (`allenai/openbookqa`)	`test` (`500`)	`25.00%`	`28.20%`	`30.00%`	`35.80%`	`42.40%`	`+12.40%` (`+6.60%` vs. Single-Pass)
PIQA (`baber/piqa`)	`val` (`1,838`)	`50.00%`	`55.28%`	`54.08%`	`55.28%`	`55.66%`	`+1.58%` (`+0.38%` vs. Single-Pass)
WinoGrande (`allenai/winogrande`)	`val` (`1,267`)	`50.00%`	`50.28%`	`50.99%`	`51.22%`	`51.22%`	`+0.23%` (`+0.00%` vs. Single-Pass)
HellaSwag (`Rowan/hellaswag`)	`val` (`1,000`)	`25.00%`	`29.20%`	`31.60%`	`31.60%`	`36.70%`	`+5.10%` (`+5.10%` vs. Single-Pass)
QNLI (`nyu-mll/glue`)	`val` (`1,000`)	`50.00%`	`51.60%`	`54.00%`	`72.00%`	`75.90%`	`+21.90%` (`+3.90%` vs. Single-Pass)
CommonsenseQA (`tau/commonsense_qa`)	`val` (`1,221`)	`20.00%`	`21.70%`	`22.77%`	`24.65%`	`28.26%`	`+5.49%` (`+3.61%` vs. Single-Pass)
9-Task Multiple-Choice Macro Average	`13,144` total	`35.56%`	`37.56%`	`39.61%`	`43.87%`	`47.16%`	`+7.55%` (`+3.30%` vs. Single-Pass)

2. Mathematical Reasoning Held-Out Perplexity & Verification (`GSM8K`, `test` split, `n = 500`)

Metric (`SLM-Bench`)	Zero-Shot	NeurIPS Baseline (`PairAdam`)	Single-Pass (`HuygensSymplectic`)	Multi-Round Curriculum	Total Improvement (Curriculum vs. Zero-Shot / `PairAdam`)
GSM8K Held-Out Perplexity (`PPL` ↓)	`27.99`	`10.65`	`9.04`	`7.60`	`3.68×` lower vs. Zero-Shot (`-28.6%` vs. `PairAdam`)
GSM8K 4-Way Numerical Verification (`Acc` ↑)	`34.20%`	`39.20%`	`98.60%`	`99.20%`	`+60.00%` vs. `PairAdam` (`+0.60%` vs. Single-Pass)

Overview of Multi-Round Curriculum Optimization

While single-pass task adaptation with HuygensSymplecticAdam already improves substantially over first-generation PairAdam, a compact 9.74M-parameter rotational backbone (12 Givens butterfly blocks on SO(D)) benefits further from multi-round curriculum training:

Norm-Preserving Rotational Geometry (SO(D)): Unlike standard dense transformers (nn.Linear) whose unconstrained weight matrices overfit small-scale benchmarks within 1–2 epochs, GRT-30M parameterizes its projections as cascades of norm-preserving 2×2 Givens rotations. Because orthogonal transformations preserve vector norms (||R(theta) x||_2 = ||x||_2), the backbone can be trained across multiple progressive curriculum stages without norm explosion or catastrophic forgetting.
Symplectic & Wavefront-Coupled Angular Updates (HuygensSymplecticAdam): SymplecticAdam and HuygensSymplecticAdam track angular velocity and cross-stage phase coherence across the Givens cascades, allowing deeper rotational blocks to align smoothly over multi-stage schedules where first-order PairAdam stalls.
Progressive Multi-Round Curriculum: Starting from the base pretrained checkpoint, the model is adapted across successive curriculum stages with cosine learning-rate scheduling, steadily refining angular phase alignment across all 12 blocks before consolidating into the published model.safetensors checkpoint.

Usage

PyTorch (`transformers`)

import torch
from transformers import AutoTokenizer, GRTForCausalLM

# Optional: register fused CUDA/ROCm Givens kernels if grt-kernels is installed
try:
    from grt_kernels.torch_ops import ensure_grt_ops
    ensure_grt_ops()
except ImportError:
    pass

model_id = "MatterWaveAI/grt-30m-curriculum"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = GRTForCausalLM.from_pretrained(model_id, dtype=torch.float32).eval()

prompt = "The history of the city"
input_ids = tokenizer(prompt, return_tensors="pt").input_ids
with torch.no_grad():
    output = model.generate(input_ids, max_new_tokens=32, do_sample=False)
print(tokenizer.decode(output[0], skip_special_tokens=True))

`llama.cpp` (`GGUF`)

./build/bin/llama-completion \
  -m grt-30m-f32.gguf \
  -p "Once upon a time" \
  -n 32 --temp 0 --seed 0

Verify file integrity against SHA256SUMS:

sha256sum --ignore-missing -c SHA256SUMS

Architecture Overview

GRT replaces dense weight matrices (nn.Linear) in both attention and FFN blocks with cascades of learned 2×2 Givens rotations arranged in a butterfly topology. At each stage k, pairs of coordinates (x0, x1) are rotated by an angle theta_k modulated by 4-component Cl(2,0) multivector features:

feat(x0, x1) = [x0^2 + x1^2,  x0,  x1,  x0 * x1_next - x1 * x0_next]
theta_k      = alpha_k + tanh(norm(feat(x0, x1)) @ C_k)
r0           = x0 * cos(theta_k) - x1 * sin(theta_k)
r1           = x0 * sin(theta_k) + x1 * cos(theta_k)

GRTEmbedding & GRTCosineHead — Token embeddings are stored in polar form (phase_angles + log_magnitude) and tied to a scaled cosine-similarity language modeling head (lm_head.output_scale = 40.23).
GRTAttention — 11-stage content-modulated Givens butterfly cascades for Q, K, V, O, paired with standard scaled dot-product attention and learnable RoPE frequencies.
GRTMixer & GRTFFN — Cross-head Givens mixing stage followed by 21-stage rotational FFN expansion/contraction and phase-preserving GRTRadialGELU.
GRTPearlNorm — Pair-aware RMS normalization that normalizes pair radii sqrt(x_{2i}^2 + x_{2i+1}^2) while preserving phase angles.

Intended Use & Limitations

Research preview. Intended for studying geometric/rotational neural architectures, kernel benchmarking, and multi-backend (PyTorch / llama.cpp / vLLM) verification.
Base causal LM only. Not instruction-tuned, no chat template, no safety filtering, and prone to repetition under greedy decoding at small parameter scale.

Complete `MatterWaveAI/grt-30m*` Research Suite on Hugging Face

Base Causal LM: MatterWaveAI/grt-30m & MatterWaveAI/grt-30m-curriculum
Chat Fine-Tune: MatterWaveAI/grt-30m-chat
4-Expert Mixture-of-Rotations (MoR): MatterWaveAI/grt-30m-mor
Circular-Pooled Sentence Embedding: MatterWaveAI/grt-30m-embedding
Cross-Encoder Reranker: MatterWaveAI/grt-30m-reranker
Vision-Language (GRTVL): MatterWaveAI/grt-30m-vl
Canonical Tokenizer & Polar Embedding Bank: MatterWaveAI/grt-30m-tokenizer
Rotational Text-to-Speech (GRTTTS): MatterWaveAI/grt-30m-tts
Multimodal Vision-Audio-Language-Speech (GRTOmni): MatterWaveAI/grt-30m-omni

Whitepaper: The Geometric Rotation Transformer (GRT): A Purely Rotational Foundation Architecture (GRT_Whitepaper.pdf)

License

Apache 2.0

Citation

@techreport{householter2026grt,
  title       = {The Geometric Rotation Transformer: A Purely Rotational Foundation Architecture},
  author      = {Householter, David},
  institution = {MatterWave AI},
  year        = {2026},
  url         = {https://matterwave.ai}
}

Downloads last month: -

Safetensors

Model size

29.8M params

Tensor type

I64

F32

MatterWaveAI
/

grt-30m-curriculum

GRT-30M-Curriculum: Multi-Round Symplectic Curriculum (`MatterWaveAI/grt-30m-curriculum`)

Model Details

24-Hour Qwen 3 Distillation Summary

4-Backend Exact Greedy Generation Parity

Benchmarks: NeurIPS Submission (`PairAdam`) → Single-Pass `HuygensSymplecticAdam` → Multi-Round Curriculum (`grt-30m-curriculum`)

1. Multiple-Choice Accuracy (`9` Core `SLM-Bench` Tasks, `13,144` Held-Out Examples)

2. Mathematical Reasoning Held-Out Perplexity & Verification (`GSM8K`, `test` split, `n = 500`)

Overview of Multi-Round Curriculum Optimization

Usage

PyTorch (`transformers`)

`llama.cpp` (`GGUF`)

Architecture Overview

Intended Use & Limitations

Complete `MatterWaveAI/grt-30m*` Research Suite on Hugging Face

License

Citation

Datasets used to train MatterWaveAI/grt-30m-curriculum

GRT-30M-Curriculum: Multi-Round Symplectic Curriculum (MatterWaveAI/grt-30m-curriculum)

Model Details

24-Hour Qwen 3 Distillation Summary

4-Backend Exact Greedy Generation Parity

Benchmarks: NeurIPS Submission (PairAdam) → Single-Pass HuygensSymplecticAdam → Multi-Round Curriculum (grt-30m-curriculum)

1. Multiple-Choice Accuracy (9 Core SLM-Bench Tasks, 13,144 Held-Out Examples)

2. Mathematical Reasoning Held-Out Perplexity & Verification (GSM8K, test split, n = 500)

Overview of Multi-Round Curriculum Optimization

Usage

PyTorch (transformers)

llama.cpp (GGUF)

Architecture Overview

Intended Use & Limitations

Complete MatterWaveAI/grt-30m* Research Suite on Hugging Face

License

Citation

Datasets used to train MatterWaveAI/grt-30m-curriculum

GRT-30M-Curriculum: Multi-Round Symplectic Curriculum (`MatterWaveAI/grt-30m-curriculum`)

Benchmarks: NeurIPS Submission (`PairAdam`) → Single-Pass `HuygensSymplecticAdam` → Multi-Round Curriculum (`grt-30m-curriculum`)

1. Multiple-Choice Accuracy (`9` Core `SLM-Bench` Tasks, `13,144` Held-Out Examples)

2. Mathematical Reasoning Held-Out Perplexity & Verification (`GSM8K`, `test` split, `n = 500`)

PyTorch (`transformers`)

`llama.cpp` (`GGUF`)

Complete `MatterWaveAI/grt-30m*` Research Suite on Hugging Face