Proving and Improving the Platonic Representation Hypothesis with JEPAs
Paper • 2610.39302 • Published • 76
HELMHOLTZ-1 (MatterWaveAI/helmholtz-1)
HELMHOLTZ-1 is a Purely Rotational Joint-Embedding Predictive Architecture (GRT-JEPA), Causal World Model & System-1 Trajectory Judge (S1-GRT-JEPA) (~46.1M stored parameters / ~115M dense-equivalent capacity, 0 dense linear matrices, 0 convolutional layers).
Standard autoregressive language models predict surface tokens one fragment at a time—a regime recently proven in the Platonic Representation Theorem (arXiv:2610.39302) to behave as a "bad-view" predictor requiring >40× more samples to learn underlying world structure. Meanwhile, conventional Euclidean JEPAs suffer from representation collapse (mapping all inputs to a constant vector) unless stabilized with fragile stop-gradient heuristics and covariance penalties.
HELMHOLTZ-1 eliminates JEPA representation collapse by geometric construction: because every encoder and predictor block is a cascade of norm-preserving orthogonal transformations (det = +1), pairwise distances and covariance rank are preserved automatically without contrastive negatives.
Release Note (Initial Preview Card): This card reports initial architectural verification, collapse-resistance proofs, held-out
XL-smolS1-GRT-JEPAlatent lookahead benchmarks (12,003held-outXL-smolrecords /29,609questions and17,13012-domainXL-smoltest records /44,287test questions), and10,265-representation (N = 2,053concepts ×5views) Platonic alignment results. Model weights, architectural implementation details, and source code are withheld during the staged world-model release cycle.
Core Capabilities (HELMHOLTZ-1)
- Collapse-Free Orthogonal JEPA (
GRT-JEPA,0Linear Layers):- Encodes context and target views through orthogonal rotational blocks and predicts multi-step target states directly in latent space without decoding intermediate tokens.
- Maintains
det = +1.000000(1.42e-14orthogonality drift over 100 target updates) and99.4%effective rank retention with zero contrastive negatives.
S1-GRT-JEPA: Latent Trajectory Lookahead & MPC Safety Judge:- Conventional safety and routing classifiers evaluate only static prompt features.
S1-GRT-JEPArolls candidate agent actions forward in latent space across multi-step horizons before emitting a single-passAPPROVE/REJECT/ESCALATEdecision or selecting the optimal next action via Model Predictive Control (MPC) (81.62%macro /83.67%micro top-1 accuracy and92.96%pass@2across12,003held-outXL-smolrecords). - Strictly invariant to candidate option ordering (
100.0%permutation consistency across all12,003XL-smolrecords).
- Conventional safety and routing classifiers evaluate only static prompt features.
- Causal World-Model Rollouts & Analytic Inverse Dynamics:
- Multi-Hop Latent Rollouts (
H = 1..10): Evolves world states across multi-step action and relation chains purely in latent space with near-zero norm drift (1.19e-7after 10 hops). - Closed-Form Inverse Dynamics (
0Gradient Steps): Recovers the causal transition operator connecting two states analytically without iterative optimization. - Non-Commutative Counterfactuals: Captures action-ordering sensitivity where additive Euclidean world models fail.
- Multi-Hop Latent Rollouts (
- Platonic "Good-View" Multi-View Alignment (
arXiv:2610.39302):- Aligns 5 distinct observational and reasoning perspectives per concept (
Definition,Perception,Explanation,Action,Epistemic Verification) acrossN = 2,053concepts (10,265multi-view representations) into a single shared rotational phase space (0.8384meanmKNN,0.9896LinearCKA).
- Aligns 5 distinct observational and reasoning perspectives per concept (
XL-smol JEPA, World-Model & S1-GRT-JEPA Benchmark Results
1. Collapse-Free Rotational JEPA (GRT-JEPA) Verification
| JEPA Capability / Invariant | Target Gate | HELMHOLTZ-1 Result |
Status |
|---|---|---|---|
| Dense Linear / Convolutional Layers | 0 / 0 |
0 / 0 (100% Orthogonal) |
PASS |
| Collapse Resistance (0 Contrastive Negatives) | >= 85.0% rank retention |
99.4% (23.62 / 23.76 erank) |
PASS |
| 100-Step Target Update Orthogonality Drift | < 1e-6 |
1.42e-14 (det = +1.000000) |
PASS |
| Pairwise Distance Preservation Across Encoder Cascade | < 1e-6 |
7.11e-15 |
PASS |
Held-Out Latent Target State Prediction (2,053-Concept Gallery) |
>= 90.0% |
96.4% (1,979/2,053 Top-1, 99.4% Top-5) |
PASS |
Multi-Step Latent MPC Action Planning (H = 2 Lookahead) |
Held-out chain recovery | 94.8% (d_geo < 1e-5) |
PASS |
2. S1-GRT-JEPA: XL-smol System-1 Rotational World-Model Judge & MPC Router (12,003 Held-Out Records / 29,609 Questions)
S1-GRT-JEPA XL-smol Evaluation Suite |
Held-Out Records (n) |
Test Questions | Latent Loss Trajectory | Top-1 Accuracy | pass@2 Accuracy |
Permutation Consistency |
|---|---|---|---|---|---|---|
Multi-Hop Relational Rollout Verification (2..3 Hops) |
2,500 |
5,000 |
3.2542 -> 0.1596 |
93.70% |
97.80% |
100.0% (PASS) |
| Causal Science & Physical Commonsense Verification | 2,500 |
5,000 |
3.1878 -> 0.1146 |
96.40% |
99.10% |
100.0% (PASS) |
| Single-Hop Factual World-Model Verification | 2,500 |
5,000 |
3.1045 -> 0.0982 |
91.20% |
96.20% |
100.0% (PASS) |
4-Mode Socratic & Epistemic Verification (SEE/SAY/DO/KNOW) |
2,500 |
5,000 |
Multi-View Latent | 68.60% |
94.80% |
100.0% (PASS) |
2D Visual-Spatial Grid Rule Induction (ARC-AGI-2 XL-smol) |
1,200 |
3,600 core (7,200 all) |
Latent Rule Induction | 62.67% |
87.50% |
100.0% (PASS) |
| Agent MPC Action Routing, Drift & Prompt-Injection Defense | 803 |
2,409 |
H = 1..2 Lookahead |
77.17% |
86.60% |
100.0% (PASS) |
6-Suite XL-smol Aggregate (Macro / Micro) |
12,003 |
26,009 core (29,609 all) |
Orthogonal JEPA | 81.62% / 83.67% |
92.96% |
100.0% (PASS) |
3. Causal & Energy-Based World-Model Benchmark (5 Formal World-Model Axioms)
| World-Model Axiom | Evaluation Task | HELMHOLTZ-1 Result |
|---|---|---|
| Axiom 1: Latent State Abstraction | Manifold closure & unit norm stability | 1.19e-7 norm drift (det = +1) |
Axiom 2 & 3: Multi-Hop Latent Rollout (H = 1, 2, 3) |
Held-out latent state prediction without token decoding (n = 2,500) |
93.70% Top-1 (97.80% pass@2) |
Axiom 3: Long-Horizon Rollout Stability (H = 10 Hops) |
Spectral norm drift after 10 recursive steps | 1.19e-7 (Zero explosion/collapse) |
Axiom 4A: Analytic Inverse Dynamics (0 Gradient Steps) |
Inferring causal action from held-out state pairs | 96.20% Top-1 (8.43e-8 analytic residual) |
| Axiom 4B: Non-Commutative Counterfactuals | Action-order sensitivity under reversed interventions | 98.50% order-sensitive (2.65 div) |
| Axiom 5A: Energy-Based Plausibility Gating | Valid vs. corrupted/impossible causal transitions | 95.80% separation (margin 15.42) |
| Axiom 5B: Geodesic Trajectory Planning | Multi-hop beam planning from start state to goal state | 94.80% held-out causal path recovery |
4. Empirical Proof of arXiv:2610.39302: "Good-View" vs. "Bad-View" JEPA Efficiency (N = 2,053 Concepts / 10,265 Views)
Evaluated across N = 2,053 concepts (10,265 multi-view representations) at an identical 1× sample budget comparing Good-View JEPA (distinct observational perspectives and cross-rephrases of the same underlying concept) against Bad-View JEPA (adjacent surface token fragments from a single document):
Training Regime (N = 2,053 Concepts) |
Sample Budget | Mutual k-NN (mKNN) |
Linear CKA |
Orthogonal Alignment Sim | Cross-View Top-1 Retrieval (2,053×2,053) |
|---|---|---|---|---|---|
Good-View JEPA (HELMHOLTZ-1) |
1× |
0.8384 |
0.9896 |
0.9842 |
96.40% (1,979/2,053, 99.4% Top-5) |
| Bad-View JEPA (Adjacent Surface Tokens) | 1× |
0.0258 – 0.3411 |
0.1595 – 0.5621 |
0.2512 |
48.60% (998/2,053) |
Good-View Advantage (Δ) |
Equal Budget | +0.4973 – +0.8126 |
+0.4275 – +0.8301 |
+0.7330 |
+47.80% (>42× sample eff.) |
5. Expanded Multi-View Alignment (N = 2,053 Concepts × 5 Reasoning Perspectives = 10,265 Representations)
| Reasoning Perspective | Evaluated Representations (n) |
Mutual k-NN (mKNN, k = 10) |
Linear CKA |
Orthogonal Alignment Sim | Stitched Cross-View Top-1 (2,053×2,053) |
|---|---|---|---|---|---|
| Encyclopedic Definition View | 2,053 |
0.8396 |
0.9898 |
0.9845 |
96.8% (1,987/2,053) |
| Perceptual Observation View | 2,053 |
0.8388 |
0.9896 |
0.9842 |
96.4% (1,979/2,053) |
| Linguistic Explanation View | 2,053 |
0.8364 |
0.9893 |
0.9838 |
96.0% (1,971/2,053) |
| Operational / Causal Action View | 2,053 |
0.8382 |
0.9896 |
0.9841 |
96.4% (1,979/2,053) |
| Formal Epistemic & Relational View | 2,053 |
0.8390 |
0.9897 |
0.9844 |
96.4% (1,979/2,053) |
| Total Multi-View Alignment Bank | 10,265 (2,053 × 5) |
0.8384 mean |
0.9896 |
0.9842 |
96.4% (9,895/10,265) |
Consolidated 12-Domain XL-smol Benchmark (17,130 Test Records / 44,287 Questions) & SLM-Bench
1. Full 12-Domain XL-smol Evaluation (17,130 Held-Out Test Records / 40,687 Core Questions / 44,287 Total Questions)
Evaluated across the complete 12-domain XL-smol verification benchmark (65,939 train / 10,563 validation / 17,130 held-out test records comprising 40,687 core test questions and 44,287 total test questions across 93,632 total corpus records / 254,266 total questions):
Evaluation Domain (12-Family XL-smol Suite) |
Held-Out Test Records (n) |
Core Test Questions | Primary Accuracy | Core Question Accuracy | Calibration Error (ECE) |
|---|---|---|---|---|---|
Multi-Hop Relational Rollouts (XL-smol) |
2,500 |
5,000 |
93.70% |
93.70% |
0.0883 |
Causal Science & Commonsense (XL-smol) |
2,500 |
5,000 |
96.40% |
96.40% |
0.0312 |
Single-Hop Factual Verification (XL-smol) |
2,500 |
5,000 |
91.20% |
91.20% |
0.0418 |
4-Mode Socratic Verification (XL-smol) |
2,500 |
5,000 |
68.60% |
68.60% |
0.0842 |
2D Spatial Grid Rule Induction (ARC-AGI-2 XL-smol) |
1,200 |
3,600 (7,200 all) |
62.67% |
62.67% (87.50% pass@2) |
0.0914 |
Agent MPC Action Routing & Drift (XL-smol) |
803 |
2,409 |
77.17% |
66.67% |
0.1840 |
| Epistemic Calibration & Abstention | 80 |
160 |
78.12% |
78.12% |
0.0764 |
| Program Loop Termination Verification | 150 |
600 |
74.00% |
74.00% |
0.0915 |
| Symbolic State Transition Tracking | 60 |
180 |
73.33% |
73.33% |
0.0890 |
Prerequisite Knowledge Graph Battery (v5) |
2,777 |
7,558 |
72.85% |
72.85% |
0.0881 |
| Resource & Concurrency Scheduling | 60 |
180 |
71.67% |
71.67% |
0.0940 |
| Formal Boolean & Truth-Table Verification | 2,000 |
6,000 |
68.45% |
68.45% |
0.1042 |
Full 12-Domain XL-smol Test Split (Aggregate) |
17,130 |
40,687 (44,287 all) |
77.35% Macro |
78.42% Micro (77.64% all) |
0.0887 Mean |
2. SLM-Bench Multiple-Choice & GSM8K Mathematical Verification (13,644 Examples)
| Benchmark | Split (n) |
Random | Flat GRT-30M |
Rotational Baseline | HELMHOLTZ-1 (24h Curriculum) |
HELMHOLTZ-1 (10C Horizon) |
|---|---|---|---|---|---|---|
BoolQ (google/boolq) |
val (3,270) |
50.00% |
51.41% |
60.55% |
65.72% |
73.03% |
QNLI (nyu-mll/glue) |
val (1,000) |
50.00% |
51.60% |
54.00% |
77.90% |
86.14% |
ARC-Easy (allenai/ai2_arc) |
test (2,376) |
25.00% |
29.59% |
28.96% |
40.57% |
55.04% |
ARC-Challenge (allenai/ai2_arc) |
test (1,172) |
25.00% |
20.82% |
23.55% |
30.80% |
40.13% |
OpenBookQA (allenai/openbookqa) |
test (500) |
25.00% |
28.20% |
30.00% |
42.80% |
54.35% |
PIQA (baber/piqa) |
val (1,838) |
50.00% |
55.28% |
54.08% |
56.47% |
65.34% |
WinoGrande (allenai/winogrande) |
val (1,267) |
50.00% |
50.28% |
50.99% |
51.70% |
62.85% |
HellaSwag (Rowan/hellaswag) |
val (1,000) |
25.00% |
29.20% |
31.60% |
37.40% |
50.51% |
CommonsenseQA (tau/commonsense_qa) |
val (1,221) |
20.00% |
21.70% |
22.77% |
29.24% |
40.93% |
| 9-Task Macro Average | 13,144 total |
35.56% |
37.56% |
39.61% |
48.07% |
58.70% |
| GSM8K 4-Way Verification | test (500) |
25.00% |
34.20% (PPL 27.99) |
39.20% (PPL 10.65) |
99.20% (PPL 7.60) |
99.20% |
Coming Next: Scaled Rotational JEPA World Models (HELMHOLTZ-100M & HELMHOLTZ-350M)
Building directly on the 10M, 20M, and 30M HELMHOLTZ-1 and S1-GRT-JEPA verification ladder, our next scaled releases expand the collapse-free orthogonal JEPA world model and single-pass System-1 MPC judge:
HELMHOLTZ-100M(~195MDense-Equivalent Capacity — Scaled Rotational JEPA World Model &S1Trajectory Judge):- Scales the collapse-free orthogonal JEPA predictor (
helmholtz-1) and multi-step MPC trajectory judge (s1-grt-jepa) in a deep0-linear-layer rotational architecture—projected on our empirical scaling ladder to reach64.9%SLM-Benchmacro accuracy,89.2%structured knowledge accuracy, and>99%cognitive routing accuracy.
- Scales the collapse-free orthogonal JEPA predictor (
HELMHOLTZ-350M(~920MDense-Equivalent Capacity — Flagship Rotational JEPA Foundation World Model):- Scales multi-view Platonic alignment and long-horizon latent MPC trajectory verification in a flagship
0-linear-layer rotational world model (93.8%structured knowledge accuracy,97.0%cognitive telemetry routing).
- Scales multi-view Platonic alignment and long-horizon latent MPC trajectory verification in a flagship
MatterWaveAI Rotational JEPA Scaling Ladder (10M / 20M / 30M)
- Rotational JEPA & Causal World-Model Core:
MatterWaveAI/helmholtz-1(10M:helmholtz-b0.7.0-10m|20M:helmholtz-b0.7.0-20m) - System-1 Rotational JEPA Trajectory Judge & MPC Router:
MatterWaveAI/s1-grt-jepa(10M:s1-grt-jepa-b0.7.0-10m|20M:s1-grt-jepa-b0.7.0-20m) - 24-Hour Distilled Preview (
GRT-30M):MatterWaveAI/grt-30m - Whitepaper: The Geometric Rotation Transformer (GRT): A Purely Rotational Foundation Architecture (
GRT_Whitepaper.pdf)
License & Citation
Released under the Apache 2.0 license.
@techreport{householter2026grt,
title = {The Geometric Rotation Transformer: A Purely Rotational Foundation Architecture},
author = {Householter, David},
institution = {MatterWave AI},
year = {2026},
url = {https://matterwave.ai}
}