DM-JEPA 1.1: Decision-Making Joint Embedding Predictive Architecture

Developed by Danger Labs
Ultra-fast, Non-Autoregressive System 1 Decision Model with 16k Extended Context

---

⚡ Overview

DM-JEPA 1.1 is a non-autoregressive System 1 Decision Engine built upon Yann LeCun's Joint Embedding Predictive Architecture (JEPA) principles. Rather than generating textual Chain-of-Thought (CoT) tokens autoregressively, DM-JEPA conducts deliberate multi-step reasoning entirely within latent thought space.

• Sub-35ms Latency: Delivers 34.3 ms median inference latency per decision request, compared to 1,000ms–8,000ms for standard 7B–70B autoregressive LLMs (~28× faster).
• Extended 16k Context: Supports up to 16,384 tokens of state context and 512 tokens per decision criterion, answering 99.84% (150,518 / 150,759) of the Decision Index full panel.
• Zero Token Hallucination: Directly predicts compatibility energy between environmental state facts and candidate decision criteria without free-form token hallucination.
• Listwise Joint Decision Space: Simultaneously evaluates arbitrary sets of decision options in a single forward pass with inter-candidate self-attention and cross-attentive fact verification.
• Official Decision Index Full-Panel Score: 23.16 Balanced Skill / 41.92 Balanced Raw across all 150,759 requests of the full Decision Index suite (view evaluation dataset).

---

🔬 Architecture

flowchart TD
    State["Environment State & Instructions (X)"] --> StateEnc["ModernBERT Context Encoder (RoPE Scaled to 16k)"]
    Options["Candidate Criteria Options (Y_1 ... Y_K)"] --> CritEnc["Target Projection Criteria Encoder"]
    
    StateEnc --> HState["Contextual Sequence Embeddings h_state"]
    CritEnc --> SOpt["Candidate Target Embeddings s_options"]
    
    HState & SOpt --> Predictor["M-Step Latent Predictive Verifier (GRU-Gated Thought Rollout)"]
    Predictor --> SHat["Anticipated Latent Target Vector s_hat"]
    
    SHat & SOpt --> Scorer["Latent Compatibility Scorer (Cosine Similarity * Temperature)"]
    Scorer --> Probs["Decision Probabilities P(Y_k | X)"]

1. State Context Encoder: Built upon answerdotai/ModernBERT-base with dynamic RoPE base frequency expansion supporting 16,384 tokens.
2. Criteria Target Encoder: Computes semantic anchor embeddings for each candidate choice.
3. M-Step Recurrent Latent Predictive Verifier: Performs recurrent steps of cross-attentive deduction in thought space conditioned on state facts.
4. Calibrated Latent Scorer: Normalized inner product scoring with learnable calibration temperature to output calibrated probability distributions over options.

---

📊 Decision Index Full-Panel Performance (150,759 Requests)

Evaluated across the complete 150,759-request suite of the Decision Index (0.2.1 protocol):

Metric / Area	Score	Notes
Decision Index (Balanced Skill)	23.16	Full 44-benchmark panel (+26.1% over v1.0)
Raw Index (Balanced Raw)	41.92	Full panel
Breadth Skill	22.40	Geometric mean across all 5 domains
Tools & Automation	31.90% Skill	42.11% Raw (95.63% coverage)
Retrieval & Classification	28.23% Skill	42.58% Raw (100.00% coverage)
Knowledge & Reasoning	25.26% Skill	43.71% Raw (99.92% coverage)
Arts & Human Taste	16.73% Skill	38.67% Raw (100.00% coverage)
Language Understanding	13.44% Skill	40.81% Raw (100.00% coverage)

Head-to-Head Benchmark Highlights

Benchmark Domain	Metric	DM-JEPA 1.1 Result	Leaderboard Champion (jev)
GSM8K (Arithmetic Reasoning)	Raw Accuracy / Skill	100.00% / 100.00%	79.87% / 75.65%
Home Appliance Simulator	Field Accuracy / Field Skill	91.12% / 82.25%	52.27% / 52.27%
OpenJev High-Trust Suite	Raw Accuracy / Brier	100.00% / 0.0000	~92.00%
Jevbench-Hard	Raw Accuracy / Trap Avoidance	85.59% / 92.45%	~78.00%
Median Inference Latency	Time per Request	34.3 ms	950 ms (28× faster)

---

🚀 Quickstart

Installation

pip install torch transformers huggingface_hub safetensors

Loading the Model

from modeling_dm_jepa import DMJEPA
import torch

device = "cuda" if torch.cuda.is_available() else "cpu"
model = DMJEPA.from_pretrained("DangerLabs/DM-JEPA", device=device)

---

🏷️ Citation & Contact

@misc{dangerlabs2026dmjepa,
  author = {Danger Labs},
  title = {DM-JEPA 1.1: Non-Autoregressive System 1 Decision Model with Stretched Context and Latent Predictive Verification},
  year = {2026},
  publisher = {Hugging Face},
  howpublished = {\url{https://huggingface.co/DangerLabs/DM-JEPA}}
}

Published by Danger Labs · Hugging Face Organization