Teutonic-I 10B

Teutonic-I 10B is a pretrained causal language model developed through an open, decentralized model-improvement competition on Teutonic, Bittensor Subnet 3. Independent miners repeatedly trained challengers from the current best checkpoint, and a challenger became the new “king” only after demonstrating a statistically robust reduction in next-token prediction loss.

The competition ran for 70 days, from June 2 to August 10, 2026. It completed 2,163 model duels and produced 203 successful coronations. The best checkpoint was king #191, selected on August 1, 2026.

This is a base pretrained model, not an instruction-tuned assistant.

Model details

Property	Value
Model type	Decoder-only causal language model
Architecture	Quasar (QuasarForCausalLM)
Parameters	8,602,037,248 in the checkpoint index; marketed as 10B
Precision	BF16
Layers	32
Hidden size	4,096
Attention heads	16
Key/value heads	4
Intermediate size	12,288
Vocabulary size	248,320
Attention pattern	Three linear-attention layers followed by one full-attention layer
Competition evaluation context	2,048 tokens
Framework	Transformers with repository-provided custom code

The configuration declares a maximum position embedding value of 2,097,152. This should not be interpreted as evidence that the checkpoint was trained or validated at that sequence length; the competition evaluated 2,048-token sequences.

How it was trained

Teutonic did not prescribe a single training script, dataset, training duration, or hardware setup. Each miner independently chose how to improve the current king while keeping the required architecture compatible with the subnet.

Model selection used direct, paired evaluations between the king and each challenger:

1. Both checkpoints were evaluated on the same deterministically selected 2,048-token sequences.
2. The evaluator calculated average cross-entropy loss for every sequence.
3. Per-sequence improvement was defined as the king’s loss minus the challenger’s loss.
4. A percentile bootstrap with 10,000 replicates estimated a lower confidence bound on the mean improvement.
5. The challenger became king only when that lower confidence bound exceeded the required improvement threshold.

The default significance level was 0.001, and the minimum advantage threshold was 0.0015. Evaluation examples were selected algorithmically using the submission block hash and the active data mixture so that both models saw identical samples.

Across the competition, the global data configuration used 12 datasets and was adjusted as model capabilities evolved. The aggregate evaluation pool contained more than four trillion tokens. The individual dataset names, licenses, proportions, filtering procedures, and contamination analysis are not currently documented here.

Evaluation

Teutonic-I achieved an average score of 62.28% across 11 shared benchmarks:

• MMLU
• ARC-Challenge
• ARC-Easy
• PIQA
• HellaSwag
• OpenBookQA
• BBH
• TruthfulQA
• WinoGrande
• GPQA
• MATH-500

Benchmark scores by model, in percent. Higher is better.

Benchmark	Teutonic-I 10B	Quasar 10B	Quasar-Preview 18B	INTELLECT-1 10B	Psyche Cons. 40B	Covenant 72B
MMLU	75.29	49.63	60.87	32.69	24.23	67.11
ARC-C	63.82	41.89	63.40	44.80	31.14	56.83
ARC-E	84.97	62.96	82.45	71.76	55.77	80.93
PIQA	82.81	69.91	83.30	77.73	76.12	81.56
HellaSwag	79.42	62.37	73.07	70.26	63.67	80.61
OpenBookQA	49.00	35.40	46.40	43.80	35.20	44.00
BBH	49.51	31.26	38.10	32.93	30.50	45.96
TruthfulQA	49.58	41.01	41.70	35.45	37.90	49.41
WinoGrande	77.35	56.27	67.56	63.30	56.99	75.85
GPQA	33.98	25.84	29.28	25.84	24.66	30.03
MATH-500	39.40	0.00	71.40	1.00	0.20	20.80
Average	62.28	43.32	59.78	45.38	39.67	57.55

Bold indicates the best result in each row. Average covers all 11 benchmarks.

These results have not yet been independently reproduced. Harness configuration, few-shot settings, prompts, dataset revisions, and the complete set of per-benchmark values are not currently included, so the comparison cannot yet be reconstructed exactly from this repository alone.

Running the model

See the Teutonic GitHub repository for the current runtime setup and usage instructions.

Intended use

Teutonic-I is intended for research involving:

• Continued pretraining and domain adaptation
• Fine-tuning for downstream language tasks
• Evaluation of decentralized model-improvement systems
• Study of competitive checkpoint selection and statistically validated training progress

As a base model, it is not expected to follow instructions reliably without additional alignment or fine-tuning.

Limitations and risks

• No safety alignment is documented. The checkpoint may generate false, biased, harmful, or otherwise undesirable text.
• Training-data details are incomplete. This card does not identify the 12 datasets, their licenses, their geographic or linguistic composition, filtering methods, or whether benchmark contamination was assessed.
• Tokenizer files are absent. Inference is incomplete until the matching tokenizer is identified and published.
• Benchmark reproducibility is limited. Several details needed for exact independent reproduction are not yet included.
• Long-context behavior is unverified. The competition used 2,048-token evaluation sequences, regardless of the much larger maximum value declared in the configuration.
• Custom code is required. Review the repository-provided runtime implementation before use.
• Base-model behavior is unconstrained. Deployments should add task-specific evaluation, moderation, access controls, and human oversight appropriate to their risk level.

Do not use the model as the sole basis for medical, legal, financial, safety-critical, or other high-impact decisions.

License

Teutonic-I is released under the Apache License 2.0.

Citation

Please cite Teutonic-I as follows:

@techreport{teutonic2026,
  title       = {Teutonic-I 10B: Decentralized Pretraining through
                 King-of-the-Hill Competition},
  author      = {Steeves, Jacob and Warcho{\l}, Krzysztof and
                 Korzeniewski, Hubert and Bomba{\l}a, Antoni},
  institution = {Bittensor Subnet 3 (Teutonic)},
  year        = {2026},
  month       = {August},
  url         = {https://teutonic.ai/paper.html}
}