One dense backbone. Sixty-four specialist minds. Morpho-180B is a 64-expert domain collective: a 72.7B dense backbone plus 64 converged specialist adapters, 180,500,471,808 parameters in total. Every expert was trained to convergence on its own routed slice of an 822,745-example corpus. The merged release is the convenient default; the specialists ship as adapters.
Read the full release report: REPORT.md — the complete
engineering account, section by section — or the
designed HTML edition, which opens in any browser with all
visuals embedded.
At a glance
| Component | Parameters |
|---|---|
| Dense backbone (80 layers) | 72,706,859,008 |
| Per expert adapter (rank 128) | 1,684,275,200 |
| 64 expert adapters | 107,793,612,800 |
| Collective total | 180,500,471,808 |
| Adapter share of collective | 59.7% |
| Stored adapter generations (64 SFT + 31 DPO) | 160,006,144,000 |
Those are sums of the actual safetensors headers, not estimates.
What actually ships
This repository is the release line. The heavy artifacts live in sibling repos:
| Artifact | What it is | Size |
|---|---|---|
| Merged | 64 adapters folded into the backbone — dense 72.7B | 145.4 GB |
| Checkpoints | 64 converged SFT adapters, rank 128 | 404.2 GB |
| DPO | 31 preference-aligned adapters | 208.9 GB |
| GGUF | f16, Q8_0, Q5_K_M, Q4_K_M | 324.5 GB |
| 7B · 1.5B | distilled students | 15.2 / 3.1 GB |
| Data | corpus, tokenized batches, code | 160 files |
Read the split carefully. 180.5B is the size of the collective. A single inference request against the merged file runs 72.7B dense parameters. You get the full 180.5B only by keeping the 64 adapters, and they are specialists, not a router: one adapter at a time, not a mixture activated per token.
The 64 experts
Eight sectors × eight specialists: Finance, Law, Medicine, Technology, Science, Business, Arts, Humanities. Each training example maps to exactly one expert — no duplication, no overlap. Counts range from 606 to 92,366 examples per expert (median 6,334), 822,745 total.
Data refinery
Raw JSONL → domain-tagged → v5 routing → tokenized NumPy batches → per-expert slices. Every byte is published, including the code that did the routing.
Pipeline
| Stage | Result |
|---|---|
| 1 · Base | 72.7B dense backbone, frozen, 4-bit NF4 QLoRA |
| 2 · Data | 822,745 examples, 17 sources, 64 routes |
| 3 · Structure | 64 expert slots, rank 128 / alpha 256, 7 target modules |
| 4 · SFT | 64 / 64 converged, patience 2,000 |
| 5 · DPO | 31 / 31 eligible experts converged; 33 thin experts SFT-only |
| 6 · Merge | gate PASS — PPL 7.69 → 7.83 (×1.02) |
| 7 · Verify | GSM8K 76.7%, PPL 7.68, 19.5 tok/s |
| 8 · GGUF | f16 + Q8_0 + Q5_K_M + Q4_K_M, 324.5 GB |
| 9 · Distill | 7B (loss 0.7188) and 1.5B (loss 1.5750) students |
Convergence
Every expert ran to a patience gate, kept only its best checkpoint, and wrote a loss file plus a step file. The vault card lists all 64 with per-expert loss, best step, and storage dtype (56 fp32, 8 bf16).
The merge gate
We did not publish a merge because it finished. We published it because it passed a perplexity gate against the untouched backbone on the same 40-document holdout, with a human read behind the number.
| Candidate | Cap | PPL | × base | Verdict |
|---|---|---|---|---|
| Plain delta mean | — | 300.88 | 39.1× | rejected — degenerate text |
| Sign consensus | 0.020 | 49.74 | 6.47× | rejected |
| Sign consensus | 0.011 | 9.34 | 1.21× | rejected — broken grammar |
| Sign consensus | 0.006 | 7.83 | 1.02× | shipped |
Method: per tensor, trim the smallest 80% of delta magnitude, elect a sign by
weighted vote, sum only consenting deltas, then apply a per-module relative-norm
cap so no single expert dominates a layer. δ = B @ A · (alpha/r = 2);
560/560 tensors merged in 15.1 minutes. Mean |δ|/|W| was 2.70% before the cap
and 0.6% after; the largest single relative change was 19.3%.
The 1.21× candidate is why the gate is 1.10×: it looked fine statistically and still produced ungrammatical text.
Verification
| Run | GSM8K | Perplexity (40 docs) | Throughput |
|---|---|---|---|
| Backbone, untouched | 25/30 · 83.3% | 7.56 | 17.1 tok/s |
| Morpho-180B merged | 23/30 · 76.7% | 7.68 | 19.5 tok/s |
| Δ | −2 items | +1.6% | +14% |
Domain audit: 8/8 sector prompts produced coherent, on-topic answers.
GGUF builds
| File | Quant | Size |
|---|---|---|
f16/Morpho-180B-f16-0000{1..4}-of-00004.gguf |
f16 (4 shards) | 145.4 GB |
Morpho-180B-q8_0.gguf |
Q8_0 | 77.3 GB |
Morpho-180B-q5_k_m.gguf |
Q5_K_M | 54.4 GB |
Morpho-180B-q4_k_m.gguf |
Q4_K_M | 47.4 GB |
Distilled students
| Student | Best loss | GSM8K | PPL (10 docs) | Loop-free |
|---|---|---|---|---|
| 7B · 15.2 GB | 0.7188 | 1/10 | 11.32 | 6/8 greedy → 8/8 at repetition_penalty=1.1 |
| 1.5B · 3.1 GB | 1.5750 | 2/10 | 13.16 | 2/8 greedy → 8/8 at repetition_penalty=1.1 |
The students inherit fluency, tone, and domain vocabulary. They do not inherit arithmetic reliability. Use the 72.7B when the number matters.
Where the specialists live
The merge does not destroy the experts. Each converged adapter is versioned separately: 64 SFT in the checkpoint vault, 31 with a DPO refresh in DPO. Load the merged model for general work; load backbone + one adapter when you want a single domain at full specialisation.
Quick start
# merged dense model
pip install -U transformers accelerate
python -c "
from transformers import AutoModelForCausalLM, AutoTokenizer
m = AutoModelForCausalLM.from_pretrained('bluemorpholimited/Morpho-180B-Merged', device_map='auto', torch_dtype='bfloat16')
t = AutoTokenizer.from_pretrained('bluemorpholimited/Morpho-180B-Merged')
print(t.decode(m.generate(**t('Explain stare decisis.', return_tensors='pt').to(m.device), max_new_tokens=200)[0], skip_special_tokens=True))
"
# quantized single-GPU path
huggingface-cli download bluemorpholimited/Morpho-180B-GGUF Morpho-180B-q4_k_m.gguf --local-dir .
llama-cli -m Morpho-180B-q4_k_m.gguf -p "Explain stare decisis." -n 256
What this is not
- Not a 180B dense model. The merged file is 72.7B dense. 180.5B is the collective across backbone plus 64 adapters.
- Not a router. There is no per-token gating in the merged weights; that is a deliberate release decision, not a missing file.
- Not a benchmark gain. The merge is quality-neutral against its backbone (+1.6% perplexity, −2 GSM8K items at n=30).
- Not long-context validated. The backbone supports 131,072 positions, but every expert trained at 2,048 tokens. Long-context behaviour after domain tuning is untested.
- Not a frontier model. This is an engineering release: a reproducible 64-expert pipeline with a documented quality gate.
Provenance and licence
Adapter weights, taxonomy, routing, merge gate, and evaluation code are
Apache 2.0. The merged weights and every artifact derived from the backbone
carry the upstream backbone terms. The required attribution notice and a full
copy of the agreement ship in NOTICE and LICENSE in this
repository and in every derived repository. Commercial use above 100 million
monthly active users requires a separate upstream licence.
Every number on this card was produced by scripts in the data repo — nothing is estimated.
Citation
@misc{morpho180b,
title = {Morpho-180B: a 64-expert domain collective with a gated merge pipeline},
author = {Blue Morpho Limited},
year = {2026},
note = {180,500,471,808 parameters across a 72.7B dense backbone and 64 expert adapters},
url = {https://huggingface.co/bluemorpholimited/Morpho-180B}
}
Release map
| Stage | Artifact |
|---|---|
| Pipeline, gate history, benchmarks | Morpho-180B |
| 4 · 64 converged SFT adapters | Checkpoints |
| 5 · 31 DPO-aligned adapters | DPO |
| 6 · Merged 72B weights | Merged |
| 8 · Quantized builds | GGUF |
| 9 · Distilled students | 7B · 1.5B |
| Data and pipeline code | Morpho-180B-Data |