Morpho-180B DPO — preference alignment artifacts
Stage 5 of the Morpho-180B pipeline. SFT backbone:
Morpho-180B-Checkpoints.
Contract
- Per-expert DPO, same 64 experts, same one-example-one-expert mapping as SFT.
- Same convergence bar: patience-gated training, best-only weights, DONE markers.
- Real preference data only — 29,983 chosen/rejected pairs (HH-RLHF, HelpSteer2, UltraFeedback, Nectar), routed per expert by the v5 taxonomy with a 200-pair floor.
- DPO adapters initialize from the converged SFT adapters — never from scratch.
Layout
dpo/raw/ # preference pairs per source (chosen/rejected JSONL)
dpo/tagged/ # per-expert routed pairs + expert_id/domain audit fields
dpo/tokenized/ # tokenized chosen/rejected batches (dpo_batch_*.npy)
dpo/adapters/ # per-expert DPO adapters (sft/expert_XX pattern)
manifest.json # per-expert pair counts + DPO losses
Status
✅ Stage 5 COMPLETE — 31/31 experts DONE (resumed 4×; all adapters and DONE
markers on Morpho-180B-DPO).
| Expert | Domain | DPO best loss | Converged step | Weights |
|---|---|---|---|---|
| 02 | Banking & Lending | 1.01e-06 | 673 | ✅ DONE + weights |
| 04 | Accounting & Auditing | 7.83e-08 | 317 | ✅ DONE + weights |
| 05 | Taxation & Compliance | 5.73e-04 | 979 | ✅ DONE + weights |
| 06 | Real Estate Finance | 0.0176 | 400 | ✅ DONE + weights |
| 09 | Criminal Law | 0.0176 | 191 | ✅ DONE + weights |
| 10 | Civil Law | 5.33e-06 | 587 | ✅ DONE + weights |
| 11 | Corporate Law | 0.00105 | 525 | ✅ DONE + weights |
| 12 | Intellectual Property | 1.38e-06 | 1695 | ✅ DONE + weights |
| 13 | International Law | 6.38e-09 | 759 | ✅ DONE + weights |
| 14 | Tax Law | 2.17e-10 | 1050 | ✅ DONE + weights |
| 15 | Regulatory Compliance | 8.82e-04 | 607 | ✅ DONE + weights |
| 17 | Pharmacology | 7.50e-16 | 739 | ✅ DONE + weights |
| 18 | Diagnostics & Imaging | 3.23e-11 | 348 | ✅ DONE + weights |
| 20 | Neurology | 0.00406 | 350 | ✅ DONE + weights |
| 21 | Cardiology | 9.27e-05 | 660 | ✅ DONE + weights |
| 22 | Oncology | 1.84e-08 | 1240 | ✅ DONE + weights |
| 23 | Public Health | 3.15e-11 | 660 | ✅ DONE + weights |
| 25 | Systems & Infrastructure | 3.61e-09 | 581 | ✅ DONE + weights |
| 27 | Databases & SQL | 1.36e-05 | 1066 | ✅ DONE + weights |
| 34 | Biology & Life Sciences | 2.37e-03 | 546 | ✅ DONE + weights |
| 35 | Mathematics | 4.11e-05 | 1191 | ✅ DONE + weights |
| 38 | Civil Engineering | 4.84e-06 | 194 | ✅ DONE + weights |
| 40 | Strategic Management | 9.45e-07 | 577 | ✅ DONE + weights |
| 43 | Operations & Logistics | 6.85e-04 | 327 | ✅ DONE + weights |
| 47 | Consulting | 4.60e-05 | 772 | ✅ DONE + weights |
| 56 | Psychology | 6.52e-03 | 388 | ✅ DONE + weights |
| 57 | Psychology (advanced) | 6.05e-04 | 1050 | ✅ DONE + weights |
| 59 | Economics | 3.77e-05 | 562 | ✅ DONE + weights |
| 60 | Philosophy | 4.02e-03 | 943 | ✅ DONE + weights |
| 62 | Political Science | 2.74e-02 | 183 | ✅ DONE + weights |
| 63 | Critical Analysis & Reasoning | 3.94e-08 | 248 | ✅ DONE + weights |
All 31 eligible experts are trained and published; 33 thin experts (<200 pairs)
keep SFT-only per policy. Each row is verified from the expert's best_loss.txt
and DONE.txt on this repo.
Parameter accounting
Each DPO adapter is rank 128 / alpha 256 with 1,120 tensors, i.e. 1,684,275,200 parameters. The 31 DPO refreshes add 52,212,531,200 parameters of stored adapter weights on top of the 64 SFT adapters. The shipped collective counts one adapter per expert (DPO where available, SFT otherwise): 180,500,471,808 total.
Where these adapters went
| Stage | Artifact |
|---|---|
| 6 · Merge | Morpho-180B-Merged — 64 adapters (DPO where available, SFT otherwise) merged with a gated TIES-style pass; PPL 7.69 → 7.83 (×1.02) |
| 7 · Verify | GSM8K 76.7% · PPL 7.68 · 19.5 tok/s |
| 8 · GGUF | Morpho-180B-GGUF — f16, Q8_0, Q5_K_M, Q4_K_M |
| 9 · Distill | Morpho-7B-Distill · Morpho-1.5B-Distill |
The 31 DPO adapters are also usable directly: load the SFT adapter from the checkpoint vault and the DPO adapter on top, per expert.
Licence
Adapter weights: Apache 2.0. The adapters are trained against the upstream
backbone and are distributed for use with it under its terms; the required notice
and a full copy of the agreement are in NOTICE and LICENSE.
Release map
| Stage | Artifact |
|---|---|
| Pipeline, gate history, benchmarks | Morpho-180B |
| 4 · 64 converged SFT adapters | Checkpoints |
| 5 · 31 DPO-aligned adapters | DPO |
| 6 · Merged 72B weights | Merged |
| 8 · Quantized builds | GGUF |
| 9 · Distilled students | 7B · 1.5B |
| Data and pipeline code | Morpho-180B-Data |