deeprcurs/MBG-1.0

MBG 1.0 — Model Bahasa Garuda

deepRcurs Labs / @deeprcurs · author: Mzed Imamkh / @mzedimamkh

MBG 1.0 ("Model Bahasa Garuda") is a research project building an Omni-Sparse Hybrid language model — a laddered-scaling architecture that combines an SSM/MLA core (Mamba-2 + DeepSeek-style multi-head latent attention), fine-grained MoE, and ternary {−1,0,1} weights (BitNet-style), with Trinity-Mirror as the core reasoning / answer-production controller.

This Hugging Face model repo is the external archive for the project: it stores the golden checkpoints and canonical weights. The dataset/corpus lives in a separate dataset repo (linked below), and the workspace snapshot holds only the controlling code and lightweight pointers.

Status: L0 (0.17M) and L1 (17M) rungs validated. This repo currently holds the L0/L1 validation checkpoints produced by the mbg_mini_gpt.py prototype, which exercises the same MoE + probe-routing + GUM-optimizer machinery used to validate the scaling, memory, and routing claims. The full Omni-Sparse Hybrid (SSM+MLA) is the target architecture and is specified in the internal design blueprint (MBG-1.0-rev-3_Blueprint.md), which is kept in the workspace snapshot only — not published to this repo.

Architecture summary (per blueprint rev-3)

• Omni-Sparse Hybrid core: interleaved Mamba-2/SSM + MLA (KV-compressed) layers with fine-grained MoE FFN, aux-loss-free bias routing, and ternary FFN-MoE weights (embeddings/attention/router stay BF16).
• Trinity-Mirror controller: Thinker → Critic → Refiner chained processor (bounded loop, confidence calibration, auditable mirror ledger). Inquiry tokens route probes to specialist MoE experts (probe→expert coupling).
• GUM optimizer (Innovation I1): GaLore-Unbiased + Muon — Muon FLOP efficiency with GaLore low-rank memory savings (~72% optimizer-state reduction, state/param ≈ 2.2 B).

Scaling ladder ([1 7, 0 8, 4 5] — mandatory mantissa pattern)

The scaling pattern is the mantissa digits [1 7, 0 8, 4 5] = 0.17M, 0.8M, 4.5M, each column multiplied by ×100 per level: 0.17M → 17M → 1.7B≈1.8B · 0.8M → 80M → 8B · 4.5M → 450M → 45B. This pattern is canonical and is reproduced here unchanged from the blueprint (§5).

Level	Parameters (active)	Status
L0	0.17M / 0.8M / 4.5M	✅ L0 validated (GUM stable, routing)
L1	17M / 80M	✅ 17M validated; 80M next
L1+	450M (active; upcycle → MoE total ~1.5–3B)	planned
L2	1.7B / 8B (MoE total larger)	if external funding (multi-GPU)
L3	45B (total MoE, ~1.7–8B active)	if external funding

Scaling beyond L1 is deferred until implementation planning is finalized and snapshot safety is guaranteed. L2 and above require external funding ("L3 ke atas = jika ada pendanaan"); capital buys compute, not debugging of an architecture that should already be validated at L0/L1.

Checkpoints in this repo

Rung	Params	Canonical weights	Checkpoint (bf16)
L0 — 0.17M	176,992	—	golden/mbg_017m.pt
L1 — 17M	16,748,928	model.safetensors	golden/mbg_l1-17m_20260901-173235.pt

.pt files are complete training checkpoints (weights + config + history) in bf16; model.safetensors is the canonical 17M weight set.

File layout

MBG-1.0/                          (external archive — clean flow)
├── README.md            # this model card
├── config.json          # current model config (L1 quality model, v2x)
├── model.safetensors    # canonical weights (follows latest model update)
├── manifest.json        # provenance: config, metrics, params, SHA-256 of every golden
├── tokenizer/
│   └── mbg_bpe.json     # BPE tokenizer artifact
├── golden/              # complete training checkpoints (bf16 .pt)
├── source/              # training/eval/data source code (reproducibility)
└── reports/             # experiment reports (markdown + JSON)

Clean flow: the workspace snapshot holds only the controller (code, scripts, docs, corpus, manifest) — no .pt/.safetensors. All model weights and checkpoints live in this repo and are downloaded on demand (e.g. internal/ops/sync.sh latest / pull).

Public vs internal: everything published here (source code, reports, weights) is intentionally public. Internal design/governance artifacts — the design blueprint, the Trinity-Mirror component spec, the internal protocols/ops documents, and the ops scripts — live under internal/ in the snapshot and are never pushed to this repo.

Dataset

The English corpora used for training/evaluation are published in the dataset repo:

deeprcurs/MBG-1.0-data — see the dataset card there.

Quick start (load 17M weights)

pip install torch safetensors

from safetensors.torch import load_file
weights = load_file("model.safetensors")   # 117 tensors (bf16; embeddings/norms fp32)
# Rebuild with src/mbg_mini_gpt.MbGPT(config.json) and load_state_dict(weights).

Integrity

manifest.json records the SHA-256 of every golden checkpoint so downloads can be verified before use.

License

Dual License (custom, source-available) — see LICENSE.md.

• Personal / Non-Commercial Use: free, for private experimentation and research. Re-branding and claiming the work as your own are prohibited.
• Commercial Use / Re-branding / derivative redistribution: requires the author's prior written permission (commercial license fee applies).

The license applies to the model weights/checkpoints, tokenizer, config, training/evaluation code, and the datasets & corpus. This is not an open-source license; it intentionally restricts redistribution and use.