qbz506/p-s-hord-m2

prabhasa-samskrutam — HORD m2

Research training checkpoints from the prabhasa-samskrutam HORD series (Pāṇinian-structured hybrid Mamba/attention language models). Published under the pseudonym p-s. These are raw PyTorch training checkpoints, not packaged inference models.

Arms (baseline vs treatment ablation)

• baseline: 199M params, step 1240, 650M tokens, d_model=768, n_layers=24
• treatment: 199M params, step 1240, 650M tokens, d_model=768, n_layers=24

What each .pt contains

Each checkpoint is a dict with keys: model (state_dict), opt (optimizer state), config (architecture), arm, step, tokens_seen, cursor. It is fully resumable.

import torch
ckpt = torch.load("treatment/final.pt", map_location="cpu", weights_only=False)
print(ckpt["config"], ckpt["step"], ckpt["tokens_seen"])
# model = build_model(**ckpt["config"]); model.load_state_dict(ckpt["model"])

Reproduce / load

Architecture and tokenizer live in the code repo: https://github.com/SharathSPhD/prabhasa-samskrutam (src/prabhasa/ for the model; tokenizer under src/prabhasa/application/tokenizer). anchor_*.pt files are the MMD magnetic anchor snapshots referenced during training.

Byte-level tokenizer (vocab_size 256), structured channels (n_roles 16).