Kenga Prophet — M2 baseline (v0.1)

The first Kenga-native trained model published externally. This release is immutable: subsequent runs ship under separate model names (kenga-prophet-m2-k16, …). Use this card as a permanent point of reference for what "6,300 parameters + Kenga corpus" did on the day of the first release.

What this model is

Linear softmax classifier: P(next_token | last_K_tokens)
Vocabulary: 28 tokens (Kenga lexemes + ID/NUM)
Window: K = 8 preceding tokens
Parameters: 6,300 trainable weights (28 × (8 × 28 + 1))
Trained in Python with numpy only, no torch, no GPU
Inference runs in Kenga Lite more VM (no GPU, no Rust)

If 6,300 / 27,000,000,000 sounds absurd, that's exactly the proportion the user wants to track: small + structurally correct versus big + general-purpose.

Numbers (held-out next-token accuracy)

kenga_seed_add   19/88  = 21.6 %
kenga_seed_fact  14/62  = 22.6 %
kenga_seed_fib   10/55  = 18.2 %
kenga_seed_max   21/88  = 25.0 %
kenga_seed_mul   16/82  = 20.7 %
kenga_seed_pow   15/68  = 22.1 %
kenga_seed_sqr   13/68  = 19.1 %
kenga_seed_sub   16/82  = 20.7 %
kenga_seed_sum   26/104 = 25.0 %
overall          149/697 = 21.4 %

These are token-accuracy numbers, not BLEU. The "trick" is that Kenga's grammar has no ambiguity in the 28-token codec, so even modest per-token accuracy can produce syntactically valid continuations.

Provenance (frozen at v0.1 release)

Kenga commit  :  993187398e8d5cda85e7c8a1fca44e648f87016a
Training   V  :  28
Context    K  :  8
Embedding features/V:  226 (K*V + bias)
Total params      :  6,300
Optimizer         :  Adam  (lr 5e-3, betas 0.9/0.999)
Epochs            :  60
Training corpus   :  168 .kenga source files, 154,000 tokens
Train/test split  :  first 90% / last 10%
Held-out program set : 9 kenga_seed_*.kenga programs

weights blob sha (16 hex):  28f7ef5c39008b52
vocab  blob sha            :  0246917ce1a8f263
train  blob sha            :  bc558fa4207b6db1
test   blob sha            :  d991ac600746b4c8
meta   blob sha            :  d13eb31ddcaba14b

Total on-disk size (all 5 artefacts):  ~ 580 KB
RAM at inference (Lite more VM):       ~ 1 MB
Wall-clock training time:             ~ 1–2 min  (numpy only)
Wall-clock per-token inference:       ~ 30 ms   (Lite more VM, single argmax)
Wall-clock full-prediction inference:  ~ 1 s    (Lite, 100 generated tokens)
CPU-only, no GPU required.

The kenga-prophet repo on Hugging Face is immutable at this SHA: subsequent improvements go to kenga-prophet-m2-k16, kenga-prophet-m2-mlp, etc. The v0.1 card stays as the first point of reference.

Weights format fix (v2 of this file): the initial upload serialized \n as literal backslash-n (single-line), which corrupted the weights, vocab, and meta files for any consumer. This revision re-serializes them with real newlines. All provenance values above (commit SHA, blob hashes, params) are unchanged — this is a serialization fix, not a retrain.

Program-validity rate (honest, measured)

tools/kenchat.py --probe runs the model and feeds the generated program to kenga-lite. Current result for v0.1:

compile-ok:    0/9 = 0.0%
run-ok:        0/9 = 0.0%
match value:   0/9 = 0.0%

The model cannot yet generate structurally valid programs: greedy decoding always predicts fn, and 21% token accuracy means 79% of tokens are wrong. This 0/9 is the honest baseline the ladder must climb — see "What this model CANNOT do" below.

What this model CAN do

Given an 8-token prefix from Kenga source, predict the next token from the 28-token codec.
Run in two or three minutes on a 1660-class GPU-less laptop (this is the entire training time).
Be inspected losslessly: weights are integers in the file at minds/mid_prophet_m2_big_w.txt, vocabulary at minds/mid_prophet_m2_big_vocab.txt, training config in minds/mid_prophet_m2_big_meta.txt.

What this model CANNOT do

Open-ended chat on natural-language queries. It was trained on Kenga source, not on English.
Pass-rate on long (multi-line) generation at this K=8 window is weak because 21% next-token accuracy means 79% wrong tokens; one wrong token later in the program bleeds into syntactic breakage.
Encode Kenga semantics. It is a next-token surface statistic. See Mid-Prophet M1 (docs/PICO_PROPHET.md) for a non-trained signature-based classifier that does better on identity classification tasks.

Why this is genuinely Kenga-native and not "just another Python model"

axis	this model	a Hugging Face PyTorch reference
Training	numpy only	PyTorch / JAX / TF
Optimizer	hand-rolled Adam (~80 lines)	torch.optim.Adam
Data pipeline	`walk('kenga/' + 'examples/')` + tokenize	datasets.load_dataset
Inference	`bootstrap/bin/kenga-lite.exe` reading weights from `minds/...`	HF pipeline / transformers
Runtime	Kenga Lite more VM (no Rust, no GPU)	CUDA / ROCm / CPU SIMD
Tokenisation	custom 28-token Kenga codec	BPE / WordPiece
File format	plain text integer weights	safetensors / ONNX

Every stage is the same Kenga: tokenisation is built around the Kenga grammar, inference runs on the kenga-lite binary that comes with the language, and there is no Python dependency in the critical path of inference. That is what makes this a Kenga-native model and not "a Python model with Kenga data".

Fixed sample predictions (token ids 0..27)

For random prefixes drawn from the held-out stream at position 32 onward, the model picks the following tokens. These are illustrative raw outputs, not corrected.

prefix  [13, 7, 14, 15, 7, ...]           predict token 7  (i64)
prefix  [11, 1, 26, 16, 12, ...]           predict token 10 (semicolon)
prefix  [0, 26, 9, 8, 7, 26, 14, ...]     predict token 11 ({)

These are toy outputs; the artefact here is provenance and ladder position, not finished quality.

Reproduce

# requires numpy only; on Windows:
git clone https://github.com/GermannM/kenga-lang
cd kenga-lang
python tools/train_m2_big.py
# produces minds/mid_prophet_m2_big_*.txt (~ 580 KB total)

# inference on a token stream:
minds/mid_prophet_m2_big_w.txt minds/mid_prophet_m2_big_vocab.txt  # explicit
bootstrap\bin\kenga-lite.exe run examples\ml\mid_prophet_m2_run.kenga

The orchestrator script scripts/mid-birth-m2.sh runs inference against the 9 held-out programs and reports the aggregate accuracy.

Citation

docs/PICO_PROPHET.md — the ladder Pico-Prophet → Mid-Prophet M1 → M2
docs/NEUROMODEL_27B.md — the six-axis stack behind the claim
tools/train_m2_big.py — the training script that produced this artefact
examples/ml/mid_prophet_m2_run.kenga — the Lite inference harness

Downloads last month: -; Downloads are not tracked for this model. How to track