vynly/mica-ember-0.3.5

MICA Ember v0.3.5

A one-of-a-kind language model. Not a transformer, not a neural network.

MICA models language with learned integer rewrite rules on a ring of cells, a cellular automaton. It is an original architecture, designed and trained from scratch. Ember is the byte-level MICA model: it reads text one UTF-8 byte at a time, suggests the next word, and finishes a word after its first two letters.

Its sibling MICA Flame-W 0.3.5 predicts whole words and continues sentences. Try both in the live playground.

What changed from v0.3.1

1. A weight-averaged automaton. Starting from the 097b native refit of Ember's learned integer rules, 60 more training rounds were run (40,000 records each, round rate 0.01) and the learned integer rule values and readout were averaged over those rounds. This is the stochastic weight averaging that produced Flame-W 0.3.5. The geometry, wiring and file size are unchanged.
2. A refit byte readout and memory. On the averaged automaton, the integer byte readout and the 14 KB integer memory sidecar were refit together on the exact integer probe states of 119,823 training records, with the recipe that produced v0.3.1 (selected on development data: the running average after 4 of 8 epochs).
3. New word readouts. The next-word and two-letter-completion readouts were retrained over the new automaton's 240 cellular probes on the same 119,823 records (the previous heads used about 8,000), with a training-only vocabulary of 8,192 words (selected on development data: weights averaged over epochs 2 to 4).

Results

Clean set	Ember v0.3.1	Ember v0.3.5	Paired change (95% CI)
Chat val1000	1.827462	1.807137	-0.020325 [-0.022023, -0.018659]
Everyday val1000	1.836078	1.812973	-0.023105 [-0.025522, -0.020739]
Equal-domain mean	1.831770	1.810055	-0.021715 [-0.023154, -0.020275]

Clean word benchmark (400 prompts)	Ember v0.3.1	Ember v0.3.5	Paired gain (95% CI)
Next word, top-1	14.25%	15.25%	+1.00 [-1.75, +3.75] points
Finish a word after two letters, top-1	48.00%	54.50%	+6.50 [+3.00, +10.00] points

The completion gain is significant. The next-word change on these 400 clean prompts is within noise; on the larger development set (1,200 prompts, used for selection) it was +3.08 [+1.67, +4.58] points.

Selection used development data only; clean sets were scored once; the sealed test set was not accessed. Development confirmation (last 500 records per domain): -0.020635 bits/target [-0.022641, -0.018629].

Why MICA is different

• No transformer inside. No attention and no stack of neural layers. Each byte is written onto a ring of integer cells, learned local rules rewrite the ring, and probes read the result.
• Integers only in the cellular updates, the probes and the word scores.
• Deterministic. The same text always gives the same suggestions.
• Tiny and CPU-only. Runs on any CPU with numpy alone.

Files

File	What it is
model.mica	The automaton and byte readout, SHA-256 c2bef375c8dcd6f944e1a44f4b92869645ddf97eb386382fa322cf9519eda506
memory.npz	Integer byte-memory sidecar, SHA-256 3827c2a38981347bff17f9666543c333178c5a89fdd089d216e5c6f8ab8847c8
word_heads.npz	Integer word readouts (8,192 words), SHA-256 1f73aa859fa9b141bdeb5a87c867cdb639a2d885166e3fd124d72989a05ee9ba
word_model.py, mica_r1/	Exact integer runtime (numpy only)
test_word_model.py, test_memory.py	Conformance tests
CHECKSUMS.json	SHA-256 of every file

pip install numpy
python test_word_model.py
python word_model.py "Thank you for "
python word_model.py --complete "Thank you for be"

Limitations

A research model with a training-only vocabulary of 8,192 common words. Top-1 accuracy does not measure relevance beyond the exact target, sentence coherence or open-vocabulary coverage. Personal and research use only; commercial use is forbidden (see LICENSE).

Code and history: github.com/Vovala14/Mica-Ai.