Glint-2.3-Mini

1,064,833 parameters. One block, looped nine times, no unique layers anywhere. It scores better on WikiText-2 than the 1.71M model it was cut down from, ties it on ARC-Easy, and loses 7.4 points of BLiMP. It carries a deletion token, <bs>, that takes back the token before it.

This card includes a negative result about that token. Read "what the deletion token actually learned" before building on it.

parameters	1,064,833 (down from 1,710,145)
architecture	one shared block of 645,312, looped 9 times
unique layers	none
vocabulary	4097 (4096 BPE plus <bs>)
attention	causal sliding window of 256
context	4096
model file	4.3 MB
wiki byte ppl	3.1216
BLiMP	62.27
ARC-Easy	36.20

quick start

pip install -r requirements.txt

python infer.py --info
python infer.py "Once upon a time"
python infer.py "The capital of France is" --raw --stats
python infer.py --repair "text you want it to check"
echo "The sun is" | python infer.py --tokens 120

pip install datasets && python evaluate.py     # reproduce the numbers on this card

Everything needed is in this directory. infer.py imports torch and tokenizers, carries its own copy of the architecture, and reads its shape from config.json.

from infer import Writer

writer = Writer()
stream = writer.generate("Once upon a time", tokens=60, seed=1)

print(writer.visible(stream))    # what a reader sees, deletions applied
print(writer.raw(stream))        # the stream as emitted, retractions bracketed
print(writer.deletions(stream))  # how many it used

Unedited, default settings:

Once upon a time, there was a little boy. He wanted to catch the holiday for him. So he went outside to join in his home. Jack's owner had an idea

The capital of France is the federal government in all partners as for[ international]⌫ing [ and][ political]⌫⌫ to the nation.

how it got to 1.06M

Not by pruning weights or lowering precision. By measuring which block earns its keep, and deleting the one that did not.

Glint-2 is one shared block looped 8 times plus a coda block of the same shape. Skipping each pass in turn and re-measuring wiki byte perplexity, with everything else running unchanged:

pass	ppl if skipped	cost
loop 0	16.5802	+13.5873
loop 1	6.8591	+3.8662
loop 7	6.5654	+3.5726
loop 2	4.5199	+1.5270
loop 3	3.7587	+0.7658
loop 6	3.7160	+0.7231
loop 4	3.6189	+0.6260
loop 5	3.4627	+0.4698
coda	3.4466	+0.4538

The coda was the cheapest block in the model -- cheaper than any single pass of the block it shares no weights with -- and it was 645,312 parameters, 38% of the total. So it went.

Two details made the replacement free. The per-loop adapter table has 16 slots and only 8 were used, so running a ninth pass in the coda's place costs zero parameters. And the retune touched only what the ablation said was cheap: two adapter slots, the 16 loop embeddings, and three RMSNorm scales. About 4,900 of 1,064,833 weights moved; every other tensor is bit-identical to the model this came from, asserted at the end of the run.

Training less beat training more. The same shape with all 1,064,833 parameters unfrozen on the same corpus reaches 3.1521 -- worse. At 18M tokens, touching everything degrades what the block already knew.

	params	trained	wiki	BLiMP	ARC-E
Glint-2	1,710,049	—	3.1270	74.49	36.20
Glint-2.3 (deletion token added)	1,710,145	—	2.9929	69.70	37.00
coda removed, nothing retrained	1,064,833	0	3.4466	—	—
this model	1,064,833	~4,900	3.1216	62.27	36.20
same, all weights trained	1,064,833	1,064,833	3.1521	62.72	37.60

about that BLiMP drop

The obvious explanation is that the model spends probability on <bs>, and <bs> is not English. But nope.

Three measurements:

Having the token is free. Bolt <bs> onto Glint-2 as a mean-of-existing-rows embedding and train nothing. The mean row produces the mean logit, so predictions are unchanged by construction. Measured: BLiMP 74.49, wiki 3.1270, ARC-E 36.20 -- identical to the base to four decimals. Adding the token costs nothing.

Benchmarks already mask it. BLiMP and ARC score fixed text containing no deletions, so <bs> is removed from the distribution before scoring. Probability parked there is not charged against the model.

This model barely puts probability there anyway. Mean P(<bs>) across 400 BLiMP sentences:

	P(<bs>) on BLiMP text	BLiMP
Glint-2.3, 1.71M	0.00029	69.70
this model	0.00052	62.27
same shape, all weights trained	0.02143	62.72
3 distinct blocks, all trained	0.02841	52.25

0.05% of the mass, and masked. It cannot account for 7.4 points.

Look at the bottom two rows, though: those models put 40x more probability on <bs>, and they are the ones where the token genuinely does damage. The mechanism is real -- it just requires the background rate to run away, which is what happens when you unfreeze everything. Keeping the retune to ~4,900 weights is what stopped it: this model's background actually went down, 0.0081 to 0.0071.

So the honest decomposition of 74.49 → 62.27 is:

step	BLiMP	cost
Glint-2	74.49	—
plus untrained <bs> row	74.49	0.00
plus deletion training (1.71M)	69.70	−4.79
minus the coda (1.06M)	62.27	−7.43

The coda is where BLiMP lived. Fine-grained grammaticality -- ranking a well-formed sentence above a minimally-corrupted one -- is what a unique final layer was buying, and adapters could not rebuild it. Perplexity and ARC-Easy came back; BLiMP did not, under any learning rate, slot choice, seeding, or extra epoch tried. If you need BLiMP, use the 1.71M model. This one is for when 4.3 MB matters more.

what the deletion token actually learned

A negative result, and the reason this card does not claim the model detects mistakes.

Earlier versions of this work reported a "deletion detector" at 41x separation and AUC 0.837: the model puts ~0.29 probability on <bs> at a repair site and ~0.007 elsewhere. Those numbers are real, and they do not mean what they look like.

Run python evaluate.py --deletion. It reads 24 held-out rows of the repair corpus, each a real stream of the form ... wrong <bs> gold ..., and reads P(<bs>) four ways:

at a repair, deletions already in context   0.2989
the same repair, first in the stream        0.0010
background, no repair                       0.0005
control, every <bs> stripped from context:
  at a repair                               0.0010
  background                                0.0005

continuing a run is 583x background; starting one is 2.0x;
with deletions stripped the mistake alone is worth 1.9x

Same mistake, same position. With deletions already visible in the context: 0.2989. As the first retraction in the stream, or with the deletions stripped out and the wrong tokens left exactly where they were: 0.0010, which is background.

The model learned that deletions come in runs. It did not learn that a token was wrong. The impressive separation was measuring the presence of <bs> in the context, not the presence of an error. This is the "statistical habit, not a decision" failure, and it is visible in generation too: without a run cap one sample took back 41 of its 60 tokens.

What survives honestly:

• The token is well-behaved: it never damages clean text, and its background rate is low enough (0.0007 on natural prose) that it does not fire spuriously.
• Retractions render correctly and the visible stream is always coherent.
• --repair will report where the model wants to retract in text you give it, but given the above, treat that output as unvalidated. It is not a grammar checker.
• An independent 49.4M judge scores the retractions this model does make at roughly neutral (−0.017 per token at --delete-bias 3.0, 4.32% deletion rate).

Fixing this needs the repairs spliced into the model's own live samples rather than into corpus text at positions the decoder never visits. That is the next experiment, not a claim this release makes.

the deletion knobs

--delete-bias (default 3.0) is added to the <bs> logit. Without it the model retracts almost nothing: it puts so little absolute mass on the token that top-k drops it before it can be drawn. Measured at temperature 0.9, top-k 40:

--delete-bias	deletion rate
0.0	0.00%
3.0 (default)	4.32%
3.5	6.46%
4.0+	runs away

The bias applies only at the first step of a run, and <bs> is blocked after six consecutive deletions -- the longest run the training corpus contains. Both guards are load-bearing: biasing every step is what produced the 41-of-60 sample.

--delete-threshold blocks <bs> below a full-distribution probability floor. It is stricter than it looks, because sampling happens after top-k renormalises over 40 tokens; anything above about 0.1 silences the model completely. Default 0.

a note on loops versus layers

At this size the loop is not a stylistic choice, it is the only way to afford depth. A 4097×96 tied embedding takes 393,312 of the 1,064,833 parameters, leaving 671,425 for blocks -- barely one full-width block (ffn_hidden 2202). Distinct depth has to be narrow: two blocks fit ffn_hidden 1037, three fit 648, four fit 454.

Spending the identical budget on separate layers instead, trained on the same corpus:

shape	params	wiki	BLiMP	ARC-E
this model (looped)	1,064,833	3.1216	62.27	36.20
2 blocks, ffn 1037	1,064,832	5.7265	51.93	28.60
3 blocks, ffn 648	1,064,448	5.6410	52.25	27.00
4 blocks, ffn 454	1,064,640	5.9695	52.42	28.60
3 blocks, 3 epochs	1,064,448	4.8983	55.79	29.40

The loop wins by ~1.8 perplexity and ~7 BLiMP. Caveat: the stacks are warm-started by truncating the shared block's feed-forward from 2112 hidden units to 1037/648/454, which throws most of it away, and they get 18M tokens where Glint-2 was pretrained on far more. Three epochs closed 0.75 of a ~2.0 gap, so they are genuinely undertrained -- not by enough to change the ranking, but a from-scratch pretrain of both shapes is the experiment that would settle it. The budget arithmetic above holds regardless.

files

Glint-2.3-Mini.pt      the weights, 4.3 MB
tokenizer.json         4096-vocab BPE; <bs> is id 4096 and lives in the model, not here
config.json            shape, loop count, deletion token id, default bias
infer.py               generation with deletions, and --repair; torch + tokenizers only
evaluate.py            reproduces this card's numbers, including the negative result
holdout_repairs.json   24 rows of held-out repair corpus, so evaluate.py is checkable
requirements.txt
README.md              you are here

limitations

• 1.06M parameters. It drifts within two sentences on anything encyclopedic and is a research artifact, not a useful assistant.
• BLiMP is 12 points below Glint-2. That capability was in the block that was removed.
• The deletion token continues runs rather than detecting errors. See above.
• loops must stay at 9. More or fewer produces gibberish; the config's loop_slots: 16 is the adapter table size, not the depth.
• No chat or instruction tuning. Base model only.

citation

Glint Research, 2026. Derived from Glint-2 by ablation-guided block removal; the deletion token is distilled from Shard-2 (49.4M).