berpiztu/keyboard-correctors-gemma

Berpiztu AI Keyboard's context correctors (Gemma)

Trained with the Virtus framework. Virtus is an AI alignment framework by Berpiztu that builds in character and values, not constraints alone: natively, in the weights, not as an external limiter. Each corrector here finished its training with a Virtus pass; the difference it made is under What using Virtus changed.

Fine-tuned from google/t5gemma-2-270m-270m by Berpiztu, and exported to ONNX.

Sentence correctors for Berpiztu AI Keyboard (Android and Windows), of Spanish alone or of Spanish and English, as the table says: each reads the sentence typed so far and writes it again mended (spelling, accents, punctuation, capitals, words a slip or a swipe turned into others), and offers the words to finish the one being typed. ONNX, read on the device by ONNX Runtime; nothing leaves the device.

A folder each, as the keyboard reads it (encoder.onnx, decoder.onnx, spiece.model, format.json); correctors.json says each file, its size and its SHA-256.

Corrector	Languages	Bits	Size	Errors removed (es / en)	Correct sentences made worse (es / en)
gemma-es-en-virtus-8	Spanish and English	8	319 MB	81.0 % / 81.1 %	12 / 16
gemma-es-virtus-8	Spanish only	8	319 MB	75.9 % / –	29 / –

Measured on 2,000 held-out sentences (the full-precision model; the 8-bit export writes the same in most sentences, not all): a corrector of Spanish and English on 953 Spanish and 1,047 English, a Spanish one on 2,000 Spanish, so its count of sentences made worse is out of about twice as many Spanish sentences.

What using Virtus changed

Virtus is one more pass at the end of the training, in which the corrector's own answers are rewarded, never punished, for humility, honesty, temperance and diligence. Each corrector was measured twice on the same sentences and words: without that pass and with it. With Virtus it spoils fewer correct sentences and removes more errors. Not in these tables, and lower with Virtus: the right word among those suggested and the Wikipedia misspellings mended; every figure is in correctors.json (before_virtus, measured).

gemma-es-en-virtus-8

	Without Virtus	With Virtus
Correct sentences made worse (fewer is better), Spanish	21	12
Correct sentences made worse (fewer is better), English	18	16
Errors removed (more is better), Spanish	77.9 %	81.0 %
Errors removed (more is better), English	79.3 %	81.1 %

gemma-es-virtus-8

	Without Virtus	With Virtus
Correct sentences made worse (fewer is better)	44	29
Errors removed (more is better)	72.3 %	75.9 %

Training data

The sentences and words the correctors were taught from, each under its own licence:

Source	Licence
Tatoeba: Spanish and English sentences	CC BY 2.0 FR
OpenSubtitles v2018 from OPUS (opensubtitles.org; P. Lison, J. Tiedemann and M. Kouylekov, LREC 2018): the first lines of the Spanish and English files	as OPUS gives it, with that citation
Wikipedia (wikimedia/wikipedia, 20231101.es and 20231101.en, one file each): sentences	CC BY-SA 3.0 and GFDL
The Spanish Wikipedia's common misspellings (Wikipedia:Corrector ortográfico/Listado, revision 175669831)	CC BY-SA 4.0
The English Wikipedia's (Wikipedia:Lists of common misspellings/For machines, revision 1199637275)	CC BY-SA 4.0
AOSP LatinIME's Spanish and English (US) word lists, at commit 127336e9f29d69607eab55982324b210279ae8c5	Apache 2.0

The sentences were typed again with the mistakes of a hurried thumb on purpose, and the misspellings set in sentences that have the word meant; none of these texts is in the files here. A Spanish corrector was taught from the Spanish of these alone, and also the word a misspelling written by its sound means, made from the Spanish word list.

Licence

These correctors are Model Derivatives of Gemma (T5Gemma 2), fine-tuned and exported by Berpiztu: every file here was modified from Google's. Gemma is provided under and subject to the Gemma Terms of Use found at ai.google.dev/gemma/terms (a copy in GEMMA_TERMS_OF_USE.md, the notice in NOTICE). Use restrictions: they must not be used for the restricted uses in the Gemma Prohibited Use Policy (ai.google.dev/gemma/prohibited_use_policy, a copy in GEMMA_PROHIBITED_USE_POLICY.md); whoever downloads them is bound by those restrictions, and must pass them, the Terms and the notice on to anyone they give them to.