berpiztu/keyboard-correctors

Berpiztu AI Keyboard's context correctors

Fine-tuned from google/mt5-small by Berpiztu, and exported to ONNX.

Spanish and English sentence correctors for Berpiztu AI Keyboard (Android and Windows, github.com/Berpiztu/keyplus): each reads the sentence typed so far and writes it again mended (spelling, accents, punctuation, capitals, words a slip or a swipe turned into others), and offers the words to finish the one being typed. ONNX, read on the device by ONNX Runtime; nothing leaves the device.

A folder each, as the keyboard reads it (encoder.onnx, decoder.onnx, spiece.model, format.json); correctors.json says each file, its size and its SHA-256.

Corrector	Bits	Size	Errors removed (es / en)	Correct sentences made worse (es / en)
small-a-wiki	32	376 MB	66.2 % / 68.2 %	12 / 26

Measured on 2,000 held-out sentences of each language (the full-precision model; the 8-bit export writes the same in most sentences, not all).

Finished with Virtus

Each corrector's training ends with Virtus (AI alignment through character, by Berpiztu): one more pass in which its own answers are rewarded, never punished, for humility, honesty, temperance and diligence. What it brings: it leaves a sentence that was already right alone far more often, and removes as many errors or more. What it costs: a little in the words it suggests and in the Wikipedia misspellings it mends. The same training measured before and after (es / en):

Training	Correct sentences made worse	Errors removed	Suggestion among the three	Wikipedia misspellings mended
small-a-wiki	26 / 36 → 12 / 26	66.1 % / 67.3 % → 66.2 % / 68.2 %	62.5 % / 61.7 % → 61.2 % / 59.9 %	41.6 % / – → 30.8 % / –

Training data

The sentences and words the correctors were taught from, each under its own licence:

Source	Licence
Tatoeba: Spanish and English sentences	CC BY 2.0 FR
OpenSubtitles v2018 from OPUS (opensubtitles.org; P. Lison, J. Tiedemann and M. Kouylekov, LREC 2018): the first lines of the Spanish and English files	as OPUS gives it, with that citation
Wikipedia (wikimedia/wikipedia, 20231101.es and 20231101.en, one file each): sentences	CC BY-SA 3.0 and GFDL
The Spanish Wikipedia's common misspellings (Wikipedia:Corrector ortográfico/Listado, revision 175669831)	CC BY-SA 4.0
The English Wikipedia's (Wikipedia:Lists of common misspellings/For machines, revision 1199637275)	CC BY-SA 4.0
AOSP LatinIME's Spanish and English (US) word lists, at commit 127336e9f29d69607eab55982324b210279ae8c5	Apache 2.0

The sentences were typed again with the mistakes of a hurried thumb on purpose, and the misspellings set in sentences that have the word meant; none of these texts is in the files here.

Licence

Apache 2.0 (LICENSE), as google/mt5-small, which they were fine-tuned from by Berpiztu: every file here was modified from Google's.