Keylime m2 (106M) — Preview
This repo has the Keylime on-device autocorrect language model. The files are in the pre-optimized ORT format of onnxruntime. The Keylime keyboard for Android and the Keylime app for macOS download these files when they start for the first time.
m2 is an autoregressive decoder language model with 106M parameters. It has a 32k token vocabulary and was trained with int4 QAT. The training data for m2 had 9B tokens with casual / standard / formal style tokens.
m2 is a preview model. It is not perfect, and can often be flaky, inconsistent, or make dumb mistakes. The reason why we have published this version of the model is to collect feedback and data regarding the model and how to better utilize it, as well as general public consensus of the model's quality.
The model can be used in our MacOS app or Android keyboard, which downloads the our Keylime models automatically. You can opt-in to a data sharing program that sends your typing data to our servers for future model training. The program sends ALL typing data, including the messages you send. If you care about your own privacy, it is not recommended to opt-in.
Style tokens: the context starts with [EOS, id]. The value of id is 249 for casual, 250 for standard and 251 for
formal.
The manifest id of m2 is m2s_int4 ("m2 106M int4"). The manifest has the file list, the step-graph layout and the
decode parameters. The manifest is at https://autocorrect.mostlime.hackclub.app/autocorrect-android/manifest.json.
A copy of the manifest is in the models-v1 release of
Mostlime12195/keylime-models.
Files
| file | bytes | sha256 | what it is |
|---|---|---|---|
m2s.step.int4m.ort |
78969984 | 28e79b14b67c25659a41ce6ac84fd498bce530310eb7ec6bff116ad2e003fe39 |
step graph (batched candidates), pre-optimized onnxruntime (ORT) format |
m2s.step.int4t.ort |
79105008 | 8a97ae16e68944f78e180c180dd3f18adf41b56dcadb21c4863e8e75b03c686f |
tree step graph (one call scores every candidate of a word), ORT format |
vocab_m2.json |
695095 | f328549e080df02ab6a57e3ecf87dc8dd6470b286516ba00f67a86f383d1c77a |
SentencePiece vocabulary |
LICENSE |
18659 | 5eebefd3682e32d661f307a28539b304e5e52fb1eb33ba214e61d7a57663e4f0 |
CC BY 4.0 license text |
To download a file, use this URL. Replace <file> with the name of the file.
https://huggingface.co/MostLime/keylime-m2-preview/resolve/main/<file>
Graph layout (manifest meta)
| key | value |
|---|---|
| params | 106,315,648 |
| d_model | 640 |
| vocab_size | 32768 |
| n_kv_heads | 2 |
| head_dim | 64 |
| conv_kernel | 7 |
| state_names | k0, v0, k1, v1, k2, v2, k3, v3, k4, v4, k5, v5, c0, c1, c2, c3, c4, c5, c6, c7, c8, c9 |
Each graph is a single-step decoder. Each graph has the KV cache (k*/v*) and the short-conv state (c*) as
inputs and outputs. Thus, the keyboard uses the shared left context again. The keyboard then scores only the candidate
words.
This repo has no code and no training data. It has only the exported weights and the vocabulary.
License
Copyright © Mostlime12195. The license for the model files is Creative Commons Attribution 4.0 International (CC BY 4.0).