bekko-embedding-v1-a25m, Q8_0 GGUF for ChatOKF

The memory encoder of ChatOKF, a pure-Swift chat app for macOS and iOS that keeps the user's notes on the device and searches them before every reply. One file:

file	bytes	sha256
bekko-embedding-v1-a25m-Q8_0.gguf	137,000,480	b077658d019ee0a8cb036f0a895ffbdf8fcd8f89836a2aa4736d49f8361c8c31

Lineage

• The model is hotchpotch/bekko-embedding-v1-a25m by Yuichi Tateno, MIT: a 25M-parameter ModernBERT (mmBERT lineage) with the Gemma-2 vocabulary of 256,000 pieces, 13 blocks, 384-d mean-pooled vectors, no query or passage prefixes, context 8192.
• The author converted it to GGUF as hotchpotch/bekko-embedding-v1-a25m-GGUF with llama.cpp 0ef6e55e (F16, BF16 and Q8_0).
• This file is that F16 requantized to Q8_0 by quantize.py, kept beside this file, over llama.cpp's gguf-py: the token table and the transformer at Q8_0, every norm f32, general.quantized_by naming the script. The tensor bytes are the author's Q8_0; the metadata carries the lineage.

Why Q8_0

Measured against the F16 file over the 157 abstracts of a sample note corpus, "same-nn" being the concepts whose nearest other concept is the one F16 finds:

table	layers	MB	cosine to f32 Python	corpus mean	same-nn
Q4_0	Q4_0	75.4	0.9572	0.9588	130/157
Q4_0	Q8_0	87.8	0.9946	0.9982	149/157
Q4_1	Q8_0	94.0	0.9969	0.9985	152/157
Q5_0	Q8_0	100.1	0.9992	0.9995	155/157
Q8_0	Q8_0	137.0	0.9999	0.9998	157/157
F16	F16	252.5	1.0000	1.0000	157/157

A 4-bit token table moves the nearest neighbour of one note in twenty; Q8_0 moves none.

How ChatOKF uses it

Queries and notes are embedded with no prefix and gated on the pooled cosine of the best hit at 0.42, a floor cut over 222 labelled questions across five note corpora (nine errors, against eleven for multilingual-e5-small under its MaxSim floor). Questions asked in ten languages other than the notes' English clear that gate 57 times in 66 where the e5 floor let 29 of 67 through. The Swift port reads this file with its own ModernBERT forward and a SentencePiece BPE tokenizer that matches the Hugging Face tokenizer id for id; its vectors agree with the f32 PyTorch model to a cosine of 0.9998.

Tokenizer notes for other GGUF readers

The vocabulary carries scores and no merge list, so merges go by score (SentencePiece BPE). The Hugging Face tokenizer prepends the U+2581 marker to every chunk, splits the text at the control and user-defined pieces first (the runs of newlines are such pieces), and never matches its space-run pieces, which it spells with markers while this file spells them with spaces. A reader that reproduces those three facts matches it byte for byte.