openmasq/ner-multilingual

On-device multilingual NER for OpenMasq, a privacy tool that redacts personal data before a prompt leaves the machine. These weights run in the desktop app and in the local proxy, on CPU, offline.

A 6-layer student of Davlan/bert-base-multilingual-cased-ner-hrl, started from the Apache-2.0 mBERT base, with the wordpiece vocabulary reduced to Latin script: 97.2 M parameters, 94 MB as int8 ONNX — against 177.3 M and 178 MB for the model it imitates, for the same score on held-out text and a little more on the product's own cases.

	Davlan (teacher)	this model
encoder blocks	12	6
vocabulary (wordpieces)	119 547	70 606
parameters	177.3 M	97.2 M
ONNX int8	178 MB	94 MB
CPU latency, 419 characters (M1, int8)	59.5 ms	32.4 ms
weights derived from the teacher	—	none

Labels

The teacher's tag set, unchanged — O, B-/I-PER, B-/I-ORG, B-/I-LOC, B-/I-DATE. A drop-in replacement: same id2label, same pipeline.

How it was made

The student is initialised from google-bert/bert-base-multilingual-cased with a fresh classification head, keeps 6 of its 12 blocks (0, 2, 4, 7, 9, 11 — evenly spaced, the XtremeDistil shape) and every Latin-script wordpiece. It is then trained with three terms:

loss = 0.7 · KL( student/T ‖ teacher/T ) · T²        T = 2, on every real token
     + 0.3 · CE( student, gold )                      on the 5 % of cases that carry gold
     + 1.0 · MSE( hidden_s[i], hidden_t[kept[i]] )    on the 7 mapped layers, embeddings included

• KL against the teacher on every token: the student copies Davlan's full distribution, hesitations included. This is what preserves everything the teacher does.
• Cross-entropy on gold where a source claims labels: 540 documents of the UNER English-EWT train split (human) and 3 000 prose documents annotated by Claude through a scored bench column (0.900 F1 against UNER's human gold, 96 % recall on organisations). A token nothing annotated is imitated, not scored as O; a token the teacher reads as a date is left to the KL term, since no source annotates dates.
• Hidden-state transfer on the mapping the block selection actually made (Udagawa et al. 2023 place it above output-distribution transfer alone). Measured: +0.002 to +0.003 F1 on every corpus over the same recipe without it.

Corpus: 65 114 documents — Wikipedia prose in six languages, synthetic records, generated credential carriers — plus 20 % shown a second time in capitals and 20 % in lower case (Mayhew et al. 2019). 3 epochs, 21 195 steps, lr 5e-5, batch 16, windows of 256 with stride 48. Final agreement with the teacher on held-out windows: 0.957 F1.

Why the pruned vocabulary is safe on Latin text

WordPiece is greedy longest-match, so a word whose pieces all survive the cut splits exactly as before; keeping every Latin-script piece makes that true of any Latin-script word. Verified rather than assumed: the teacher itself with the pruned vocabulary and no training returns predictions identical to the original on 127 of 127 documents of the Text Anonymization Benchmark.

Measured

Character-level F1 on the categories the product exposes, in the product (rules + model, the level a user gets). TAB (European Court of Human Rights judgments) and UNER (the English Web Treebank's test and validation splits: e-mails, newsgroups, reviews, weblogs) are real text held out from everything either model was trained on; the other three are the public PII benchmarks' scoring splits.

corpus	documents	previous model (Davlan)	this model
ai4privacy	2 000	0.827	0.825
Gretel	2 000	0.646	0.646
Nemotron	2 000	0.928	0.938
TAB — real, held out	127	0.862	0.861
OpenMasq internal	907	0.923	0.925
UNER English-EWT — real, held out	634	0.674	0.682

Three corpora up, two down by a thousandth, one level. Where precision was measured separately (TAB, internal, UNER) it is 0.001 to 0.017 above the teacher's, and the gains are precision rather than recall — the supervision taught the student where entities are not more than where they are, which is what 91 % of the supervised tokens say.

Limitations — read this before deploying

• Non-Latin scripts are gone. Chinese, Arabic, Cyrillic, Japanese, Korean, Hebrew, Bengali and every other non-Latin writing system were removed with the vocabulary; their characters become [UNK] and entities in them are not detected at all. The teacher covered ten languages including Chinese and Arabic. A deliberate trade for a product serving six European languages — disqualifying for anything else.
• Brand knowledge is weaker than the teacher's. Well-formed company names it has not seen (Royal Caribbean, Hobby Lobby, Travelocity, BBC) are recognised less often: 70 % of organisation characters on UNER against the teacher's 74 %. Three training runs with five times more supervised organisations did not move that figure — it looks like a property of the 2018 pre-training, not of the corpus. In OpenMasq a rules-side lexicon is the planned answer; on its own, this model is not the one to pick for entity linking over web text.
• Quantisation is part of the measurement. Every figure above is the int8 ONNX graph, which is what ships — not the fp32 checkpoint.
• It predicts DATE, which OpenMasq maps to an opt-in category; ignore that tag if you do not want it.

Files

file	what reads it
onnx/model_quantized.onnx	onnxruntime, in the desktop app and the local proxy
config.json	the label set
tokenizer.json, tokenizer_config.json	the pruned wordpiece tokenizer
vocab.txt	the same vocabulary for the slow tokenizer

OpenMasq pins each of the first four by sha256 and verifies them before onnxruntime parses a byte, so this repository is the origin, never the trust anchor: a change here can fail a build, it cannot substitute bytes.

onnx/model_quantized.onnx  7470819a0961b246614ceedd940934b8d8ae7c39070cb15508883fe0e6f93012
config.json                b942516293db81db39bc12b734888dc5e7a7e38c2f5f6f7ecacb337fd716c4c3
tokenizer.json             01c1e79cf5f607729b4e64af721b961d41981c77c91849a9fd4f7262b4cc9437
tokenizer_config.json      5c1d1f0110ec8761dc91247fb20bebfea28aa577e35b2f0eb02c55cfb69871b1

Licence

Apache-2.0. The weights are initialised from google-bert/bert-base-multilingual-cased (Apache-2.0) and contain no parameter of any other model. They were trained to imitate the outputs of Davlan/bert-base-multilingual-cased-ner-hrl (AFL-3.0); whether learning from a model's answers makes a derivative of that model is a contested question that this card does not settle — it states the provenance so the reader can judge. The earlier release in this repository (commit fa87beb) was initialised from Davlan's weights and was declared AFL-3.0 for that reason.

Training data, for the record: WikiNER (CC-BY 4.0), Universal NER English-EWT train split (CC-BY-SA 4.0), ai4privacy PII-Masking-300k, Nemotron-PII and Gretel PII Finance under their respective terms, and synthetic credential carriers generated in-house. TAB and UNER's scoring splits were used for evaluation only.

Citation

• Mukherjee & Hassan Awadallah, XtremeDistil, ACL 2020 — the shallow-student shape.
• Hinton, Vinyals & Dean, Distilling the Knowledge in a Neural Network, 2015 — the KL term.
• Udagawa et al., A Comparative Analysis of Task-Agnostic Distillation Methods, 2023 — hidden-state transfer over output transfer alone.
• Mayhew, Tsygankova & Roth, ner and pos when nothing is capitalized, EMNLP 2019 — the casing augmentation.
• Monaikul et al., 2021; Zhang et al., 2023 — the old model as teacher in the loss, and the semantic shift of the non-entity type it prevents.
• Adelani et al. / Davlan — the teacher. Mayhew et al., Universal NER, 2024; Pilán et al., The Text Anonymization Benchmark, CL 2022 — the held-out corpora.