mmbert-small-pii-redaction

Tags the words of personal details in a message: names, postal addresses, dates of birth, e-mail addresses, phone numbers, IBANs, payment cards and national ID numbers, in English, Polish, German, French, Spanish and Lithuanian. The first 12 of mmBERT-small's 22 layers, fine-tuned with BIO tags over words; ONNX weight-only int8 for transformers.js, 36 MB plus a 1.6 MB tokenizer.

It is meant to run together with a rule layer for identifiers with check digits (IBAN, cards, PESEL and other national IDs), as the widget on balticml.eu does. The model labels words, not tokens: a word is a run of letters and digits or one other character, tokenized with the whitespace before it. Use the code in the repo (github.com/balticml/pii-redaction, web/model.js and src/pii_redaction/predict.py) rather than the generic pipeline, which tokenizes the whole text at once.

Results

60 hand-written messages, 10 per language, 172 details; "caught" means every letter and digit of a detail is under a bar.

	caught	bars on a detail	latency
this model + rules, int8 with the pruned tokenizer (what the widget runs)	98.3	91.4	18 ms (ORT CPU, 4 threads)
this model + rules, fp32	98.3	94.4
rules alone	39.0	100.0
Qwen 2.5 7B instruct (int4, Ollama), asked for a JSON list of details	94.8	94.8	498 ms (GPU)

The test messages were written by the person who wrote the prompts for the training data, so the result is an upper bound. Tables per language and per kind are in the repo's README.

Training data

No real personal data. A local LLM (gpt-oss-20b) wrote 2,074 messages (support e-mails, chat lines, CRM notes, contract excerpts, forms) with placeholders where the details go; each was filled twelve times with made-up names, addresses and identifiers with valid check digits, 17,914 training texts. Messages in which the LLM wrote out a detail itself were rejected.

Limits

Six languages tested. Company names, customer and order numbers, licence plates and health details are not tagged; customer numbers are sometimes tagged as IDs. Missing a detail is possible: review redacted text before sending anything sensitive.