flisak-anon-en 0.3.0

flisak-anon-en 0.3.0 is Kurdzik/flisak-anon-en 0.1.0 with two labels added to its token-classification head, B-ADDRESS and I-ADDRESS, so that the anonymisation service can replace a postal address as well as a person's name. Its top four encoder layers (8-11) and the head were fine-tuned, with 0.1.0's own person outputs as a teacher; the embeddings, layers 0-7, the labels O, B-PER, I-PER and the tokenizer are 0.1.0's, byte for byte.

Status: staged, not released. It passes every line of its release gate (below, Evaluation), two of them only just: address recall on OpenPII-1M's English validation split (0.9519 against 0.95) and the name leak on the synthetic mail (47 -> 53 characters, inside the paired bootstrap's interval, whose low end is 0). It was developed for the anonymisation feature of Flisak, a document-management system.

Model details

Developed by	Mateusz Kurdziel
Version	0.3.0 (0.2.0, the head rows alone, was staged and not released)
Architecture	RobertaForTokenClassification (RoBERTa base: 12 layers, hidden size 768, 125 M parameters), 5 labels
Base model	flisak-anon-en 0.1.0 (a fine-tune of FacebookAI/roberta-base, MIT)
Tokenizer	roberta-base's byte-level BPE, unchanged
Language	English
Licence	CC BY 4.0

Labels

O, B-PER, I-PER (PERSON), B-ADDRESS, I-ADDRESS (ADDRESS). An address is any postal address with a house number or a postal code: street, number, flat, postcode and town written together (221B Baker Street, London NW1 6XE), a person's or a company's, as the text rarely says whose it is. A street or a town alone stays readable.

Intended use

The name and address stage of the anonymisation service (flisak_anon.server with ANON_LANGUAGE=en, API contract 1.3, code 0.3.0): the regex stage first, then pipeline("ner", aggregation_strategy="simple") in windows of at most 512 tokens sharing 64, entities above 0.4 kept. A request asking for ADDRESS gets <ADDRESS_n>; one not asking for it runs the model with the two address rows switched off and keeps every address readable. Out of scope: a guarantee of anonymity, identifying or tracking people, other languages (Polish: flisak-anon).

Training

Two steps, each checkpoint picked on validation sets only. First (0.2.0) the head's two new rows, initialised at zero, trained over the frozen encoder (5,000 steps). Then encoder layers 8-11 and the whole head (lr 2e-5, batch 32, warm-up 6%, weight decay 0.01, 256-token windows; early stopping ended it at step 2,500 of 7,078, 1,131 s on one GPU), the loss on the gold labels plus the KL divergence from 0.1.0's own distribution over its three labels on every labelled token, on: synthetic English mail with address blocks and confusable non-addresses (20,000 mails), OpenPII-1M's English train records (20,000) and Nemotron-PII's 20,000 training records (their address parts grouped into addresses, persons given no weight), MultiCoNER v2, WNUT-17 and Broad Twitter Corpus train text with their persons, and 0.1.0's synthetic English mail train set. Every text went through the regex stage first. The checkpoint (step 500 of evals every 500) was the one with no failed guard (names leaked and F1 on three English validation sets, with every feature and with names alone, addresses marked in real text), then the highest worst-set address recall.

Evaluation (test sets, read once after the pick; through the service's pipeline, 0.1.0 beside it)

Set	Person F1 0.1.0 / 0.3.0	Name characters leaked, every feature	PERSON alone
synthetic mail test v3 (2,000)	0.9901 / 0.9904	47 / 53	47 / 53
UNER English-EWT test	0.8916 / 0.8939	284 / 268	284 / 268
Nemotron-PII test (1,000)	0.9898 / 0.9898	22 / 22	33 / 33
WNUT-17 test	0.6028 / 0.6037	2,000 / 2,000	2,000 / 2,000
Broad Twitter Corpus f	0.8409 / 0.8418	574 / 560	574 / 561

Address set	Recall (address words in an ADDRESS span)	Other words marked
synthetic address mail test (2,000 mails, held-out templates, streets and towns)	0.9934 (13,579 / 13,669)	0.019%
OpenPII-1M English validation (2,000 records)	0.9519 (3,623 / 3,806; 0.95 needs 3,616)	0.26%
Nemotron-PII test (1,000 records)	0.9897 (770 / 778)	0.075%
UNER English-EWT test (no addresses)		0.37%

The regex features and phone numbers answer exactly as 0.1.0 (payment cards 3,000 of 3,000). CPU, 2 cores, 200 requests of one 2,000-character text: p50 1.058 s, p95 1.171 s, peak RSS 1,278 MB (0.1.0: 1.104 s, 1.172 s, 1,275 MB).

Limitations

• OpenPII's free-form addresses pass the 0.95 bar by 7 words: one address word in 21 stays readable there. On the 53 validation records sharing half their phrasing with OpenPII training records it finds every address word, and without them the recall is 0.9495; a document bootstrap puts it between 0.942 and 0.960, and the validation records used to pick the checkpoint read 0.940.
• Marking addresses costs a little elsewhere: 0.47% of the synthetic mail's other words and 0.37% of EWT's are marked ADDRESS when it is asked for.
• Everything 0.1.0's card says about names holds.

Files

pytorch_model.bin and model.safetensors (the same tensors), config.json (0.1.0's with the two labels), the tokenizer files (0.1.0's, byte-identical), LICENSE, ATTRIBUTION.md, manifest.json (the sha256 of every file, the tensors that moved and the training record; the service refuses to start unless the directory holds exactly these files).

Licence and attribution

CC BY 4.0. Changes made to flisak-anon-en 0.1.0: two head rows added, encoder layers 8-11 and the head fine-tuned. Credits for RoBERTa base (MIT, Copyright (c) Facebook, Inc. and its affiliates), Nemotron-PII, MultiCoNER v2, WNUT-17, the Broad Twitter Corpus, UNER English-EWT, OpenPII-1M (Ai4Privacy / Ai Suisse SA) and Faker are in ATTRIBUTION.md. The original authors do not endorse this model.