FranJCastilloC/distilbert-pii-ner-es-en

DistilBERT multilingual — PII token classification (ES/EN)

Fine-tuned distilbert-base-multilingual-cased for detecting personally identifiable information in Spanish and English business documents. It is the learned component of the PII Detection & Redaction Pipeline, where it is combined with a checksum-backed rule engine and Presidio.

Entity types

10 BIO classes: PERSON, EMAIL, PHONE, ADDRESS, DATE_OF_BIRTH, GOV_ID, USERNAME, CREDENTIAL, IP_ADDRESS, DATE_TIME.

CREDIT_CARD and BANK_ACCOUNT are deliberately absent. The training corpus annotates a card issuer but contains no card numbers and no bank account numbers at all, so the model cannot learn them. In the parent pipeline those two classes are handled by the rule engine, where Luhn and the IBAN mod-97 checksum are arithmetic proofs rather than estimates.

Results

Validation (seqeval, entity level, 2,000 held-out samples):

	Precision	Recall	F1
micro	0.9236	0.9468	0.9350

Entity	Precision	Recall	F1	Support
EMAIL	0.976	0.991	0.984	669
GOV_ID	0.943	0.973	0.958	2536
ADDRESS	0.954	0.941	0.948	2864
PHONE	0.927	0.945	0.936	526
PERSON	0.887	0.953	0.919	1338
USERNAME	0.928	0.909	0.918	680
DATE_TIME	0.897	0.940	0.918	1450
DATE_OF_BIRTH	0.900	0.933	0.916	510
IP_ADDRESS	0.885	0.935	0.909	585
CREDENTIAL	0.809	0.851	0.830	403

On complete business documents (out-of-distribution, 250 synthetic invoices, contracts, emails, tickets and forms) F1 drops to 0.651, while the rule baseline reaches 0.712 and the full ensemble 0.765. Formal documents carry explicit field labels (Phone:, Name:) that regexes handle well and that the sentence-level training corpus never contained. Use this model as one component of an ensemble, not on its own.

Usage

from transformers import AutoModelForTokenClassification, AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained("FranJCastilloC/distilbert-pii-ner-es-en")
model = AutoModelForTokenClassification.from_pretrained("FranJCastilloC/distilbert-pii-ner-es-en")

For span decoding, sliding windows over long documents, per-span confidence and redaction, use the pipeline in the linked repository.

Training data

ai4privacy/pii-masking-300k, filtered to English and Spanish. The corpus is fully synthetic: no real person's data was used at any point. Character offsets were re-aligned to this model's own tokenisation rather than reusing the corpus's precomputed BIO tags.

Config: 15,057 training windows, batch 16, lr 5e-05, max length 192, 1 epoch.

Limitations

• Spanish and English only.
• Trained on synthetic sentences; real documents bring OCR noise, abbreviations and inconsistent layout that this evaluation does not capture.
• Degrades on document-style input relative to sentence-style input (see above).
• Should not be the only safeguard in a redaction system. The parent pipeline pairs it with deterministic rules, calibrated confidence scores and a human review queue for exactly this reason.