luispoveda93/MiniCPM5-1B-PII-tagger-lora

MiniCPM5-1B PII Tagger (LoRA pilot)

A LoRA adapter on openbmb/MiniCPM5-1B that turns the model into a token-classification PII/PHI detector, trained on a 20K-example pilot of nvidia/Nemotron-PII (train split).

What it does

Given English text, the model predicts one of 55 PII/PHI category labels per token (IO scheme): first_name, last_name, date_of_birth, ssn, street_address, email, phone_number, credit_debit_card, blood_type, medical_record_number, api_key, password, ipv4, mac_address, and 40 more.

Label list (sorted): account_number, age, api_key, bank_routing_number, biometric_identifier, blood_type, certificate_license_number, city, company_name, coordinate, country, county, credit_debit_card, customer_id, cvv, date, date_of_birth, date_time, device_identifier, education_level, email, employee_id, employment_status, fax_number, first_name, gender, health_plan_beneficiary_number, http_cookie, ipv4, ipv6, language, last_name, license_plate, mac_address, medical_record_number, national_id, occupation, password, pin, political_view, postcode, race_ethnicity, religious_belief, sexuality, ssn, state, street_address, swift_bic, tax_id, time, unique_id, url, user_name, vehicle_identifier.

Results (pilot)

Evaluated on 2,000 held-out examples from the nvidia/Nemotron-PII test split (seed 42 shuffle). Metrics are exact token/span match against char-offset-derived gold labels; a span is a maximal run of consecutive tokens with the same non-O label.

Metric	Score
Token precision / recall / F1	0.9635 / 0.9626 / 0.9631
Span precision / recall / F1	0.8645 / 0.9046 / 0.8841
eval_loss	0.0409

After only 25 training steps (smoke test) the model already reached token F1 0.47, so most of the score was learned, not inherited.

Usage

import ast, torch
from transformers import AutoTokenizer, AutoModelForTokenClassification
from peft import PeftModel

base = AutoModelForTokenClassification.from_pretrained(
    "openbmb/MiniCPM5-1B",
    id2label={...},  # the 55-label mapping in this card
    label2id={...},
    dtype="bfloat16",
)
model = PeftModel.from_pretrained(base, "luispoveda93/MiniCPM5-1B-PII-tagger-lora")
tok = AutoTokenizer.from_pretrained("luispoveda93/MiniCPM5-1B-PII-tagger-lora")

text = "My date of birth is 1987-05-22 and I live at 87 Avenida De La Estrella."
enc = tok(text, return_tensors="pt", return_offsets_mapping=True, truncation=True, max_length=1024)
offsets = enc.pop("offset_mapping")[0].tolist()
with torch.no_grad():
    preds = model(**enc).logits.argmax(-1)[0].tolist()
for (s, e), p in zip(offsets, preds):
    if p != 0 and not (s == 0 and e == 0):
        print(text[s:e], "->", id2label[p])

Adjacent predicted tokens with the same label form one entity span.

Training details

• Base: openbmb/MiniCPM5-1B (LlamaForTokenClassification head, 55 labels, bf16, SDPA attention)
• Data: first 20,000 examples of nvidia/Nemotron-PII default/train after a seed-42 shuffle; spans mapped to tokens via fast-tokenizer offset_mapping (IO scheme, -100 on special tokens); max_length=1024 (only ~0.07% of spans lost to truncation)
• LoRA: r=16, alpha=32, dropout=0.1, target modules q/k/v/o/gate/up/down_proj — 11.3M trainable params (1.27%)
• Hyperparameters: lr 2e-4 (cosine, no warmup), effective batch 32 (8 × grad-accum 4), 1 epoch (625 optimizer steps), bf16, seed 42
• Hardware: 1× A10G (24 GB), peak GPU memory 22.4 GB, ~38 min wall-clock

Known data caveats (from nvidia/Nemotron-PII)

• ~1.5% of spans have a char-level mismatch between spans[i].text and text[start:end] in the source dataset (measured over 182,169 spans in the pilot sample). Offsets were treated as ground truth for labeling.
• The label vocabulary was built from the pilot sample; the full 100K train split may contain categories this model has not seen.

Limitations

• Pilot scale (20K of 100K train rows), single epoch, no per-label breakdown — rare categories likely underperform.
• Synthetic, persona-grounded text only; real-world documents may shift.
• Token-level boundaries are exact-match; adjacent same-label spans merge into one prediction.

Job provenance

Trained with HF Jobs (pii-tagger-pilot-20k-minicpm5-a10g-v2-push, job 6aa0653d32d5d0c22c5ae52c), A10G-small, 2026-09-08.