adapterops-pii

Lists the personal information in a text as LABEL: value lines, over 19 labels.

Part of AdapterOps: four LoRA adapters over one Qwen2.5-1.5B base, served together with vLLM multi-LoRA. Portfolio project — no real users or customer data.

The scores below describe revision 5315b1dcd98663c2ce7292299cff2e511639480a (adapter weights sha256 66631c40fcc17994…), the revision the project serves. Load that revision rather than main.

Prompt

List every piece of personal information in the text, one per line, as LABEL: value.
Text: {text}
Found:

Raw text, no chat template. Greedy decoding, at most 384 new tokens. Replace {text} with the input.

Evaluation

Golden sets are frozen random held-out splits; every system below was run on the same items. The hard-cases split is mined from this adapter's own failures, so it is report-only and sits near zero by construction for classification.

system	split (n)	metric	score
this adapter	golden (300)	span_f1_strict	0.9421
previous adapter e0bde68f, same session	golden (300)	span_f1_strict	0.9421
base model, 5 demonstrations	golden (300)	span_f1_strict	0.5700
GPT-4o-mini (frontier reference)	golden	span_f1_strict	0.6663
this adapter	golden (300)	documents with all personal text masked	0.8700
this adapter	golden (300)	gold spans left wholly unmasked	0.0113
this adapter	PII-free texts (928)	texts with a reported span	4
previous adapter	PII-free texts (928)	texts with a reported span	5
this adapter	held-out PII-free sentences (491)	texts with a reported span	4
previous adapter	held-out PII-free sentences (491)	texts with a reported span	0
this adapter	hard cases (150), report-only	span_f1_strict	0.8686

Latency with all four adapters served at once on one A10 (vLLM, concurrency 16): P50 858 ms · P95 2,259 ms, measured with the previous revision.

Caveats

• Trained with PII-free sentences and empty answers so it can report nothing; on 928 PII-free texts it still reports a span in 4. The previous revision, trained only on documents containing PII, reported one in every text.
• Trained and evaluated on synthetic spans only. Not a compliance control.
• Strict scoring requires each value to match its span exactly.

Training

QLoRA (4-bit NF4) on Qwen/Qwen2.5-1.5B-Instruct, LoRA rank 16, alpha 32, on all attention and MLP projections; prompt tokens masked from the loss. 13,910 training rows from ai4privacy/pii-masking-openpii-1m (cc-by-4.0).

Full decision log, results and negative findings: https://github.com/tpawar03/AdapterOps.