gliner2-sa-names-lora

A LoRA fine-tune of fastino/gliner2-privacy-filter-PII-multi (GLiNER2, 205M params) that fixes a measured recall gap on South African names in cue-free, label-style text — e.g. address/attention lines and email signatures, where the model previously had little contextual signal to lean on beyond the token itself.

This is the model behind privacy-ext, an on-device PII redactor for web forms. The deployed variant (onnx_int8/) is what privacy-ext's Rust inference engine (gliner2-rs) actually runs.

Problem this fixes

The base PII model was trained mostly on cue-rich text ("My name is ...", "Contact: ..."). On South African names appearing in address/attn lines or email signatures — with no explicit naming cue — recall dropped sharply, especially for isiZulu, isiXhosa, Sesotho, Setswana, Tshivenda, and Xitsonga names.

Training

• Base checkpoint: fastino/gliner2-privacy-filter-PII-multi (the trainable PyTorch checkpoint the deployed ONNX model was originally exported from — not the bare gliner2-base-v1, to preserve existing PII tuning).
• Method: LoRA, lora_target_modules=["encoder"] only (r=16, alpha=32, dropout=0.1). 0.86% of parameters trainable (2.65M / 309.75M).
• Data: ~2,500 templated examples built from real name/place pools sourced from nwu-ctext (CC-BY 2.5 SA) and MphayaNER (Apache 2.0), covering 7 label types (name, street address, email, phone_num, id_num, url, username) across 7 context styles (address_attn, email_sig, form_field, narrative_no_cue, cue_explicit, billing, third_person), weighted toward the address_attn/email_sig failure modes while keeping the other contexts well-represented.
• Schedule: batch_size=16, task_lr=5e-4, up to 15 epochs, early stopping (patience=3). Best checkpoint at epoch 3 (eval_loss=0.238); training halted at epoch 7.

Regression eval

266-case eval spanning 6 South African name groups (isiZulu, isiXhosa, Sesotho/Setswana, Afrikaans, Tshivenda/Xitsonga, Western control) x 7 contexts, 1,491 total name instances. Recall = a name span overlapping the expected span was returned by the model.

	Baseline (shipped)	Fine-tuned (fp16)	Fine-tuned + INT8 encoder
address_attn recall	48.4%	100.0%	98.1%
email_sig recall	58.7%	99.5%	99.1%
Overall recall	—	99.7% (1486/1491)	99.5% (1484/1491)
Exact-span match	—	1479/1491	1458/1491
isiZulu recall	76.4%	100.0%	98.2%
isiXhosa recall	80.7%	100.0%	100.0%
Sesotho/Setswana recall	85.0%	100.0%	99.6%
Afrikaans recall	85.4%	98.2%	99.6%
Tshivenda/Xitsonga recall	93.4%	100.0%	100.0%
Western control recall	97.1%	100.0%	100.0%

No material regression on the other 6 contexts (form_field, narrative_no_cue, cue_explicit, billing, third_person), which were already at 97-100% on the baseline.

INT8 quantization (dynamic, weight-only, encoder fragment only, reduce_range=True) trades a small amount of exact-span precision for a 39% smaller encoder (556 MB → 337 MB fp16 vs int8) and ~2x throughput (1.88 → 3.66 req/s, mean latency 532ms → 273ms, single-request, CPU, batch size 1) — measured against the running pii-server daemon on an 8-core dev machine. Overall recall is unchanged (99.5%, same as without reduce_range).

reduce_range=True matters for correctness, not just accuracy: the first int8 export (default 8-bit weight range) passed every local test but reproducibly returned zero entities on GitHub Actions runners lacking AVX-VNNI — a documented ONNX Runtime caveat where u8s8 quantized matmul can overflow the accumulator on non-VNNI x86 CPUs. reduce_range=True quantizes weights to 7 bits instead of 8, avoiding the overflow at negligible cost. Confirmed stable across Intel Xeon Platinum 8370C, AMD EPYC 9V74, and AMD EPYC 7763 CI runners after the fix. If you re-quantize this model yourself, keep reduce_range=True unless you can guarantee every deployment target has AVX-VNNI (most current-gen server/desktop CPUs do; many older or low-power CPUs don't).

Files

• config.json, encoder_config/, model.safetensors, tokenizer* — merged PyTorch checkpoint (LoRA weights already merged into the base). Load with:

  from gliner2 import GLiNER2
  model = GLiNER2.from_pretrained("stefanj0/gliner2-sa-names-lora")
  

• onnx_int8/ — the 8-fragment ONNX split (encoder, token_gather, span_rep, schema_gather, count_pred_argmax, count_lstm_fixed, scorer, classifier) that gliner2-rs expects, exported with the fragment splitter vendored in the gliner2-sa-names-finetune repo. Note: despite the _fp16.onnx filenames (required by gliner2-rs's autodetection, which keys off this suffix), encoder_fp16.onnx in this folder is dynamically INT8-quantized (weight-only, onnxruntime.quantization.quantize_dynamic) — the other 7 fragments are fp16 as named. This is the exact fragment set currently deployed in privacy-ext.

To run with gliner2-rs / privacy-ext:

PII_MODELS_DIR=<path-to-onnx_int8-dir> PII_TOKEN=<secret> ./pii-server

Limitations

• LoRA was applied to the encoder only; span-boundary/classification behavior is unchanged from the base checkpoint, so entity types other than name were not targeted by this fine-tune (though the eval shows no regression on them).
• Training data is templated/synthetic (real name pools, synthetic sentence templates), not naturally-occurring text — may not generalize to name contexts/templates far outside the 7 covered here.
• Evaluated on 266 constructed cases; no held-out real-world corpus eval yet.

License

Apache 2.0, inherited from the base fastino/gliner2-privacy-filter-PII-multi checkpoint and the Apache/CC-BY-2.5-SA training data sources.