plingampally/meridianpii-hi-v1

MeridianPII — Hindi / Hinglish PII Detection (v1)

On-device PII (personally identifiable information) detection for Hindi (Devanagari), Hinglish (romanized/code-switched), and Indian-English text. Fine-tuned from microsoft/Multilingual-MiniLM-L12-H384, vocabulary-trimmed to a 56 MB INT8 ONNX that runs in the browser via transformers.js.

It extends the architecture of Rampart (National Design Studio, CC BY 4.0) to Indic scripts. Rampart's tokenizer contains 70 Devanagari pieces out of 19,730 and cannot represent Hindi; this model is built on a genuinely multilingual base to close that gap.

Designed to run alongside a deterministic recognizer layer. Structured IDs with checksums/fixed formats — Aadhaar (Verhoeff), PAN, IFSC, emails, URLs — are handled by regex+validation, not the model, and are premasked during training so the model spends its capacity on what rules can't do: names, addresses, cities, phones in free-form, code-switched text. See "Recommended pipeline" below.

Evaluation

Span-level, on a frozen 2,000-row test set (1,000 Devanagari · 625 Hinglish · 375 Indian-English), template-disjoint from all training data. Recall is the headline metric for a PII tool — a missed entity is a leak.

Redaction metrics (private-term recall = "did any PII leak?")

Metric	Overall	Hindi	Hinglish	EN-India
Private-term recall	99.6%	99.9%	99.2%	99.5%
Public-term retention (keep-set CITY/STATE/ZIP)	99.2%	99.7%	97.5%	99.7%
Span-F1 (relaxed IoU≥0.5)	0.990	—	—	—
Span-F1 (strict IoU=1.0)	0.973	—	—	—

For comparison, on the same set the English-vocab Rampart baseline scores 62.6% overall / 50.0% Hindi private-term recall (2,242 leaks vs this model's 24).

Per-entity detection (entity-level P/R/F1, IoU≥0.5)

Entity	Precision	Recall	F1	Support
GIVEN_NAME	0.950	0.985	0.967	1500
SURNAME	0.985	0.967	0.976	1050
PHONE	0.981	0.988	0.985	1025
STREET_NAME	0.982	0.986	0.984	550
BUILDING_NUMBER	0.992	0.977	0.985	525
SECONDARY_ADDRESS	0.976	0.987	0.981	375
ZIP_CODE (PIN)	1.000	0.997	0.999	350
BANK_ACCOUNT	1.000	0.891	0.943	350
CITY	0.652	0.798	0.718	675
GOVERNMENT_ID (voter-ID)	0.980	0.960	0.970	150
PASSPORT	0.973	1.000	0.986	250
DRIVERS_LICENSE	0.995	0.884	0.936	225
micro avg	0.932	0.915	0.923	7350

Per-language micro-F1: Hindi 0.926 · Hinglish 0.933 · EN-India 0.906.

STATE is omitted from the table: the model detects state names (198/200 in a probe) but labels them CITY — see Limitations. Both are keep-set, so this has no effect on redaction.

Out-of-distribution check (human-written text)

Because training data is synthetically generated, we also evaluate on HiNER — human-annotated Hindi from Wikipedia, an entirely different distribution (NER, so person-names only):

Person-name recall (HiNER, human Hindi)
MeridianPII v1	79.7% (216/271)
Rampart baseline	0.7% (2/271)

The model generalizes to real Hindi names it never saw. (HiNER LOCATION spans countries/rivers/regions and is not comparable to this model's Indian-city/state PII scope.)

Recommended pipeline

This model is the contextual half of a two-layer redactor. Ship it with the deterministic layer:

1. Deterministic recognizers (run first, mask before the model): Aadhaar (12-digit + Verhoeff checksum), PAN ([A-Z]{5}[0-9]{4}[A-Z], holder-type char), IFSC ([A-Z]{4}0[A-Z0-9]{6}), +91 phones, emails, URLs. These have exact structure — a checksum beats a learned tagger on the highest-stakes IDs, with near-zero false positives. Reference implementation (JS + Python): github.com/plingampally/meridianpii.
2. This model: everything contextual — names (both scripts), street/building/secondary address, city/state, phones, bank accounts, voter-ID, passport, driver's license.
3. Policy: CITY / STATE / ZIP_CODE are a keep-set (detected but retained, not redacted) so downstream assistants keep coarse geography. Tune per your compliance needs.

Usage

transformers.js (browser / Node, on-device):

import { pipeline } from '@huggingface/transformers';
const pipe = await pipeline('token-classification', 'plingampally/meridianpii-hi-v1', { dtype: 'q8' });
const out = await pipe('मेरा नाम प्रिया शर्मा है, फ़ोन 9876543210', { aggregation_strategy: 'simple' });
// -> spans with entity/word/start/end; merge B-/I- with aggregation_strategy:'simple'

Python (transformers):

from transformers import pipeline
nlp = pipeline("token-classification", model="plingampally/meridianpii-hi-v1",
               aggregation_strategy="simple")
print(nlp("मेरा नाम प्रिया शर्मा है, पता 12 MG Road Bengaluru 560001"))

Use aggregation_strategy="simple" — raw BIO subword tags fragment names (प्रि+##या) and trip people up. The ONNX file is onnx/model_quantized.onnx (INT8).

Decision threshold

A confidence floor of 0.15 was calibrated on a held-out dev split (recall-biased: it maximizes public-term retention while keeping recall ≥0.985). INT8 quantization flattens softmax scores, so the borrowed 0.4 floor from Rampart is wrong for this model. For compliance use, keep the floor low (≤0.15) and bias toward recall — a false positive is a redacted non-secret; a false negative is a leak.

Intended use & limitations

Intended use: local-first redaction of Hindi/Hinglish/Indian-English text before it leaves a device (e.g. scrubbing chat input before it reaches a hosted LLM). Recall-biased by design.

Limitations / known failure modes:

• CITY vs STATE are frequently interchanged (both geographic); no redaction impact (both keep-set), but don't rely on the distinction as a label.
• Non-Indian geography (countries, rivers, world cities) is out of scope — the model targets Indian city/state names in PII contexts.
• Structured IDs (Aadhaar/PAN/IFSC/email/URL) are not detected by the model by design — they are premasked and must be handled by the deterministic layer. Without it, those classes will be missed.
• Romanized names colliding with common English words (a person literally named "Chicken"/"College") can be missed or over-fired; the training name pool carries a small amount of such noise.
• Very long, fragmented addresses split across a message and dense code-switch boundaries are the hardest cases.
• Scope is Indian locales by design — this is a locale pack. Route European text to Rampart (or a European pack), Indian text here. We deliberately don't claim European-language performance; keeping each model small and focused is the point.
• Test set is synthetic (though OOD-validated on HiNER); treat numbers as strong evidence, not a substitute for evaluation on your own data.

Does not detect: medical/health identifiers, biometric data beyond the listed IDs, dates of birth (treated as non-PII, per the keep policy), or non-Indian national IDs.

Labels (35 BIO tags = O + B-/I- × 17 types)

GIVEN_NAME, SURNAME, EMAIL, PHONE, URL, TAX_ID, BANK_ACCOUNT, ROUTING_NUMBER, GOVERNMENT_ID, PASSPORT, DRIVERS_LICENSE, BUILDING_NUMBER, STREET_NAME, SECONDARY_ADDRESS, CITY, STATE, ZIP_CODE

(EMAIL / URL / TAX_ID / ROUTING_NUMBER are present in the schema but premasked in training — handled by the deterministic layer at serve time.)

Training

• Base: microsoft/Multilingual-MiniLM-L12-H384 (Apache 2.0), vocabulary trimmed 250,002 → 96,029 pieces (Latin + Devanagari + digits/punct), 100% output parity vs the full-vocab model.
• Data (~40k rows): 40% Hindi, 25% Hinglish, 15% Indian-English (synthetic, template expansion over Indian name/geo/format fillers) + 20% ai4privacy openpii-1.5m (CC BY 4.0) anti-forgetting slice. NFC normalization (never NFKD — it strips Devanagari matras).
• Recipe: 2 epochs, fp16, single T4; best checkpoint by dev span-F1. Deterministic classes premasked (train/serve symmetry, per the Rampart whitepaper).

License & attribution

CC BY 4.0. Derives from Rampart (National Design Studio, CC BY 4.0) — architecture, label schema, deterministic-layer and premasking methodology. Base model microsoft/Multilingual-MiniLM-L12-H384 (Apache 2.0). Anti-forgetting data from ai4privacy (CC BY 4.0).