IndoPII Mini
IndoPII Mini detects and redacts Indonesian personal data. Coverage follows UU No. 27/2022 on personal data protection (Pelindungan Data Pribadi, UU PDP). It runs entirely on CPU and needs 8β18 ms for a ~1,000-character document on a laptop, with no GPU and no external API.
It is a hybrid system:
- A rule engine handles structured identifiers. It validates them instead of just pattern-matching:
- NIK and KK region codes and dates of birth;
- NPWP in both the 15- and 16-digit formats;
- Luhn checks for card numbers;
- operator prefixes for phone numbers;
- NIP structure;
- context keywords.
- A key-value form parser handles KTP/KK OCR dumps and filled-in forms.
- A 4-layer MiniLMv2 token tagger (19.6M parameters, INT8 ONNX, 27 MB) handles contextual personal data: names, addresses, birth places, gender, religion, marital status, and identifiers that appear without keywords.
| Developed by | AIbnuHibban |
| Language | Indonesian (id) |
| Labels | 21 personal-data types (below) |
| Model | 4-layer MiniLMv2 (H=384), distilled from an XLM-R-base teacher, pruned 32k vocabulary, INT8 ONNX |
| Runtime | onnxruntime + tokenizers on CPU (no PyTorch) |
| Accurate variant | AIbnuHibban/indopii-base |
| Benchmark | AIbnuHibban/indopii-bench |
| Demo | IndoPII Space |
| License | Apache-2.0 |
Quick start
pip install "git+https://huggingface.co/AIbnuHibban/indopii-mini"
from indopii import Detector
det = Detector(model="mini") # downloads this repository on first use; model="base" for the accurate variant
text = ("Halo kak, saya Rina Wulandari. Tolong kirim ke Jl. Kenanga II No. 7, RT 03/RW 05, Depok. "
"WA 0857-2210-9384, transfer ke BCA 8730012345 a.n. Rina Wulandari.")
for e in det.detect(text):
print(e.label, e.text, round(e.score, 2))
print(det.redact(text, mode="tag", indexed=True).text)
# Halo kak, saya [NAME_1]. Tolong kirim ke [ADDRESS_1]. WA [PHONE_1], transfer ke BCA [BANK_ACCOUNT_1] a.n. [NAME_1].
Redaction modes:
| Mode | Output |
|---|---|
tag |
[NIK], or with indexed=True a consistent [NAME_1], [NAME_2], β¦ |
mask |
************0001 (keep_last=4 keeps the last digits) |
hash |
[NIK:9f2c1a7e3b], a keyed HMAC-SHA256 that is consistent across documents |
fake |
Consistent, format-preserving pseudonyms. A fake NIK is still a structurally valid NIK. |
There is also a command line interface for .txt and .jsonl files:
indopii redact --model mini --mode mask --keep-last 4 chat_export.txt > chat_export.redacted.txt
indopii detect --model mini records.jsonl --field text > spans.jsonl
Detector() without a model runs the rule engine only: about 0.4 ms per document, high precision, structured identifiers only.
Labels
| Category (UU PDP) | Labels |
|---|---|
| General personal data | NAME, ADDRESS, BIRTH_DATE, BIRTH_PLACE, GENDER, RELIGION, MARITAL_STATUS |
| Identity numbers | NIK, KK, NPWP, PASSPORT, SIM, BPJS, NIP, PLATE |
| Contact and online identifiers | PHONE, EMAIL, IP_ADDRESS, USERNAME |
| Personal financial data | BANK_ACCOUNT, CARD_NUMBER |
The annotation guideline is published with the benchmark. It defines minimal spans and keeps honorifics such as Bapak, Ibu and mas outside names. It does not tag a bare city, or organisations and their office contacts.
Evaluation
All scores come from IndoPII-Bench test: 480 Indonesian documents in 16 genres with 2,591 personal-data spans. The genres include chat, WhatsApp, forms, KTP/KK OCR, logs, CSV/JSON, medical, finance, legal, HR, news and hard negatives. Metrics:
- Strict F1 is exact span plus label.
- Leak is the share of personal-data values that were not fully covered by any predicted span, whatever its label. It is what a redaction pipeline actually lets through.
| System | Strict F1 | Leak β |
|---|---|---|
Microsoft Presidio (spaCy en_core_web_lg) |
0.194 | 44.8% |
0xcynyx/ner-pii-indonesian-mini |
0.262 | 25.4% |
NVIDIA gliner-PII |
0.400 | 46.4% |
AIbnuHibban/gliner2.5-indonesian v2 (zero-shot PII prompts) |
0.614 | 22.5% |
fastino/gliner2.5-multi-v1 (zero-shot PII prompts) |
0.633 | 35.7% |
urchade/gliner_multi_pii-v1 |
0.636 | 28.5% |
| IndoPII rules only | 0.622 | 53.3% |
| IndoPII Mini | 0.897 | 7.9% |
| IndoPII Base | 0.930 | 4.0% |
Baselines use their own entity types, mapped onto IndoPII labels. Leak is label-agnostic and therefore the fairest cross-system number.
IndoPII scores are for one-document-at-a-time inference. With batched inference (detect_batch) Mini scores 0.900, because dynamic INT8 computes its activation scales per batch.
Speed (same hardware)
The same 100 test documents (~1,000 characters each) were run on an AMD Ryzen 5 6600H laptop CPU with 4 threads, using the full pipeline:
| System | Strict F1 | ms / document |
|---|---|---|
| IndoPII rules only | 0.626 | 0.4 |
| IndoPII Mini | 0.887 | 8.5 |
| IndoPII Base | 0.930 | 116 |
0xcynyx/ner-pii-indonesian-mini |
0.317 | 63 |
fastino/gliner2.5-multi-v1 |
0.607 | 428 |
AIbnuHibban/gliner2.5-indonesian v3 |
0.629 | 449 |
The tagger alone, on a 40-token sentence, takes 2.4 ms with 4 threads and 6.1 ms with 1 thread while the CPU is boosting. When the laptop is thermally throttled this rises to about 6 ms and 10 ms.
Per label (test, strict F1)
| Label | F1 | Label | F1 | Label | F1 |
|---|---|---|---|---|---|
NAME |
0.883 | NIK |
0.944 | PHONE |
0.962 |
ADDRESS |
0.805 | KK |
0.816 | EMAIL |
0.968 |
BIRTH_DATE |
0.933 | NPWP |
0.895 | IP_ADDRESS |
0.907 |
BIRTH_PLACE |
0.940 | PASSPORT |
0.692 | USERNAME |
0.754 |
GENDER |
0.911 | SIM |
1.000 | BANK_ACCOUNT |
0.879 |
RELIGION |
0.982 | BPJS |
0.933 | CARD_NUMBER |
0.947 |
MARITAL_STATUS |
0.885 | NIP |
1.000 | PLATE |
1.000 |
Additional test sets
| Test set | Mini | Base |
|---|---|---|
Real-text track: 450 real sentences from news, Wikipedia and reviews (NAME, ADDRESS only), strict F1 |
0.737 | 0.820 |
| Real-text track: leak | 21.2% | 4.0% |
| Name-ambiguity challenge (200 minimal pairs), pair accuracy | 0.665 | 0.790 |
| Name-ambiguity challenge: names found | 85.6% | 92.1% |
| Name-ambiguity challenge: ordinary words wrongly tagged | 21.5% | 14.0% |
The real-text track reuses the person and address annotations of the Indonesian GLiNER benchmark. On this track gliner2.5-indonesian scores 0.877, because it was trained under that same annotation guideline.
The challenge set pairs the same word used as a name and as an ordinary word ("bunga lagi sakit" against "bunga mawar sudah mekar"), plus honorifics without names, name particles, and heroes against streets named after them. All of its words were held out of training.
Training
Data:
- 75k synthetic documents from 1,320 hand-written templates covering 16 genres, filled with structurally valid fake identifiers (NIK and KK with real region codes, Luhn-valid cards, valid NPWP check digits) and with names and addresses from real Indonesian region data;
- NERGrit news with human
PERSONannotations; - 2.6k silver-annotated real sentences;
- contrastive templates for ambiguous names and honorifics.
The benchmark and the challenge set were written independently and never used for training.
Teacher: XLM-R base fine-tuned on the same data (test F1 0.930).
Student:
- Prune the XLM-R vocabulary from 250k to 32k pieces.
- Fine-tune a 6-layer MiniLMv2 with logit distillation from the teacher (Ξ± 0.8, T 2).
- Drop the top two layers.
- Distil again for 8 epochs (Ξ± 0.9).
The span threshold (0.8) was selected on the dev set.
Quantisation: dynamic INT8 for embeddings, attention projections and the first FFN projection. The second FFN projection, which receives GELU activations with large outliers, and the classifier stay in fp32. Full INT8 cost about 4 F1 points; this recipe is within 0.2 points of fp32.
Hardware: a single RTX 4090 for about 40 minutes, covering the teacher and both students.
Intended use and limitations
Intended use:
- redacting Indonesian text before it is logged, shared, used for analytics, or sent to an external LLM API;
- auditing datasets for personal data;
- supporting UU PDP data-minimisation processes.
Limitations:
- The main benchmark is LLM-written and LLM-adjudicated, not annotated by humans. The real-text track shows that recall on real news and Wikipedia text is lower (21% of names and addresses leak with Mini). Use IndoPII Base when recall matters more than speed.
- Lower-case chat names that are also ordinary words are hard. Mini over-tags such words in about one in five cases on the challenge set.
- Heavily degraded OCR, Indonesian regional languages, and speech transcripts are out of scope.
- Office contacts of organisations (call centres,
info@mailboxes) are intentionally not treated as personal data. - Detection is not a compliance guarantee. Review the output for high-stakes uses.
License
The code and model weights are released under Apache-2.0. The base models are MIT (MiniLMv2, XLM-R). Training sources:
| Source | License |
|---|---|
| NERGrit | BSD-3-Clause |
| NusaX | Apache-2.0 |
| Wikipedia text | CC BY-SA 4.0 |
| Region codes (cahyadsn/wilayah) | MIT |