VEILGATETURKISH/veilgate-turkish

VeilGate — Multilingual privacy NER models

English | Türkçe

This repository is the shared distribution location for the Light, Medium and High families and their FP32/INT8 variants. turkish in the repository name is historical; Light v1 and Medium v1 have been evaluated across 13 languages.

Selected model: Light v1, the strong privacy-common-v1 baseline. Rejected health continuation experiments are not included in these weights. This is an experimental model release, not enterprise DLP, medical diagnosis, Turkish data-protection compliance, or independent product/platform acceptance.

Families and variants

Family	Base model	Status	Distribution
Light v1	multilingual DistilBERT	Trained, selected baseline	Native FP32: light/; ONNX: light/fp32/, light/int8/
Medium v1	multilingual BERT	Trained, evaluated, user-approved application model option; not the default	Native FP32: medium/; ONNX: medium/fp32/, medium/int8/
High	XLM-R base	Training not started	Future high/fp32/, high/int8/

High is not trained or released. Medium has 177,271,307 parameters and uses its own base/tokenizer initialization, not Light weights. FP32 and INT8 are evaluated separately. Root ONNX files are Light FP32 compatibility aliases. Native Transformers use requires subfolder="light" or subfolder="medium"; the root directory is not a native Transformers checkpoint.

Light v1 specifications

• 134,742,539 parameters, including 8,459 classifier parameters.
• 6 encoder blocks, hidden size 768, 12 attention heads, vocabulary size 119,547.
• 512-token context, overflow stride 64, simple BIO aggregation.
• PERSON, ADDRESS, LOCATION, ORGANIZATION, HEALTH_INFORMATION; 11 BIO outputs including O.
• Base: distilbert/distilbert-base-multilingual-cased, revision 45c032ab32cc946ad88a166f7cb282f58c753c2e (Apache-2.0).
• Native MC2+WikiNER encoder stage, followed by 144,147 common-privacy training steps.
• FP32 source weights SHA256: 1828864a5c7bc7ae3a3400b15604a5e3000933d44d908b3565e42107254196a4.

Languages: Turkish, Bengali, German, English, Spanish, Persian, French, Hindi, Italian, Portuguese, Swedish, Ukrainian and Chinese. Language/type combinations do not have equal annotation coverage; this set has no Turkish health gold.

Measured results

The following first table is Light v1; Medium results appear in the dedicated Medium section below. Both use the same development records and decoder.

12,374 preserved development records, 13 languages, threshold 0.6. Evaluation uses raw text, a gold-independent overflow/simple decoder and source annotation scopes. ONNX FP32 exactly reproduced all aggregate native-baseline scores. The threshold is not calibrated.

Metric	ONNX FP32	ONNX INT8
Person–location–organization macro F1, 13 languages	84.3293%	84.2948%
Person–location–organization micro F1	85.0021%	84.9404%
Turkish person–location–organization F1	84.1770%	83.9593%
Person F1 / exact annotated-positive hits	89.3948% · 6244/7120	89.2221% · 6229/7120
Location F1 / exact annotated-positive hits	82.2040% · 2928/3543	82.1335% · 2929/3543
Organization F1 / exact annotated-positive hits	78.8375% · 1831/2342	79.0166% · 1832/2342
Address exact-positive hits	912/1003	913/1003
Health F1 / exact annotated-positive hits	69.5313% · 623/896	69.0848% · 619/896
ONNX file size	539,097,176 bytes / 514.12 MiB	411,979,831 bytes / 392.89 MiB

Address annotations are positive-only: defensible scoped precision/F1 is unavailable. Scoped F1 and all annotated-positive hit counts use different scopes. The common macro F1 excludes health/address. This development set was reused; results are not independent final acceptance, statistical significance, or a guarantee for real traffic. The full text-free report is evaluation.json.

FP32 is the recommended default. INT8 is about 23.58% smaller, but matched 15 fewer person and 4 fewer health annotations on the same set. Quantization is dynamic, per-channel MatMul INT8; embeddings remain FP32. RAM/latency gains and success on all platforms are not guaranteed. INT8 is an explicit experimental option, not lossless or independently release-approved.

Methods and decision rationale

We used full fine-tuning of a pretrained multilingual base, not language pretraining from random weights. First, the encoder was adapted using separate MC2+WikiNER native heads; then it was trained on five common privacy types. The corpora overlap substantially: the second stage unifies labels and adds sources, rather than using entirely unrelated text. This lineage has not been proven superior to every single-stage alternative.

Partial-label loss avoids treating types unannotated by a source as false negatives. Exposure balancing controls how often ADDRESS/background examples are encountered. Health continuations were measured and rejected. INT8 uses post-training dynamic, per-channel MatMul quantization: a simple, measurable starting point requiring no additional training, not a claim of optimal speed or quality on every device.

QAT was not used. Quantization-aware training adapts weights to simulated low-precision effects; it does not guarantee faster INT8 or better quality. Static INT8 calibration, pruning, LoRA/adapters and GPU/FP16 acceleration were not used in these released weights. Frozen-parent KL distillation was tried only in rejected, separately preserved TC/TCKN correction experiments; those weights are not distributed. These target different problems: LoRA can reduce training costs, QAT can reduce quantization-related quality loss, and distillation can transfer behavior to a smaller model. DistilBERT's upstream distilled origin does not mean that we performed an additional distillation stage.

Possible future work — not implemented or automatically scheduled

Option	Evidence needed before choosing it	Additional training?
Human-reviewed, targeted data	Measured annotation/coverage gaps, difficult boundaries or false alarms	Yes, to learn from new data
Static INT8 PTQ	Calibration and operator support benefit the target device	Usually calibration only
QAT	INT8 quality loss warrants adapting weights to low precision	Yes
Teacher–student distillation	A better teacher can improve a smaller model under the same acceptance gates	Yes
LoRA/adapters	Training memory/cost, rather than inference latency, is the bottleneck	Yes, fewer trainable parameters
Pruning	Structured/sparse execution produces a real hardware speedup	Often recovery fine-tuning
FP16, GPU providers, ONNX graph/thread tuning	Target-device benchmarks show gains without unacceptable quality loss	Usually no

Choose by critical misses, false alarms, per-language/type quality, p50/p95 latency and peak RAM, not parameter count alone. No option guarantees improved F1 or speed. Input must not be silently truncated or detection coverage reduced to claim a speedup. Independent final gold is excluded from tuning, distillation and calibration. These are candidate directions, not an automatic queue after training completes.

Training data and attribution

The main encoded corpus contains 192,163 TRAIN / 12,374 development records. Native pretraining is part of the weight lineage; these counts do not mean that all of OpenPII was used.

Source / creators	Use	Declared license
MultiCoNER v2, Besnik Fetahu, Zhiyu Chen, Sudipta Kar, Oleg Rokhlenko, Shervin Malmasi / Amazon	12-language native pretraining; person/organization/location/medical common mapping	Paper: CC BY-SA4; HF/AWS: CC BY4; discrepancy retained
Turkish-WikiNER, Duygu Altinok; Co-one re-annotation; Kuzgunlar/TWNERTC source lineage	Turkish native pretraining and common mapping	CC BY-SA4
BTX24 Turkish Privacy PII NER, Boran Toktay	Turkish person/address, positive-only TRAIN/dev	CC BY4
OpenPII1.5M, Ai4Privacy / Ai Suisse SA	Bounded train/dev prefixes, not the complete dataset	CC BY4 declaration; metadata: other
VeilGate authored background	50 separate fictional TRAIN negative examples	Not provider corpus or independent human gold

Processing includes explicit label mapping, partial-label loss, preservation of original text/gold/partitions, TRAIN-only alignment quarantine, exact/template reservations, offset-safe chunking and exposure balancing. Text reconstructed from native tokens is marked accordingly. Provider test files were not used. General document/entity/near-duplicate/translation-family isolation and independent human annotation are not claimed complete.

Vitamins/supplements (Duygu Altinok / Co-one, CC BY-SA4) and authored-health additions were used only in rejected continuation experiments, not in the selected Light weights. Raw TWNERTC was not used as new TRAIN data; WikiNER lineage credit is retained. Gated/commercial sources, unpublished corpora and Turkish33 drafts are excluded from the main corpus.

Papers/DOIs, exact revisions, creators, licenses and modifications are listed in ATTRIBUTION.md; relevant full license texts are under licenses/. The base-model license does not remove dataset obligations. This card is not legal clearance: manifests retain license_status=unverified. Weight licensing and redistribution limits are in LICENSE.md. Raw source corpora are not uploaded here.

Usage — native FP32

from transformers import AutoModelForTokenClassification, AutoTokenizer, pipeline

repo = "VEILGATETURKISH/veilgate-turkish"
# Add revision="<full-HF-commit>" for reproducible use.
tokenizer = AutoTokenizer.from_pretrained(repo, subfolder="light", trust_remote_code=False)
model = AutoModelForTokenClassification.from_pretrained(
    repo, subfolder="light", trust_remote_code=False,
)
ner = pipeline("token-classification", model=model, tokenizer=tokenizer,
               aggregation_strategy="simple", stride=64, device=-1)
predictions = [hit for hit in ner("Fictional example text.") if hit["score"] >= 0.6]

After the initial download, the local cache supports offline use. Private text must not be sent to an external service for model download or inference. This call is NER-only: it does not automatically add regex/TCKN checks, the TC filter, policy or restoration.

Usage — ONNX and gateway

Gateway source: alperenkra/llm-privacy-gateway. Access permission may be required. Download all five manifest-listed files from light/fp32 or light/int8; model.onnx alone is insufficient. Graph inputs are int64 input_ids and attention_mask; the output is logits. Do not pass token_type_ids to the DistilBERT graph. Decoding and chunk merging are required.

MODEL_VARIANT=lightfp32 docker compose up --build -d
# Smaller explicit option with measured quality differences:
MODEL_VARIANT=lightint8 docker compose up -d

MODEL_HUB_REVISION=main updates only the selected variant; Light never automatically switches to Medium/High. The gateway isolates caches by repository and variant, checks checksums, warms up at startup, then activates atomically. Unpublished Medium/High selections do not silently fall back to another tier. Use a full commit pin and MODEL_AUTO_UPDATE=false for reproducible installation.

Medium v1 — specifications, training and results

Medium is a user-approved model option for the application; Light remains the default. The bundle's technical experimental status denotes unfinished independent/platform acceptance, not an automatic plan to replace Medium with another model family.

• 177,271,307 parameters, including 8,459 classifier parameters; 12 encoder blocks, hidden size 768, 12 attention heads and vocabulary size 119,547.
• Base: google-bert/bert-base-multilingual-cased, pinned revision 3f076fdb1ab68d5b2880cb87a0886f315b8146f8 (Apache-2.0).
• Five privacy types / 11 BIO outputs, 512-token context, overflow stride 64, original character offsets and the same simple decoder/threshold as Light.
• Stage 1: MC2 + Turkish-WikiNER joint encoder, separate native heads, three epochs. This is supervised NER adaptation of a pretrained base, not language pretraining from scratch.
• Stage 2: original native encoder + randomly initialized common privacy head; full fine-tuning, seed42, batch4, LR2e-5, weight decay0.01, 10% warmup/linear decay, 144,147 optimizer steps, checkpoint/monitor every5,000 steps; FP16/BF16 disabled.
• The corpus has192,163 TRAIN/12,374 development records. There are192,145 supervised base windows plus50 authored background windows. Exposure balancing yields261,303 sampled windows/pass:25% ADDRESS and1.8% authored background. Partial-label loss preserves source-specific annotation scope; native literal-O guidance uses confidence0.95.
• Epoch accounting: the common stage is step-budgeted, not three complete passes over the resampled loader: saved Trainer epoch 2.20658,576,588 sample exposures. Repetition does not create new unique records. The training monitor uses512 dev records; the final raw-text development comparison uses all12,374.
• The same sources, creators, license caveats and split limitations in the training-data section apply to Medium. No health-continuation or identity-correction additions were merged into its baseline. Reproduction settings/provenance: ml/results/medium_v1/training.json.

Development results

Same 12,374 raw development records, 13 languages, simple/stride decoder, threshold 0.6:

Metric	Medium FP32	Medium INT8
Person–location–organization language-macro F1	86.9927%	86.9871%
Same task micro F1	87.3027%	87.3654%
Turkish same-task F1	84.5492%	85.0048%
ADDRESS exact positive hits / 1,003	910	906
HEALTH_INFORMATION scoped F1	73.4994%	72.9238%
ONNX graph bytes	709,321,719	455,101,944

Type	FP32 scoped F1 / annotated-positive hits	INT8 scoped F1 / annotated-positive hits
PERSON	91.3171% ·6396/7120	91.2813% ·6391/7120
LOCATION	84.1895% ·3008/3543	84.2446% ·3006/3543
ORGANIZATION	82.5397% ·1924/2342	82.8266% ·1934/2342
ADDRESS	unavailable ·910/1003	unavailable ·906/1003
HEALTH_INFORMATION	73.4994% ·649/896	72.9238% ·641/896

Scoped F1 and all annotated-positive hits have different denominators. Medium FP32 improves common-three F1 in all13 languages versus Light, but Turkish gains only0.3722 percentage points and ADDRESS loses two exact hits. Not every capability improved.

Medium native weights SHA256: 1a793d6958c6abdb367d8227274e730b554309d6004557e01eca4b23110fb407. FP32 reproduces native aggregate development metrics. INT8 has measured address/health regressions and did not speed up medium/long texts in the local macOS ARM CPU smoke. Both exceeded the 1 GiB peak-process RSS target in that smoke; Windows/Linux product acceptance is unperformed. Evidence: ml/results/medium_v1/. This is repeatedly used development, not independent acceptance. ADDRESS is positive-only: precision/F1 are not inferred.

These are the original completed Medium baseline weights, not the rejected identity correction experiments. Light remains the default. Neither model has learned the TC/TCKN field-label correction reliably; the correction below is application code.

Native usage: select subfolder="medium" for both tokenizer and model. ONNX files are under medium/fp32/ and medium/int8/; gateway MODEL_VARIANT=mediumfp32 or mediumint8 selects the corresponding verified local bundle. No root/Light weight is replaced.

Open acceptance work

Independent human-reviewed final gold, Turkish health gold, calibrated thresholds, document/entity/near-duplicate family isolation, legal clearance and four-platform native packaging/resource acceptance remain open. Local resource results are not Windows/Linux guarantees. The model detects mentions, not company policy, secrets, person identity/coreference or permission to share. These gaps do not imply new training has been scheduled; High training requires a separate explicit start.

TC field filtering and safety boundaries

The gateway context filter suppresses incorrect model person/organization/location/health predictions fully contained within a recognized TCKN or TC kimlik numaram field label. Genuine institutions and adjacent identity numbers are preserved. This is not a correction learned by the weights. Direct Transformers/ONNX calls can still produce the raw errors.

The model may detect disease mentions; it does not decide personal diagnosis, consent, sharing permission or utility. Gender/identity/orientation classes were not trained in this head. This is not a general secret scanner, file/OCR protection, system-wide egress or identity authorization. Native Windows/Linux/macOS packaging and performance acceptance are incomplete. Model scores alone do not guarantee that sensitive data cannot leak.