Hasanblanka/polygraf-ner-stage3-first-model

Polygraf NER Stage 3 First Model

This is the first English NER model produced for Stage 3 of the Polygraf Applied NLP / NER technical task. It fine-tunes distilbert/distilbert-base-cased on the manually reviewed Stage 2 dataset.

Labels

The model predicts BIO tags for eight entity types:

• PERSON
• ORGANIZATION
• LOCATION
• TIMEDATE
• PRODUCT
• WORKOFART
• JOB
• AMOUNT

The exact id2label and label2id mappings are stored in config.json.

Training data

• Dataset: Hasanblanka/polygraf-ner-stage_2-dataset
• Dataset revision: bce1f32778a333297ba1bf9749c13dd394e6256d
• Train records: 506
• Test records: 127
• Base model revision: 6ea81172465e8b0ad3fddeed32b986cdcdcffcf0

No validation split was created. The held-out test split was not used for early stopping or hyperparameter selection.

Training configuration

• Epochs: 4
• Batch size: 8
• Gradient accumulation: 2
• Effective batch size: 16
• Optimizer: AdamW
• Learning rate: 5e-5
• Weight decay: 0.01
• Warmup ratio: 0.1
• Seed: 42
• Maximum sequence length: 512
• Truncation: disabled
• Device used for this run: CPU

Evaluation

Metric	Train	Test
Precision	0.7455	0.5473
Recall	0.7713	0.6260
Entity F1	0.7582	0.5840
Macro F1	0.6735	0.5348
Token accuracy	0.9635	0.9201

Test results by entity type:

Label	Precision	Recall	F1
PERSON	0.6071	0.7816	0.6834
ORGANIZATION	0.4795	0.5785	0.5243
LOCATION	0.6398	0.8306	0.7228
TIMEDATE	0.7302	0.6434	0.6840
PRODUCT	0.0938	0.0732	0.0822
WORKOFART	0.4136	0.7165	0.5245
JOB	0.5306	0.3662	0.4333
AMOUNT	0.7358	0.5417	0.6240

The metrics/ directory contains the complete machine-readable results, benchmark, manual wild-test metrics, and test error breakdown.

Resource measurements

• Parameters: 65,203,985
• Saved model directory: approximately 261.5 MB
• CPU throughput at batch size 1: 19.15 sequences/s
• CPU token throughput: 1,792.72 tokens/s
• Peak process RAM: approximately 639 MB
• Incremental peak RAM: approximately 244 MB

The benchmark measured 100 samples in a fresh process on an Intel Core i5-13420H CPU. Results will vary by hardware and input length.

Usage

from transformers import pipeline

model_id = "Hasanblanka/polygraf-ner-stage3-first-model"
ner = pipeline(
    task="token-classification",
    model=model_id,
    tokenizer=model_id,
    aggregation_strategy="simple",
)

text = "Maya Ortiz joined Northstar Robotics in Lisbon on March 12, 2026."
for entity in ner(text):
    print(entity)

Wild-sample QA

Ten fresh, manually annotated samples outside the main dataset were used as a targeted stress test. Entity F1 was 0.5376. Observed problems included excessive WORKOFART predictions, missed PRODUCT and JOB entities, organization/location confusion, organization/job confusion, fragmented multiword entities, subword fragmentation, and boundary errors.

This small manual set is useful for diagnosis but is not a statistically representative benchmark.

Limitations

• This is a first baseline and is not suitable for production.
• PRODUCT recognition is particularly weak.
• WORKOFART is overpredicted.
• News-heavy training data limits domain coverage.
• The project uses a custom flat, non-overlapping annotation policy.
• Nested entities are not supported.

License and data terms

The base DistilBERT checkpoint is Apache-2.0. The fine-tuning dataset combines sources with different terms, including evaluation-use-only starter data and CC BY / CC BY-SA Open Newswire records. This model is therefore published with license: other and is intended for evaluation and research. Users must review and follow the source and licensing notes in the Stage 2 dataset card.