BERT + CRF for Named Entity Recognition
A BERT-base model with a Conditional Random Field (CRF) layer fine-tuned on CoNLL-2003 for Named Entity Recognition.
Model Description
- Base Model: bert-base-cased
- Architecture: BERT encoder + Linear projection + CRF decoder
- Task: Named Entity Recognition (NER)
- Dataset: CoNLL-2003 (eriktks/conll2003)
Training
- Epochs: 10
- Batch Size: 16
- Learning Rate: 2e-5
- Max Sequence Length: 128
- Optimizer: AdamW with linear warmup (10%)
Performance
| Split | Precision | Recall | F1 |
|---|---|---|---|
| Validation | 95.27% | 95.72% | 95.50% |
| Test | 91.02% | 92.31% | 91.66% |
Training Curves
| Epoch | Train Loss | Val F1 |
|---|---|---|
| 1 | 5.72 | 92.15% |
| 2 | 0.59 | 94.12% |
| 3 | 0.31 | 94.75% |
| 4 | 0.21 | 95.18% |
| 5 | 0.13 | 95.19% |
| 6 | 0.07 | 95.21% |
| 7 | 0.05 | 95.44% |
| 8 | 0.02 | 95.34% |
| 9 | 0.02 | 95.52% |
| 10 | 0.01 | 95.50% |
Labels
The model recognizes 4 entity types in BIO format (9 classes total):
- PER: Person names
- ORG: Organizations
- LOC: Locations
- MISC: Miscellaneous entities
Usage
from model import BertCrfForTokenClassification
from transformers import AutoTokenizer
from huggingface_hub import hf_hub_download
import torch
# Load model and tokenizer
MODEL_REPO = "sebastiao-teixeira/bert-crf-ner-conll2003"
tokenizer = AutoTokenizer.from_pretrained(MODEL_REPO)
model = BertCrfForTokenClassification("bert-base-cased", num_labels=9)
weights_path = hf_hub_download(repo_id=MODEL_REPO, filename="pytorch_model.bin")
model.load_state_dict(torch.load(weights_path, map_location="cpu"))
model.eval()
# Inference
sentence = "John works at Google in New York."
inputs = tokenizer(sentence, return_tensors="pt")
with torch.no_grad():
outputs = model(**inputs)
predictions = outputs["logits"][0].tolist()
Citation
@misc{bert-crf-ner,
author = {Sebastiao Teixeira},
title = {BERT + CRF for Named Entity Recognition},
year = {2026},
publisher = {Hugging Face},
journal = {Hugging Face Hub},
}