---
license: apache-2.0
base_model: bert-base-cased
tags:
- generated_from_trainer
metrics:
- precision
- recall
- f1
- accuracy
model-index:
- name: bert-base-cased-finetuned-conll2003-ner
  results: []
datasets:
- conll2003
language:
- en
library_name: transformers
pipeline_tag: token-classification
---

# bert-base-cased-finetuned-conll2003-ner

This model is a fine-tuned version of BERT ([bert-base-cased](https://huggingface.co/bert-base-cased)) on the CoNLL-2003 (Conference on Computational Natural Language Learning) dataset.

The model performs named entity recognition (NER). It pertains to section 2 of chapter 7 of the Hugging Face "NLP Course" (https://huggingface.co/learn/nlp-course/chapter7/2).

It was trained using the Trainer API of Hugging Face Transformers.

Code: https://github.com/sambitmukherjee/huggingface-notebooks/blob/main/course/en/chapter7/section2_pt.ipynb

Experiment tracking: https://wandb.ai/sadhaklal/bert-base-cased-finetuned-conll2003-ner

## Usage

```
from transformers import pipeline

model_checkpoint = "sadhaklal/bert-base-cased-finetuned-conll2003-ner"
token_classifier = pipeline("token-classification", model=model_checkpoint, aggregation_strategy="simple")

print(token_classifier("My name is Sylvain and I work at Hugging Face in Brooklyn."))
```

## Dataset

From the dataset page:

> The shared task of CoNLL-2003 concerns language-independent named entity recognition. We will concentrate on four types of named entities: persons, locations, organizations and names of miscellaneous entities that do not belong to the previous three groups.

Examples: https://huggingface.co/datasets/conll2003/viewer

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 8
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 3

### Training results

| Training Loss | Epoch | Step | Validation Loss | Precision | Recall | F1     | Accuracy |
|:-------------:|:-----:|:----:|:---------------:|:---------:|:------:|:------:|:--------:|
| 0.0125        | 1.0   | 1756 | 0.0729          | 0.9095    | 0.9339 | 0.9215 | 0.9810   |
| 0.0001        | 2.0   | 3512 | 0.0558          | 0.9265    | 0.9487 | 0.9375 | 0.9862   |
| 0.0001        | 3.0   | 5268 | 0.0578          | 0.9366    | 0.9515 | 0.9440 | 0.9867   |

### Framework versions

- Transformers 4.37.2
- PyTorch 2.1.0+cu121
- Datasets 2.17.1
- Tokenizers 0.15.2