Edit model card

Model Description

This model is a fine-tuned version of distilroberta-base on ConLL2003 dataset. It achieves the following results on the evaluation set in Named Entity Recognition (NER)/Token Classification task:

  • Loss: 0.0585
  • F1: 0.9536

Model Performance

Model Usage

from transformers import AutoTokenizer, AutoModelForTokenClassification
from transformers import pipeline

tokenizer = AutoTokenizer.from_pretrained("jinhybr/distilroberta-ConLL2003")
model = AutoModelForTokenClassification.from_pretrained("jinhybr/distilroberta-ConLL2003")

nlp = pipeline("ner", model=model, tokenizer=tokenizer, grouped_entities=True)
example = "My name is Tao Jin and live in Canada"
ner_results = nlp(example)
print(ner_results)

[{'entity_group': 'PER', 'score': 0.99686015, 'word': ' Tao Jin', 'start': 11, 'end': 18}, {'entity_group': 'LOC', 'score': 0.9996836, 'word': ' Canada', 'start': 31, 'end': 37}]

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 5e-05
  • train_batch_size: 32
  • eval_batch_size: 16
  • seed: 24
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • num_epochs: 6.0

Training results

Training Loss Epoch Step Validation Loss F1
0.1666 1.0 439 0.0621 0.9345
0.0499 2.0 878 0.0564 0.9391
0.0273 3.0 1317 0.0553 0.9469
0.0167 4.0 1756 0.0553 0.9492
0.0103 5.0 2195 0.0572 0.9516
0.0068 6.0 2634 0.0585 0.9536

Framework versions

  • Transformers 4.35.2
  • Pytorch 2.1.0+cu121
  • Datasets 2.17.0
  • Tokenizers 0.15.1
Downloads last month
2
Safetensors
Model size
81.5M params
Tensor type
F32
·

Finetuned from

Space using jinhybr/distilroberta-ConLL2003 1