---
license: mit
base_model: numind/entity-recognition-general-sota-v1
tags:
- generated_from_trainer
metrics:
- precision
- recall
- f1
- accuracy
model-index:
- name: entity-recognition-general-sota-v1-finetuned-ner
  results: []
datasets:
- Babelscape/multinerd
language:
- en
library_name: transformers
pipeline_tag: token-classification
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

## Model description
# entity-recognition-general-sota-v1-finetuned-ner

This model is a fine-tuned version of [numind/entity-recognition-general-sota-v1](https://huggingface.co/numind/entity-recognition-general-sota-v1) on Babelscape/MultiNerd dataset.

It achieves the following results on the validation set:
- Loss: 0.0396
- Precision: 0.9138
- Recall: 0.9146
- F1: 0.9142
- Accuracy: 0.9857


## Training and evaluation data

The dataset if filtered on english language and sampled first 1M on train and 100k on validation.
further filtered with data containing atleast one tag from labels2ids mentioned below.
Training data - 131280 items
Validation data - 16410 items

Trained on all tags from the MultiNERD dataset.

labels2ids = {
    "O": 0,
    "B-PER": 1,
    "I-PER": 2,
    "B-ORG": 3,
    "I-ORG": 4,
    "B-LOC": 5,
    "I-LOC": 6,
    "B-ANIM": 7,
    "I-ANIM": 8,
    "B-BIO": 9,
    "I-BIO": 10,
    "B-CEL": 11,
    "I-CEL": 12,
    "B-DIS": 13,
    "I-DIS": 14,
    "B-EVE": 15,
    "I-EVE": 16,
    "B-FOOD": 17,
    "I-FOOD": 18,
    "B-INST": 19,
    "I-INST": 20,
    "B-MEDIA": 21,
    "I-MEDIA": 22,
    "B-MYTH": 23,
    "I-MYTH": 24,
    "B-PLANT": 25,
    "I-PLANT": 26,
    "B-TIME": 27,
    "I-TIME": 28,
    "B-VEHI": 29,
    "I-VEHI": 30,
  }

## Training procedure

HF Trainer module

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 20
- eval_batch_size: 20
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 1

### Training & Test set evaluation results

| Training Loss | Epoch | Step | Validation Loss | Precision | Recall | F1     | Accuracy |
|:-------------:|:-----:|:----:|:---------------:|:---------:|:------:|:------:|:--------:|
| 0.0323        | 1.0   | 6564 | 0.0396          | 0.9138    | 0.9146 | 0.9142 | 0.9857   |

Evaluation on test set:
{'eval_loss': 0.02707073651254177,
 'eval_precision': 0.9378337879893957,
 'eval_recall': 0.9518034704620784,
 'eval_f1': 0.9447669917943954,
 'eval_accuracy': 0.9901678553418342,
 'eval_runtime': 133.0665,
 'eval_samples_per_second': 247.305,
 'eval_steps_per_second': 30.917}

### Framework versions

- Transformers 4.35.2
- Pytorch 2.1.0+cu118
- Datasets 2.15.0
- Tokenizers 0.15.0