---
license: mit
base_model: xlm-roberta-base
tags:
- generated_from_trainer
datasets:
- conll2003
metrics:
- precision
- recall
- f1
- accuracy
model-index:
- name: XLM-RoBERTa-Base-Conll2003-English-NER-Finetune-BinaryClass-WeightedLoss
  results:
  - task:
      name: Token Classification
      type: token-classification
    dataset:
      name: conll2003
      type: conll2003
      config: conll2003
      split: test
      args: conll2003
    metrics:
    - name: Precision
      type: precision
      value: 0.952513966480447
    - name: Recall
      type: recall
      value: 0.9660056657223796
    - name: F1
      type: f1
      value: 0.9592123769338959
    - name: Accuracy
      type: accuracy
      value: 0.9906536018089803
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# XLM-RoBERTa-Base-Conll2003-English-NER-Finetune-BinaryClass-WeightedLoss

This model is a fine-tuned version of [xlm-roberta-base](https://huggingface.co/xlm-roberta-base) on the conll2003 dataset.
It achieves the following results on the evaluation set:
- Loss: 0.1110
- Precision: 0.9525
- Recall: 0.9660
- F1: 0.9592
- Accuracy: 0.9907

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 5e-06
- train_batch_size: 4
- eval_batch_size: 4
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_ratio: 0.05
- num_epochs: 10

### Training results

| Training Loss | Epoch  | Step  | Validation Loss | Precision | Recall | F1     | Accuracy |
|:-------------:|:------:|:-----:|:---------------:|:---------:|:------:|:------:|:--------:|
| 0.3351        | 0.3333 | 1441  | 0.1107          | 0.9131    | 0.9377 | 0.9252 | 0.9814   |
| 0.0357        | 0.6667 | 2882  | 0.0501          | 0.9500    | 0.9446 | 0.9473 | 0.9882   |
| 0.0239        | 1.0    | 4323  | 0.0662          | 0.9481    | 0.9479 | 0.9480 | 0.9883   |
| 0.0178        | 1.3333 | 5764  | 0.0667          | 0.9489    | 0.9603 | 0.9546 | 0.9899   |
| 0.0188        | 1.6667 | 7205  | 0.0712          | 0.9488    | 0.9575 | 0.9531 | 0.9895   |
| 0.018         | 2.0    | 8646  | 0.0605          | 0.9524    | 0.9559 | 0.9541 | 0.9902   |
| 0.0119        | 2.3333 | 10087 | 0.0840          | 0.9487    | 0.9662 | 0.9574 | 0.9901   |
| 0.0124        | 2.6667 | 11528 | 0.0758          | 0.9486    | 0.9641 | 0.9563 | 0.9901   |
| 0.0112        | 3.0    | 12969 | 0.0664          | 0.9559    | 0.9628 | 0.9593 | 0.9910   |
| 0.0082        | 3.3333 | 14410 | 0.0939          | 0.9483    | 0.9603 | 0.9543 | 0.9899   |
| 0.0083        | 3.6667 | 15851 | 0.0681          | 0.9555    | 0.9591 | 0.9573 | 0.9907   |
| 0.0077        | 4.0    | 17292 | 0.0686          | 0.9555    | 0.9572 | 0.9563 | 0.9902   |
| 0.0055        | 4.3333 | 18733 | 0.0852          | 0.9498    | 0.9642 | 0.9569 | 0.9905   |
| 0.005         | 4.6667 | 20174 | 0.0795          | 0.9530    | 0.9653 | 0.9591 | 0.9907   |
| 0.0049        | 5.0    | 21615 | 0.0871          | 0.9526    | 0.9614 | 0.9570 | 0.9900   |
| 0.0042        | 5.3333 | 23056 | 0.1054          | 0.9482    | 0.9658 | 0.9569 | 0.9898   |
| 0.0045        | 5.6667 | 24497 | 0.0764          | 0.9559    | 0.9598 | 0.9579 | 0.9905   |
| 0.0043        | 6.0    | 25938 | 0.0996          | 0.9510    | 0.9662 | 0.9585 | 0.9905   |
| 0.0037        | 6.3333 | 27379 | 0.0909          | 0.9539    | 0.9641 | 0.9590 | 0.9908   |
| 0.003         | 6.6667 | 28820 | 0.1010          | 0.9519    | 0.9639 | 0.9579 | 0.9905   |
| 0.003         | 7.0    | 30261 | 0.0944          | 0.9510    | 0.9632 | 0.9571 | 0.9905   |
| 0.0037        | 7.3333 | 31702 | 0.1041          | 0.9514    | 0.9642 | 0.9578 | 0.9903   |
| 0.0021        | 7.6667 | 33143 | 0.1048          | 0.9520    | 0.9658 | 0.9589 | 0.9907   |
| 0.0029        | 8.0    | 34584 | 0.1001          | 0.9526    | 0.9651 | 0.9588 | 0.9907   |
| 0.0019        | 8.3333 | 36025 | 0.1098          | 0.9525    | 0.9653 | 0.9588 | 0.9906   |
| 0.0019        | 8.6667 | 37466 | 0.1027          | 0.9538    | 0.9651 | 0.9594 | 0.9906   |
| 0.0019        | 9.0    | 38907 | 0.0990          | 0.9543    | 0.9653 | 0.9598 | 0.9908   |
| 0.0018        | 9.3333 | 40348 | 0.1086          | 0.9537    | 0.9655 | 0.9595 | 0.9907   |
| 0.0014        | 9.6667 | 41789 | 0.1090          | 0.9533    | 0.9658 | 0.9595 | 0.9907   |
| 0.0014        | 10.0   | 43230 | 0.1110          | 0.9525    | 0.9660 | 0.9592 | 0.9907   |


### Framework versions

- Transformers 4.41.1
- Pytorch 2.3.0+cu121
- Datasets 2.19.1
- Tokenizers 0.19.1