---
license: mit
base_model: facebook/xlm-roberta-xl
tags:
- generated_from_trainer
metrics:
- precision
- recall
- f1
- accuracy
model-index:
- name: xlm-roberta-xl-final-lora152520
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# xlm-roberta-xl-final-lora152520

This model is a fine-tuned version of [facebook/xlm-roberta-xl](https://huggingface.co/facebook/xlm-roberta-xl) on an unknown dataset.
It achieves the following results on the evaluation set:
- Loss: 1.5503
- Precision: 0.9267
- Recall: 0.9291
- F1: 0.9279
- Accuracy: 0.9386

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 0.0001
- train_batch_size: 16
- eval_batch_size: 16
- seed: 42
- distributed_type: multi-GPU
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 40
- num_epochs: 40
- mixed_precision_training: Native AMP
- label_smoothing_factor: 0.2

### Training results

| Training Loss | Epoch | Step  | Validation Loss | Precision | Recall | F1     | Accuracy |
|:-------------:|:-----:|:-----:|:---------------:|:---------:|:------:|:------:|:--------:|
| 2.6885        | 1.0   | 250   | 1.9699          | 0.7944    | 0.8343 | 0.8139 | 0.8442   |
| 1.7948        | 2.0   | 500   | 1.6854          | 0.8702    | 0.8791 | 0.8746 | 0.8949   |
| 1.6148        | 3.0   | 750   | 1.6185          | 0.8827    | 0.8998 | 0.8911 | 0.9096   |
| 1.5365        | 4.0   | 1000  | 1.5710          | 0.9031    | 0.9054 | 0.9043 | 0.9195   |
| 1.4852        | 5.0   | 1250  | 1.5524          | 0.9124    | 0.9129 | 0.9126 | 0.9255   |
| 1.4538        | 6.0   | 1500  | 1.5431          | 0.9112    | 0.9176 | 0.9144 | 0.9272   |
| 1.4306        | 7.0   | 1750  | 1.5390          | 0.9145    | 0.9221 | 0.9183 | 0.9297   |
| 1.4132        | 8.0   | 2000  | 1.5358          | 0.9191    | 0.9219 | 0.9205 | 0.9321   |
| 1.4004        | 9.0   | 2250  | 1.5365          | 0.9174    | 0.9262 | 0.9218 | 0.9337   |
| 1.3883        | 10.0  | 2500  | 1.5407          | 0.9176    | 0.9263 | 0.9220 | 0.9332   |
| 1.3803        | 11.0  | 2750  | 1.5326          | 0.9218    | 0.9278 | 0.9248 | 0.9358   |
| 1.3727        | 12.0  | 3000  | 1.5353          | 0.9187    | 0.9245 | 0.9216 | 0.9329   |
| 1.3674        | 13.0  | 3250  | 1.5392          | 0.9202    | 0.9254 | 0.9228 | 0.9350   |
| 1.3609        | 14.0  | 3500  | 1.5384          | 0.9220    | 0.9259 | 0.9239 | 0.9347   |
| 1.3572        | 15.0  | 3750  | 1.5382          | 0.9201    | 0.9240 | 0.9220 | 0.9334   |
| 1.3522        | 16.0  | 4000  | 1.5410          | 0.9197    | 0.9270 | 0.9233 | 0.9342   |
| 1.3502        | 17.0  | 4250  | 1.5449          | 0.9245    | 0.9268 | 0.9256 | 0.9355   |
| 1.3456        | 18.0  | 4500  | 1.5439          | 0.9233    | 0.9278 | 0.9256 | 0.9360   |
| 1.3423        | 19.0  | 4750  | 1.5435          | 0.9259    | 0.9248 | 0.9253 | 0.9346   |
| 1.3419        | 20.0  | 5000  | 1.5432          | 0.9270    | 0.9282 | 0.9276 | 0.9371   |
| 1.3397        | 21.0  | 5250  | 1.5398          | 0.9250    | 0.9284 | 0.9267 | 0.9369   |
| 1.3377        | 22.0  | 5500  | 1.5411          | 0.9253    | 0.9270 | 0.9262 | 0.9358   |
| 1.3351        | 23.0  | 5750  | 1.5471          | 0.9274    | 0.9284 | 0.9279 | 0.9374   |
| 1.3369        | 24.0  | 6000  | 1.5542          | 0.9214    | 0.9240 | 0.9227 | 0.9339   |
| 1.3348        | 25.0  | 6250  | 1.5479          | 0.9268    | 0.9288 | 0.9278 | 0.9374   |
| 1.334         | 26.0  | 6500  | 1.5492          | 0.9268    | 0.9294 | 0.9281 | 0.9384   |
| 1.3334        | 27.0  | 6750  | 1.5471          | 0.9299    | 0.9287 | 0.9293 | 0.9377   |
| 1.3327        | 28.0  | 7000  | 1.5438          | 0.9291    | 0.9309 | 0.9300 | 0.9394   |
| 1.3314        | 29.0  | 7250  | 1.5445          | 0.9304    | 0.9315 | 0.9310 | 0.9403   |
| 1.3318        | 30.0  | 7500  | 1.5456          | 0.9291    | 0.9310 | 0.9301 | 0.9399   |
| 1.3312        | 31.0  | 7750  | 1.5474          | 0.9278    | 0.9295 | 0.9287 | 0.9386   |
| 1.3304        | 32.0  | 8000  | 1.5489          | 0.9273    | 0.9302 | 0.9288 | 0.9388   |
| 1.3298        | 33.0  | 8250  | 1.5469          | 0.9286    | 0.9299 | 0.9293 | 0.9388   |
| 1.3295        | 34.0  | 8500  | 1.5474          | 0.9291    | 0.9312 | 0.9302 | 0.9398   |
| 1.3288        | 35.0  | 8750  | 1.5518          | 0.9280    | 0.9300 | 0.9290 | 0.9386   |
| 1.3292        | 36.0  | 9000  | 1.5484          | 0.9271    | 0.9308 | 0.9289 | 0.9388   |
| 1.3287        | 37.0  | 9250  | 1.5487          | 0.9278    | 0.9297 | 0.9287 | 0.9382   |
| 1.328         | 38.0  | 9500  | 1.5492          | 0.9290    | 0.9305 | 0.9298 | 0.9394   |
| 1.3281        | 39.0  | 9750  | 1.5496          | 0.9278    | 0.9293 | 0.9285 | 0.9387   |
| 1.3285        | 40.0  | 10000 | 1.5503          | 0.9267    | 0.9291 | 0.9279 | 0.9386   |


### Framework versions

- Transformers 4.35.2
- Pytorch 2.0.1+cu118
- Datasets 2.15.0
- Tokenizers 0.15.0