---
library_name: transformers
license: mit
base_model: xlnet/xlnet-base-cased
tags:
- generated_from_trainer
metrics:
- f1
- accuracy
model-index:
- name: UIT-NO-PRExlnet-base-cased-finetuned
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# UIT-NO-PRExlnet-base-cased-finetuned

This model is a fine-tuned version of [xlnet/xlnet-base-cased](https://huggingface.co/xlnet/xlnet-base-cased) on the None dataset.
It achieves the following results on the evaluation set:
- Loss: 0.6397
- F1: 0.7187
- Roc Auc: 0.7856
- Accuracy: 0.4531

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 16
- eval_batch_size: 16
- seed: 42
- optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
- lr_scheduler_type: cosine
- lr_scheduler_warmup_steps: 100
- num_epochs: 30

### Training results

| Training Loss | Epoch | Step | Validation Loss | F1     | Roc Auc | Accuracy |
|:-------------:|:-----:|:----:|:---------------:|:------:|:-------:|:--------:|
| 0.5578        | 1.0   | 139  | 0.5043          | 0.3149 | 0.5892  | 0.2022   |
| 0.4414        | 2.0   | 278  | 0.4142          | 0.6034 | 0.7089  | 0.3827   |
| 0.3813        | 3.0   | 417  | 0.3941          | 0.6657 | 0.7419  | 0.4061   |
| 0.2291        | 4.0   | 556  | 0.4109          | 0.6806 | 0.7463  | 0.4260   |
| 0.23          | 5.0   | 695  | 0.4438          | 0.6914 | 0.7605  | 0.4386   |
| 0.15          | 6.0   | 834  | 0.4506          | 0.7001 | 0.7664  | 0.4513   |
| 0.1324        | 7.0   | 973  | 0.4842          | 0.6910 | 0.7646  | 0.4603   |
| 0.0803        | 8.0   | 1112 | 0.5092          | 0.7164 | 0.7915  | 0.4513   |
| 0.0772        | 9.0   | 1251 | 0.5448          | 0.7000 | 0.7734  | 0.4404   |
| 0.0459        | 10.0  | 1390 | 0.5791          | 0.6958 | 0.7693  | 0.4332   |
| 0.0308        | 11.0  | 1529 | 0.5954          | 0.7102 | 0.7836  | 0.4386   |
| 0.0343        | 12.0  | 1668 | 0.6288          | 0.7030 | 0.7780  | 0.4350   |
| 0.0254        | 13.0  | 1807 | 0.6397          | 0.7187 | 0.7856  | 0.4531   |
| 0.0269        | 14.0  | 1946 | 0.6693          | 0.6932 | 0.7705  | 0.4368   |
| 0.0144        | 15.0  | 2085 | 0.6778          | 0.6949 | 0.7711  | 0.4386   |
| 0.0104        | 16.0  | 2224 | 0.7106          | 0.6943 | 0.7684  | 0.4368   |
| 0.006         | 17.0  | 2363 | 0.7217          | 0.7053 | 0.7749  | 0.4422   |
| 0.0109        | 18.0  | 2502 | 0.7313          | 0.7124 | 0.7826  | 0.4368   |
| 0.0063        | 19.0  | 2641 | 0.7416          | 0.7116 | 0.7796  | 0.4314   |
| 0.0043        | 20.0  | 2780 | 0.7470          | 0.7124 | 0.7795  | 0.4332   |
| 0.0031        | 21.0  | 2919 | 0.7647          | 0.7012 | 0.7710  | 0.4296   |
| 0.0041        | 22.0  | 3058 | 0.7516          | 0.7019 | 0.7722  | 0.4350   |
| 0.0028        | 23.0  | 3197 | 0.7591          | 0.7035 | 0.7721  | 0.4386   |
| 0.0027        | 24.0  | 3336 | 0.7587          | 0.7052 | 0.7748  | 0.4386   |
| 0.0028        | 25.0  | 3475 | 0.7644          | 0.7099 | 0.7770  | 0.4422   |
| 0.0027        | 26.0  | 3614 | 0.7656          | 0.7081 | 0.7758  | 0.4422   |
| 0.0033        | 27.0  | 3753 | 0.7629          | 0.7097 | 0.7764  | 0.4477   |
| 0.0026        | 28.0  | 3892 | 0.7622          | 0.7082 | 0.7758  | 0.4458   |
| 0.0025        | 29.0  | 4031 | 0.7626          | 0.7082 | 0.7758  | 0.4458   |
| 0.0023        | 30.0  | 4170 | 0.7627          | 0.7082 | 0.7758  | 0.4458   |


### Framework versions

- Transformers 4.48.1
- Pytorch 2.4.0
- Datasets 3.0.1
- Tokenizers 0.21.0