---
license: mit
base_model: MoritzLaurer/DeBERTa-v3-base-mnli-fever-anli
tags:
- generated_from_trainer
datasets:
- sem_eval_2024_task_2
metrics:
- accuracy
- precision
- recall
- f1
model-index:
- name: results2
  results:
  - task:
      name: Text Classification
      type: text-classification
    dataset:
      name: sem_eval_2024_task_2
      type: sem_eval_2024_task_2
      config: sem_eval_2024_task_2_source
      split: validation
      args: sem_eval_2024_task_2_source
    metrics:
    - name: Accuracy
      type: accuracy
      value: 0.715
    - name: Precision
      type: precision
      value: 0.7186959617536364
    - name: Recall
      type: recall
      value: 0.7150000000000001
    - name: F1
      type: f1
      value: 0.7137907659862921
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# results2

This model is a fine-tuned version of [MoritzLaurer/DeBERTa-v3-base-mnli-fever-anli](https://huggingface.co/MoritzLaurer/DeBERTa-v3-base-mnli-fever-anli) on the sem_eval_2024_task_2 dataset.
It achieves the following results on the evaluation set:
- Loss: 1.7766
- Accuracy: 0.715
- Precision: 0.7187
- Recall: 0.7150
- F1: 0.7138

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 5e-05
- train_batch_size: 16
- eval_batch_size: 16
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 500
- num_epochs: 20
- mixed_precision_training: Native AMP

### Training results

| Training Loss | Epoch | Step | Validation Loss | Accuracy | Precision | Recall | F1     |
|:-------------:|:-----:|:----:|:---------------:|:--------:|:---------:|:------:|:------:|
| 0.6998        | 1.0   | 107  | 0.6713          | 0.6      | 0.6214    | 0.6000 | 0.5815 |
| 0.7015        | 2.0   | 214  | 0.6502          | 0.68     | 0.7143    | 0.6800 | 0.6667 |
| 0.6755        | 3.0   | 321  | 0.6740          | 0.53     | 0.6579    | 0.53   | 0.4107 |
| 0.6605        | 4.0   | 428  | 0.6061          | 0.64     | 0.6502    | 0.64   | 0.6338 |
| 0.5918        | 5.0   | 535  | 0.5675          | 0.695    | 0.7023    | 0.6950 | 0.6922 |
| 0.5717        | 6.0   | 642  | 0.5945          | 0.685    | 0.6953    | 0.685  | 0.6808 |
| 0.4655        | 7.0   | 749  | 0.5644          | 0.68     | 0.6801    | 0.6800 | 0.6800 |
| 0.3407        | 8.0   | 856  | 0.7529          | 0.7      | 0.7029    | 0.7    | 0.6989 |
| 0.3539        | 9.0   | 963  | 0.7211          | 0.69     | 0.6901    | 0.69   | 0.6900 |
| 0.2695        | 10.0  | 1070 | 0.7760          | 0.685    | 0.6905    | 0.685  | 0.6827 |
| 0.1666        | 11.0  | 1177 | 1.1053          | 0.71     | 0.7188    | 0.71   | 0.7071 |
| 0.1648        | 12.0  | 1284 | 1.1662          | 0.72     | 0.7258    | 0.72   | 0.7182 |
| 0.1229        | 13.0  | 1391 | 1.2760          | 0.735    | 0.7438    | 0.735  | 0.7326 |
| 0.0737        | 14.0  | 1498 | 1.5943          | 0.7      | 0.7029    | 0.7    | 0.6989 |
| 0.1196        | 15.0  | 1605 | 1.5407          | 0.705    | 0.7085    | 0.7050 | 0.7037 |
| 0.0389        | 16.0  | 1712 | 1.6411          | 0.69     | 0.7016    | 0.69   | 0.6855 |
| 0.0199        | 17.0  | 1819 | 1.7139          | 0.685    | 0.6919    | 0.685  | 0.6821 |
| 0.0453        | 18.0  | 1926 | 1.6549          | 0.71     | 0.7121    | 0.71   | 0.7093 |
| 0.0536        | 19.0  | 2033 | 1.7612          | 0.71     | 0.7142    | 0.71   | 0.7086 |
| 0.0035        | 20.0  | 2140 | 1.7766          | 0.715    | 0.7187    | 0.7150 | 0.7138 |


### Framework versions

- Transformers 4.35.2
- Pytorch 2.1.0+cu121
- Datasets 2.16.1
- Tokenizers 0.15.0