---
library_name: transformers
license: mit
base_model: microsoft/deberta-v3-large
tags:
- generated_from_trainer
model-index:
- name: deberta_large
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# deberta_large

This model is a fine-tuned version of [microsoft/deberta-v3-large](https://huggingface.co/microsoft/deberta-v3-large) on the None dataset.
It achieves the following results on the evaluation set:
- Loss: 0.6965

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 5e-05
- train_batch_size: 4
- eval_batch_size: 8
- seed: 3407
- gradient_accumulation_steps: 2
- total_train_batch_size: 8
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: cosine
- lr_scheduler_warmup_steps: 10
- num_epochs: 1

### Training results

| Training Loss | Epoch  | Step | Validation Loss |
|:-------------:|:------:|:----:|:---------------:|
| 0.471         | 0.0173 | 20   | 0.2712          |
| 1.4977        | 0.0346 | 40   | 0.7702          |
| 0.7066        | 0.0519 | 60   | 0.4904          |
| 0.6817        | 0.0692 | 80   | 1.0408          |
| 0.897         | 0.0865 | 100  | 1.7531          |
| 0.6793        | 0.1039 | 120  | 0.8040          |
| 0.7946        | 0.1212 | 140  | 0.6972          |
| 0.7391        | 0.1385 | 160  | 0.6933          |
| 0.7441        | 0.1558 | 180  | 0.7431          |
| 0.656         | 0.1731 | 200  | 0.8636          |
| 0.5681        | 0.1904 | 220  | 0.7142          |
| 0.9031        | 0.2077 | 240  | 0.7183          |
| 0.5978        | 0.2250 | 260  | 0.7371          |
| 0.6308        | 0.2423 | 280  | 0.7036          |
| 0.6829        | 0.2596 | 300  | 0.6932          |
| 0.6472        | 0.2769 | 320  | 0.7010          |
| 0.6331        | 0.2942 | 340  | 0.7316          |
| 0.7698        | 0.3116 | 360  | 0.6938          |
| 0.6704        | 0.3289 | 380  | 0.7128          |
| 0.7532        | 0.3462 | 400  | 0.6992          |
| 0.703         | 0.3635 | 420  | 0.6932          |
| 0.7747        | 0.3808 | 440  | 0.7437          |
| 0.7063        | 0.3981 | 460  | 0.6931          |
| 0.6013        | 0.4154 | 480  | 0.7127          |
| 0.768         | 0.4327 | 500  | 0.6955          |
| 0.682         | 0.4500 | 520  | 0.6948          |
| 0.7419        | 0.4673 | 540  | 0.6976          |
| 0.6928        | 0.4846 | 560  | 0.6963          |
| 0.6246        | 0.5019 | 580  | 0.6995          |
| 0.6742        | 0.5193 | 600  | 0.6965          |
| 0.6419        | 0.5366 | 620  | 0.6947          |
| 0.7268        | 0.5539 | 640  | 0.6996          |
| 0.7054        | 0.5712 | 660  | 0.6938          |
| 0.6931        | 0.5885 | 680  | 0.7010          |
| 0.7344        | 0.6058 | 700  | 0.6975          |
| 0.7151        | 0.6231 | 720  | 0.6950          |
| 0.7106        | 0.6404 | 740  | 0.7027          |
| 0.6829        | 0.6577 | 760  | 0.6941          |
| 0.6883        | 0.6750 | 780  | 0.7005          |
| 0.6268        | 0.6923 | 800  | 0.6978          |
| 0.6674        | 0.7096 | 820  | 0.6979          |
| 0.6704        | 0.7270 | 840  | 0.7024          |
| 0.7527        | 0.7443 | 860  | 0.7010          |
| 0.7239        | 0.7616 | 880  | 0.7060          |
| 0.7611        | 0.7789 | 900  | 0.7033          |
| 0.7522        | 0.7962 | 920  | 0.7010          |
| 0.773         | 0.8135 | 940  | 0.6992          |
| 0.6564        | 0.8308 | 960  | 0.6959          |
| 0.7369        | 0.8481 | 980  | 0.6970          |
| 0.7119        | 0.8654 | 1000 | 0.6971          |
| 0.7181        | 0.8827 | 1020 | 0.6971          |
| 0.7011        | 0.9000 | 1040 | 0.6975          |
| 0.7414        | 0.9174 | 1060 | 0.6968          |
| 0.7732        | 0.9347 | 1080 | 0.6968          |
| 0.6499        | 0.9520 | 1100 | 0.6965          |
| 0.6681        | 0.9693 | 1120 | 0.6965          |
| 0.7831        | 0.9866 | 1140 | 0.6965          |


### Framework versions

- Transformers 4.44.2
- Pytorch 2.4.1+cu121
- Datasets 3.0.1
- Tokenizers 0.19.1