---
tags:
- generated_from_trainer
model-index:
- name: DNADebertaK6b
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# DNADebertaK6b

This model is a fine-tuned version of [](https://huggingface.co/) on the None dataset.
It achieves the following results on the evaluation set:
- Loss: 1.4362

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 5e-05
- train_batch_size: 64
- eval_batch_size: 64
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 15
- mixed_precision_training: Native AMP

### Training results

| Training Loss | Epoch | Step    | Validation Loss |
|:-------------:|:-----:|:-------:|:---------------:|
| 4.061         | 0.25  | 20000   | 1.7733          |
| 1.7344        | 0.5   | 40000   | 1.6608          |
| 1.6651        | 0.75  | 60000   | 1.6319          |
| 1.6359        | 0.99  | 80000   | 1.6092          |
| 1.6131        | 1.24  | 100000  | 1.5932          |
| 1.5959        | 1.49  | 120000  | 1.5753          |
| 1.5827        | 1.74  | 140000  | 1.5624          |
| 1.5719        | 1.99  | 160000  | 1.5534          |
| 1.5617        | 2.24  | 180000  | 1.5454          |
| 1.5551        | 2.49  | 200000  | 1.5403          |
| 1.5477        | 2.74  | 220000  | 1.5322          |
| 1.5414        | 2.98  | 240000  | 1.5262          |
| 1.5366        | 3.23  | 260000  | 1.5220          |
| 1.5308        | 3.48  | 280000  | 1.5184          |
| 1.5274        | 3.73  | 300000  | 1.5121          |
| 1.5224        | 3.98  | 320000  | 1.5085          |
| 1.5194        | 4.23  | 340000  | 1.5050          |
| 1.5164        | 4.48  | 360000  | 1.5027          |
| 1.5126        | 4.72  | 380000  | 1.4984          |
| 1.5086        | 4.97  | 400000  | 1.4947          |
| 1.5048        | 5.22  | 420000  | 1.4914          |
| 1.5025        | 5.47  | 440000  | 1.4914          |
| 1.5006        | 5.72  | 460000  | 1.4877          |
| 1.4982        | 5.97  | 480000  | 1.4840          |
| 1.4952        | 6.22  | 500000  | 1.4825          |
| 1.4926        | 6.46  | 520000  | 1.4800          |
| 1.4907        | 6.71  | 540000  | 1.4778          |
| 1.4886        | 6.96  | 560000  | 1.4761          |
| 1.4864        | 7.21  | 580000  | 1.4746          |
| 1.4854        | 7.46  | 600000  | 1.4730          |
| 1.484         | 7.71  | 620000  | 1.4709          |
| 1.4826        | 7.96  | 640000  | 1.4676          |
| 1.4794        | 8.21  | 660000  | 1.4674          |
| 1.479         | 8.45  | 680000  | 1.4658          |
| 1.4777        | 8.7   | 700000  | 1.4661          |
| 1.4751        | 8.95  | 720000  | 1.4649          |
| 1.4742        | 9.2   | 740000  | 1.4614          |
| 1.4728        | 9.45  | 760000  | 1.4602          |
| 1.472         | 9.7   | 780000  | 1.4603          |
| 1.4703        | 9.95  | 800000  | 1.4577          |
| 1.4694        | 10.19 | 820000  | 1.4578          |
| 1.4662        | 10.44 | 840000  | 1.4557          |
| 1.4668        | 10.69 | 860000  | 1.4545          |
| 1.466         | 10.94 | 880000  | 1.4548          |
| 1.465         | 11.19 | 900000  | 1.4513          |
| 1.4626        | 11.44 | 920000  | 1.4511          |
| 1.4616        | 11.69 | 940000  | 1.4509          |
| 1.4609        | 11.93 | 960000  | 1.4485          |
| 1.4595        | 12.18 | 980000  | 1.4474          |
| 1.4588        | 12.43 | 1000000 | 1.4470          |
| 1.4588        | 12.68 | 1020000 | 1.4452          |
| 1.4565        | 12.93 | 1040000 | 1.4443          |
| 1.4556        | 13.18 | 1060000 | 1.4433          |
| 1.4543        | 13.43 | 1080000 | 1.4409          |
| 1.453         | 13.68 | 1100000 | 1.4409          |
| 1.4524        | 13.92 | 1120000 | 1.4397          |
| 1.4511        | 14.17 | 1140000 | 1.4402          |
| 1.4501        | 14.42 | 1160000 | 1.4385          |
| 1.4484        | 14.67 | 1180000 | 1.4373          |
| 1.449         | 14.92 | 1200000 | 1.4360          |


### Framework versions

- Transformers 4.19.2
- Pytorch 1.11.0
- Datasets 2.2.2
- Tokenizers 0.12.1