---
tags:
- generated_from_trainer
model-index:
- name: dna_bert_3_1000seq-finetuned
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# dna_bert_3_1000seq-finetuned

This model is a fine-tuned version of [armheb/DNA_bert_3](https://huggingface.co/armheb/DNA_bert_3) on an unknown dataset.
It achieves the following results on the evaluation set:
- Loss: 0.4684

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 16
- eval_batch_size: 16
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 100
- mixed_precision_training: Native AMP

### Training results

| Training Loss | Epoch | Step | Validation Loss |
|:-------------:|:-----:|:----:|:---------------:|
| 0.8607        | 1.0   | 62   | 0.6257          |
| 0.6177        | 2.0   | 124  | 0.6120          |
| 0.6098        | 3.0   | 186  | 0.6062          |
| 0.604         | 4.0   | 248  | 0.6052          |
| 0.5999        | 5.0   | 310  | 0.6040          |
| 0.5982        | 6.0   | 372  | 0.5996          |
| 0.5985        | 7.0   | 434  | 0.5985          |
| 0.5956        | 8.0   | 496  | 0.5968          |
| 0.5936        | 9.0   | 558  | 0.5950          |
| 0.5908        | 10.0  | 620  | 0.5941          |
| 0.5904        | 11.0  | 682  | 0.5932          |
| 0.59          | 12.0  | 744  | 0.5917          |
| 0.5877        | 13.0  | 806  | 0.5921          |
| 0.5847        | 14.0  | 868  | 0.5903          |
| 0.5831        | 15.0  | 930  | 0.5887          |
| 0.5852        | 16.0  | 992  | 0.5878          |
| 0.5805        | 17.0  | 1054 | 0.5872          |
| 0.5795        | 18.0  | 1116 | 0.5853          |
| 0.5754        | 19.0  | 1178 | 0.5869          |
| 0.5757        | 20.0  | 1240 | 0.5839          |
| 0.5722        | 21.0  | 1302 | 0.5831          |
| 0.5693        | 22.0  | 1364 | 0.5811          |
| 0.5667        | 23.0  | 1426 | 0.5802          |
| 0.5652        | 24.0  | 1488 | 0.5775          |
| 0.5608        | 25.0  | 1550 | 0.5788          |
| 0.5591        | 26.0  | 1612 | 0.5724          |
| 0.5538        | 27.0  | 1674 | 0.5736          |
| 0.552         | 28.0  | 1736 | 0.5689          |
| 0.5483        | 29.0  | 1798 | 0.5689          |
| 0.5442        | 30.0  | 1860 | 0.5671          |
| 0.5405        | 31.0  | 1922 | 0.5658          |
| 0.537         | 32.0  | 1984 | 0.5605          |
| 0.5349        | 33.0  | 2046 | 0.5575          |
| 0.5275        | 34.0  | 2108 | 0.5569          |
| 0.5227        | 35.0  | 2170 | 0.5537          |
| 0.52          | 36.0  | 2232 | 0.5509          |
| 0.5173        | 37.0  | 2294 | 0.5504          |
| 0.5123        | 38.0  | 2356 | 0.5435          |
| 0.5088        | 39.0  | 2418 | 0.5472          |
| 0.5037        | 40.0  | 2480 | 0.5383          |
| 0.501         | 41.0  | 2542 | 0.5379          |
| 0.4931        | 42.0  | 2604 | 0.5365          |
| 0.4923        | 43.0  | 2666 | 0.5328          |
| 0.4879        | 44.0  | 2728 | 0.5301          |
| 0.482         | 45.0  | 2790 | 0.5295          |
| 0.4805        | 46.0  | 2852 | 0.5261          |
| 0.4772        | 47.0  | 2914 | 0.5221          |
| 0.4738        | 48.0  | 2976 | 0.5234          |
| 0.4674        | 49.0  | 3038 | 0.5210          |
| 0.4646        | 50.0  | 3100 | 0.5169          |
| 0.4621        | 51.0  | 3162 | 0.5142          |
| 0.4574        | 52.0  | 3224 | 0.5129          |
| 0.4552        | 53.0  | 3286 | 0.5127          |
| 0.4539        | 54.0  | 3348 | 0.5124          |
| 0.4506        | 55.0  | 3410 | 0.5076          |
| 0.4457        | 56.0  | 3472 | 0.5082          |
| 0.4454        | 57.0  | 3534 | 0.5027          |
| 0.4398        | 58.0  | 3596 | 0.5019          |
| 0.4386        | 59.0  | 3658 | 0.4998          |
| 0.4332        | 60.0  | 3720 | 0.4970          |
| 0.4277        | 61.0  | 3782 | 0.4995          |
| 0.4273        | 62.0  | 3844 | 0.4962          |
| 0.4235        | 63.0  | 3906 | 0.4909          |
| 0.4201        | 64.0  | 3968 | 0.4913          |
| 0.4198        | 65.0  | 4030 | 0.4899          |
| 0.4182        | 66.0  | 4092 | 0.4919          |
| 0.4157        | 67.0  | 4154 | 0.4902          |
| 0.4104        | 68.0  | 4216 | 0.4881          |
| 0.4095        | 69.0  | 4278 | 0.4881          |
| 0.4077        | 70.0  | 4340 | 0.4861          |
| 0.4064        | 71.0  | 4402 | 0.4868          |
| 0.4041        | 72.0  | 4464 | 0.4826          |
| 0.4029        | 73.0  | 4526 | 0.4833          |
| 0.3976        | 74.0  | 4588 | 0.4819          |
| 0.3997        | 75.0  | 4650 | 0.4809          |
| 0.3974        | 76.0  | 4712 | 0.4801          |
| 0.3953        | 77.0  | 4774 | 0.4783          |
| 0.3938        | 78.0  | 4836 | 0.4775          |
| 0.3934        | 79.0  | 4898 | 0.4762          |
| 0.3923        | 80.0  | 4960 | 0.4742          |
| 0.3893        | 81.0  | 5022 | 0.4742          |
| 0.3909        | 82.0  | 5084 | 0.4740          |
| 0.3856        | 83.0  | 5146 | 0.4739          |
| 0.3904        | 84.0  | 5208 | 0.4740          |
| 0.3883        | 85.0  | 5270 | 0.4701          |
| 0.3865        | 86.0  | 5332 | 0.4727          |
| 0.3809        | 87.0  | 5394 | 0.4736          |
| 0.3853        | 88.0  | 5456 | 0.4704          |
| 0.3821        | 89.0  | 5518 | 0.4704          |
| 0.3809        | 90.0  | 5580 | 0.4701          |
| 0.3814        | 91.0  | 5642 | 0.4698          |
| 0.3795        | 92.0  | 5704 | 0.4702          |
| 0.3804        | 93.0  | 5766 | 0.4692          |
| 0.377         | 94.0  | 5828 | 0.4683          |
| 0.3812        | 95.0  | 5890 | 0.4692          |
| 0.3806        | 96.0  | 5952 | 0.4683          |
| 0.3745        | 97.0  | 6014 | 0.4690          |
| 0.3825        | 98.0  | 6076 | 0.4684          |
| 0.374         | 99.0  | 6138 | 0.4687          |
| 0.3795        | 100.0 | 6200 | 0.4684          |


### Framework versions

- Transformers 4.21.1
- Pytorch 1.12.0+cu113
- Datasets 2.4.0
- Tokenizers 0.12.1