---
license: mit
tags:
- generated_from_trainer
metrics:
- precision
- recall
- f1
- accuracy
model-index:
- name: Variome_5e-05_0404_ES6
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# Variome_5e-05_0404_ES6

This model is a fine-tuned version of [microsoft/BiomedNLP-PubMedBERT-base-uncased-abstract-fulltext](https://huggingface.co/microsoft/BiomedNLP-PubMedBERT-base-uncased-abstract-fulltext) on the None dataset.
It achieves the following results on the evaluation set:
- Loss: 0.0695
- Precision: 0.6584
- Recall: 0.6123
- F1: 0.6345
- Accuracy: 0.9866

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 5e-05
- train_batch_size: 16
- eval_batch_size: 16
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- training_steps: 2000

### Training results

| Training Loss | Epoch | Step | Validation Loss | Precision | Recall | F1     | Accuracy |
|:-------------:|:-----:|:----:|:---------------:|:---------:|:------:|:------:|:--------:|
| 0.56          | 0.13  | 25   | 0.1838          | 0.0       | 0.0    | 0.0    | 0.9759   |
| 0.1843        | 0.26  | 50   | 0.1810          | 0.0       | 0.0    | 0.0    | 0.9759   |
| 0.1822        | 0.39  | 75   | 0.1823          | 0.0       | 0.0    | 0.0    | 0.9759   |
| 0.1797        | 0.52  | 100  | 0.1800          | 0.0       | 0.0    | 0.0    | 0.9759   |
| 0.1618        | 0.65  | 125  | 0.1838          | 0.0       | 0.0    | 0.0    | 0.9759   |
| 0.1735        | 0.79  | 150  | 0.1784          | 0.0       | 0.0    | 0.0    | 0.9759   |
| 0.1654        | 0.92  | 175  | 0.1673          | 0.0       | 0.0    | 0.0    | 0.9759   |
| 0.1171        | 1.05  | 200  | 0.1367          | 0.1538    | 0.0038 | 0.0075 | 0.9760   |
| 0.1366        | 1.18  | 225  | 0.1333          | 0.1881    | 0.1881 | 0.1881 | 0.9752   |
| 0.1262        | 1.31  | 250  | 0.1110          | 0.2137    | 0.1411 | 0.1699 | 0.9771   |
| 0.1102        | 1.44  | 275  | 0.1016          | 0.3440    | 0.2582 | 0.2950 | 0.9789   |
| 0.099         | 1.57  | 300  | 0.0918          | 0.4712    | 0.3695 | 0.4142 | 0.9813   |
| 0.0843        | 1.7   | 325  | 0.0868          | 0.4654    | 0.3359 | 0.3902 | 0.9812   |
| 0.0833        | 1.83  | 350  | 0.0830          | 0.5755    | 0.3512 | 0.4362 | 0.9818   |
| 0.0689        | 1.96  | 375  | 0.0807          | 0.5766    | 0.3973 | 0.4705 | 0.9828   |
| 0.0656        | 2.09  | 400  | 0.0781          | 0.5361    | 0.4923 | 0.5133 | 0.9839   |
| 0.0625        | 2.23  | 425  | 0.0769          | 0.5802    | 0.4722 | 0.5206 | 0.9839   |
| 0.0707        | 2.36  | 450  | 0.0748          | 0.5979    | 0.4453 | 0.5105 | 0.9837   |
| 0.0568        | 2.49  | 475  | 0.0732          | 0.5515    | 0.5345 | 0.5429 | 0.9844   |
| 0.0659        | 2.62  | 500  | 0.0703          | 0.6244    | 0.5154 | 0.5647 | 0.9845   |
| 0.0601        | 2.75  | 525  | 0.0674          | 0.6174    | 0.5528 | 0.5833 | 0.9848   |
| 0.0554        | 2.88  | 550  | 0.0667          | 0.6230    | 0.5614 | 0.5906 | 0.9855   |
| 0.0634        | 3.01  | 575  | 0.0699          | 0.6605    | 0.5509 | 0.6007 | 0.9856   |
| 0.0449        | 3.14  | 600  | 0.0694          | 0.6249    | 0.5691 | 0.5957 | 0.9850   |
| 0.0435        | 3.27  | 625  | 0.0693          | 0.5868    | 0.5710 | 0.5788 | 0.9847   |
| 0.0338        | 3.4   | 650  | 0.0710          | 0.6321    | 0.5672 | 0.5979 | 0.9858   |
| 0.0405        | 3.53  | 675  | 0.0669          | 0.6245    | 0.5825 | 0.6028 | 0.9859   |
| 0.0459        | 3.66  | 700  | 0.0665          | 0.6808    | 0.5710 | 0.6211 | 0.9862   |
| 0.0471        | 3.8   | 725  | 0.0669          | 0.6162    | 0.5902 | 0.6029 | 0.9845   |
| 0.0477        | 3.93  | 750  | 0.0657          | 0.6655    | 0.5595 | 0.6079 | 0.9859   |
| 0.0476        | 4.06  | 775  | 0.0652          | 0.6483    | 0.6084 | 0.6277 | 0.9859   |
| 0.0419        | 4.19  | 800  | 0.0679          | 0.6134    | 0.5969 | 0.6051 | 0.9854   |
| 0.036         | 4.32  | 825  | 0.0693          | 0.6598    | 0.5864 | 0.6209 | 0.9854   |
| 0.0372        | 4.45  | 850  | 0.0666          | 0.6474    | 0.6008 | 0.6232 | 0.9859   |
| 0.0309        | 4.58  | 875  | 0.0702          | 0.6811    | 0.5883 | 0.6313 | 0.9864   |
| 0.0301        | 4.71  | 900  | 0.0659          | 0.6804    | 0.5864 | 0.6299 | 0.9866   |
| 0.0364        | 4.84  | 925  | 0.0651          | 0.6738    | 0.6008 | 0.6352 | 0.9867   |
| 0.0321        | 4.97  | 950  | 0.0640          | 0.6638    | 0.5931 | 0.6265 | 0.9867   |
| 0.0296        | 5.1   | 975  | 0.0657          | 0.6920    | 0.5845 | 0.6337 | 0.9868   |
| 0.0278        | 5.24  | 1000 | 0.0668          | 0.6739    | 0.5988 | 0.6341 | 0.9866   |
| 0.0273        | 5.37  | 1025 | 0.0673          | 0.6568    | 0.5969 | 0.6254 | 0.9867   |
| 0.0254        | 5.5   | 1050 | 0.0677          | 0.6969    | 0.5893 | 0.6386 | 0.9871   |
| 0.0271        | 5.63  | 1075 | 0.0658          | 0.6667    | 0.6104 | 0.6373 | 0.9861   |
| 0.025         | 5.76  | 1100 | 0.0695          | 0.6584    | 0.6123 | 0.6345 | 0.9866   |


### Framework versions

- Transformers 4.27.4
- Pytorch 2.0.0+cu118
- Datasets 2.11.0
- Tokenizers 0.13.2