---
license: apache-2.0
base_model: GanjinZero/biobart-v2-base
tags:
- generated_from_trainer
metrics:
- rouge
model-index:
- name: fine-tuned-BioBART-20-epochs-wang-lab
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# fine-tuned-BioBART-20-epochs-wang-lab

This model is a fine-tuned version of [GanjinZero/biobart-v2-base](https://huggingface.co/GanjinZero/biobart-v2-base) on the None dataset.
It achieves the following results on the evaluation set:
- Loss: 1.0360
- Rouge1: 0.308
- Rouge2: 0.1254
- Rougel: 0.2791
- Rougelsum: 0.2797
- Gen Len: 15.85

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 0.0001
- train_batch_size: 4
- eval_batch_size: 4
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_ratio: 0.1
- num_epochs: 20

### Training results

| Training Loss | Epoch | Step | Validation Loss | Rouge1 | Rouge2 | Rougel | Rougelsum | Gen Len |
|:-------------:|:-----:|:----:|:---------------:|:------:|:------:|:------:|:---------:|:-------:|
| No log        | 1.0   | 301  | 0.7404          | 0.2534 | 0.1025 | 0.2218 | 0.221     | 15.29   |
| 0.851         | 2.0   | 602  | 0.7099          | 0.2907 | 0.1069 | 0.255  | 0.2557    | 15.04   |
| 0.851         | 3.0   | 903  | 0.6956          | 0.2652 | 0.1049 | 0.2313 | 0.23      | 16.04   |
| 0.5436        | 4.0   | 1204 | 0.7288          | 0.3168 | 0.129  | 0.286  | 0.2872    | 15.11   |
| 0.3356        | 5.0   | 1505 | 0.7856          | 0.2869 | 0.1041 | 0.2569 | 0.2601    | 15.4    |
| 0.3356        | 6.0   | 1806 | 0.8174          | 0.2863 | 0.1214 | 0.2447 | 0.2444    | 15.95   |
| 0.1807        | 7.0   | 2107 | 0.8477          | 0.3048 | 0.1169 | 0.2787 | 0.2797    | 15.82   |
| 0.1807        | 8.0   | 2408 | 0.8816          | 0.3118 | 0.1227 | 0.2726 | 0.2733    | 15.66   |
| 0.1072        | 9.0   | 2709 | 0.9081          | 0.2988 | 0.1169 | 0.2665 | 0.2668    | 14.73   |
| 0.0649        | 10.0  | 3010 | 0.9342          | 0.2869 | 0.1175 | 0.2531 | 0.2535    | 15.63   |
| 0.0649        | 11.0  | 3311 | 0.9588          | 0.3094 | 0.1212 | 0.2722 | 0.2747    | 15.77   |
| 0.0376        | 12.0  | 3612 | 0.9761          | 0.3147 | 0.1197 | 0.2867 | 0.2882    | 15.62   |
| 0.0376        | 13.0  | 3913 | 0.9870          | 0.3144 | 0.1172 | 0.2843 | 0.285     | 16.09   |
| 0.0244        | 14.0  | 4214 | 0.9918          | 0.3217 | 0.1267 | 0.2931 | 0.2942    | 15.94   |
| 0.0145        | 15.0  | 4515 | 1.0044          | 0.3102 | 0.1196 | 0.2801 | 0.2815    | 15.91   |
| 0.0145        | 16.0  | 4816 | 1.0152          | 0.3094 | 0.1316 | 0.2796 | 0.2804    | 16.01   |
| 0.0094        | 17.0  | 5117 | 1.0290          | 0.317  | 0.1133 | 0.2838 | 0.2857    | 15.77   |
| 0.0094        | 18.0  | 5418 | 1.0337          | 0.3006 | 0.1216 | 0.2712 | 0.272     | 15.9    |
| 0.0066        | 19.0  | 5719 | 1.0346          | 0.307  | 0.1254 | 0.2785 | 0.2797    | 15.85   |
| 0.0047        | 20.0  | 6020 | 1.0360          | 0.308  | 0.1254 | 0.2791 | 0.2797    | 15.85   |


### Framework versions

- Transformers 4.36.2
- Pytorch 1.12.1+cu113
- Datasets 2.15.0
- Tokenizers 0.15.0