File size: 3,462 Bytes
95fd42a c8516e2 95fd42a 8437cde 95fd42a c8516e2 95fd42a c8516e2 95fd42a |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 |
---
tags:
- generated_from_trainer
metrics:
- rouge
model-index:
- name: ViT-Bert_Mimic
results: []
---
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->
# ViT-Bert_Mimic
This model is a fine-tuned version of [](https://huggingface.co/) on an unknown dataset.
It achieves the following results on the evaluation set:
- Loss: 0.1305
- Rouge1: 34.725
- Rouge2: 21.4916
- Rougel: 33.3614
- Rougelsum: 34.1142
- Gen Len: 20.706
## Model description
More information needed
## Intended uses & limitations
More information needed
## Training and evaluation data
More information needed
## Training procedure
### Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 5e-05
- train_batch_size: 8
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 20
### Training results
| Training Loss | Epoch | Step | Validation Loss | Rouge1 | Rouge2 | Rougel | Rougelsum | Gen Len |
|:-------------:|:-----:|:------:|:---------------:|:-------:|:-------:|:-------:|:---------:|:-------:|
| 0.0684 | 1.0 | 7500 | 0.0752 | 34.4312 | 25.586 | 34.2067 | 34.2816 | 14.065 |
| 0.0626 | 2.0 | 15000 | 0.0694 | 38.0498 | 26.9882 | 37.2064 | 37.6682 | 19.492 |
| 0.0599 | 3.0 | 22500 | 0.0676 | 37.9403 | 26.7796 | 37.0514 | 37.571 | 21.805 |
| 0.054 | 4.0 | 30000 | 0.0661 | 38.1215 | 26.8065 | 37.3608 | 37.7763 | 18.883 |
| 0.0484 | 5.0 | 37500 | 0.0658 | 39.0689 | 27.489 | 38.0601 | 38.8175 | 20.556 |
| 0.043 | 6.0 | 45000 | 0.0679 | 38.5537 | 26.6503 | 37.4722 | 38.1314 | 20.994 |
| 0.0378 | 7.0 | 52500 | 0.0701 | 37.8821 | 26.1994 | 36.7872 | 37.4123 | 19.978 |
| 0.0324 | 8.0 | 60000 | 0.0741 | 38.5791 | 26.2187 | 37.3411 | 38.0767 | 21.761 |
| 0.0269 | 9.0 | 67500 | 0.0787 | 36.2698 | 24.3513 | 35.1553 | 35.7864 | 20.512 |
| 0.0199 | 10.0 | 75000 | 0.0848 | 34.8266 | 22.0111 | 33.591 | 34.3348 | 19.67 |
| 0.0158 | 11.0 | 82500 | 0.0921 | 34.5083 | 21.5876 | 33.273 | 34.0396 | 20.663 |
| 0.0114 | 12.0 | 90000 | 0.0990 | 33.6601 | 20.3509 | 32.3799 | 33.1785 | 21.574 |
| 0.0078 | 13.0 | 97500 | 0.1057 | 33.5222 | 20.262 | 32.3084 | 33.0449 | 20.7 |
| 0.0057 | 14.0 | 105000 | 0.1122 | 32.9482 | 19.0875 | 31.6809 | 32.4176 | 21.562 |
| 0.0037 | 15.0 | 112500 | 0.1172 | 33.2572 | 19.0712 | 31.8675 | 32.7193 | 21.432 |
| 0.0027 | 16.0 | 120000 | 0.1215 | 34.0583 | 20.5815 | 32.5961 | 33.4699 | 21.379 |
| 0.0019 | 17.0 | 127500 | 0.1257 | 34.3046 | 21.1929 | 33.0026 | 33.6992 | 20.687 |
| 0.0013 | 18.0 | 135000 | 0.1280 | 34.9621 | 21.8578 | 33.6017 | 34.3908 | 21.249 |
| 0.001 | 19.0 | 142500 | 0.1298 | 35.1328 | 21.8242 | 33.7634 | 34.5288 | 20.567 |
| 0.0007 | 20.0 | 150000 | 0.1305 | 34.725 | 21.4916 | 33.3614 | 34.1142 | 20.706 |
### Framework versions
- Transformers 4.37.1
- Pytorch 1.13.1+cu117
- Datasets 2.15.0
- Tokenizers 0.15.1
|