Hugging Face's logo

tanatapanun
/
fine-tuned-BioBART-20-epochs-1500-input-256-output

Text2Text Generation
Transformers
TensorBoard
Safetensors
bart
Generated from Trainer
Inference Endpoints
Model card Files Metrics Community
tanatapanun commited on
Commit
98fe9a4
·
verified ·
1 Parent(s): 2665546

Model save

Browse files
Files changed (4) hide show
  1. README.md +38 -3
  2. generation_config.json +0 -1
  3. model.safetensors +1 -1
  4. runs/Jan24_20-16-20_william-gpu-3090-10-8vlnc/events.out.tfevents.1706127381.william-gpu-3090-10-8vlnc.4542.0 +2 -2
README.md CHANGED
@@ -3,6 +3,8 @@ license: apache-2.0
3
  base_model: GanjinZero/biobart-base
4
  tags:
5
  - generated_from_trainer
 
 
6
  model-index:
7
  - name: fine-tuned-BioBART-20-epochs-1500-input-256-output
8
  results: []
@@ -14,6 +16,13 @@ should probably proofread and complete it, then remove this comment. -->
14
  # fine-tuned-BioBART-20-epochs-1500-input-256-output
15
 
16
  This model is a fine-tuned version of [GanjinZero/biobart-base](https://huggingface.co/GanjinZero/biobart-base) on the None dataset.
 
 
 
 
 
 
 
17
 
18
  ## Model description
19
 
@@ -33,13 +42,39 @@ More information needed
33
 
34
  The following hyperparameters were used during training:
35
  - learning_rate: 0.0001
36
- - train_batch_size: 4
37
- - eval_batch_size: 4
38
  - seed: 42
39
  - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
40
  - lr_scheduler_type: linear
41
  - lr_scheduler_warmup_ratio: 0.1
42
- - num_epochs: 15
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
43
 
44
  ### Framework versions
45
 
 
3
  base_model: GanjinZero/biobart-base
4
  tags:
5
  - generated_from_trainer
6
+ metrics:
7
+ - rouge
8
  model-index:
9
  - name: fine-tuned-BioBART-20-epochs-1500-input-256-output
10
  results: []
 
16
  # fine-tuned-BioBART-20-epochs-1500-input-256-output
17
 
18
  This model is a fine-tuned version of [GanjinZero/biobart-base](https://huggingface.co/GanjinZero/biobart-base) on the None dataset.
19
+ It achieves the following results on the evaluation set:
20
+ - Loss: 0.9257
21
+ - Rouge1: 0.1655
22
+ - Rouge2: 0.0291
23
+ - Rougel: 0.1256
24
+ - Rougelsum: 0.1266
25
+ - Gen Len: 34.62
26
 
27
  ## Model description
28
 
 
42
 
43
  The following hyperparameters were used during training:
44
  - learning_rate: 0.0001
45
+ - train_batch_size: 8
46
+ - eval_batch_size: 8
47
  - seed: 42
48
  - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
49
  - lr_scheduler_type: linear
50
  - lr_scheduler_warmup_ratio: 0.1
51
+ - num_epochs: 20
52
+
53
+ ### Training results
54
+
55
+ | Training Loss | Epoch | Step | Validation Loss | Rouge1 | Rouge2 | Rougel | Rougelsum | Gen Len |
56
+ |:-------------:|:-----:|:----:|:---------------:|:------:|:------:|:------:|:---------:|:-------:|
57
+ | No log | 1.0 | 151 | 6.1052 | 0.0511 | 0.0 | 0.047 | 0.0474 | 22.48 |
58
+ | No log | 2.0 | 302 | 1.1483 | 0.077 | 0.0156 | 0.0673 | 0.0678 | 11.56 |
59
+ | No log | 3.0 | 453 | 0.9767 | 0.0744 | 0.0182 | 0.0537 | 0.0557 | 23.57 |
60
+ | 4.0217 | 4.0 | 604 | 0.9160 | 0.1355 | 0.033 | 0.1053 | 0.1042 | 37.77 |
61
+ | 4.0217 | 5.0 | 755 | 0.8850 | 0.1682 | 0.0352 | 0.1342 | 0.1342 | 41.92 |
62
+ | 4.0217 | 6.0 | 906 | 0.8736 | 0.1342 | 0.0308 | 0.1037 | 0.1037 | 35.34 |
63
+ | 0.761 | 7.0 | 1057 | 0.8582 | 0.144 | 0.0361 | 0.1082 | 0.1095 | 39.27 |
64
+ | 0.761 | 8.0 | 1208 | 0.8551 | 0.165 | 0.0392 | 0.1233 | 0.1254 | 39.55 |
65
+ | 0.761 | 9.0 | 1359 | 0.8623 | 0.141 | 0.0302 | 0.1169 | 0.1179 | 23.69 |
66
+ | 0.5257 | 10.0 | 1510 | 0.8642 | 0.1715 | 0.0436 | 0.1249 | 0.1267 | 45.78 |
67
+ | 0.5257 | 11.0 | 1661 | 0.8705 | 0.1702 | 0.0331 | 0.1386 | 0.1385 | 30.28 |
68
+ | 0.5257 | 12.0 | 1812 | 0.8761 | 0.169 | 0.035 | 0.1247 | 0.1254 | 42.74 |
69
+ | 0.5257 | 13.0 | 1963 | 0.8938 | 0.1719 | 0.0376 | 0.139 | 0.1389 | 29.73 |
70
+ | 0.368 | 14.0 | 2114 | 0.8907 | 0.1716 | 0.0402 | 0.1371 | 0.1377 | 36.07 |
71
+ | 0.368 | 15.0 | 2265 | 0.9027 | 0.1677 | 0.0324 | 0.1329 | 0.134 | 36.82 |
72
+ | 0.368 | 16.0 | 2416 | 0.9141 | 0.16 | 0.0322 | 0.1268 | 0.1281 | 32.87 |
73
+ | 0.2635 | 17.0 | 2567 | 0.9177 | 0.1702 | 0.0324 | 0.1312 | 0.1323 | 35.4 |
74
+ | 0.2635 | 18.0 | 2718 | 0.9194 | 0.1713 | 0.0333 | 0.1297 | 0.1312 | 37.75 |
75
+ | 0.2635 | 19.0 | 2869 | 0.9234 | 0.1693 | 0.0294 | 0.1293 | 0.1299 | 35.69 |
76
+ | 0.2141 | 20.0 | 3020 | 0.9257 | 0.1655 | 0.0291 | 0.1256 | 0.1266 | 34.62 |
77
+
78
 
79
  ### Framework versions
80
 
generation_config.json CHANGED
@@ -1,5 +1,4 @@
1
  {
2
- "_from_model_config": true,
3
  "bos_token_id": 0,
4
  "decoder_start_token_id": 2,
5
  "early_stopping": true,
 
1
  {
 
2
  "bos_token_id": 0,
3
  "decoder_start_token_id": 2,
4
  "early_stopping": true,
model.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:1ca7bddf50311a47c656bfee63f848518796316384e9182d0d4f3fbc045f8f67
3
  size 560837164
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ebcc535997b8a62694eb537636219f9ee0128b430fb90c23d8d9aa21c7420b29
3
  size 560837164
runs/Jan24_20-16-20_william-gpu-3090-10-8vlnc/events.out.tfevents.1706127381.william-gpu-3090-10-8vlnc.4542.0 CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:6233b3e51740815cc006003ba995aad944f151294cbdf56b9a999b452b8fdcef
3
- size 16642
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:33c4b8fe71a520c483b68bb879d15b5c0ff6bb5e8b964d1f5e29b7465167c60a
3
+ size 17521