Hugging Face's logo

tanatapanun
/
fine-tuned-BioBART-50-epochs-1024-input-128-output

Text2Text Generation
Transformers
TensorBoard
Safetensors
bart
Generated from Trainer
Inference Endpoints
Model card Files Metrics Community
fine-tuned-BioBART-50-epochs-1024-input-128-output / README.md
tanatapanun's picture
tanatapanun
Model save
efd745d verified
preview code
|
raw
history blame contribute delete
6.55 kB
metadata
license: apache-2.0
base_model: GanjinZero/biobart-base
tags:
  - generated_from_trainer
metrics:
  - rouge
model-index:
  - name: fine-tuned-BioBART-50-epochs-1024-input-128-output
    results: []

fine-tuned-BioBART-50-epochs-1024-input-128-output

This model is a fine-tuned version of GanjinZero/biobart-base on the None dataset. It achieves the following results on the evaluation set:

  • Loss: 1.9109
  • Rouge1: 0.1191
  • Rouge2: 0.0252
  • Rougel: 0.105
  • Rougelsum: 0.1059
  • Gen Len: 16.2

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 0.0001
  • train_batch_size: 8
  • eval_batch_size: 8
  • seed: 42
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • lr_scheduler_warmup_ratio: 0.1
  • num_epochs: 50

Training results

Training Loss Epoch Step Validation Loss Rouge1 Rouge2 Rougel Rougelsum Gen Len
No log 1.0 151 8.7986 0.0 0.0 0.0 0.0 14.54
No log 2.0 302 4.6009 0.007 0.0022 0.0066 0.0067 4.73
No log 3.0 453 1.9851 0.1025 0.0246 0.091 0.0906 13.95
6.1578 4.0 604 1.7001 0.0763 0.0172 0.0666 0.0674 10.25
6.1578 5.0 755 1.6023 0.1303 0.0277 0.1167 0.1164 15.08
6.1578 6.0 906 1.5322 0.0795 0.0176 0.0732 0.0736 14.54
1.4113 7.0 1057 1.4998 0.0972 0.0241 0.0839 0.0838 13.47
1.4113 8.0 1208 1.4808 0.0992 0.0238 0.0894 0.0898 14.28
1.4113 9.0 1359 1.4964 0.1249 0.0214 0.111 0.1106 12.36
0.8834 10.0 1510 1.4858 0.1459 0.0363 0.1235 0.1237 15.63
0.8834 11.0 1661 1.4990 0.1578 0.0403 0.1379 0.139 15.92
0.8834 12.0 1812 1.5210 0.1327 0.0253 0.1212 0.1209 15.11
0.8834 13.0 1963 1.5381 0.1372 0.038 0.1255 0.1251 15.45
0.5229 14.0 2114 1.5559 0.1383 0.0348 0.1263 0.1263 16.49
0.5229 15.0 2265 1.5824 0.1509 0.0369 0.1336 0.1325 15.78
0.5229 16.0 2416 1.6369 0.128 0.0298 0.1176 0.1185 14.12
0.2708 17.0 2567 1.6393 0.1362 0.0429 0.1229 0.1229 15.77
0.2708 18.0 2718 1.6599 0.1521 0.0402 0.1329 0.1333 15.34
0.2708 19.0 2869 1.6705 0.1293 0.0265 0.1165 0.1166 16.51
0.1203 20.0 3020 1.6943 0.141 0.0289 0.1273 0.1275 15.69
0.1203 21.0 3171 1.6969 0.1253 0.0337 0.1081 0.1085 16.35
0.1203 22.0 3322 1.7431 0.1319 0.0272 0.1185 0.1185 15.63
0.1203 23.0 3473 1.7434 0.1357 0.0343 0.1253 0.125 16.39
0.0509 24.0 3624 1.7507 0.1375 0.0325 0.1233 0.1231 16.79
0.0509 25.0 3775 1.7776 0.1222 0.0328 0.1121 0.1121 16.18
0.0509 26.0 3926 1.7733 0.1265 0.0216 0.1166 0.117 16.25
0.0257 27.0 4077 1.8001 0.1238 0.0239 0.1116 0.1113 16.44
0.0257 28.0 4228 1.7955 0.1173 0.0221 0.103 0.1046 16.64
0.0257 29.0 4379 1.8143 0.1311 0.0273 0.1186 0.1183 16.78
0.0164 30.0 4530 1.8108 0.1331 0.0296 0.1219 0.1226 15.64
0.0164 31.0 4681 1.8184 0.1245 0.0339 0.1134 0.1143 16.55
0.0164 32.0 4832 1.8545 0.1101 0.0217 0.0982 0.0998 16.09
0.0164 33.0 4983 1.8550 0.1421 0.0322 0.1292 0.1296 16.07
0.0117 34.0 5134 1.8573 0.1309 0.0292 0.1192 0.1193 16.0
0.0117 35.0 5285 1.8453 0.1254 0.0238 0.1133 0.1139 16.55
0.0117 36.0 5436 1.8724 0.1167 0.0241 0.1024 0.1035 15.89
0.0089 37.0 5587 1.8761 0.1345 0.0275 0.1206 0.1208 15.87
0.0089 38.0 5738 1.8772 0.1338 0.0301 0.1216 0.1228 16.78
0.0089 39.0 5889 1.8654 0.134 0.0264 0.1193 0.1196 16.85
0.0071 40.0 6040 1.8812 0.129 0.0287 0.1181 0.1177 16.12
0.0071 41.0 6191 1.8838 0.1238 0.0274 0.1134 0.1134 16.29
0.0071 42.0 6342 1.8752 0.1334 0.0262 0.1209 0.1214 16.66
0.0071 43.0 6493 1.8993 0.1238 0.0254 0.1111 0.1113 16.31
0.0056 44.0 6644 1.8963 0.1279 0.0346 0.1133 0.1154 16.07
0.0056 45.0 6795 1.9079 0.1225 0.0261 0.108 0.1084 16.09
0.0056 46.0 6946 1.9132 0.129 0.025 0.1157 0.1154 16.26
0.0045 47.0 7097 1.9120 0.1419 0.0362 0.1275 0.1278 15.78
0.0045 48.0 7248 1.9069 0.1316 0.0253 0.1161 0.1165 16.38
0.0045 49.0 7399 1.9099 0.1206 0.0259 0.1074 0.1077 16.32
0.0041 50.0 7550 1.9109 0.1191 0.0252 0.105 0.1059 16.2

Framework versions

  • Transformers 4.36.2
  • Pytorch 1.12.1+cu113
  • Datasets 2.16.1
  • Tokenizers 0.15.0