Hugging Face's logo

tanatapanun
/
fine-tuned-BioBART-50-epochs-1024-input-192-output

Text2Text Generation
Transformers
TensorBoard
Safetensors
bart
Generated from Trainer
Inference Endpoints
Model card Files Metrics Community
Edit model card

fine-tuned-BioBART-50-epochs-1024-input-192-output

This model is a fine-tuned version of GanjinZero/biobart-base on the None dataset. It achieves the following results on the evaluation set:

  • Loss: 1.4193
  • Rouge1: 0.1671
  • Rouge2: 0.0236
  • Rougel: 0.1346
  • Rougelsum: 0.135
  • Gen Len: 39.47

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 0.0001
  • train_batch_size: 8
  • eval_batch_size: 8
  • seed: 42
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • lr_scheduler_warmup_ratio: 0.1
  • num_epochs: 50

Training results

Training Loss Epoch Step Validation Loss Rouge1 Rouge2 Rougel Rougelsum Gen Len
No log 1.0 151 8.7921 0.0814 0.0009 0.0732 0.0736 37.05
No log 2.0 302 4.1000 0.0056 0.0014 0.0055 0.0054 4.67
No log 3.0 453 1.4452 0.1244 0.0299 0.1124 0.1108 21.03
5.9057 4.0 604 1.2534 0.1025 0.0216 0.0814 0.0812 23.24
5.9057 5.0 755 1.1841 0.1353 0.0266 0.1003 0.1 51.9
5.9057 6.0 906 1.1373 0.0884 0.0207 0.0708 0.0711 20.47
1.0961 7.0 1057 1.1075 0.1223 0.0317 0.0901 0.0905 33.73
1.0961 8.0 1208 1.1046 0.1597 0.036 0.1304 0.1309 32.43
1.0961 9.0 1359 1.0959 0.1452 0.0259 0.1183 0.1181 30.69
0.7053 10.0 1510 1.1090 0.1672 0.0374 0.1244 0.124 40.15
0.7053 11.0 1661 1.1060 0.1739 0.032 0.1392 0.1379 30.7
0.7053 12.0 1812 1.1295 0.1754 0.0341 0.1313 0.1296 48.33
0.7053 13.0 1963 1.1343 0.1911 0.0546 0.1616 0.1591 36.76
0.4358 14.0 2114 1.1602 0.1759 0.0355 0.1406 0.1408 31.15
0.4358 15.0 2265 1.1586 0.1957 0.045 0.1537 0.153 33.9
0.4358 16.0 2416 1.1901 0.1782 0.0474 0.1507 0.1498 35.08
0.2375 17.0 2567 1.2039 0.1868 0.0419 0.155 0.1537 31.81
0.2375 18.0 2718 1.2239 0.1882 0.0497 0.154 0.1518 34.44
0.2375 19.0 2869 1.2359 0.1684 0.0316 0.1386 0.138 34.03
0.1161 20.0 3020 1.2349 0.1773 0.0447 0.1482 0.1467 34.32
0.1161 21.0 3171 1.2541 0.1933 0.0442 0.1632 0.1621 35.39
0.1161 22.0 3322 1.2735 0.1809 0.0315 0.1492 0.1478 30.5
0.1161 23.0 3473 1.2864 0.1737 0.0311 0.1381 0.1377 41.44
0.0529 24.0 3624 1.2933 0.1748 0.0317 0.1417 0.1395 38.1
0.0529 25.0 3775 1.2963 0.1691 0.0346 0.1402 0.1395 38.66
0.0529 26.0 3926 1.3108 0.1754 0.0362 0.1435 0.1422 36.24
0.0272 27.0 4077 1.3213 0.1844 0.0369 0.1438 0.143 38.48
0.0272 28.0 4228 1.3274 0.1867 0.0263 0.1494 0.1492 48.51
0.0272 29.0 4379 1.3518 0.1799 0.0395 0.1466 0.146 37.97
0.0169 30.0 4530 1.3385 0.1693 0.0223 0.1408 0.1404 38.08
0.0169 31.0 4681 1.3528 0.1646 0.0419 0.1298 0.1304 39.36
0.0169 32.0 4832 1.3581 0.1618 0.0266 0.133 0.1324 39.97
0.0169 33.0 4983 1.3608 0.1686 0.0305 0.1359 0.1366 40.58
0.0112 34.0 5134 1.3631 0.1849 0.0361 0.1548 0.1526 39.22
0.0112 35.0 5285 1.3679 0.1773 0.0237 0.1419 0.1412 42.64
0.0112 36.0 5436 1.3674 0.1906 0.0319 0.1454 0.1447 42.92
0.009 37.0 5587 1.3893 0.169 0.0286 0.1364 0.1368 42.74
0.009 38.0 5738 1.3892 0.1617 0.0239 0.1265 0.1267 43.14
0.009 39.0 5889 1.3976 0.1378 0.02 0.1063 0.1074 36.67
0.0065 40.0 6040 1.4050 0.1699 0.0284 0.1344 0.1351 44.07
0.0065 41.0 6191 1.4019 0.1687 0.0296 0.1305 0.1311 40.0
0.0065 42.0 6342 1.4126 0.14 0.0202 0.1102 0.1115 36.65
0.0065 43.0 6493 1.4029 0.1716 0.0275 0.13 0.1308 43.95
0.0052 44.0 6644 1.4133 0.1784 0.0255 0.1431 0.1432 40.4
0.0052 45.0 6795 1.4173 0.1647 0.0226 0.1336 0.1332 40.33
0.0052 46.0 6946 1.4155 0.1743 0.0251 0.1407 0.141 38.74
0.0043 47.0 7097 1.4197 0.1659 0.0257 0.1325 0.1336 38.49
0.0043 48.0 7248 1.4213 0.1752 0.0265 0.1397 0.1401 40.02
0.0043 49.0 7399 1.4202 0.1703 0.0239 0.1377 0.1379 38.96
0.0039 50.0 7550 1.4193 0.1671 0.0236 0.1346 0.135 39.47

Framework versions

  • Transformers 4.36.2
  • Pytorch 1.12.1+cu113
  • Datasets 2.16.1
  • Tokenizers 0.15.0
Downloads last month
3
Safetensors
Model size
139M params
Tensor type
F32
·
Inference Examples
Text2Text Generation
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social visibility and check back later, or deploy to Inference Endpoints (dedicated) instead.

Model tree for tanatapanun/fine-tuned-BioBART-50-epochs-1024-input-192-output

Base model

GanjinZero/biobart-base
Finetuned
(12)
this model

Evaluation results

Metadata error: specify a dataset to view leaderboard