Hugging Face's logo

gayanin
/
t5-small-mlm-pubmed

Text2Text Generation
Transformers
PyTorch
TensorBoard
t5
generated_from_trainer
Inference Endpoints
text-generation-inference
Model card Files Metrics Community
1
gayanin commited on
Commit
d33459f
•
1 Parent(s): 5cf6019

update model card README.md

Browse files
Files changed (1) hide show
  1. README.md +58 -36
README.md CHANGED
@@ -14,10 +14,10 @@ should probably proofread and complete it, then remove this comment. -->
14
 
15
  This model is a fine-tuned version of [t5-small](https://huggingface.co/t5-small) on an unknown dataset.
16
  It achieves the following results on the evaluation set:
17
- - Loss: 0.8727
18
- - Rouge2 Precision: 0.5862
19
- - Rouge2 Recall: 0.435
20
- - Rouge2 Fmeasure: 0.4863
21
 
22
  ## Model description
23
 
@@ -42,44 +42,66 @@ The following hyperparameters were used during training:
42
  - seed: 42
43
  - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
44
  - lr_scheduler_type: linear
45
- - num_epochs: 24
46
  - mixed_precision_training: Native AMP
47
 
48
  ### Training results
49
 
50
  | Training Loss | Epoch | Step | Validation Loss | Rouge2 Precision | Rouge2 Recall | Rouge2 Fmeasure |
51
  |:-------------:|:-----:|:-----:|:---------------:|:----------------:|:-------------:|:---------------:|
52
- | 1.5638 | 0.75 | 500 | 1.1867 | 0.5614 | 0.3818 | 0.4424 |
53
- | 1.2804 | 1.51 | 1000 | 1.0872 | 0.5661 | 0.4018 | 0.4581 |
54
- | 1.1983 | 2.26 | 1500 | 1.0417 | 0.5653 | 0.4066 | 0.4602 |
55
- | 1.1469 | 3.02 | 2000 | 1.0130 | 0.572 | 0.4171 | 0.4696 |
56
- | 1.1154 | 3.77 | 2500 | 0.9924 | 0.5683 | 0.4166 | 0.468 |
57
- | 1.0932 | 4.52 | 3000 | 0.9755 | 0.5734 | 0.4181 | 0.4711 |
58
- | 1.0628 | 5.28 | 3500 | 0.9610 | 0.5729 | 0.424 | 0.4751 |
59
- | 1.0494 | 6.03 | 4000 | 0.9471 | 0.5681 | 0.4213 | 0.4715 |
60
- | 1.0333 | 6.79 | 4500 | 0.9366 | 0.572 | 0.4244 | 0.4743 |
61
- | 1.0252 | 7.54 | 5000 | 0.9303 | 0.5752 | 0.4267 | 0.4772 |
62
- | 1.0098 | 8.3 | 5500 | 0.9203 | 0.5681 | 0.4221 | 0.4715 |
63
- | 0.9944 | 9.05 | 6000 | 0.9162 | 0.5752 | 0.4283 | 0.4782 |
64
- | 0.9864 | 9.8 | 6500 | 0.9115 | 0.5755 | 0.4292 | 0.4788 |
65
- | 0.9853 | 10.56 | 7000 | 0.9048 | 0.5759 | 0.4282 | 0.4783 |
66
- | 0.9736 | 11.31 | 7500 | 0.9004 | 0.577 | 0.4282 | 0.4788 |
67
- | 0.9611 | 12.07 | 8000 | 0.8965 | 0.5798 | 0.4319 | 0.4823 |
68
- | 0.9594 | 12.82 | 8500 | 0.8930 | 0.583 | 0.4335 | 0.4845 |
69
- | 0.9462 | 13.57 | 9000 | 0.8910 | 0.5822 | 0.4351 | 0.485 |
70
- | 0.9527 | 14.33 | 9500 | 0.8868 | 0.5818 | 0.432 | 0.4828 |
71
- | 0.9471 | 15.08 | 10000 | 0.8848 | 0.5831 | 0.4335 | 0.4843 |
72
- | 0.9426 | 15.84 | 10500 | 0.8827 | 0.5857 | 0.4357 | 0.4867 |
73
- | 0.9394 | 16.59 | 11000 | 0.8803 | 0.5833 | 0.4329 | 0.4837 |
74
- | 0.94 | 17.35 | 11500 | 0.8793 | 0.5858 | 0.4354 | 0.4865 |
75
- | 0.9264 | 18.1 | 12000 | 0.8781 | 0.5856 | 0.4355 | 0.4864 |
76
- | 0.9269 | 18.85 | 12500 | 0.8762 | 0.5834 | 0.4337 | 0.4845 |
77
- | 0.926 | 19.61 | 13000 | 0.8754 | 0.5862 | 0.4352 | 0.4863 |
78
- | 0.926 | 20.36 | 13500 | 0.8741 | 0.5853 | 0.4347 | 0.4859 |
79
- | 0.9295 | 21.12 | 14000 | 0.8740 | 0.5863 | 0.4351 | 0.4865 |
80
- | 0.9209 | 21.87 | 14500 | 0.8732 | 0.5865 | 0.4359 | 0.487 |
81
- | 0.9174 | 22.62 | 15000 | 0.8732 | 0.5866 | 0.4359 | 0.4871 |
82
- | 0.9203 | 23.38 | 15500 | 0.8727 | 0.5862 | 0.435 | 0.4863 |
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
83
 
84
 
85
  ### Framework versions
 
14
 
15
  This model is a fine-tuned version of [t5-small](https://huggingface.co/t5-small) on an unknown dataset.
16
  It achieves the following results on the evaluation set:
17
+ - Loss: 0.8008
18
+ - Rouge2 Precision: 0.6071
19
+ - Rouge2 Recall: 0.4566
20
+ - Rouge2 Fmeasure: 0.5079
21
 
22
  ## Model description
23
 
 
42
  - seed: 42
43
  - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
44
  - lr_scheduler_type: linear
45
+ - num_epochs: 40
46
  - mixed_precision_training: Native AMP
47
 
48
  ### Training results
49
 
50
  | Training Loss | Epoch | Step | Validation Loss | Rouge2 Precision | Rouge2 Recall | Rouge2 Fmeasure |
51
  |:-------------:|:-----:|:-----:|:---------------:|:----------------:|:-------------:|:---------------:|
52
+ | 0.914 | 0.75 | 500 | 0.8691 | 0.5901 | 0.4357 | 0.4879 |
53
+ | 0.9093 | 1.51 | 1000 | 0.8646 | 0.5867 | 0.4372 | 0.488 |
54
+ | 0.895 | 2.26 | 1500 | 0.8618 | 0.5891 | 0.4387 | 0.49 |
55
+ | 0.8842 | 3.02 | 2000 | 0.8571 | 0.5899 | 0.4374 | 0.4891 |
56
+ | 0.8796 | 3.77 | 2500 | 0.8544 | 0.5903 | 0.4406 | 0.4916 |
57
+ | 0.8759 | 4.52 | 3000 | 0.8513 | 0.5921 | 0.4395 | 0.4912 |
58
+ | 0.8621 | 5.28 | 3500 | 0.8485 | 0.5934 | 0.4413 | 0.493 |
59
+ | 0.8613 | 6.03 | 4000 | 0.8442 | 0.5944 | 0.4428 | 0.4944 |
60
+ | 0.8537 | 6.79 | 4500 | 0.8406 | 0.594 | 0.4414 | 0.4932 |
61
+ | 0.8518 | 7.54 | 5000 | 0.8399 | 0.5956 | 0.4424 | 0.4945 |
62
+ | 0.8438 | 8.3 | 5500 | 0.8365 | 0.5953 | 0.4452 | 0.4964 |
63
+ | 0.8339 | 9.05 | 6000 | 0.8353 | 0.5983 | 0.4468 | 0.4983 |
64
+ | 0.8307 | 9.8 | 6500 | 0.8331 | 0.5979 | 0.4461 | 0.4976 |
65
+ | 0.8328 | 10.56 | 7000 | 0.8304 | 0.5975 | 0.4465 | 0.4979 |
66
+ | 0.8263 | 11.31 | 7500 | 0.8283 | 0.5977 | 0.4467 | 0.4981 |
67
+ | 0.8168 | 12.07 | 8000 | 0.8267 | 0.5971 | 0.4463 | 0.4976 |
68
+ | 0.8165 | 12.82 | 8500 | 0.8248 | 0.5969 | 0.4462 | 0.4976 |
69
+ | 0.8084 | 13.57 | 9000 | 0.8245 | 0.6018 | 0.4527 | 0.5035 |
70
+ | 0.8136 | 14.33 | 9500 | 0.8219 | 0.6023 | 0.4509 | 0.5023 |
71
+ | 0.8073 | 15.08 | 10000 | 0.8206 | 0.6002 | 0.4486 | 0.5001 |
72
+ | 0.808 | 15.84 | 10500 | 0.8185 | 0.6009 | 0.4506 | 0.5019 |
73
+ | 0.8027 | 16.59 | 11000 | 0.8173 | 0.5978 | 0.4478 | 0.4989 |
74
+ | 0.8061 | 17.35 | 11500 | 0.8169 | 0.6022 | 0.4513 | 0.5026 |
75
+ | 0.7922 | 18.1 | 12000 | 0.8152 | 0.6016 | 0.4501 | 0.5016 |
76
+ | 0.7928 | 18.85 | 12500 | 0.8141 | 0.6009 | 0.45 | 0.5012 |
77
+ | 0.7909 | 19.61 | 13000 | 0.8143 | 0.6019 | 0.4521 | 0.5028 |
78
+ | 0.7909 | 20.36 | 13500 | 0.8115 | 0.5997 | 0.4505 | 0.5011 |
79
+ | 0.7949 | 21.12 | 14000 | 0.8115 | 0.6043 | 0.4536 | 0.5048 |
80
+ | 0.7853 | 21.87 | 14500 | 0.8095 | 0.6033 | 0.4527 | 0.5038 |
81
+ | 0.7819 | 22.62 | 15000 | 0.8095 | 0.6054 | 0.4541 | 0.5056 |
82
+ | 0.7828 | 23.38 | 15500 | 0.8075 | 0.6036 | 0.453 | 0.5042 |
83
+ | 0.787 | 24.13 | 16000 | 0.8068 | 0.6031 | 0.4528 | 0.504 |
84
+ | 0.7739 | 24.89 | 16500 | 0.8072 | 0.6043 | 0.4529 | 0.5045 |
85
+ | 0.7782 | 25.64 | 17000 | 0.8073 | 0.606 | 0.4551 | 0.5063 |
86
+ | 0.7772 | 26.4 | 17500 | 0.8063 | 0.6055 | 0.4549 | 0.5062 |
87
+ | 0.7718 | 27.15 | 18000 | 0.8057 | 0.606 | 0.4546 | 0.5059 |
88
+ | 0.7747 | 27.9 | 18500 | 0.8045 | 0.6046 | 0.4543 | 0.5054 |
89
+ | 0.7738 | 28.66 | 19000 | 0.8035 | 0.6059 | 0.4549 | 0.506 |
90
+ | 0.7642 | 29.41 | 19500 | 0.8041 | 0.6053 | 0.4545 | 0.5058 |
91
+ | 0.7666 | 30.17 | 20000 | 0.8039 | 0.6066 | 0.457 | 0.508 |
92
+ | 0.7686 | 30.92 | 20500 | 0.8027 | 0.6075 | 0.4571 | 0.5081 |
93
+ | 0.7664 | 31.67 | 21000 | 0.8026 | 0.6062 | 0.4566 | 0.5076 |
94
+ | 0.77 | 32.43 | 21500 | 0.8022 | 0.6068 | 0.4571 | 0.5081 |
95
+ | 0.7618 | 33.18 | 22000 | 0.8015 | 0.6065 | 0.4563 | 0.5072 |
96
+ | 0.7615 | 33.94 | 22500 | 0.8013 | 0.6064 | 0.4565 | 0.5074 |
97
+ | 0.7611 | 34.69 | 23000 | 0.8017 | 0.607 | 0.4567 | 0.5078 |
98
+ | 0.7611 | 35.44 | 23500 | 0.8013 | 0.608 | 0.4565 | 0.5082 |
99
+ | 0.7604 | 36.2 | 24000 | 0.8012 | 0.6069 | 0.4561 | 0.5072 |
100
+ | 0.7599 | 36.95 | 24500 | 0.8013 | 0.6078 | 0.4571 | 0.5085 |
101
+ | 0.7542 | 37.71 | 25000 | 0.8016 | 0.6083 | 0.4579 | 0.5091 |
102
+ | 0.7637 | 38.46 | 25500 | 0.8009 | 0.6072 | 0.4569 | 0.5081 |
103
+ | 0.7596 | 39.22 | 26000 | 0.8008 | 0.6069 | 0.4566 | 0.5078 |
104
+ | 0.7604 | 39.97 | 26500 | 0.8008 | 0.6071 | 0.4566 | 0.5079 |
105
 
106
 
107
  ### Framework versions