t5_clinical_SA

This model is a fine-tuned version of luqh/ClinicalT5-base on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.4740
• Rouge1: 0.2395
• Rouge2: 0.0748
• Rougel: 0.2314
• Rougelsum: 0.2315
• Gen Len: 10.3363

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 1
• eval_batch_size: 1
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 12

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum	Gen Len
2.1954	1.0	527	0.5438	0.0035	0.0	0.0035	0.0035	0.1504
0.5373	2.0	1054	0.5078	0.1198	0.0337	0.1153	0.1155	11.0796
0.5116	3.0	1581	0.4901	0.1741	0.0618	0.1682	0.1709	11.6549
0.4576	4.0	2108	0.4798	0.1725	0.0576	0.1698	0.1728	12.1416
0.4626	5.0	2635	0.4758	0.2184	0.0723	0.2133	0.215	10.4867
0.435	6.0	3162	0.4765	0.2343	0.0796	0.2234	0.2245	10.6195
0.4018	7.0	3689	0.4746	0.2281	0.0765	0.2199	0.2206	10.0442
0.4046	8.0	4216	0.4711	0.2452	0.0769	0.2317	0.2329	11.0531
0.4128	9.0	4743	0.4726	0.2358	0.0712	0.2269	0.2276	10.6106
0.3885	10.0	5270	0.4734	0.2362	0.0719	0.2281	0.2284	10.5664
0.4003	11.0	5797	0.4738	0.243	0.08	0.235	0.2351	10.2655
0.362	12.0	6324	0.4740	0.2395	0.0748	0.2314	0.2315	10.3363

Framework versions

• Transformers 4.26.1
• Pytorch 1.13.1+cu116
• Datasets 2.10.1
• Tokenizers 0.13.2