t5-small-finetuned-medical_knowledge_from_extracts

This model is a fine-tuned version of t5-small on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 1.1561
• Rouge1: 32.9864
• Rouge2: 13.23
• Rougel: 32.3943
• Rougelsum: 32.4479
• Gen Len: 19.0

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 10
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum	Gen Len
No log	1.0	87	1.7900	22.2671	0.7152	21.395	21.4052	19.0
No log	2.0	174	1.4078	31.5412	9.5847	31.1049	31.1295	18.9986
No log	3.0	261	1.3030	32.0827	11.4061	31.531	31.5738	19.0
No log	4.0	348	1.2511	32.3393	11.8795	31.7307	31.7918	19.0
No log	5.0	435	1.2163	33.0328	12.6708	32.4155	32.4638	19.0
1.7948	6.0	522	1.1927	32.9176	12.915	32.3593	32.3931	19.0
1.7948	7.0	609	1.1757	32.809	12.932	32.263	32.3132	19.0
1.7948	8.0	696	1.1642	32.9525	13.0878	32.3805	32.4297	19.0
1.7948	9.0	783	1.1582	32.9297	13.1084	32.3697	32.4219	19.0
1.7948	10.0	870	1.1561	32.9864	13.23	32.3943	32.4479	19.0

Framework versions

• Transformers 4.35.2
• Pytorch 2.1.0+cu121
• Datasets 2.16.1
• Tokenizers 0.15.1