Jammal7/t5-small-finetuned-Big-Patents

This model is a fine-tuned version of t5-small on big_patents datasets. It achieves the following results on the evaluation set:

• Train Loss: 2.1148
• Validation Loss: 2.0812
• Train Rouge1: 23.0058
• Train Rouge2: 12.5142
• Train Rougel: 19.9076
• Train Rougelsum: 20.8720
• Train Gen Len: 19.0
• Epoch: 14

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• optimizer: {'name': 'AdamWeightDecay', 'learning_rate': 2e-05, 'decay': 0.0, 'beta_1': 0.9, 'beta_2': 0.999, 'epsilon': 1e-07, 'amsgrad': False, 'weight_decay_rate': 0.01}
• training_precision: float32

Training results

Train Loss	Validation Loss	Train Rouge1	Train Rouge2	Train Rougel	Train Rougelsum	Train Gen Len	Epoch
2.6552	2.3220	21.3700	10.2105	17.5338	18.8802	18.9929	0
2.4395	2.2535	22.2253	11.2822	18.7238	19.9181	19.0	1
2.3755	2.2174	22.2327	11.5575	18.8348	19.9379	19.0	2
2.3329	2.1886	22.5328	11.8225	19.2582	20.3227	19.0	3
2.3004	2.1681	22.7872	12.1642	19.5194	20.5907	19.0	4
2.2696	2.1579	22.6969	12.0848	19.5012	20.5564	19.0	5
2.2487	2.1385	22.7300	12.2347	19.5685	20.5822	19.0	6
2.2249	2.1347	22.7250	12.2077	19.5103	20.5246	19.0	7
2.2060	2.1265	22.8921	12.4139	19.6903	20.7126	19.0	8
2.1892	2.1126	22.9440	12.4268	19.7717	20.8156	19.0	9
2.1740	2.1084	22.9201	12.4491	19.8107	20.8059	19.0	10
2.1580	2.0980	22.9947	12.3798	19.7494	20.7852	19.0	11
2.1414	2.0934	23.0885	12.5899	19.9531	20.9145	19.0	12
2.1302	2.0809	23.0854	12.5580	19.9468	20.8737	19.0	13
2.1148	2.0812	23.0058	12.5142	19.9076	20.8720	19.0	14

Framework versions

• Transformers 4.30.1
• TensorFlow 2.12.0
• Datasets 2.12.0
• Tokenizers 0.13.3