muo-ahn/my_awesome_billsum_model

This model is a fine-tuned version of google-t5/t5-small on an unknown dataset. It achieves the following results on the evaluation set:

• Train Loss: 3.0118
• Validation Loss: 2.8311
• Train Rouge1: 0.5405
• Train Rouge2: 0.2035
• Train Rougel: 0.423
• Train Rougelsum: 0.4234
• Train Gen Len: 126.4758
• Epoch: 17

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• optimizer: {'name': 'AdamWeightDecay', 'learning_rate': 2e-05, 'decay': 0.0, 'beta_1': 0.9, 'beta_2': 0.999, 'epsilon': 1e-07, 'amsgrad': False, 'weight_decay_rate': 0.01}
• training_precision: float32

Training results

Train Loss	Validation Loss	Train Rouge1	Train Rouge2	Train Rougel	Train Rougelsum	Train Gen Len	Epoch
4.4387	3.6485	0.4627	0.1328	0.3418	0.3424	128.0	0
3.8209	3.4001	0.487	0.1549	0.3647	0.3651	128.0	1
3.6230	3.2513	0.4985	0.1622	0.3759	0.3762	127.3065	2
3.4889	3.1677	0.5059	0.1666	0.3846	0.3849	126.6129	3
3.4166	3.1117	0.5111	0.1709	0.3905	0.3908	126.2298	4
3.3556	3.0683	0.5147	0.1748	0.3954	0.3958	126.1048	5
3.3031	3.0318	0.5171	0.1795	0.3997	0.4001	126.3266	6
3.2666	3.0008	0.5205	0.1847	0.4055	0.4059	126.5	7
3.2273	2.9744	0.5255	0.1888	0.4088	0.4093	126.5	8
3.1942	2.9524	0.5263	0.1911	0.4097	0.4104	126.5484	9
3.1685	2.9321	0.5275	0.1929	0.4117	0.4123	126.5282	10
3.1408	2.9130	0.5304	0.1942	0.4138	0.4145	126.4677	11
3.1154	2.8960	0.5312	0.1961	0.4161	0.4166	126.4718	12
3.0876	2.8813	0.5339	0.1976	0.4178	0.4183	126.4637	13
3.0742	2.8672	0.5346	0.1987	0.4185	0.4191	126.4718	14
3.0464	2.8539	0.5361	0.2003	0.4198	0.4204	126.4677	15
3.0293	2.8424	0.5384	0.2015	0.4206	0.4212	126.4758	16
3.0118	2.8311	0.5405	0.2035	0.423	0.4234	126.4758	17

Framework versions

• Transformers 4.42.0.dev0
• TensorFlow 2.15.0
• Datasets 2.19.1
• Tokenizers 0.19.1