t5-base-extraction-cnndm_10000-all

This model was trained from scratch on the None dataset. It achieves the following results on the evaluation set:

• Loss: 1.8182
• Rouge1: 33.8286
• Rouge2: 14.4919
• Rougel: 28.8935
• Rougelsum: 28.9581
• Gen Len: 19.0

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 24
• eval_batch_size: 48
• seed: 1799
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 10

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum	Gen Len
1.9662	0.48	200	1.9092	33.2564	14.236	28.2044	28.3269	18.992
1.9495	0.96	400	1.8775	33.7516	14.2246	28.9019	28.9507	19.0
1.9062	1.44	600	1.8580	33.7533	14.2196	28.3873	28.4658	19.0
1.8713	1.92	800	1.8496	33.6921	14.4532	28.5695	28.6573	19.0
1.85	2.4	1000	1.8327	34.1551	14.7671	28.9492	28.9885	19.0
1.8232	2.88	1200	1.8182	33.8286	14.4919	28.8935	28.9581	19.0
1.8004	3.36	1400	1.8299	34.5099	14.8659	29.1119	29.1544	19.0
1.7832	3.84	1600	1.8252	34.5877	15.1259	29.3368	29.3638	19.0
1.7677	4.32	1800	1.8226	34.4487	15.0361	29.2962	29.3431	19.0

Framework versions

• Transformers 4.18.0
• Pytorch 1.10.0+cu111
• Datasets 2.5.1
• Tokenizers 0.12.1