North-T5_large_NO_CNN-idun

This model is a fine-tuned version of north/t5_large_NCC_modern on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 1.7705
• Rouge1: 32.1183
• Rouge2: 12.1864
• Rougel: 22.008
• Rougelsum: 29.6903
• Gen Len: 96.8196

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 2
• eval_batch_size: 2
• seed: 42
• gradient_accumulation_steps: 4
• total_train_batch_size: 8
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 5

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum	Gen Len
2.1173	1.0	6118	1.8719	31.1024	11.4005	21.1829	28.6286	91.2659
1.9812	2.0	12236	1.8214	31.7965	11.9251	21.6532	29.3509	95.1182
1.8761	3.0	18354	1.7854	31.8821	12.0616	21.7162	29.4558	98.6727
1.8243	4.0	24472	1.7747	32.0071	12.1025	21.8843	29.5732	96.5254
1.7848	5.0	30590	1.7705	32.1183	12.1864	22.008	29.6903	96.8196

Framework versions

• Transformers 4.32.1
• Pytorch 2.3.0+cu121
• Datasets 2.12.0
• Tokenizers 0.13.2