ADL_test

This model is a fine-tuned version of google/mt5-small on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 3.9924
• Rouge-1: 19.8975
• Rouge-2: 7.0134
• Rouge-l: 18.2845
• Gen Len: 14.3786

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 1
• eval_batch_size: 1
• seed: 42
• gradient_accumulation_steps: 16
• total_train_batch_size: 16
• optimizer: Adafactor
• lr_scheduler_type: linear
• num_epochs: 4.0

Training results

Training Loss	Epoch	Step	Gen Len	Validation Loss	Rouge-1	Rouge-2	Rouge-l
6.0773	1.0	1356	11.8835	4.1822	17.4884	6.039	16.4024
5.0033	2.0	2713	13.385	4.0972	18.9145	6.6873	17.5342
4.7707	3.0	4068	14	4.019	19.62	6.908	18.08
4.7391	4.0	5424	14.3786	3.992	19.9	7.013	18.28

Framework versions

• Transformers 4.34.0
• Pytorch 1.13.1+cu116
• Datasets 2.14.5
• Tokenizers 0.14.1