NOT SELF REPORTED VALUES FOR THE LEADERBOARD, I HAVE NO CLUE WHY ITS BROKE. CHECK PULL REQUEST
Use summarization without adding summarize to the start of the string.
Trained on Samsum train split.
Parameters for training:
no_decay = ["bias", "LayerNorm.weight", "layer_norm.weight"] optimizer_grouped_parameters = [ { "params": [p for n, p in model.named_parameters() if not any(nd in n for nd in no_decay)], "weight_decay": 0.0, }, { "params": [p for n, p in model.named_parameters() if any(nd in n for nd in no_decay)], "weight_decay": 0.0, }, ]
lr = 0.00005 optimizer = torch.optim.RAdam(optimizer_grouped_parameters, lr=lr)
lr_scheduler = get_scheduler( name="linear", optimizer=optimizer, num_warmup_steps=0, num_training_steps=50005)
This was only for 10K steps with a batch size of 10
If you want more info, feel free to message me or email me at: samuelfipps@gmail.com
- Downloads last month
- 46
Space using Samuel-Fipps/t5-efficient-large-nl36_fine_tune_sum_V2 1
Evaluation results
- ROUGE-1 on samsumtest set self-reported50.505
- ROUGE-2 on samsumtest set self-reported25.647
- ROUGE-L on samsumtest set self-reported41.754
- ROUGE-LSUM on samsumtest set self-reported46.206
- loss on samsumtest set self-reported1.516
- gen_len on samsumtest set self-reported24.034
- ROUGE-1 on cnn_dailymailtest set self-reported34.406
- ROUGE-2 on cnn_dailymailtest set self-reported14.127
- ROUGE-L on cnn_dailymailtest set self-reported24.335
- ROUGE-LSUM on cnn_dailymailtest set self-reported31.658
- loss on cnn_dailymailtest set self-reported2.446
- gen_len on cnn_dailymailtest set self-reported45.928
- ROUGE-1 on samsumself-reported54.933
- ROUGE-2 on samsumself-reported31.797
- ROUGE-L on samsumself-reported47.006
- ROUGE-LSUM on samsumself-reported51.203
- loss on samsumself-reported1.131
- gen_len on samsumself-reported23.799
- ROUGE-1 on scientific_papersverified23.670
- ROUGE-2 on scientific_papersverified7.569
- ROUGE-L on scientific_papersverified15.607
- ROUGE-LSUM on scientific_papersverified21.456
- loss on scientific_papersverified3.937
- gen_len on scientific_papersverified65.999