metadata
language: de
tags:
- summarization
datasets:
- mlsum
mT5-small fine-tuned on German MLSUM
This model was finetuned for 3 epochs with a max_len (input) of 768 tokens and target_max_len of 192 tokens.
It was fine-tuned on all German articles present in the train split of the MLSUM dataset having less than 384 "words" after splitting on whitespace, which resulted in 80249 articles.
The exact expression to filter the dataset was the following:
dataset = dataset.filter(lambda e: len(e['text'].split()) < 384)
Evaluation results
The fine-tuned model was evaluated on 2000 random articles from the validation set. Mean f1 ROUGE scores were calculated for both the fine-tuned model and the lead-3 baseline (which simply produces the leading three sentences of the document) and are presented in the following table.
| Model | Rouge-1 | Rouge-2 | Rouge-L |
|---|---|---|---|
| mt5-small | 0.399 | 0.318 | 0.392 |
| lead-3 | 0.343 | 0.263 | 0.341 |