README.md

---
language: de
tags:
- summarization
datasets:
- mlsum
---

# mT5-small fine-tuned on German MLSUM
This model was finetuned for 3 epochs with a max_len (input) of 768 tokens and target_max_len of 192 tokens.  
It was fine-tuned on all German articles present in the train split of the [MLSUM dataset](https://huggingface.co/datasets/mlsum) having less than 384 "words" after splitting on whitespace, which resulted in 80249 articles.  
The exact expression to filter the dataset was the following:
```python
dataset = dataset.filter(lambda e: len(e['text'].split()) < 384)
```

## Evaluation results
The fine-tuned model was evaluated on 2000 random articles from the validation set.
Mean [f1 ROUGE scores](https://github.com/pltrdy/rouge) were calculated for both the fine-tuned model and the lead-3 baseline (which simply produces the leading three sentences of the document) and are presented in the following table.

| Model         | Rouge-1 | Rouge-2  | Rouge-L |
| ------------- |:-------:| --------:| -------:|
| mt5-small     | 0.399   | 0.318    | 0.392   |
| lead-3        | 0.343   | 0.263    | 0.341   |