---
language:
- ru
- ru-RU
tags:
- summarization
- t5
datasets:
- IlyaGusev/gazeta
license: apache-2.0
---

# RuT5SumGazeta

## Model description

This is the model for abstractive summarization for Russian based on [rut5-base](https://huggingface.co/cointegrated/rut5-base).


## Intended uses & limitations

#### How to use

Colab: [link](https://colab.research.google.com/drive/1re5E26ZIDUpAx1gOCZkbF3hcwjozmgG0)

```python
from transformers import AutoTokenizer, T5ForConditionalGeneration

model_name = "IlyaGusev/rut5_base_sum_gazeta"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = T5ForConditionalGeneration.from_pretrained(model_name)

article_text = "..."

input_ids = tokenizer(
    [article_text],
    add_special_tokens=True,
    padding="max_length",
    truncation=True,
    return_tensors="pt"
)["input_ids"]

output_ids = model.generate(
    input_ids=input_ids,
    no_repeat_ngram_size=3,
    early_stopping=True
)[0]

summary = tokenizer.decode(output_ids, skip_special_tokens=True)
print(summary)
```

## Training data

- Dataset: https://github.com/IlyaGusev/gazeta

## Training procedure

- Training script: https://github.com/IlyaGusev/summarus/blob/master/external/hf_scripts/train.py

## Eval results

Gazeta v1 train -> Gazeta v1 test

| Model                     | R-1-f | R-2-f | R-L-f | chrF | METEOR | BLEU |
|:--------------------------|:------|:------|:------|:-------|:-------|:-----|
| mbart_ru_sum_gazeta       | 32.1  | 14.3  | **27.9**  | **39.7** | **25.7**   | **12.4** |
| rut5_base_sum_gazeta      | **32.3**  | **14.5**  | 27.9 | 39.6 | 25.1   | 11.5 |

Gazeta v1 train -> Gazeta v2 test

| Model                     | R-1-f | R-2-f | R-L-f | chrF | METEOR | BLEU |
|:--------------------------|:------|:------|:------|:-------|:-------|:-----|
| mbart_ru_sum_gazeta       | **28.8**  | **11.1**  | **24.5**  | **37.4** | **22.7**   | **9.5** |
| rut5_base_sum_gazeta      | 28.5  | 11.0  | 24.2  | 36.8 | 21.2   | 8.6 |

Predicting all summaries:
```python
import json
import torch
from transformers import AutoTokenizer, T5ForConditionalGeneration
from datasets import load_dataset


def gen_batch(inputs, batch_size):
    batch_start = 0
    while batch_start < len(inputs):
        yield inputs[batch_start: batch_start + batch_size]
        batch_start += batch_size


def predict(
    model_name,
    input_records,
    output_file,
    max_source_tokens_count=400,
    max_target_tokens_count=200,
    batch_size=16
):
    device = "cuda" if torch.cuda.is_available() else "cpu"

    tokenizer = AutoTokenizer.from_pretrained(model_name)
    model = T5ForConditionalGeneration.from_pretrained(model_name).to(device)
    
    predictions = []
    for batch in gen_batch(input_records, batch_size):
        texts = [r["text"] for r in batch]
        input_ids = tokenizer(
            texts,                                                                                                     
            add_special_tokens=True,
            max_length=max_source_tokens_count,
            padding="max_length",
            truncation=True,
            return_tensors="pt"
        )["input_ids"].to(device)
        
        output_ids = model.generate(
            input_ids=input_ids,
            max_length=max_target_tokens_count,
            no_repeat_ngram_size=3,
            early_stopping=True
        )
        summaries = tokenizer.batch_decode(output_ids, skip_special_tokens=True)
        for s in summaries:
            print(s)
        predictions.extend(summaries)
    with open(output_file, "w") as w:
        for p in predictions:
            w.write(p.strip().replace("\n", " ") + "\n")

gazeta_test = load_dataset('IlyaGusev/gazeta', script_version="v1.0")["test"]
predict("IlyaGusev/rut5_base_sum_gazeta", list(gazeta_test), "t5_predictions.txt")
```

Evaluation: https://github.com/IlyaGusev/summarus/blob/master/evaluate.py

Flags: --language ru --tokenize-after --lower