|
--- |
|
language: |
|
- ru |
|
- ru-RU |
|
tags: |
|
- summarization |
|
- t5 |
|
datasets: |
|
- IlyaGusev/gazeta |
|
license: apache-2.0 |
|
--- |
|
|
|
# RuT5SumGazeta |
|
|
|
## Model description |
|
|
|
This is the model for abstractive summarization for Russian based on [rut5-base](https://huggingface.co/cointegrated/rut5-base). |
|
|
|
|
|
## Intended uses & limitations |
|
|
|
#### How to use |
|
|
|
Colab: [link](https://colab.research.google.com/drive/1re5E26ZIDUpAx1gOCZkbF3hcwjozmgG0) |
|
|
|
```python |
|
from transformers import AutoTokenizer, T5ForConditionalGeneration |
|
|
|
model_name = "IlyaGusev/rut5_base_sum_gazeta" |
|
tokenizer = AutoTokenizer.from_pretrained(model_name) |
|
model = T5ForConditionalGeneration.from_pretrained(model_name) |
|
|
|
article_text = "..." |
|
|
|
input_ids = tokenizer( |
|
[article_text], |
|
add_special_tokens=True, |
|
padding="max_length", |
|
truncation=True, |
|
return_tensors="pt" |
|
)["input_ids"] |
|
|
|
output_ids = model.generate( |
|
input_ids=input_ids, |
|
no_repeat_ngram_size=3, |
|
early_stopping=True |
|
)[0] |
|
|
|
summary = tokenizer.decode(output_ids, skip_special_tokens=True) |
|
print(summary) |
|
``` |
|
|
|
## Training data |
|
|
|
- Dataset: https://github.com/IlyaGusev/gazeta |
|
|
|
## Training procedure |
|
|
|
- Training script: https://github.com/IlyaGusev/summarus/blob/master/external/hf_scripts/train.py |
|
|
|
## Eval results |
|
|
|
Gazeta v1 train -> Gazeta v1 test |
|
|
|
| Model | R-1-f | R-2-f | R-L-f | chrF | METEOR | BLEU | |
|
|:--------------------------|:------|:------|:------|:-------|:-------|:-----| |
|
| mbart_ru_sum_gazeta | 32.1 | 14.3 | **27.9** | **39.7** | **25.7** | **12.4** | |
|
| rut5_base_sum_gazeta | **32.3** | **14.5** | 27.9 | 39.6 | 25.1 | 11.5 | |
|
|
|
Gazeta v1 train -> Gazeta v2 test |
|
|
|
| Model | R-1-f | R-2-f | R-L-f | chrF | METEOR | BLEU | |
|
|:--------------------------|:------|:------|:------|:-------|:-------|:-----| |
|
| mbart_ru_sum_gazeta | **28.8** | **11.1** | **24.5** | **37.4** | **22.7** | **9.5** | |
|
| rut5_base_sum_gazeta | 28.5 | 11.0 | 24.2 | 36.8 | 21.2 | 8.6 | |
|
|
|
Predicting all summaries: |
|
```python |
|
import json |
|
import torch |
|
from transformers import AutoTokenizer, T5ForConditionalGeneration |
|
from datasets import load_dataset |
|
|
|
|
|
def gen_batch(inputs, batch_size): |
|
batch_start = 0 |
|
while batch_start < len(inputs): |
|
yield inputs[batch_start: batch_start + batch_size] |
|
batch_start += batch_size |
|
|
|
|
|
def predict( |
|
model_name, |
|
input_records, |
|
output_file, |
|
max_source_tokens_count=400, |
|
max_target_tokens_count=200, |
|
batch_size=16 |
|
): |
|
device = "cuda" if torch.cuda.is_available() else "cpu" |
|
|
|
tokenizer = AutoTokenizer.from_pretrained(model_name) |
|
model = T5ForConditionalGeneration.from_pretrained(model_name).to(device) |
|
|
|
predictions = [] |
|
for batch in gen_batch(input_records, batch_size): |
|
texts = [r["text"] for r in batch] |
|
input_ids = tokenizer( |
|
texts, |
|
add_special_tokens=True, |
|
max_length=max_source_tokens_count, |
|
padding="max_length", |
|
truncation=True, |
|
return_tensors="pt" |
|
)["input_ids"].to(device) |
|
|
|
output_ids = model.generate( |
|
input_ids=input_ids, |
|
max_length=max_target_tokens_count, |
|
no_repeat_ngram_size=3, |
|
early_stopping=True |
|
) |
|
summaries = tokenizer.batch_decode(output_ids, skip_special_tokens=True) |
|
for s in summaries: |
|
print(s) |
|
predictions.extend(summaries) |
|
with open(output_file, "w") as w: |
|
for p in predictions: |
|
w.write(p.strip().replace("\n", " ") + "\n") |
|
|
|
gazeta_test = load_dataset('IlyaGusev/gazeta', script_version="v1.0")["test"] |
|
predict("IlyaGusev/rut5_base_sum_gazeta", list(gazeta_test), "t5_predictions.txt") |
|
``` |
|
|
|
Evaluation: https://github.com/IlyaGusev/summarus/blob/master/evaluate.py |
|
|
|
Flags: --language ru --tokenize-after --lower |
|
|
