File size: 3,167 Bytes
200c2b4 1fbd29d 200c2b4 1fbd29d 200c2b4 025d269 200c2b4 1fbd29d 200c2b4 3563536 1fbd29d 200c2b4 1fbd29d 200c2b4 1fbd29d 200c2b4 1fbd29d 200c2b4 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 |
---
tags:
- Summarization
- mT5
datasets:
- csebuetnlp/xlsum
language:
- id
license:
- apache-2.0
---
<!-- This model card has been generated automatically according to the information Keras had access to. You should
probably proofread and complete it, then remove this comment. -->
# mt5-small-indonesian-sum
This model is a fine-tuned version of [google/mt5-small](https://huggingface.co/google/mt5-small) on an [csebuetnlp/xlsum](https://huggingface.co/datasets/csebuetnlp/xlsum) dataset.
## Training results
| Train Loss | Validation Loss | Train Rouge1 | Train Rouge2 | Train Rougel | Train Rougelsum | Train Gen Len |
|:----------:|:---------------:|:------------:|:------------:|:------------:|:---------------:|:-------------:|
| 4.6665 | 2.6526 | 19.7876 | 6.6344 | 16.7880 | 16.8459 | 18.92 |
## Training hyperparameters
Optimizer | VALUE
---------|----------------------------
learning_rate| 2e-05
decay| 0.0
beta_1| 0.9
beta_2| 0.999
epsilon| 1e-07
amsgrad| False
weight_decay_rate| 0.01
- training_precision: float32
## Using this model in `transformers` (tested on 4.19.2)
```python
from transformers import AutoTokenizer, AutoModelForSeq2SeqLM
import re
news = """
Anggota Unit Perlindungan Rakyat Kurdi di kota Rabia, pada perbatasan Irak-Suriah. Pasukan Kurdi Irak dilaporkan sudah menguasai kembali kota Rabia meskipun banyak korban jatuh. Pejabat senior Kurdi Irak mengatakan pasukan Kurdi Peshmerga mencatat kemajuan lewat serangan dini hari di Rabia. Sementara itu, milisi ISIS berusaha memukul mundur pasukan Kurdi Suriah di bagian lain perbatasan. Hal ini terjadi saat koalisi pimpinan Amerika terus melanjutkan serangan udara terhadap sasaran ISIS di Suriah dan Irak. Hari Selasa (30 September) dilaporkan juga terjadi serangkaian serangan bom di ibu kota Irak, Baghdad dan kota suci Syiah, Karbala. Dalam perkembangan terpisah, sejumlah tank Turki berada di bukit di sepanjang perbatasan dekat kota Kobane, Suriah setelah sejumlah bom mengenai wilayah Turki saat terjadi bentrokan dengan milisi ISIS dan pejuang Kurdi. Pemerintah Turki diperkirakan akan menyampaikan mosi ke parlemen, agar menyetujui aksi militer terhadap ISIS di Irak dan Suriah.
"""
tokenizer = AutoTokenizer.from_pretrained("assamim/mt5-small-indonesian-sum")
model = AutoModelForSeq2SeqLM.from_pretrained("assamim/mt5-small-indonesian-sum", from_tf=True)
WHITESPACE_HANDLER = lambda k: re.sub('\s+', ' ', re.sub('\n+', ' ', k.strip()))
input_ids = tokenizer.encode(WHITESPACE_HANDLER(news1), return_tensors='pt')
summary_ids = model.generate(input_ids,
min_length=20,
max_length=200,
num_beams=7,
repetition_penalty=2.5,
length_penalty=1.0,
early_stopping=True,
no_repeat_ngram_size=2,
use_cache=True,
do_sample = True,
temperature = 0.8,
top_k = 50,
top_p = 0.95)
summary_text = tokenizer.decode(summary_ids[0], skip_special_tokens=True)
print(summary_text)
```
## Framework versions
- Transformers 4.19.2
- TensorFlow 2.8.0
- Datasets 2.2.2
- Tokenizers 0.12.1
|