|
--- |
|
language: hu |
|
metrics: rouge |
|
--- |
|
|
|
[Paper](https://hlt.bme.hu/en/publ/foszt2oszt) |
|
|
|
We publish an abstractive summarizer for Hungarian, an |
|
encoder-decoder model initialized with [huBERT](huggingface.co/SZTAKI-HLT/hubert-base-cc), and fine-tuned on the |
|
[ELTE.DH](https://elte-dh.hu/) corpus of former Hungarian news portals. The model produces fluent output in the correct topic, but it hallucinates frequently. |
|
Our quantitative evaluation on automatic and human transcripts of news |
|
(with automatic and human-made punctuation, [Tündik et al. (2019)](https://www.isca-speech.org/archive/interspeech_2019/tundik19_interspeech.html), [Tündik and Szaszák (2019)](https://www.isca-speech.org/archive/interspeech_2019/szaszak19_interspeech.html)) shows that the model is |
|
robust with respect to errors in either automatic speech recognition or |
|
automatic punctuation restoration. In fine-tuning and inference, we followed [a jupyter notebook by Patrick von |
|
Platen](https://github.com/patrickvonplaten/notebooks/blob/master/BERT2BERT_for_CNN_Dailymail.ipynb). Most hyper-parameters are the same as those by von Platen, but we |
|
found it advantageous to change the minimum length of the summary to 8 word- |
|
pieces (instead of 56), and the number of beams in beam search to 5 (instead |
|
of 4). Our model was fine-tuned on a server of the [SZTAKI-HLT](hlt.bme.hu/) group, which kindly |
|
provided access to it. |
