hu PWKP : A H ungarian Text Simplification Corpus (2023)

Abstract: In this article we introduce huPWKP, the first parallel corpus consisting of Hungarian standard language-simplified sentence pairs. As Hungarian is a quite low-resource language in regards to text simplification, we opted for translating an already existing corpus, PWKP (Zhu et al., 2010), on which we performed some cleaning in order to improve its quality. We evaluated the corpus both with the help of human evaluators and by training a seq2seq model on both the Hungarian corpus and the original (cleaned) English corpus. The Hungarian model performed slightly worse in terms of automatic metrics; however, the English model attains a SARI score close to the state of the art on the official PWKP set. According to the human evaluation, the corpus performs at around 3 on a scale ranging from 1 to 5 in terms of information retention and increase in simplification and around 3.7 in terms of grammaticality.

## introduction the most important function and goal of human communication is joint meaning construction @xcite : we want every person who participates in the discourse to understand the referential scene @xcite . what we are talking about -exactly (or as similarly as possible) as we intended it to be understood. in order to achieve this, we sometimes need to simplify what we are saying and how we are phrasing it: meaning, we need to reduce "the linguistic complexity of a text, while still retaining the original information content and meaning" @xcite . simplified texts can be of use for several groups of people, e. g. for people with (communicative or other) disabilities @xcite , non-native speakers @xcite or children @xcite . however, as text simplification is a fairly time-and resource-consuming task for humans, it seems beneficial to try to automate this task. there have been multiple successful attempts at creating text simplificaton systems: most of them for english, e.g. @xcite or @xcite or @xcite . less-resourced languages, such as hungarian, have been largely ignored in the literature. in this paper, we introduce the first (albeit translated) hungarian parallel corpus consisting of standard language -simplified sentence pairs, as well as a simplification model trained on it. the work of @xcite has brought a breakthrough in abstractive text simplification: they used a sequence-based model, originally designed for machine translation, using standard-language material as source text and simplified texts as target text -this allowed more complex automatic changes to take place that could greatly affect the syntactic structure of the sentence. since then, numerous different attempts were made to better the existing ts methods. these mainly focus on lexical simplification (such as @xcite or sheang et al. ( 2022 )), however some of them concentrate on paragraph-level or document-level simplification (for example, @xcite successfully attempt both document-and paragraph-level simplification). however, what seems to be similar in most -although not all -of these attempts is the need for data, as a lot of these systems are fine-tuned on large parallel corpora. ## creating the corpus due to the limited financial and human resources available to us, as well as the lack of existing hungarian parallel data, building an original corpus was out of the scope of this research. instead, following the already existing literature, such as @xcite , we opted for the translation of an already existing english corpus. this obviously influences further studies on the corpus: since it does not consist of authentic hungarian data, it cannot be used to determine e.g. the strategies that hungarians use to simplify texts. however, assuming that the simplifications in the original english corpus are adequate and the translation is good enough, the resulting corpus can still be used to train simplification models on. ## evaluation we evaluated the translated corpus in two different ways. first, we trained a seq2seq model on both the english and the hungarian corpora and compared the results. second, we conducted a questionnaire study in order to include the human perspective in the evaluation. ## conclusion in this paper, we have introduced hupwkp, a hungarian translation of the pwkp corpus. the translation was performed automatically, based on a cleaned version of pwkp, which we also publish. the translation was evaluated both manually and automatically: the latter by training a seq2seq simplification models initialized from native bert-base checkpoints for both languages. the english and hungarian models performed similarly, at around the best sari score reported by other models on the official pwkp test set. the manual evaluation was carried out using a questionnaire survey. it shows that the model can produce averages of around 3 for meaning preservation and increasing the degree of simplicity, and 3.7 for grammaticality. while some of the most glaring issues in pwkp have been addressed, the corpus could be improved further by tackling the more involved cases of referential subjects and simplified sentence duplication. we plan to incorporate such changes in future releases of the corpus.