# Automatic Simplification of Lithuanian Administrative Texts
## 2 Related Work  
Text simplification has advanced significantly in recent years, evolving from rule-based methods (e.g., [5; 6]) to data-driven approaches (e.g., [7; 8]). These techniques encompass various strategies for simplification, including lexical and syntactic simplification, while taking into consideration different aspects of text complexity. Lexical simplification focuses on simplifying the vocabulary of the text by substituting complex words with easier-to-understand alternatives. This approach generally involves several sub-tasks, such as identifying difficult words, then selecting appropriate replacements, refining and ranking them by simplicity [9]. These tasks employ techniques that range from using simple synonym dictionaries [10] to employing machine learning [11] and deep learning models like LSBERT [12]. Meanwhile, syntactic simplification aims to improve text's readability by modifying syntactic structures, such as converting passive constructions into active ones or reducing the number of clauses. As with lexical simplification, syntactic simplification methods vary from rule-based approaches to data-driven ones. The former ones involve parsing sentences, identifying complex structures, applying rewrite rules, and post-processing [13]. Data-driven methods employ statistical or neural machine learning models, such as statistical machine translation [14]. Neural Machine Translation (NMT) has become popular for text simplification, treating text simplification task as a translation problem, i.e., translating complex text into its simpler equivalent (e.g., [15; 16]). This approach has been applied for text simplification in low-resource scenarios, such as in [17; 18; 19]. Text simplification can also be framed as a text generation task, where sequence-to-sequence models have been applied for this purpose [20; 21; 22; 23]. The Transformer architecture has been particularly successful in text simplification as it processes entire input sequences to extract essential information (e.g., [24; 25]). Furthermore, large language models (LLMs) are now able to generate simplified texts that avoid complex and linked sentences [26; 27]. Recent studies have highlighted the potential of these models to simplify texts through various techniques, such as specifying the desired reading grade level [28] or directly indicating necessary simplification operations [29]. To name a few, the BERT model has been employed for lexical simplification [30], text simplification via monolingual machine translation [31], and hybrid text simplification approaches [21]. Similarly, the T5 model has been used for controllable text simplification [32; 33] and for simplification in low-resource scenarios [34; 35]. Also, BART has been applied to the tasks of controllable text simplification [32], as well as paragraph- [36] and document-level simplification [37]. Among the various LLMs, GPT models have gained particular attention, especially in low-resource scenarios [38; 39; 40]. In such cases, ChatGPT has demonstrated a potential in simplifying texts [41; 42]. However, evaluations of ChatGPT's performance indicate that while it simplifies text effectively, it may omit relevant information [43] and lack the depth and accuracy of human experts [44]. Nevertheless, when additional context is provided, ChatGPT may simplify complex texts effectively [45] and provide most of the factually correct information [26]. Some of the latest models and approaches for text simplification include SIMSUM, designed for automated document-level simplification [46], SimpleBART, which employs a specific pre-training strategy for this task [27], and KGSimple, which applies an unsupervised approach that leverages knowledge graphs to generate compressed text [47], to name a few. Also, instruction-based fine-tuning has been used for text simplification, where pre-trained models are designed to take instructions in natural language, specifying the characteristics the simplified text should enlist [48; 49; 50].