# Automatic Simplification of Lithuanian Administrative Texts
## 7 Discussion  
Our study on the simplification of Lithuanian administrative texts to plain language brings several findings in terms of model performance, data augmentation, evaluation metrics, and the capabilities of different language models.  
_Contrary to common expectations, data augmentation did not improve model performance in our case._ The original dataset produced superior results, suggesting that the quality and distribution of the initial training data were sufficient for the current configuration of simplification experiments. This finding reinforces the importance of high-quality, representative data in text simplification tasks, particularly for less-resourced languages like Lithuanian.  
While automated metrics such as BLEU and ROUGE are widely used, our findings support their limitations in capturing the nuances of text simplification. Therefore, human evaluators proved to be crucial in assessing grammaticality, semantic accuracy, and simplification. This aligns with broader discussions in the NLP community about the need for more comprehensive evaluation methods that combine automated metrics with human judgment [94; 95; 96].  
However, analysis of inter-rater reliability revealed fair agreement on meaning retention and grammaticality, but only slight agreement on simplicity. This discrepancy highlights the subjective nature of assessing simplification, particularly in the context of Plain Lithuanian, which is a relatively new example of tailored language. Future research should focus on developing more standardized criteria for evaluating simplicity in plain language contexts.  
In our analysis of the relationship between automatic metrics and human scores, we observed moderate positive correlations for BERTScore and ROUGE metrics, particularly for the mBART model. SARI also showed promise, especially in assessing simplicity and meaning preservation. However, BLEU exhibited lower correlations, suggesting its limitations for this specific task. Interestingly, readability indices (LIX and RIX) showed negative correlations with human evaluations, highlighting a discrepancy between surface-level textual features and human judgment of simplified text quality.  
Furthermore, both fine-tuned models, especially mBART, while mT5 to a lesser extent, demonstrated adequate results in applying certain linguistic structures characteristic of Plain Lithuanian, such as adding subject and possessive pronouns and transforming passive voice to active. However, both models struggled with consistently achieving full simplification scores, indicating a need for more diverse training data. The mT5 model, in particular, struggled with longer passages, sometimes producing incoherent outputs or omitting significant portions of essential information.  
Our experiments with ChatGPT revealed limitations in its ability to apply specific linguistic rules for Lithuanian text simplification. While ChatGPT performed adequately with simple prompts, it struggled to incorporate more complex, language-specific rules. This aligns with findings from related studies on ChatGPT's limited metalinguistic awareness for less-resourced languages [92; 97; 98], which are usually less supported in large language models.  
To summarize, our findings support the need for task-specific and language-specific evaluation metrics that would better align with human judgments. Also, the importance of human evaluation in combination with automated metrics for a more comprehensive assessment of text simplification quality has been stressed. Finally, our results highlight the necessity for larger, more diverse training datasets to improve model performance on complex sentences and ensure consistent simplification.