# Automatic Simplification of Lithuanian Administrative Texts
## 6 Results
### Automatic Evaluation
#### 6.1.2 Text Simplification by Fine-Tuned mT5 and mBART  
The results of the simplification of Lithuanian administrative texts by mBART and mT5 models are summarized in Table 2. The performance of both models was evaluated using various metrics, including BLEU, SARI, BERTScore, ROUGE, and readability indices such as LIX and RIX. For testing our models, we used our test dataset (see Section 3.) Figure 2: Comparison of evaluation metrics for mT5. (**a**) SacreBLEU Score (**b**) Rougel score (**c**) Rougel Score (**d**) Rouge2 Score. The BLEU score, which measures the similarity between the model output and the reference text in terms of n-gram overlap, was higher for the old mBART (0.4196), fine-tuned on Parallel Corpus 1, compared to the new mBART (0.2688) and both versions of mT5, indicating that the old mBART, n-gram wise, produced outputs more closely aligned with n-grams in the reference simplifications. SARI, another metric that we used to evaluate the quality of text simplification, was also higher for the old mBART model (72.9781) compared to the new mBART model (57.2374) and both versions of mT5. This suggests that the old mBART was more effective in simplifying the text in terms of words kept, added, and deleted during simplification. Following this tendency, old mBART achieved the highest score of BERIscore as well, outperforming the other three models. This further supports the conclusion that the old mBART generates outputs that are semantically closer to the reference. ROUGE metrics, which measure the overlap of n-grams between the model outputs and reference texts, were consistently higher for the old mBART across all ROUGE variants: ROUGE-1 (0.7797), ROUGE-2 (0.6753), and ROUGE-L (0.7555). This demonstrates the old mBART's better performance in capturing the relevant information from the source text. In terms of readability, the _Original Sentence_ LIX and RIX scores, calculated for original sentences in both datasets (Parallel Corpus 1 and Parallel Corpus 2) separately, were identical for both models (76.3475 and 9.8409, respectively), reflecting the complexity of the input text. Meanwhile, the _Human Simplified_ LIX and RIX scores, calculated for simplifications made by experts, were significantly lower (68.2252 and 7.6104), indicating that the simplified texts were easier to read and understand. Although according to the LIX score, expert simplified sentences still fell into the category of difficult text, the RIX score showed that these sentences approached the universal readability level. The _Model Simplified_ LIX and RIX scores revealed differences in readability in terms of outputs produced by our models. The new mBART achieved a LIX score of 69.7718 and a RIX score of 7.9385, which are closer to the _Human Simplified_ scores, indicating better readability compared to the old mBART, which scored 70.2625 for LIX and 8.1929 for RIX. In contrast, the new mT5's scores were higher (82.5804 for LIX and 15.1725 for RIX), suggesting that its outputs were less simplified and more complex. The old mT5 had better readability scores compared to the new mT5, with scores of 81.1109 for LIX and 12.5187 for RIX. Overall, the results indicate that _old mBART outperforms new mBART and both versions of mT5 in most metrics for simplification of Lithuanian administrative texts, providing outputs that are more accurate, semantically more similar to reference sentences, and easier to read._ However, the _new mBART shows improved readability over the old mBART in terms of LIX and RIX scores.