# Automatic Simplification of Lithuanian Administrative Texts
## 6 Results
### Correlation  
To evaluate the effectiveness of our mBART and mT5 models in simplifying Lithuanian text, we used a combination of automatic metrics and human assessments. Spearman's rank correlation coefficient was particularly appropriate for our analysis compared to other correlation measures, such as Pearson's correlation, because it does not assume a linear relationship between variables, making it well-suited for handling our ordinal human evaluation scores. Additionally, it is less sensitive to outliers, which is beneficial given the inherent variability in human judgments [93].  
Spearman correlation coefficients between automatic evaluation metrics and human scores (Figure 3) across three dimensions of text simplification (simplicity, meaning retention, and grammaticality). In both heatmaps, warmer colors indicate stronger positive correlations, while cooler colors represent negative correlations.  
For the mBART model, BERScore shows moderate positive correlations with all three human evaluation criteria, around 0.40, indicating a reasonable alignment with human judgments on simplicity, meaning preservation, and grammaticality. ROUGE metrics also displayed moderate positive correlations, with ROUGE-L having the highest correlations, up to 0.42 for simplicity. SARI also performed well, particularly in simplicity and meaning preservation, with correlations of 0.42 and 0.37, respectively.  
Conversely, BLEU exhibited lower correlations, around 0.25, suggesting it might not be as reliable for this specific task. Readability indices (LIX and RIX) generally showed negative correlations, indicating that higher readability scores did not necessarily correspond to better human-judged text quality. This may have happened because readability indices focus on surface-level features like sentence length and word difficulty, which do not fully capture the nuanced aspects that human evaluators prioritize, such as meaning preservation and grammaticality. Human evaluations also consider the overall fluency of the text, an aspect that readability metrics are not designed to measure, making it difficult to directly compare these indices with human judgments.  
The mT5 model exhibited a similar trend, with BERTScore and ROUGE metrics showing moderate to strong positive correlations with human evaluations. ROUGE-1, in particular, had the highest correlation with grammaticality at 0.44. Meanwhile, SARI showed slightly lower correlations compared to mBART, particularly in simplicity. BLEU again showed relatively low correlations. The readability indices for the mT5 model  
Figure 3: Comparison of Spearman correlations between mBART and mT5 evaluation metrics and human scores.  
mirrored the results seen with mBART, with negative correlations, suggesting that these metrics might not effectively reflect human judgments in this context.  
_In summary, our analysis revealed that while automatic metrics like BERTScore and ROUGE align moderately well with human evaluations, others, like BLEU and readability indices, show lower or even negative correlations. This indicates that while automatic metrics are useful, they have limitations and should be complemented with human evaluations to ensure the nuanced aspects of text simplification are adequately captured._