# Beyond Lexical Boundaries: LLM-Generated Text Detection for Romanian Digital Libraries
## 5 Discussion
### Human-Machine Textual Similarity
\begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Domain** & **Model** & \begin{tabular}{c} **Cosine** \\ **M (SD)** \\ \end{tabular} & \
\begin{tabular}{c} **BLEU** \\ **M (SD)** \\ \end{tabular} &
\begin{tabular}{c} **ROUGE** \\ **M (SD)** \\ \end{tabular} \\ \hline \multirow{6}{*}{Books} & RoGPT2 & 0.359 (0.118) & 0.138 (0.124) & 0.251 (0.120) \\  & GPT-Neo-Ro & 0.358 (0.076) & 0.138 (0.018) & 0.148 (0.029) \\  & Flan-T5 & 0.545 (0.239) & 0.296 (0.237) & 0.395 (0.243) \\  & Davinci-TC & 0.375 (0.082) & 0.164 (0.048) & 0.202 (0.049) \\  & Davinci-BT & 0.410 (0.131) & 0.079 (0.063) & 0.133 (0.075) \\  & Opus-MT & 0.557 (0.147) & 0.237 (0.098) & 0.314 (0.099) \\  & mBART & 0.440 (0.141) & 0.118 (0.073) & 0.178 (0.081) \\ \hline \multirow{6}{*}{Legal} & RoGPT2 & 0.467 (0.127) & 0.196 (0.131) & 0.313 (0.125) \\  & GPT-Neo-Ro & 0.417 (0.088) & 0.151 (0.037) & 0.187 (0.051) \\  & Flan-T5 & 0.680 (0.160) & 0.405 (0.194) & 0.519 (0.170) \\  & Davinci-TC & 0.418 (0.160) & 0.139 (0.019) & 0.186 (0.031) \\  & Davinci-BT & 0.356 (0.121) & 0.054 (0.062) & 0.112 (0.078) \\  & Opus-MT & 0.530 (0.167) & 0.218 (0.124) & 0.294 (0.128) \\  & mBART & 0.348 (0.160) & 0.065 (0.073) & 0.105 (0.097) \\ \hline \multirow{6}{*}{Medical} & RoGPT2 & 0.375 (0.107) & 0.077 (0.072) & 0.212 (0.064) \\  & GPT-Neo-Ro & 0.387 (0.082) & 0.140 (0.013) & 0.137 (0.018) \\ \cline{1-1}  & Flan-T5 & 0.471 (0.234) & 0.219 (0.199) & 0.298 (0.226) \\ \cline{1-1}  & Davinci-TC & 0.403 (0.077) & 0.132 (0.034) & 0.182 (0.032) \\ \cline{1-1}  & Davinci-BT & 0.421 (0.121) & 0.075 (0.055) & 0.139 (0.068) \\ \cline{1-1}  & Opus-MT & 0.273 (0.127) & 0.019 (0.064) & 0.026 (0.081) \\ \cline{1-1}  & mBART & 0.392 (0.154) & 0.098 (0.064) & 0.143 (0.075) \\ \hline \hline \end{tabular}
\end{table} 
Table 5: Mean (M) and standard deviation (SD) values for similarities per model. Summarizing these results, we notice that Flan-T5 produces texts that are similar to human-produced ones while also having the highest variability (standard deviation), thus denoting potential inconsistencies. Since our data do not follow a normal distribution across the five domains, non-parametric statistical methods are required to compare and analyze the results of human and machine similarity. These tests are valuable when working with similarity scores, allowing us to make statistical inferences about the performance of different text generation models without making strong assumptions about the underlying data distribution. The Friedman test is a non-parametric statistical test that represents an alternative to repeated measures analysis of variance (ANOVA) and represents a measure of the variability between multiple models. Table 6 shows the Friedman test results for the five domains applied to cosine similarity scores. In hypothesis testing, a low \(p\)-value (typically less than 0.05) means the null hypothesis is rejected, and significant differences exist among the tested models. According to the results across all five domains, we have high Friedman scores and low \(p\)-values, which suggests the compared language models perform differently in each domain, and further analysis or pairwise comparisons can be conducted to understand the nature of these differences. We further perform the Wilcoxon test, targeting pairwise comparisons. The Wilcoxon signed-rank test is used to compare two paired groups and check if there are significant differences between them. Similarly to the Friedman test, the Wilcoxon test is used in cases where data does not follow a normal distribution, and it denotes the degree of differences between the two pairs analyzed. The Wilcoxon test results for our dataset across the five domains for cosine similarity scores are presented in Appendix D, Figure A8. The high Wilcoxon scores in the matrix (see Figure A8) argue substantial differences in cosine similarity scores between pairs of models. There is strong evidence to suggest that the variations in performance are statistically significant.