## 4 Results and Discussion
### RQ6: GenAI or Machine Translators?
This suggests that GenAI translations occasionally align closely with the original semantic meaning, yet at other times, they deviate significantly, which accounts for the broader variability range observed. Regarding GenAI's inferior performance in the META and style adjustment tests but superior results in the other two models, it can be inferred that the META model is more stringent in assessing semantic similarity, thus imposing harsher penalties on GenAI translations, particularly when deviations in structure or word order occur. In contrast, the other models (DB and Corpus) may be less sensitive to such variations, allowing GenAI translations to perform better in terms of overall semantic similarity. Additionally, when compared to INCIBE, GenAI's higher performance in the other two models likely reflects its ability to align more closely with the general semantic meaning, even if it occasionally struggles with consistency in structure--an aspect that is crucial for the evaluation within the META model. Furthermore, the results of the style adjustment test to align with INCIBE, employing the new grouping, are provided in Table 19. The corresponding ranking can be found in Table 20.  
The results are unequivocal. GenAI systems generate translations that, across all metrics, exhibit significantly lower alignment with the INCIBE style in comparison to the translations produced by machine translators. On average, machine translators outperform GenAI across all metrics, demonstrating a higher degree of similarity to the reference translation. In the similarity ranking with INCIBE, machine translators consistently occupy the top position across all metrics, while GenAI ranks second in each case. The results from BLEU, GLEU, CHRF, and METEOR \begin{table}
\begin{tabular}{l c c c c c c c c} \hline \hline  & **BLEU** & **COMET** & **GLEU** & **RIBES** & **TER** & **YiSi** & **METEOR** & **CHRF** \\ \hline GenAI & 0.479 & 0.644 & 0.510 & 0.907 & 0.666 & 0.693 & 0.724 & 0.757 & 0.672 \\ Machine translators & 0.485 & 0.687 & 0.517 & 0.915 & 0.677 & 0.714 & 0.751 & 0.776 & 0.690 \\ \hline \hline \end{tabular}
\end{table} 
Table 20: Similarity ranking of the translations produced by GenAI and machine translators compared to those provided by INCIBE. \begin{table}
\begin{tabular}{l c c c c c c c c} \hline \hline  & **BLEU** & **COMET** & **GLEU** & **RIBES** & **TER** & **YiSi** & **METEOR** & **CHRF** \\ \hline GenAI & 2\({}^{\circ}\) & 2\({}^{\circ}\) & 2\({}^{\circ}\) & 2\({}^{\circ}\) & 2\({}^{\circ}\) & 2\({}^{\circ}\) & 2\({}^{\circ}\) & 2\({}^{\circ}\) \\ Machine translators & 1\({}^{\circ}\) & 1\({}^{\circ}\) & 1\({}^{\circ}\) & 1\({}^{\circ}\) & 1\({}^{\circ}\) & 1\({}^{\circ}\) & 1\({}^{\circ}\) & 1\({}^{\circ}\) \\ \hline \hline \end{tabular}
\end{table} 
Table 19: Values obtained for each evaluation metric when the reference translation is provided by INCIBE. confirm that machine translators exhibit greater fidelity to the structure, order, and vocabulary of the reference translation. In terms of more advanced metrics, such as COMET and YiSi, machine translators continue to surpass GenAI, indicating not only a closer structural alignment but also a more accurate preservation of the text's overall meaning. Although it is commonly believed that GenAI produces more natural and semantically enriched translations, the results of this study suggest that machine translators more accurately convey the reference meaning, as measured by these metrics. This discrepancy may be attributed to traditional machine translators (such as DeepL, Google Translate, and Bing) being specifically trained on aligned translation corpora optimized for reference text matching. In contrast, GenAI models (such as ChatGPT and Copilot) are not exclusively trained for translation tasks but for text generation, which may lead to more creative outputs that deviate from word-for-word alignment with the reference. Given that this evaluation emphasizes fidelity to the reference, GenAI's inherent flexibility may result in penalization.