## 3 Materials and Methods
### Evaluation Metrics  
In order to evaluate the translations obtained without the necessity of a reference translation, several word embedding models were trained using the FastText library [55] with various available corpora. These models had previously been trained on large volumes of text to learn word relationships, particularly focusing on English-Spanish language pairs. The first model pair utilized all translations from the original dataset provided by INCIBE [8], which included vulnerabilities and their corresponding translations. The second pair employed a publicly available translation corpus, also provided by INCIBE [57]. Finally, the third model pair consisted of pretrained models, _wiki.en.bin_ and _wiki.es.bin_, which are available within the FastText library and were developed as part of the MUSE research project [58] by Meta.  
For the anchor word dictionaries of the first and second model pairs, a script was developed to identify repeated words within each pair of descriptions (English-Spanish) associated with the same vulnerability. This script follows a procedure to analyze and  
\begin{table}
\begin{tabular}{c c} \hline \hline
**GenAI** & **Prompt** \\ \hline \hline \multirow{2}{*}{Copilot} & Translate the lines of this text into Spanish independently. \\  & Respond only with the translations. \\ \hline \hline \multirow{2}{*}{ChatGPT} & Translate the lines of this text into Spanish independently. \\  & Respond only with the translations. \\ \hline \hline \end{tabular}
\end{table}
Table 8: Prompts used for AI-generated translation samples.  
compare two text files in different languages, identifying common words and generating structured results. Initially, both files are processed simultaneously, with each line split into individual words and repetitions within the same line removed. A comprehensive word count is then performed in each file, recording the frequency of occurrences and the lines in which these words appear. Subsequently, words appearing in both lines of the files are identified as common words. These words are stored alongside their frequency of occurrence and the lines where they match in both texts. After all lines are processed, words that occur with identical frequency in both files are filtered, consolidating the relevant data. For the third model pair, an anchor word dictionary provided by the same sources as the models was used. After obtaining the three model pairs and their respective anchor word dictionaries, the script was executed for each of them. This script first extracts vector representations for a set of shared key words, referred to as anchors, in both languages. These anchors are then used to compute an orthogonal transformation matrix, which aligns the source language's vector space with that of the target language, employing the Orthogonal Procrustes technique. For each sentence pair, the semantic similarity between the source text and its possible translations is computed by obtaining their respective vector representations and calculating the average of the word vectors that compose them. The vector spaces for each sentence pair are then aligned, and their similarity is measured using cosine similarity, leveraging tools from the SciPy library [59].  
The selection of evaluation metrics with a reference translation (specifically, the translation provided by INCIBE) for the experiment was guided by the objective of ensuring a diverse range of aspects were assessed, including metrics of different types. Furthermore, the selection was influenced by resource constraints and the requests of the expert team, prioritizing metrics deemed most applicable for practical use. Initially, BERTScore and Trados Studio were considered; however, they were excluded from the evaluation due to their paid nature, high execution time, and request limitations. The final selection of the metrics is included in Table 9.