## 5 Results
### Documents in Slovenian  
When compared to results for documents in English, similar findings can be observed for the Slovenian language (Tables 8-10; Figures 5-7). Mistral-8x7B-Chat again outperforms all other models in the 4-7B parameter size group. There are also no indications implying that the Slovenian language is less represented with the models in this group. We attribute this to the fact that all of the models in this group are open-weight and have likely been trained on multilingual corpora, which sufficiently represent the Slovenian language. Additionally, efforts by several research groups in Slovenia have been made to improve the availability and accessibility of high quality open-sourced text corpora in the Slovenian language. The understanding of the document context for the task of UDC recommendation in a language like Slovenian, which has limited digital corpora and existing language models, seems to be on par with a well-resourced language such as English. The group with 70B parameter models consisting of the Llama family models from different generations again shows the effects of model improvements over time. The latest model in the group (Llama-3.3-70B) outperforms both Llama-3-70B and Llama-2-70B on all metrics. Overall, for this group of models, the metric values obtained from testing on the Slovenian language are very similar to those obtained from testing on the English language. These results could imply that the newer Llama family of models has been trained with focus on better multilingual support, including the Slovenian language; however, the best performing models still tend to have higher metric values for the English language. \begin{table}
\begin{tabular}{c c c c c c c} \hline \hline \multirow{2}{*}{**Model**} & \multicolumn{2}{c}{**[PERSON]**} & \multicolumn{2}{c}{**Jaro–Winkler**} & \multicolumn{2}{c}{**Adj. Hier. Sim.**} \\ \cline{2-7}  & **Median** & **SD** & **Median** & **SD** & **Median** & **SD** \\ \hline GPT-3.5-Turbo & 0.210 & 0.129 & 0.591 & 0.130 & 0.532 & 0.147 \\ \hline GPT-4o & 0.265 & 0.158 & 0.672 & 0.114 & 0.596 & 0.147 \\ \hline o1-mini & 0.188 & 0.102 & 0.569 & 0.107 & 0.511 & 0.127 \\ \hline Gemini-1.0-Pro & 0.259 & 0.123 & 0.621 & 0.101 & 0.535 & 0.156 \\ \hline Gemini-1.5-Flash & 0.238 & 0.116 & 0.608 & 0.098 & 0.544 & 0.128 \\ \hline Gemini-2.0-Flash & 0.250 & 0.142 & 0.639 & 0.113 & 0.563 & 0.146 \\ \hline Claude-3-Haiku & 0.232 & 0.149 & 0.630 & 0.125 & 0.562 & 0.147 \\ \hline Claude-3.5-Haiku & 0.259 & 0.140 & 0.638 & 0.107 & 0.570 & 0.144 \\ \hline \hline \end{tabular}
\end{table} 
Table 7: Median values and standard deviations (SD) of selected metrics for models with unknown parameter size evaluated on documents in English. \begin{table}
\begin{tabular}{c c c c c c c} \hline \hline \multirow{2}{*}{**Model**} & \multicolumn{2}{c}{**Levenshtein**} & \multicolumn{2}{c}{**[PERSON]–[PERSON]**} & \multicolumn{2}{c}{**Adj. Hier. Sim.**} \\ \cline{2-7}  & **Median** & **SD** & **Median** & **SD** & **Median** & **SD** \\ \hline Mistral-8x7B-Chat & 0.207 & 0.105 & 0.592 & 0.105 & 0.514 & 0.137 \\ \hline Gemma-7B-Instruct & 0.200 & 0.126 & 0.509 & 0.103 & 0.450 & 0.117 \\ \hline Llama-3.1-7B & 0.188 & 0.116 & 0.567 & 0.127 & 0.499 & 0.148 \\ \hline Llama-3.2-7B & 0.148 & 0.105 & 0.531 & 0.135 & 0.469 & 0.155 \\ \hline DeepSeek-R1-7B & 0.100 & 0.079 & 0.522 & 0.202 & 0.457 & 0.205 \\ \hline Gemma-3.4B & 0.200 & 0.094 & 0.569 & 0.110 & 0.504 & 0.127 \\ \hline \hline \end{tabular}
\end{table} 
Table 8: Median values and standard deviations (SD) of selected metrics for models of size 4-7B parameters evaluated on documents in Slovenian. Finally, in the group with proprietary models and unknown parameter sizes, GPT-4o performs the best on the selected metrics, followed by Claude-3.5-Haiku and the Gemini family of models (Gemini-1.0-Pro, Gemini-1.5-Flash, and Gemini-2.0-Flash). Again, the reasoning model o1-mini underperforms in comparison with other models. Similar to our test on the English documents, we attribute this to the size of this particular model.