## 5 Results
### Documents In English  
For the document text features in the English language, the selected models seemed to work well (Figures 2-4); however, they struggled with the UDC codes that represent very specific fields of expertise. We found that the tested models performed well in recommending UDC codes up to the third hierarchical level of the UDC (e.g., UDC codes like 5, 54, and 545). For further hierarchical levels of the UDC (e.g., UDC codes like 004.738.5), the models sometimes failed to grasp the concept of decimal notation with the period separator (.) and, therefore, returned invalid UDC codes in those cases (e.g., codes like 0047). In the evaluation, invalid UDC codes were not removed or treated in any special way. They were processed like every other valid UDC code. With the used metrics, parts of invalid codes still showed a promising attempt at giving a potentially useful UDC code, due to the hierarchical structure of the UDC. In some cases, the separator was used correctly; however, the numbers behind it were not defined in the UDC catalogue. These specific cases can be classified as hallucinations since the models completely made up UDC classes, which do not exist in practice. Additionally, the most common mistakes made by the models were due to order-switching of disciplines in documents with multidisciplinary content (e.g., models returned UDC code 004:621.3 instead of 621.3:004; computer science in electrical engineering instead of electrical engineering in computer science). This is a common problem even in manual assignment of UDC codes, since the cataloguing process takes place in a domain-specific library. For example, a thesis on online banking could be assigned a code 004:336 (computer science in finance) if it were catalogued by a computer science faculty library, or it could be assigned a code 336:004 (finance in computer science) if it were catalogued by an economics faculty library. This is a consequence of different processes of cataloguing within libraries as they commonly anchor to specific information about the documents such as the author, thesis supervisor, and organization of origin. Figure 2: Boxplot visualization of the Levenshtein similarity for models evaluated on documents in English. Table 5 presents the measured median values and standard deviations of selected metrics for models of size 4-7B parameters. The best performing model on all metrics in this group was Mistral-8x7B-Chat. All models in this group are open-weight. Surprisingly, the later released models did not perform as well--for example, the newest two models, DeepSeek-R1-7B and Gemma-3-4B, were underperforming, especially on the Levenshtein similarity. This might be due to a change in training data and the parameter size, type, and architecture of the LLM itself (e.g., DeepSeek-R1-7B is a reasoning model with 7B parameters). In general, the lower scores of the Levenshtein similarity stem from its properties and, in our use-case, it can be interpreted as a stricter measure than the other two metrics. The metric results for the group with larger (70B) parameter size LLMs are given in Table 6. All 70B models were the Llama models of the 2 nd and 3 rd generation. This is an interesting insight, as it shows the improvements in this particular model family over time. With Llama-2-70B being the oldest and least performant, both Llama-3-70B and Llama-3.3-70B outperformed it on all metrics. The newest model (Llama-3.3-70B) performed the  
Figure 4: Boxplot visualization of the adjusted hierarchical similarity for models evaluated on documents in English. Figure 3: Boxplot visualization of the Jaro–Winkler similarity for models evaluated on documents in English. best. In comparison with the 4-7B models, the 70B models perform better on all metrics as expected, due to the increased complexity of the models. Table 7 presents the metric scores for the group of models where the parameter sizes were not disclosed publicly.