## 4 Methodology And Implementation
### Chatbot Recommender Architecture  
We designed our UDC recommendation in the form of a chatbot application since we decided to investigate how well conversational systems perform as recommenders. The chatbot application supports several different chatbots as well as proprietary and open-weight large language models that are available either through the Poe chatbot platform, or the Ollama framework (Figure 1). The Poe chatbot platform hosts a variety of proprietary and open-weight models that can be paid with credits. This can be a limitation since the quota of free credits is limited per day. However, a paid plan can be used to remove this limitation. The Ollama framework on the other hand is a completely local service which can be used to serve open-weight models; however, its limitation is the hardware capabilities on the local device. It is important to note that chatbot API requests using the Poe chatbot platform send the input text to the chatbot workflow, which is usually not just a generative large language model. Chatbot workflows can be very complex and can include different preprocessing methods, additional classifiers, and even supporting models. We opted to test the chatbot APIs of popular chatbot solutions ChatGPT (models GPT-3.5 Turbo, GPT-4o, and o1-mini), Claude (models Claude-3-Haiku and Claude-3.5-Haiku) and Gemini (models Gemini-1.0-Pro. Gemini-1.5-Flash and Gemini-2.0-Flash). Figure 1: Our UDC recommendation framework using proprietary and open-weight models. The Poe chatbot platform also hosts a variety of open-weight LLMs that can be used via APIs which return the generated text response. To further investigate the disparity between performance of proprietary chatbots with complex workflows and chatbots using open-weight LLMs, we tested some well-known LLMs. We used a combination of Poe platform and a local Ollama setup while testing these models. The models included Llama-2-70B, Llama-3-70B, Llama-3-1-7B, Llama-3.2-7B, Llama-3-3-70B, Gemma-7B-Instruct, Gemma-3-4B, Mixtral-8x7B-Chat, and DeepSeek-R1-7B. The details of all LLMs are given in Table 4. The technical details on computing resources, inference time, and costs required during the evaluation are given in Appendix B. From the metadata of the documents, we prepared prompts, performed requests to the Poe platform and Ollama APIs using a console application, and saved the resulting responses in JSON files. As context, a prompt included the title, keywords, and the abstract of the document. We used the zero-shot prompting technique since few-shot prompting did not make sense as we did not have predefined examples of suitable UDC codes in terms of documents containing the exact same contexts. The zero-shot prompting approach also closely resembles the manual process of librarians when they classify the documents into the UDC. Consequently, the models received only an instruction to recommend a UDC code, the document's context in form of a title, keywords, and an abstract, and further limiting instructions as part of our chosen prompting framework. The expected result was either a simple or a complex UDC expression for each document, which was stored into intermediate JSON files. The model outputs were not modified, edited, filtered, or corrected in any way before evaluation. The resulting JSON files that contained the result UDC code, used LLM, and document ID were processed during the analysis where a ground truth UDC value was compared to the result and a similarity metric value was calculated.