# Using Large Language Models for Goal-Oriented Dialogue Systems
## 3 Materials and Methods  
In this study we evaluated and compared the performances of seven LLMs, including GhatGPT ([[https://chatgpt.com/](https://chatgpt.com/)]([https://chatgpt.com/](https://chatgpt.com/)) (accessed on 20 August 2024)), Mistral ([[https://mistral.ai/](https://mistral.ai/)]([https://mistral.ai/](https://mistral.ai/)) (accessed on 20 August 2024)), GigaChat ([[https://giga.chat/](https://giga.chat/)]([https://giga.chat/](https://giga.chat/)) (accessed on 20 August 2024)), Yandex GPT ([[https://ya.ru/ai/gpt](https://ya.ru/ai/gpt)]([https://ya.ru/ai/gpt](https://ya.ru/ai/gpt)) (accessed on 20 August 2024)), LLAMA ([[https://www.llama.com/](https://www.llama.com/)]([https://www.llama.com/](https://www.llama.com/)) (accessed on 20 August 2024)), MIXTRAL ([[https://mistral.ai/](https://mistral.ai/)]([https://mistral.ai/](https://mistral.ai/)) (accessed on 20 August 2024)), and Google Gemini ([[https://gemini.google.com/](https://gemini.google.com/)]([https://gemini.google.com/](https://gemini.google.com/)) (accessed on 20 August 2024)). The ChatGPT 3.5 language model was developed by Open AI based on the fine-tuned GPT 3.0 model, which contains 175 billion parameters. The mistral-Saiga language model is based on the Mistral-7B-v0.1 large language model, which is a pre-trained generative text model with 7 billion parameters. The Russian language model GigaChat is based on the ruGPT-3.5-13B large language model with 13 billion parameters. The Yandex GPT language model (YaLM 100B) uses 100 billion parameters. The Google Gemini language model was obtained by the fine-tuned PaLM model, which contains 540 billion parameters. The LLAMA and MIXTRAL language models each contain 7 billion parameters. The results of the comparison of the seven models are presented in Section 4.1. We used the prompt engineering technique to prepare a limited number of text prompts to classify user intent and perform named-entity recognition (NER) with an LLM. We used the English and Russian prompts presented in Figure 1. \begin{table}
\begin{tabular}{c c c c c c c} \hline \hline
**LLM** & **Dataset/Database** & **Language** & **RAG** & **Dialogue Generation** & **Metric** & **Ref.** \\ \hline \multirow{3}{*}{OPT, LLama2} & \multirow{3}{*}{CONVAI2} & \multirow{3}{*}{English} & \multirow{3}{*}{+} & Selective prompt tuning-based & F1, BLEU & \multirow{3}{*}{[48]} \\  & & & & dialogue generation & ROUGE-1 & \\ \cline{1-1} \cline{6-6}  & & & & & ROUGE-2 & \\ \hline \multirow{3}{*}{LLaMA} & \multirow{3}{*}{MultiWOZ 2.2, MANTS} & \multirow{3}{*}{Russian, English} & \multirow{3}{*}{+} & Context-based, LLM-based iterative & BERTScore, & \multirow{3}{*}{Our} \\  & & & dialogue generation with and & BLEU, Meteor, & & \\ \cline{1-1} \cline{6-6}  & & & without additional training on & human & \multirow{3}{*}{research} \\ \cline{1-1} \cline{6-6}  & & & labeled dialogues & evaluation & \\ \hline \hline \end{tabular}
\end{table} 
Table 1: _Cont._We evaluated the effectiveness of the proposed approaches on the following datasets:
* The English-language dataset MultiWOZ 2.2 (Multi-Domain Wizard-of-Oz) [49] contains text dialogues between people in seven different categories of service provision. The data under study contain a turn-by-turn dialogue of an average length of 14 responses between the USER and the SYSTEM, with each turn of the dialogue representing one utterance of the user or the system. In total, the dataset contains 113,748 utterances. * The multi-domain information-seeking dialogue MANtIS dataset [50] contains 80,000 information-seeking conversations. In total, the dataset contains 6701 labeled utterances.