title: Ð Ð°Ð·ÑÐ°Ð±Ð¾ÑÐºÐ° Ð¼ÐµÑÐ¾Ð´Ð¸ÐºÐ¸ Ð´Ð»Ñ ÑÑÐ¼Ð¼Ð°ÑÐ¸Ð·Ð°ÑÐ¸Ð¸ Ð´Ð¸Ð°Ð»Ð¾Ð³Ð¾Ð² Ð½Ð° Ð¾ÑÐ½Ð¾Ð²Ðµ Ð±Ð¾Ð»ÑÑÐ¾Ð¹ ÑÐ·ÑÐºÐ¾Ð²Ð¾Ð¹ Ð¼Ð¾Ð´ÐµÐ»Ð¸ Ð² ÑÑÐ»Ð¾Ð²Ð¸ÑÑ Ð´ÐµÑÐ¸ÑÐ¸ÑÐ° ÑÐ°Ð·Ð¼ÐµÑÐµÐ½Ð½ÑÑ Ð´Ð°Ð½Ð½ÑÑ
title: Ð²ÑÐ¿ÑÑÐºÐ½Ð°Ñ ÐºÐ²Ð°Ð»Ð¸ÑÐ¸ÐºÐ°ÑÐ¸Ð¾Ð½Ð½Ð°Ñ ÑÐ°Ð±Ð¾ÑÐ° Ð±Ð°ÐºÐ°Ð»Ð°Ð²ÑÐ°
titleType: Subtitle
title: Developing a Method for Dialogue Summarization Using a LLM with Limited Labeled Data
Creators: name: ÐÐµÐ½ÑÑÐ¾Ð², ÐÐ¼Ð¸ÑÑÐ¸Ð¹
Publisher: name: Ð¡Ð°Ð½ÐºÑ-ÐÐµÑÐµÑÐ±ÑÑÐ³ÑÐºÐ¸Ð¹ Ð¿Ð¾Ð»Ð¸ÑÐµÑÐ½Ð¸ÑÐµÑÐºÐ¸Ð¹ ÑÐ½Ð¸Ð²ÐµÑÑÐ¸ÑÐµÑ ÐÐµÑÑÐ° ÐÐµÐ»Ð¸ÐºÐ¾Ð³Ð¾
Year: 2025
Subjects: ÑÑÐ¼Ð¼Ð°ÑÐ¸Ð·Ð°ÑÐ¸Ñ Ð´Ð¸Ð°Ð»Ð¾Ð³Ð¾Ð²
Ð±Ð¾Ð»ÑÑÐ¸Ðµ ÑÐ·ÑÐºÐ¾Ð²ÑÐµ Ð¼Ð¾Ð´ÐµÐ»Ð¸
ÑÐ¸Ð½ÑÐµÑÐ¸ÑÐµÑÐºÐ¸Ðµ Ð´Ð°Ð½Ð½ÑÐµ
Ð´ÐµÑÐ¸ÑÐ¸Ñ ÑÐ°Ð·Ð¼ÐµÑÐµÐ½Ð½ÑÑ Ð´Ð°Ð½Ð½ÑÑ
dialogue summarization
large language models
synthetic data
limited labeled data

description: Ð Ð´Ð°Ð½Ð½Ð¾Ð¹ ÑÐ°Ð±Ð¾ÑÐµ Ð¾Ð±ÑÐµÐºÑÐ¾Ð¼ Ð¸ÑÑÐ»ÐµÐ´Ð¾Ð²Ð°Ð½Ð¸Ñ Ð²ÑÑÑÑÐ¿Ð°ÐµÑ Ð¿ÑÐ¾ÑÐµÑÑ Ð°Ð²ÑÐ¾Ð¼Ð°ÑÐ¸ÑÐµÑÐºÐ¾Ð¹ Ð°Ð½Ð½Ð¾ÑÐ°ÑÐ¸Ð¸ Ð´Ð¸Ð°Ð»Ð¾Ð³Ð¾Ð² Ð½Ð° Ð±Ð°Ð·Ðµ LLM. ÐÑÐµÐ´Ð¼ÐµÑ Ð¸ÑÑÐ»ÐµÐ´Ð¾Ð²Ð°Ð½Ð¸Ñ - Ð¼ÐµÑÐ¾Ð´Ñ ÐºÐ¾Ð½ÑÐµÐºÑÑÐ½Ð¾Ð³Ð¾ Ð¾Ð±ÑÑÐµÐ½Ð¸Ñ Ð¸ Ð¼ÐµÑÐ¾Ð´Ñ RAG Ð´Ð»Ñ Ð¾ÑÐ±Ð¾ÑÐ° Ð´ÐµÐ¼Ð¾Ð½ÑÑÑÐ°ÑÐ¸Ð¾Ð½Ð½ÑÑ Ð¿ÑÐ¸Ð¼ÐµÑÐ¾Ð², Ð¿Ð¾Ð²ÑÑÐ°ÑÑÐ¸Ñ ÐºÐ°ÑÐµÑÑÐ²Ð¾ ÑÑÐ¼Ð¼Ð°ÑÐ¸Ð·Ð°ÑÐ¸Ð¸. Ð¦ÐµÐ»Ñ - ÑÐ°Ð·ÑÐ°Ð±Ð¾ÑÐºÐ° Ð¼ÐµÑÐ¾Ð´Ð¸ÐºÐ¸ Ð¸ ÑÑÐµÐ¹Ð¼Ð²Ð¾ÑÐºÐ°, Ð¿Ð¾Ð·Ð²Ð¾Ð»ÑÑÑÐµÐ³Ð¾ Ð°Ð²ÑÐ¾Ð¼Ð°ÑÐ¸ÑÐµÑÐºÐ¸ ÑÐ¾ÑÐ¼Ð¸ÑÐ¾Ð²Ð°ÑÑ Ð¾Ð±ÑÑÐ°ÑÑÐ¸Ðµ Ð½Ð°Ð±Ð¾ÑÑ Ð¸ Ð°Ð½Ð½Ð¾ÑÐ°ÑÐ¸Ð¸ Ñ Ð¼Ð¸Ð½Ð¸Ð¼Ð°Ð»ÑÐ½ÑÐ¼Ð¸ Ð·Ð°ÑÑÐ°ÑÐ°Ð¼Ð¸ Ð½Ð° ÑÑÑÐ½ÑÑ ÑÐ°Ð·Ð¼ÐµÑÐºÑ, Ð° ÑÐ°ÐºÐ¶Ðµ Ð¾ÑÐµÐ½ÐºÐ° ÐµÐ³Ð¾ ÑÑÑÐµÐºÑÐ¸Ð²Ð½Ð¾ÑÑÐ¸ Ð¾ÑÐ½Ð¾ÑÐ¸ÑÐµÐ»ÑÐ½Ð¾ Ð°Ð»ÑÑÐµÑÐ½Ð°ÑÐ¸Ð²Ð½ÑÑ ÑÐµÑÐµÐ½Ð¸Ð¹. ÐÐµÑÐ¾Ð´Ð¾Ð»Ð¾Ð³Ð¸ÑÐµÑÐºÐ°Ñ Ð¾ÑÐ½Ð¾Ð²Ð° ÑÐ°Ð±Ð¾ÑÑ - Ð¿ÑÐ¸Ð¼ÐµÐ½ÐµÐ½Ð¸Ðµ Ð¼ÐµÑÐ¾Ð´Ð¾Ð² Ð¼Ð°ÑÐ¸Ð½Ð½Ð¾Ð³Ð¾ Ð¾Ð±ÑÑÐµÐ½Ð¸Ñ, ÐºÐ¾Ð½ÑÐµÐºÑÑÐ½Ð¾Ð³Ð¾ Ð¾Ð±ÑÑÐµÐ½Ð¸Ñ Ð¸ Ð¼ÐµÑÐ°Ð½Ð¸Ð·Ð¼Ð¾Ð² RAG. Ð ÑÐ¾Ð´Ðµ Ð¸ÑÑÐ»ÐµÐ´Ð¾Ð²Ð°Ð½Ð¸Ñ Ð¿ÑÐ¾Ð°Ð½Ð°Ð»Ð¸Ð·Ð¸ÑÐ¾Ð²Ð°Ð½Ñ Ð²Ð¾Ð·Ð¼Ð¾Ð¶Ð½Ð¾ÑÑÐ¸ ÑÐ¾Ð²Ð¼ÐµÑÑÐ½Ð¾Ð¹ Ð¸Ð½ÑÐµÐ³ÑÐ°ÑÐ¸Ð¸ ÐºÐ¾Ð½ÑÐµÐºÑÑÐ½Ð¾Ð³Ð¾ Ð¾Ð±ÑÑÐµÐ½Ð¸Ñ Ð¸ RAG, Ð¿ÑÐ¾Ð²ÐµÐ´ÑÐ½ ÑÑÐ°Ð²Ð½Ð¸ÑÐµÐ»ÑÐ½ÑÐ¹ Ð°Ð½Ð°Ð»Ð¸Ð· ÑÑÑÐ°ÑÐµÐ³Ð¸Ð¹ Ð¾ÑÐ±Ð¾ÑÐ° Ð´ÐµÐ¼Ð¾Ð½ÑÑÑÐ°ÑÐ¸Ð¹ Ð¸ ÑÐµÐ°Ð»Ð¸Ð·Ð¾Ð²Ð°Ð½Ñ ÑÐºÑÐ¿ÐµÑÐ¸Ð¼ÐµÐ½ÑÑ Ð½Ð° Ð°Ð½Ð³Ð»Ð¾ÑÐ·ÑÑÐ½ÑÑ, ÑÑÑÑÐºÐ¾ÑÐ·ÑÑÐ½ÑÑ Ð¸ Ð¼ÐµÐ´Ð¸ÑÐ¸Ð½ÑÐºÐ¸Ñ Ð´Ð¸Ð°Ð»Ð¾Ð³Ð°Ñ Ñ Ð¸ÑÐ¿Ð¾Ð»ÑÐ·Ð¾Ð²Ð°Ð½Ð¸ÐµÐ¼ Ð¼ÐµÑÑÐ¸Ðº ROUGE Ð¸ BERTScore. Ð ÐµÐ·ÑÐ»ÑÑÐ°ÑÑ Ð¿Ð¾Ð´ÑÐ²ÐµÑÐ´Ð¸Ð»Ð¸ ÑÑÑÐµÑÑÐ²ÐµÐ½Ð½Ð¾Ðµ ÑÐ»ÑÑÑÐµÐ½Ð¸Ðµ ÐºÐ°ÑÐµÑÑÐ²Ð° ÑÐµÐ·ÑÐ¼Ð¸ÑÐ¾Ð²Ð°Ð½Ð¸Ñ, Ð¾ÑÐ¾Ð±ÐµÐ½Ð½Ð¾ Ð² ÑÐ·ÐºÐ¾ÑÐ¿ÐµÑÐ¸Ð°Ð»Ð¸Ð·Ð¸ÑÐ¾Ð²Ð°Ð½Ð½ÑÑ ÑÑÐµÐ½Ð°ÑÐ¸ÑÑ, Ð¸ Ð¿Ð¾ÐºÐ°Ð·Ð°Ð»Ð¸, ÑÑÐ¾ Ð¿ÑÐµÐ´Ð»Ð¾Ð¶ÐµÐ½Ð½ÑÐ¹ Ð¿Ð¾Ð´ÑÐ¾Ð´ Ð¼Ð¾Ð¶ÐµÑ Ð¾Ð±ÑÐ¾Ð´Ð¸ÑÑÑÑ Ð±ÐµÐ· Ð¼Ð°ÑÑÑÐ°Ð±Ð½Ð¾Ð¹ ÑÑÑÐ½Ð¾Ð¹ Ð°Ð½Ð½Ð¾ÑÐ°ÑÐ¸Ð¸ Ð¿ÑÐ¸ ÑÐ¾ÑÐ¼Ð¸ÑÐ¾Ð²Ð°Ð½Ð¸Ð¸ Ð¾Ð±ÑÑÐ°ÑÑÐ¸Ñ Ð²ÑÐ±Ð¾ÑÐ¾Ðº. Ð Ð°Ð·ÑÐ°Ð±Ð¾ÑÐ°Ð½Ð½ÑÐ¹ Ð¿Ð¾Ð´ÑÐ¾Ð´ Ð¿ÑÐ¸Ð¼ÐµÐ½Ð¸Ð¼ Ð´Ð»Ñ Ð°Ð²ÑÐ¾Ð¼Ð°ÑÐ¸ÑÐµÑÐºÐ¾Ð¹ ÑÐ°Ð·Ð¼ÐµÑÐºÐ¸ Ð´Ð¸Ð°Ð»Ð¾Ð³Ð¾Ð² Ð² ÐºÐ¾Ð»Ð»-ÑÐµÐ½ÑÑÐ°Ñ, Ð¼ÐµÐ´Ð¸ÑÐ¸Ð½ÑÐºÐ¸Ñ Ð¸ Ð¸Ð½ÑÑ ÑÐµÑÐ²Ð¸ÑÐ°Ñ, Ð¾Ð±Ð»ÐµÐ³ÑÐ°Ñ Ð°Ð½Ð½Ð¾ÑÐ°ÑÐ¸Ñ Ð¸ Ð¿Ð¾Ð²ÑÑÐ°Ñ ÑÐ¾ÑÐ½Ð¾ÑÑÑ Ð¸ÑÐ¾Ð³Ð¾Ð²ÑÑ ÑÐµÐºÑÑÐ¾Ð². ÐÑÐ²Ð¾Ð´Ñ Ð¿Ð¾Ð´ÑÐ²ÐµÑÐ¶Ð´Ð°ÑÑ, ÑÑÐ¾ Ð¸ÑÐ¿Ð¾Ð»ÑÐ·Ð¾Ð²Ð°Ð½Ð¸Ðµ RAG Ð¸ ÐºÐ¾Ð½ÑÐµÐºÑÑÐ½Ð¾Ð³Ð¾ Ð¾Ð±ÑÑÐµÐ½Ð¸Ñ LLM Ð¿Ð¾Ð²ÑÑÐ°ÐµÑ ÐºÐ°ÑÐµÑÑÐ²Ð¾ ÑÑÐ¼Ð¼Ð°ÑÐ¸Ð·Ð°ÑÐ¸Ð¹ Ð¸ ÑÐ¿ÑÐ¾ÑÐ°ÐµÑ ÑÐ¾ÑÐ¼Ð¸ÑÐ¾Ð²Ð°Ð½Ð¸Ðµ Ð¾Ð±ÑÑÐ°ÑÑÐ¸Ñ Ð½Ð°Ð±Ð¾ÑÐ¾Ð². ÐÑÐµÐ´Ð»Ð¾Ð¶ÐµÐ½Ð½Ð°Ñ Ð¼ÐµÑÐ¾Ð´Ð¸ÐºÐ° Ð¸ ÑÑÐµÐ¹Ð¼Ð²Ð¾ÑÐº Ð¼Ð¾Ð³ÑÑ Ð±ÑÑÑ Ð²Ð½ÐµÐ´ÑÐµÐ½Ñ Ð² ÑÐ¸ÑÐ¾ÐºÐ¸Ð¹ ÑÐ¿ÐµÐºÑÑ Ð¿ÑÐ¸Ð»Ð¾Ð¶ÐµÐ½Ð¸Ð¹, ÑÑÐµÐ±ÑÑÑÐ¸Ñ ÑÐ¾ÑÐ½Ð¾Ð¹ Ð¸ ÑÐºÐ¾Ð½Ð¾Ð¼Ð¸ÑÐ½Ð¾Ð¹ Ð´Ð¸Ð°Ð»Ð¾Ð³Ð¾Ð²Ð¾Ð¹ ÑÑÐ¼Ð¼Ð°ÑÐ¸Ð·Ð°ÑÐ¸Ð¸.
description: In this paper, the object of the study is the process of automatic annotation of dialogues based on LLM. The subject of the study is contextual learning methods and RAG methods for selecting demonstration examples that improve the quality of summarization. The goal is to develop a methodology and framework that allows for automatic formation of training sets and annotations with minimal costs for manual labeling, as well as to evaluate its effectiveness relative to alternative solutions. The methodological basis of the work is the use of machine learning methods, contextual learning and RAG mechanisms. During the study, the possibilities of joint integration of contextual learning and RAG were analyzed, a comparative analysis of demonstration selection strategies was carried out, and experiments were carried out on English-language, Russian-language and medical dialogues using the ROUGE and BERTScore metrics. The results confirmed a significant improvement in the quality of summarization, especially in highly specialized scenarios, and showed that the proposed approach can do without large-scale manual annotation when forming training samples. The developed approach is applicable for automatic dialog labeling in call centers, medical and other services, facilitating annotation and increasing the accuracy of the resulting texts. The findings confirm that the use of RAG and contextual LLM learning improves the quality of summaries and simplifies the formation of training sets. The proposed methodology and framework can be implemented in a wide range of applications requiring accurate and cost-effective dialog summaries.