D ialog QAE : N-to-N Question Answer Pair Extraction from Customer Service Chatlog (2023)

Abstract: Harvesting question-answer (QA) pairs from customer service chatlog in the wild is an efficient way to enrich the knowledge base for customer service chatbots in the cold start or continuous integration scenarios. Prior work attempts to obtain 1-to-1 QA pairs from growing customer service chatlog, which fails to integrate the incomplete utterances from the dialog context for composite QA retrieval. In this paper, we propose N-to-N QA extraction task in which the derived questions and corresponding answers might be separated across different utterances. We introduce a suite of generative/discriminative tagging based methods with end-to-end and two-stage variants that perform well on 5 customer service datasets and for the first time setup a benchmark for N-to-N DialogQAE with utterance and session level evaluation metrics. With a deep dive into extracted QA pairs, we find that the relations between and inside the QA pairs can be indicators to analyze the dialogue structure, e.g. information seeking, clarification, barge-in and elaboration. We also show that the proposed models can adapt to different domains and languages, and reduce the labor cost of knowledge accumulation in the real-world product dialogue platform.

## introduction the development of natural language processing and conversational intelligence has radically redefined the customer service landscape. the customer service chatbots empowered by knowledge bases or frequently asked questions (faqs) drastically enhance the efficiency of customer support (e.g. @xcite @xcite @xcite in the cold start or continuous integration scenarios, harvesting qa pairs from existing or growing customer service chatlog is an efficient way to enrich knowledge bases. besides the retrieved qa pairs can be valuable resources to improve dialogue summarization @xcite , gain insights into the prevalent customer concerns and figure out new customer intents or sales trends @xcite , which are of vital importance to business growth. prior work on question-answer extraction follows the utterance matching paradigm, e.g., matching the answers to the designated questions in a dialogue session @xcite in the offline setting or figuring out the best response to the specific user query @xcite @xcite in the online setting. within this framework, 1-to-1 qa extraction has been explored by @xcite , however, we argue that users might not cover all the details in a single query while interacting with the customer service agents, which means that a certain qa pair might involve multiple utterances in the dialogue session. in this paper, we extend 1-to-1 qa matching to n-to-n qa extraction, where the challenges are two-fold: 1) cluster-to-cluster qa matching with no prior knowledge of the number of utterances involved in each qa pair and the number of qa pairs in each dialogue session, as the question might be distributed in single or multiple user queries. 2) session-level representation learning with a longer context, as the paired questions and answers might be separated within the dialogue, and the model shall detect multiple same-topic questions and then check if the answer is related to any one of the questions. we propose session-level tagging-based methods to deal with the two challenges. our method is not only compatible with the n-n qa extraction task setting but also 1-1 and 1-n, which is generic. switching from matching to tagging, we feed the entire dialogue session into powerful pre-trained models, like bert @xcite 送货什么时间能到？ when will the delivery arrive? @xmath0 do you have any follow-up question on our latest feedback, or other new questions? @xmath1. when will it arrive? i have been waiting for a whole day. ## conclusion in this paper, we propose n-to-n question and answer (qa) pair extraction from customer service dialogue history, where each question or answer may involve more than one dialogue utterance. we introduce a suite of end-to-end and two-stage tagging-based methods that perform well on 5 customer service datasets, as well as utterance and session level evaluation metrics for dialogqae. with further analysis, we find that the extracted qa pairs characterize the dialogue structure, e.g. information seeking, clarification, barge-in, and elaboration. extensive experiments show that the proposed models can adapt to different domains and languages and largely accelerate knowledge accumulation in the real-world dialogue platform.