Insurance Question Answering via Single-turn Dialogue Modeling (2022)

Abstract: With great success in single-turn question answering (QA), conversational QA is currently receiving considerable attention. Several studies have been conducted on this topic from different perspectives. However, building a real-world conversational system remains a challenge. This study introduces our ongoing project, which uses Korean QA data to develop a dialogue system in the insurance domain. The goal is to construct a system that provides informative responses to general insurance questions. We present the current results of single-turn QA. A unique aspect of our approach is that we borrow the concepts of intent detection and slot filling from task-oriented dialogue systems. We present details of the data construction process and the experimental results on both learning tasks.

## introduction although there has been significant progress in single-turn question answering (qa), it cannot cover complex questions of realistic scenarios @xcite . recently, multi-turn (conversational) qa has emerged as an alternative to address this problem by clarifying the questions via conversation @xcite @xcite . conversational qa is a category of dialogue systems which are divided into task-oriented, chitchat, and qa systems @xcite . however, qa is not always distinct from the other two categories. in this study, we are interested in building a dialogue system in a restricted domain, insurance. the system aims to provide users with general descriptions of cancer insurance. we assumed the task is not pre-defined and should be specified from the data. a significant difficulty is that complete conversational data does not exist. therefore, we needed to find other types of source data similar to * corresponding author: young-min kim the dialogues between users and experts on cancer insurance. the first is q&a data from a korean online qa service. although our goal is to construct a multi-turn dialogue system, this study covers only the singleturn qa corresponding to the target system's front part. the novelty of the present study is that we designed the system considering the further extension to multi-turns. therefore, unlike the existing kb-based or neural qa systems, we borrow the concept of intent detection and slot filling from task-oriented dialogue systems @xcite . the transformer-based pre-trained models such as bidirectional encoder representations from transformers (bert) achieved excellent performance for nlp tasks. bert is one of the pioneers of the pre-trained language representation models @xcite . since it was proposed in 2018, a paradigm shift has taken place in the nlp domain. most nlp tasks now are based on pre-trained language models. meanwhile, there are also previous studies using directly bert embeddings directly to express queries for conversational qa or faq retrieval @xcite @xcite . we use a korean version of electra @xcite , a variant of bert, for intent detection and slot filling. this study introduces intermediate results of our ongoing project on dialogue system construction in the insurance domain. we encountered many challenging situations from the first stage, data collection. we designed the system to be constructed using single-turn qa data but to finally serve as a multi-turn dialogue system. in the remainder of this paper, we describe the process of constructing training data for insurance qa in section 2. then the methods used for intent detection, slot filling, and the other approaches for the answer retrieval are presented in section 3. section 4 presents the experimental results for both learning tasks, including a quantitative analysis of the answer retrieval result. then we conclude with some future works in section 5. ## experiments first, we present the experimental results for intent detection and slot filling. the former is a typical classification problem, and the latter can be interpreted as a sequence labeling task, as introduced in section 3. koelectra was for the training in both tasks. second, we described the performance of answer retrieval from the faqs. the weighted sum of the three scores was the similarity score of the utterance-faq pair: we selected the faq that had the highest similarity score given an utterance. ## conclusions in this study, we built a single-turn dialogue system corresponding to the front part of our target system for the insurance domain. our final goal is to construct a multi-turn dialogue system that can return informative counselors about insurance. for future scalability, the concept of intention detection and slot filling was borrowed, therefore, for this purpose, training data and kb was constructed on their own. we obtained an encouraging result for both tasks despite the limited quantity of the source. to enhance the performance of the slots with low occurrence and high-value diversity, we proposed a slot replacement method through dictionary mapping. the method also provided a good result. future work first includes modifying category definitions and improving answer retrieval performance. furthermore, we will re-design the system for the multi-turn dialogues. we expect the extracted intents and slot values to be effectively used for the multi-turn system.