Learning as Conversation: Dialogue Systems Reinforced for Information Acquisition (2022)

Abstract: We propose novel AI-empowered chat bots for learning as conversation where a user does not read a passage but gains information and knowledge through conversation with a teacher bot. Our information acquisition-oriented dialogue system employs a novel adaptation of reinforced self-play so that the system can be transferred to various domains without in-domain dialogue data, and can carry out conversations both informative and attentive to users.

## introduction communication is the central process of education @xcite . in learning as conversation @xcite , a student does not read a passage but gains information and knowledge through conversation with a teacher who reads the passage. compared to the traditional learning by reading, learning as conversation has the advantages that conversation helps students stay engaged and that information is provided piece by piece, which helps strengthen learning with a shorter attention span. the advantages of learning as conversation have been verified with educational evidence @xcite @xcite . for example, studies have shown that when children read storybooks, parents' guided conversation, e.g., posing questions and providing responsive feedback, substantially amplifies the learning benefits. while high-quality conversations with experts are not always available, it would be help- * * work done during internship at ibm research ai 1 code and data at: https://github.com/ibm/ reinforced-dialog-system-for-learning ful if ai-empowered chat bots could be applied to facilitate users to gain information or knowledge. in recent years, there has been significant research in content-grounded dialogue generation, where external passages are used to inspire knowledge intensive dialogues. however, these systems or datasets are either for chit chat @xcite or for goal-oriented information seeking @xcite , little work has explored applying chat bots for the learning as conversation purpose. in this work we propose a novel task for learning as conversation: information-acquisition-oriented dialogue generation. given a passage, our chat bot actively engages with an end-user to form a coherent conversation, so that the user could gain knowledge without reading the passage. our task has a broad range of potential application venues in which people traditionally rely on reading to obtain information, including: as shown in figure 1 , for our task, a good conversation should have the following characteristics: our contributions include: 1) a novel task of information-acquisition-oriented dialogue system; 2) a novel unsupervised learning framework which enables a teacher bot to carry out informative and coherent conversations with human users for information acquisition purpose; 3) extensive experiments with human evaluation demonstrate the effectiveness of our proposed approach. ## approach in order to obtain an informative and attentive teaching dialogue system, we propose a framework that consists of two chat bots in different roles, and leverage both supervised learning and unsupervised reinforcement learning, as illustrated in figure 2 . the unsupervised reinforcement learning enables the system to be fine-tuned on other text corpus where no annotation or dialogue data is required. ## experimental settings we proceed by describing our datasets, comparison systems and evaluation metrics. we then show the performance of our proposed approach compared to state-of-the-art in §4. ## objective results results on test sets. trading off coverage for coherence. in figure 4 , we plot the learning curves of coverage and coherence scores when the reinforcer adopts a single reward (+cov, +coh) or both (full). we use r cov and r coh to approximate coverage and coherence scores. these plots are generated using 50 validation instances from the paper abstracts dataset. we observe that with only the coverage reward (+cov), teacher bot tends to aggressively copy content from the passage, while disregarding the conversation history. this inevitably leads to incoherent conversations. conversely, +coh can improve on coherence, but falls short on delivering informative conversations. finally, our full teacher bot trades off coverage for substantially higher coherence, thus achieving a significant improvement over the baseline doha model. figure 3 illustrates the gain of information for each of the three turns. the average rouge gain is reported for each turn, using conversations produced for the paper abstracts dataset. we observe that there is a general tendency across turns that information gain is decreasing. this is in part because that at the beginning of a conversation, teacher bot has no constraints regarding content selection, it could rephrase any content selected from the supporting passage to initiate a dialogue. in subsequent turns, teacher bot has to exercise caution in response generation considering both the conversation history and overall coherence of the conversation. consequently, we find that the average length of the utterances also decreases in subsequent turns. ## subjective results we demonstrate the subjective evaluation results in ## conclusion we propose an information-acquisition-oriented dialogue system that transfers information and knowledge in passages to users through conversations. an unsupervised self-talk approach is introduced leveraging novel rewards to enable teacher bots to deliver informative and attentive conversations. experiments with automatic and human evaluations demonstrate the effectiveness of our approach. some interesting future directions include extending the conversations to be based on a set of documents and specializing our dialogue systems for in specific domain, e.g. patient education.