Restatement and Question Generation for Counsellor Chatbot (2021)

Abstract: Amidst rising mental health needs in society, virtual agents are increasingly deployed in counselling. In order to give pertinent advice, counsellors must first gain an understanding of the issues at hand by eliciting sharing from the counsellee. It is thus important for the counsellor chatbot to encourage the user to open up and talk. One way to sustain the conversation flow is to acknowledge the counsellee’s key points by restating them, or probing them further with questions. This paper applies models from two closely related NLP tasks — summarization and question generation — to restatement and question generation in the counselling context. We conducted experiments on a manually annotated dataset of Cantonese post-reply pairs on topics related to loneliness, academic anxiety and test anxiety. We obtained the best performance in both restatement and question generation by fine-tuning BertSum, a state-of-the-art summarization model, with the in-domain manual dataset augmented with a large-scale, automatically mined open-domain dataset.

## introduction advances in dialog modeling have facilitated chatbot use in many domains @xcite @xcite . they are now also increasingly deployed for mental health assistance, including counselling @xcite . dialogs in counselling share some common characteristics with those in other domains. advice generation, for example, can be implemented with a q&a model that retrieves counselling materials from a knowledge base @xcite . empathetic language -words that reflect the feelings of one's interlocutors -is conducive to establishing rapport with the counsellee. research in empathetic response generation has led to systems that can recognize the emotional state of the user, and generate responses tailored to that state @xcite . the counsellor must also encourage the counsellee to open up and talk in order to gain an adequate understanding of the issues at hand. a common strategy to sustain the conversation flow is to use "encouragers" @xcite , such as backchannel phrases, restatements and questions. a good restatement acknowledges main points from the counsellee by paraphrasing or summarizing them. a helpful question elicits elaboration on a key point and invites collaborative problem solving. this paper focuses on automatic generation of restatements and questions for counselling dialogs. specifically, it addresses two research questions: our experiments compare a number of summarization, question generation and dialog models for the single-turn reply generation task. we obtained the strongest model by fine-tuning bertsum @xcite , a state-of-the-art summarization model, with an in-domain, manually annotated dataset augmented with a large-scale, automatically mined open-domain dataset. after summarizing previous work (section 2) and presenting our dataset (section 3), we describe our approach for restatement and question generation (section 4). we then report experimen- post restatement question (a) 每逢測驗都一定會夜晚唔食飯 你一定唔可以輸 你同邊個比賽呀？ 專心溫習 同自己講 我一定唔可以輸 before a test, i skip dinner to study and i you must not lose who are you competing say to myself, "i must not lose" with? (b) professor教書教得咁廢考試又出勁難 考試勁難 你考試係咪唔識做？ the professor teaches poorly and gives exam is extremely are there questions you a really hard exam hard can't answer in the exam? (c) 我估我到考試果陣會頭痛，我以前 你擔心考試時會頭痛 你有冇試過去搵醫生 都試過系咁 睇睇呢? i just knew i'll get a headache during you worry you'll get a have you tried to the exam, like i did before. headache during the exam consult a doctor? (d) 朋友真係咁易識咩...唔想要損友... 你覺得唔容易識朋友 係咪覺得損友好冇益？ making friends is not so easy ... [i] don't you think it's not easy you think bad friends want bad friends ... to make friends are bad for you? ## previous work while chatbot response generation has exploited models from machine translation @xcite and question answering @xcite , there has been less effort in leveraging those from other nlp tasks such as text summarization and question generation. this section reviews research in these two fields. ## data our data consists of post-reply pairs, a term that will be used henceforth to refer to both postrestatement and post-question pairs. this section describes the construction process of two datasets, which contain in-domain, manually crafted (section 3.1) and open-domain, automatically mined (section 3.2) post-reply pairs, respectively. ## approach we first construct and evaluate models for restatement generation and for question generation separately (section 4.1). we then combine these models to interleave restatements and questions in a counselling dialog (section 4.2). ## conclusion restatements and questions are common conversation strategies in counselling. this paper has investigated automatic generation of these two reply types by exploiting models of two closely related nlp tasks -summarization and question generation. we obtained the best generation performance for both reply types by fine-tuning bertsum, a stateof-the-art summarization model, with an in-domain, manually annotated dataset augmented with a largescale, automatically mined open-domain dataset. we then showed that restatements and questions can be interleaved with a confidence score threshold. to the best of our knowledge, this is the first reported application of summarization models on chatbot response generation in the counselling domain. it is hoped that our proposed techniques can improve the quality of a counsellor chatbot for the public. further research is needed to take into account the progress of the counselling session when selecting a reply @xcite , and to measure correlation with counselling outcomes.