DIRECT : Direct and Indirect Responses in Conversational Text Corpus (2021)

Abstract: We create a large-scale dialogue corpus that provides pragmatic paraphrases to advance technology for understanding the underlying intentions of users. While neural conversation models acquire the ability to generate fluent responses through training on a dialogue corpus, previous corpora have mainly focused on the literal meanings of utterances. However, in reality, people do not always present their intentions directly. For example, if a person said to the operator of a reservation service “I don’t have enough budget.”, they, in fact, mean “please find a cheaper option for me.” Our corpus provides a total of 71,498 indirect–direct utterance pairs accompanied by a multi-turn dialogue history extracted from the MultiWoZ dataset. In addition, we propose three tasks to benchmark the ability of models to recognize and generate indirect and direct utterances. We also investigated the performance of state-of-the-art pre-trained models as baselines.

## introduction we create a large-scale dialogue corpus that discloses users' hidden intentions to advance techniques for natural language understanding in dialogue systems. neural conversation models have been able to generate high-quality responses @xcite and achieve dialogue state tracking @xcite . these previous studies have been based on the literal meanings of user utterances. little attention has been given to the implied intention of the utterances considered. however, during conversation, humans often respond to others with indirect expressions, without directly telling them their requests or intentions @xcite . when humans receive an indirect response, they infer the intention implied in the response based on context, such as dialogue history. for example, in the example would you like to make a reservation for this restaurant? i don't have enough budget... ## related work paraphrases have been applied in a dialogue system's research in the context of data augmentation @xcite . despite its importance in understanding users' intentions, the pragmatic paraphrases have been overlooked. only a few recent studies have focused on pragmatic paraphrases to advance the understanding of users' intentions. pragst and ultes (2018) proposed a rule-based approach to automatically construct a corpus consisting of pairs of indirect and direct utterances. they demonstrated that the neural conversation model could accurately extract utterances with opposing directness. because of their rulebased approach, patterns of indirect/direct utterances in their corpus are limited. @xcite used crowdsourcing to build a corpus comprising indirect answers to yes/no questions, annotating whether the answers were yes or no. this corpus provides natural answers written by crowdsourcing workers; however, it is limited to context-free yes/no questions. in contrast to these studies, di-rect provides natural utterances written by humans with rich dialogue histories. furthermore, it covers various types of utterances. while there are several paraphrase corpora @xcite , all have focused on context-free paraphrases. hence, none provide pragmatic paraphrases that emerge in contexts. corpora for natural language inference are also relevant to our study @xcite @xcite . similar to the paraphrase corpora, they do not provide contexts. this means that these corpora rely on world knowledge to deter-mine whether a text entails a hypothesis. in contrast, context is a crucial element in determining paraphrasal relationships in pragmatic paraphrases. our direct is the first corpus that provides largescale pragmatic paraphrases. it would be a valuable resource also for research on paraphrase identification and generation to make a step forward from literal paraphrases. ## direct corpus a pragmatic paraphrase is a pair of texts that have equivalent outcomes in a given context, which frequently emerge in conversations. expanding a dialogue corpus is a promising approach for building a corpus that collects such pragmatic paraphrases as such a corpus is conversational by nature and often provides conversation histories. specifically, we employed multiwoz2.1 @xcite and collected pragmatic paraphrases using crowdsourcing. we describe how we collected pragmatic paraphrases in section 3.1 with careful quality control as described in section 3.2. section 3.3 describes the statistics of the collected corpus. section 3.4 presents a comparative analysis between our corpus and existing paraphrase corpora using conventional paraphrase identification models. ## conclusion we created direct, a dialogue corpus providing 71, 498 pairs of pragmatic paraphrases with context. in addition, we proposed three benchmark tasks and showed the performance of state-of-theart pre-trained models as the baseline. in a future, we will apply direct to a taskoriented dialogue system to handle indirect responses in an end-to-end manner. we also intend to investigate the relations between pragmatic paraphrases and other features of dialogue acts and belief states.