Machines Getting with the Program: Understanding Intent Arguments of Non-Canonical Directives (2020)

Abstract: Modern dialog managers face the challenge of having to fulfill human-level conversational skills as part of common user expectations, including but not limited to discourse with no clear objective. Along with these requirements, agents are expected to extrapolate intent from the user’s dialogue even when subjected to non-canonical forms of speech. This depends on the agent’s comprehension of paraphrased forms of such utterances. Especially in low-resource languages, the lack of data is a bottleneck that prevents advancements of the comprehension performance for these types of agents. In this regard, here we demonstrate the necessity of extracting the intent argument of non-canonical directives in a natural language format, which may yield more accurate parsing, and suggest guidelines for building a parallel corpus for this purpose. Following the guidelines, we construct a Korean corpus of 50K instances of question/command-intent pairs, including the labels for classification of the utterance type. We also propose a method for mitigating class imbalance, demonstrating the potential applications of the corpus generation method and its multilingual extensibility.

## introduction the advent of smart agents such as amazon echo and google home has shown relatively wide market adoption. users have been familiarized with formulating questions and orders in a way that these agents can easily comprehend and take actions. given this trend, particularly for cases where questions have various forms such as yes/no, alternative, wh-, echo and embedded @xcite , a number of analysis techniques have been studied in the domain of semantic role labeling @xcite and entity recognition (molla * both authors contributed equally to this manuscript. unlike in the korean example that is to be investigated, in english translation, the wh-related noun (here, destination) is placed at the head part of the output. @xcite . nowadays, various question answering tasks have been proposed @xcite and have yielded systems that demonstrate significant advances in performance. studies on the parsing of canonical imperatives @xcite have also been done for many intelligent agents. however, discerning the intention from a conversational and non-canonical directive (question or command) and correctly extracting its intent argument is still a challenge. it usually matters when the user is not familiar with the canonical commands, namely where the direct speech act meets the genuine intention. that is, sometimes, the speech act can be hard to guess merely from the sentence form, as in inferring (1), (1) why don't you just call the police as a representation of the to-do list 'to call the police'. although advanced dialog managing systems may generate a plausible reaction regarding the input utterance, it is different from extracting the exact intent argument (a question set or a to-do-list) that should be investigated for actual operation. additional complexity is introduced when the target text is in a speech recognition context, as the input text may not contain punctuation. for example, given an unclear declarative question @xcite such as (2), (2) you know where we should go today a human listener can interpret the subject of inquiry as 'the destination of today', while this can be challenging for a machine. the basis of our work is that if a system is trained to extract a structured natural language (nl) query from directive sentences, it may help the language understanding systems be more robust at understanding non-canonical expressions in executing the command. some may argue that the structured information retrieval we aim to support may benefit from the data augmentation technologies that are concurrent with the studies on paraphrasing @xcite . however, complexities as in the examples above have not seen much exploration outside of english, especially in the context of languages with a distinguished syntax or cases which do not use latin-like alphabets. also, it is not guaranteed that such technologies fit with less explored languages, where sufficient pre-training resources may not be readily available. as a more concrete example, in the korean language, the morphology is agglutinative, the syntax is head-final, and scrambling (non-deterministic permutations of word/phrase ordering) is a common practice between native speakers. primarily, the agglutinative property of korean requires additional morphological analysis, which makes it challenging to identify the component of the sentence that has the most substantial connection to core intent. moreover, the head-finality characteristic introduces an additional layer of complexity, where an under-specified sentence ender incorporates a prosodic cue which requires disambiguation to comprehend the original intent @xcite . finally, considering the scrambling aspect, which frequently happens in spoken utterances, further analysis is required on top of recognizing the entities and extracting the relevant phrases. these make it difficult for dialog managers to directly apply conventional analysis methods that have been used in germanic or other indo-european languages. in this paper, based on such aspects of the conversation-style utterances of korean, we propose a structured nl query foot_0 extraction scheme, which can help enrich the human-like conversation with artificial intelligence (ai). for automation, we construct a corpus of sentence-phrase pairs via annotation and then augment the dataset to mitigate class imbalance, demonstrating the flexibility, practicality, and extensibility of the proposed methods. to further prove that the scheme is not limited to a specific language, we demonstrate the methodology using english examples and supplement specific cases with korean. we describe the followings as our contribution to the field: ## concept and related work the theoretical background of this proposal builds on literature from speech act @xcite and formal semantics @xcite . although many task-oriented systems identify the intents as a specific action that the agent should take @xcite , to make our approach generic in the aspect of the domain and sentence structures, we hypothesized that it would be beneficial for the natural language understanding (nlu) modules first to recognize the directiveness and represent the core content in a structured format. once an utterance is identified to be directive, conventional systems rely on slot-filling to extract the item and argument @xcite , where the number of the categories is generally restricted. instead, for non-task-oriented dialogues, we hypothesized that the arguments should be attained in nl format rather than structured data, by, e.g., rewriting the utterances into some nominalized or simplified terms which correspond to the source text. there have been studies on paraphrasing of questions concerning the core content @xcite , but little has been done on nl formalization. also, our study targets the extraction of commands, which is equivalently essential but has not been widely explored outside of the robotics domain @xcite . the closest problem to this task is probably semantic parsing @xcite and structured query language (sql) generation, @xcite which propose seq2seq @xcite -like architectures to transform nl input into a structured format. these approaches provide the core content of the directive utterances as a sequence of queries, both utilizing it in paraphrasing @xcite or code generation @xcite . however, the proposed source sentence formats are usually canonical and mostly information-seeking, rather than being in a colloquial context. our motivation builds on the basis that realworld utterances as input (e.g., smart home commands from the less tech-familiar audience), in particular for korean, can diverge from the expected input form, to the non-canonical utterances that require actual comprehension for classifying as a question or command. moreover, as we discuss in the latter part of our work, we intend the extracted nl queries to be reusable as building blocks for efficient paraphrasing, following the approach in @xcite . recently, in a related view, or stronger linguistic context emphasis, guidelines for identifying noncanonical natural language questions or commands have been suggested for korean @xcite . we build on top of this corpus for the initial dataset creation, and extend the dataset with additional human-generated sentences. ## proposed scheme in this section, we describe the corpus construction scheme along with the motivation of this work. as discussed in the first section, our goal is to propose a guideline for discerning the intent argument for conversational and non-canonical questions and commands. these appear a lot in everyday life, but unlike cases where the input is in a canonical form, algorithmically extracting the core intent is not straightforward. we suggest that a data-driven methodology should be introduced for this task, which can be done by creating a corpus annotated with the core content of the utterances. while our work in this paper is for korean, the example sentences and the proposed structured scheme are provided in english, for demonstrative purposes. ## experiments here, we validate the usefulness of the constructed dataset with multiple sequence-to-sequence (seq2seq) @xcite architectures. we would like to note that as we propose both a new dataset accompanied by a new task, there is no baseline or proven evaluation metric as of the time of writing. for these reasons, we used existing evaluation frameworks used by other generation tasks. ## conclusion the significance of this research is in proposing the construction and augmentation schemes for rewriting of less explored sentence units, and making it an open, permissive resource for the general public. the sentence set consists of directive utterances in the korean language, where the morpho-syntactic property often provide difficulties in information retrieval. additionally, we propose baselines for the constructed dataset using multiple seq2seq architectures, exhibiting that our methodology is practically meaningful in real-world applications. our next work is to extend this more typologically by showing that the annotation/generation scheme applies to other languages. while the scope of our work is limited to korean, we hope that the proposed annotation scheme and resources from our work can be reused as a common protocol for intent-argument extraction tasks in other languages.