Memory-Based Semantic Parsing (2021)

Abstract: We present a memory-based model for context- dependent semantic parsing. Previous approaches focus on enabling the decoder to copy or modify the parse from the previous utterance, assuming there is a dependency between the current and previous parses. In this work, we propose to represent contextual information using an external memory. We learn a context memory controller that manages the memory by maintaining the cumulative meaning of sequential user utterances. We evaluate our approach on three semantic parsing benchmarks. Experimental results show that our model can better process context-dependent information and demonstrates improved performance without using task-specific decoders.

## introduction semantic parsing is the task of converting natural language utterances into machine interpretable meaning representations such as executable queries or logical forms. it has emerged as an important component in many natural language interfaces @xcite with applications in robotics @xcite , question answering @xcite , dialogue systems @xcite , and the internet of things @xcite . neural network based approaches have led to significant improvements in semantic parsing @xcite @xcite @xcite across domains and semantic formalisms. the majority of existing studies focus on parsing utterances in isolation, and as a result they cannot readily transfer in more realistic settings where users ask multiple inter-related questions to satisfy an information need. in this work, we study context-dependent semantic parsing focusing specifically on text-to-sql generation, which has emerged as a popular application area in recent years. figure 1 shows a sequence of utterances in an interaction. the discourse focuses on a specific topic serving a specific information need, namely, finding out which continental flights leave from chicago on a given date and time. importantly, interpreting each of these utterances, and mapping them to a database query to retrieve an answer, needs to be situated in a particular context as the exchange proceeds. the topic further evolves as the discourse transitions from one utterance to the next and constraints (e.g., time or place) are added or revised. for example, in q2 the time constraint before 10am from q1 is revised to before noon, and in q3 to before 2pm. aside from such topic extensions @xcite , the interpretation of q2 and q3 depends on q1, as it is implied that the questions concern continental flights that go from chicago to seattle, not just any continental flights, however the phrase from chicago to seattle is elided from q2 and q3. the interpretation of q4 depends on q3, which in turn depends on q1. interestingly, q5 introduces information with no dependencies on previous discourse and, in this case, relying on information from previous utterances will lead to incorrect sql queries. the problem of contextual language processing has been most widely studied within dialogue systems where the primary goal is to incrementally fill pre-defined slot-templates, which can be then used to generate appropriate natural language responses @xcite . but the rich semantics of sql queries makes the task of contextual text-to-sql parsing substantially different. previous approaches @xcite tackle this problem by enabling the decoder to copy or modify the previous queries under the assumption that they contain all necessary context for generating the current sql query. the figure 1 : example utterances from a user interaction in the atis dataset. utterance segments referring to the same entity or objects are in same color. sql queries corresponding to q2-q5 follow a pattern similar to q1 and are not shown for the sake of brevity. our own work draws inspiration from @xcite text comprehension model. in their system the process of comprehension involves three levels of operations. firstly, smaller units of meaning (i.e., propositions) are extracted and organized into a coherent whole (microstructure); some of these are stored in a working memory buffer and allow to decide whether new input overlaps with already processed propositions. secondly, the gist of the whole is condensed (macrostructure). and thirdly, the previous two operations generate new texts in working with the memory. in other words, the (short and long term) memory of the reader gives meaning to the text read. they propose three macro rules, namely, deletion, generalization, and construction, as essential to reduce and organize the detailed information of the microstructure of the text. furthermore, previous knowledge and experience are central to the interpretation of text enabling the reader to fill information gaps. our work borrows several key insights from @xcite without being a direct implementation of their model. specifically, we also break down input utterances into smaller units, namely, phrases, and argue that this infor-mation can be effectively utilized in maintaining contextual information in an interaction. furthermore, the notion of a memory buffer that can be used to store and process new and old information plays a prominent role in our approach. we propose a memory-based context model (which we call memce for short) for keeping track of contextual information, and learn a context memory controller that manages the memory. each interaction (sequence of user utterances) maintains its context using a memory matrix. user utterances are segmented into a sequence of phrases representing either new information to be added into the memory (e.g., that have a meal in figure 1 ) or old information which might conflict with current information in memory and needs to be updated (e.g., before 10 am should be replaced with before noon in figure 1 ). our model can inherently add new content to memory, read existing content by accessing the memory, and update old information. we evaluate our approach on the atis @xcite , sparc @xcite , and cosql @xcite datasets. we observe performance improvements when we combine memce with existing models underlying the importance of more specialized mechanisms for processing context information. in addition, our model brings interpretability in how the context is processed. we are able to inspect the learned memory controller and analyze whether important discourse phenomena such as coreference and ellipsis are modeled. in this paper we are concerned with the lesser studied problem of contextualized semantic parsing where previous utterances are taken into account in the interpretation of the current utterance. earlier work @xcite @xcite has focused on symbolic features for representing contextfor example, by explicitly modeling discourse referents, or the flow of discourse. more recent neural methods extend the sequence-to-sequence architecture to incorporate contextual information either by modifying the encoder or the decoder. context-aware encoders resort to concatenating the current utterance with the utterances preceding it @xcite or focus on the history of the utterances most relevant to the current decoder state @xcite . the decoders take context representations as additional input and often copy segments from the previous query @xcite . hybrid approaches @xcite @xcite employ neural networks for representation learning but use a grammar for decoding (e.g., a sequence of actions or an intermediate representation). a tremendous amount of work has taken place in the context of discourse modeling focusing on extended texts @xcite and dialogue @xcite . @xcite study the mental operations underlying the comprehension and summarization of text. they introduce propositions as the basic unit of text representation, and a model of how incoming text is processed given memory limitations; texts are reduced to important propositions (to be recalled later) using macro-operators (e.g., addition, deletion). their model has met with popularity in cognitive psychology @xcite and has also found application in summarization @xcite . our work proposes a new encoder for contextualized semantic parsing. at the heart of our approach is a memory controller that keeps track of context via writing new information and updating old information. our memory-based approach is inspired by @xcite and is closest to @xcite , who use a memory augmented neural network @xcite for meta-learning. specifically, they introduce a method for accessing external memory which functions as short-term storage for meta-learning. although we report experiments solely on semantic parsing, our encoder is fairly general and could be applied to other context-dependent tasks such as conversational information seeking @xcite ) and information retrieval @xcite . ## model our model is based on the encoder-decoder architecture @xcite with the addition of a memory component @xcite for incorporating context. as shown in figure 2 , our model consists of four components: (1) a memory matrix retains discourse information, (2) a memory controller, which learns to access and manipulate the memory such that correct discourse information is retained, (3) utterance and phrase encoders, and (4) a decoder that interacts with the memory and utterance encoder using an attention mechanism to generate sql output. ## experimental setup we evaluated memce, our memory-based context model, on various settings by integrating it with multiple open-source models. we achieve this by replacing the discourse component of related models with memce subject to minor or no additional changes. all base models in our experiments use a turn-level hierarchical encoder 1 in experiments we found that using the (raw) memory directly is empirically better to encoding it with an lstm. to capture previous language context. for primary evaluation, we use the atis @xcite dataset but also present results on sparc @xcite and cosql @xcite . ## results in this section, we assess the effectiveness of the memce encoder at handling contextual information. we present our results, evaluation methodology, and comparisons against the state of the art. ## analysis in this section, we analyze our model's ability to handle important discourse phenomena such as focus shift, referring expressions, and ellipsis. we also showcase its interpretability by examining the behavior of the (learned) memory controller. ## conclusions in this paper, we presented a memory-based model for context-dependent semantic parsing and evaluated its performance on a text-to-sql task. analysis of model output revealed that our approach is able to handle several discourse related phenomena to a large extent. we also analyzed the behavior of the memory controller and observed that it correlates with the model's output decisions. our study indicates that explicitly modeling context can be helpful for contextual language processing tasks. our model manipulates information at the phrase level which can be too rigid for fine-grained updates. in the future, we would like to experiment with learning the right level of utterance segmentation for context modeling as well as learning when to reinstate a constraint.