Value-Agnostic Conversational Semantic Parsing (2021)

Abstract: Conversational semantic parsers map user utterances to executable programs given dialogue histories composed of previous utterances, programs, and system responses. Existing parsers typically condition on rich representations of history that include the complete set of values and computations previously discussed. We propose a model that abstracts over values to focus prediction on type- and function-level context. This approach provides a compact encoding of dialogue histories and predicted programs, improving generalization and computational efficiency. Our model incorporates several other components, including an atomic span copy operation and structural enforcement of well-formedness constraints on predicted programs, that are particularly advantageous in the low-data regime. Trained on the SMCalFlow and TreeDST datasets, our model outperforms prior work by 7.3% and 10.6% respectively in terms of absolute accuracy. Trained on only a thousand examples from each dataset, it outperforms strong baselines by 12.4% and 6.4%. These results indicate that simple representations are key to effective generalization in conversational semantic parsing.

## introduction conversational semantic parsers, which translate natural language utterances into executable programs while incorporating conversational context, play an increasingly central role in systems for interactive data analysis @xcite , instruction following @xcite , and task-oriented dialogue @xcite ). an example of this task is shown in figure 1 . typical models are based on an autoregressive sequence prediction approach, in which a detailed representation of the dialogue history is concatenated to the input sequence, and predictors condition on this sequence and all previously generated components of the output @xcite . while this approach can capture arbitrary dependencies between inputs and outputs, it comes at the cost of sample-and computational inefficiency. we propose a new "value-agnostic" approach to contextual semantic parsing driven by type-based representations of the dialogue history and functionbased representations of the generated programs. types and functions have long served as a foundation for formal reasoning about programs, but their use in neural semantic parsing has been limited, e.g., to constraining the hypothesis space @xcite , guiding data augmentation @xcite , and coarsening in coarse-to-fine models @xcite . we show that representing conversation histories and partial programs via the types and functions they contain enables fast, accurate, and sample-efficient contextual semantic parsing. we propose a neural encoderdecoder contextual semantic parsing model which, in contrast to prior work: our model substantially improves upon the best published results on the smcalflow @xcite and treedst @xcite conversational semantic parsing datasets, improving model performance by 7.3% and 10.6%, respectively, in terms of absolute accuracy. in further experiments aimed at quantifying sample efficiency, ## proposed model our goal is to map natural language utterances to programs while incorporating context from dialogue histories (i.e., past utterances and their asso-ciated programs and execution results). we model a program as a sequenceo of function invocations, each consisting of a function and zero or more argument values, as illustrated at the lower right of figure 1 . the argument values can be either literal values or references to results of previous function invocations. the ability to reference previous elements of the sequence, sometimes called a target-side copy, allows us to construct programs that involve re-entrancies. owing to this referential structure, a program can be equivalently represented as a directed acyclic graph (see e.g., @xcite . we propose a transformer-based @xcite encoder-decoder model that predicts programs by generating function invocations sequentially, where each invocation can draw its arguments from an inventory of values ( §2.5)-possibly copied from the utterance-and the results of previous function invocations in the current program. the encoder ( §2.2) transforms a natural language utterance and a dialogue history to a continuous representation. subsequently, the decoder ( §2.3) uses this representation to define an autoregressive distribution over function invocation sequences and chooses a high-probability sequence by performing beam search. as our experiments ( §3) will show, a naïve encoding of the complete dialogue history and program results in poor model accuracy. current program with revision revise( @xmath0, @xmath1, @xmath2, @xmath3, @xmath4, @xmath5, @xmath6, @xmath7, @xmath8, @xmath9, @xmath10, @xmath11." contains information that is not mentioned in the current utterance only contains information that is mentioned in the current utterance ## experiments we first report results on smcalflow @xcite and treedst @xcite , two recently released large-scale conversational semantic parsing datasets. our model makes use of type information in the programs, so we manually constructed a set of type declarations for each dataset and then used a variant of the hindley-milner type inference algorithm @xcite to annotate programs with types. as mentioned in §2.1, we also transformed treedst to introduce meta-computation operators for references and revisions (more details can be found in appendix c). 7 we also report results on nonconversational semantic parsing datasets in §3.2. we use the same hyperparameters across all experiments (see appendix e), and we use bertmedium @xcite to initialize our encoder. ## related work our approach builds on top of a significant amount of prior work in neural semantic parsing and also context-dependent semantic parsing. ## conclusion we showed that abstracting away values while encoding the dialogue history and decoding programs significantly improves conversational semantic parsing accuracy. in summary, our goal in this work is to think about types in a new way. similar to previous neural and non-neural methods, types are an important source of constraints on the behavior of the decoder. here, for the first time, they are also the primary ingredient in the representation of both the parser actions and the dialogue history. our approach, which is based on type-centric encodings of dialogue states and function-centric encodings of programs ( §2), outperforms prior work by 7.3% and 10.6%, on smcalflow and treedst, respectively ( §3), while also being more computationally efficient than competing methods. perhaps more importantly, it results in even more significant gains in the low-data regime. this indicates that choosing our representations carefully and making appropriate independence assumptions can result in increased accuracy and computational efficiency.