Semantic Parsing of Ambiguous Input using Multi Synchronous Grammars

Semantic parsing (SP) is the problem of parsing a given natural language (NL) sentence into a meaning representation (MR) conducive to further processing by applications. One of the major challenges in SP stems from the fact that NL is rife with ambiguities. Previous works using statistical models along with formalisms such as combinatorial categorial grammars, synchronous context free grammars, and dependency-based compositional semantics have shown notable success in resolving these ambiguities [7, 9, 12, 11, 14]. However, in many cases, the input for NL applications is underspecified and ungrammatical. We illustrate the example of search queries in Table 1. From these queries (Column 1) and their MRs (Column 2), we can see that there are several kinds of ambiguity. For example the parser must make the distinction between Kobe as a city or a basketball player. This problem of word sense ambiguity occurs in standard semantic parsing tasks, but there are also more pernicious problems unique to the more ambiguous input posed by search queries. Focusing on the queries “Kobe hotels” and “Kobe flight” we can see that it is also necessary to estimate the latent relationship between words, such as “location” or “destination. ” However it should be noted that if we take the keyword query and re-express it as a more explicit paraphrase, we can reduce this ambiguity to the point where there is only one reasonable interpretation. For example, in the second line, if we add the preposition “to” the user is likely asking for flights that arriving in Kobe, and if we add “from” the user is asking for departures. In this paper, we focus on SP of ambiguous input and propose a new method for dealing with the problem of ambiguity. Specifically we describe a framework where an ambiguous input (Column 1) is simultaneously transformed into both its MR (Column 2) and a more explicit, less ambiguous paraphrase (Column 3). The advantage of this method is that it is then possible to verify that the paraphrase indeed expresses the intended meaning of the underspecified input. Specifically, this verification can be done either manually by the system user or automatically using a probabilistic model trained to judge the naturalness of the paraphrases. As a concrete approach, we build upon the formalism of synchronous context free grammars (SCFG). Unlike traditional SCFGs, which usually only generate one target string (in semantic parsing, an MR), we introduce a new variety of SCFGs called “tri-synchronous” grammars, which generate multiple strings on the target side. This allows us to not only generate the MR, but also jointly generate the more explicit paraphrase. We then use a language model (LM) over the paraphrases generated by each derivation to help determine which derivations, and consequently which MRs, are more likely. Evaluation was performed using the Geoquery benchmark of 880 query-logic pairs representing questions about US geography [13]. The baseline SCFG parser achieves reasonable accuracy on regular questions but when the same method is used with underspecified input, the system accuracy decreases significantly. On the other hand, when incorporating the proposed tri-synchronous grammar to generate paraphrases and verify them with an LM, we find that it is possible to recover the loss of accuracy, resulting in a model that is able to parse the ambiguous input with significantly better accuracy. 1