Disambiguate First, Parse Later: Generating Interpretations for Ambiguity Resolution in Semantic Parsing (2025)

Abstract: Handling ambiguity and underspecification is an important challenge in natural language interfaces, particularly for tasks like text-to-SQL semantic parsing. We propose a modular approach that resolves ambiguity using natural language interpretations before mapping these to logical forms (e.g., SQL queries). Although LLMs excel at parsing unambiguous utterances, they show strong biases for ambiguous ones, typically predicting only preferred interpretations. We constructively exploit this bias to generate an initial set of preferred disambiguations and then apply a specialized infilling model to identify and generate missing interpretations. To train the infilling model, we introduce an annotation method that uses SQL execution to validate different meanings. Our approach improves interpretation coverage and generalizes across datasets with different annotation styles, database structures, and ambiguity types.

## introduction natural language utterances are often ambiguous, vague, or underspecified, giving rise to multiple valid interpretations. figure 1 shows an ambiguous request ("return the rating of each hotel") in the context of natural language interfaces. in the example, "rating" could refer to the number of stars hotels receive as an indication of their quality (e.g., 4 stars) or guest reviews on booking websites (e.g., 8.5 out of 10), or both. ignoring such ambiguity can lead to incomplete or incorrect results, undermining user trust and limiting the practical usefulness of any conversational system. while state-of-the-art large language models (llms) demonstrate remarkable performance on tasks like question answering, text-to-sql parsing, and natural language inference, recent studies @xcite ) have shown they 1 our code and data can be downloaded from github.com/saparina/disambiguate-then-parse. are not adept at handling ambiguity. they display systematic biases in their choice of interpretation @xcite @xcite , typically defaulting to a single interpretation when multiple ones exist. what are the response strategies llms should adopt to address ambiguity? an approach might be to respond with a clarification question, which directly engages users and ensures accurate resolution of the ambiguity but introduces additional interaction turns. alternatively, presenting multiple possible interpretations (an "overton response") would allow users to select the most relevant answer themselves. this approach minimizes interruptions, caters to users with different levels of expertise, and provides interpretability by making the system's reasoning explicit. for tasks like semantic parsing (see figure 1 ), a hypothetical response should not only include interpretations in their final form (e.g., sql queries) but also their different readings in natural language. from a modeling perspective, interpretations serve as explanations (of the system's output) and as intermediate repre-sentations, providing a way to decompose complex semantic parsing problems into simpler steps. in this work, we focus on text-to-sql parsing of ambiguous questions, and propose a two-stage approach that first disambiguates by generating all possible meanings in natural language, and then parses each unambiguous interpretation. separating the disambiguation and parsing tasks allows us to use existing semantic parsers that perform generally well on unambiguous inputs. we obtain interpretations, by prompting an llm to generate all possible meanings for an utterance. these initial interpretations are often incomplete due to inherent biases stemming from statistical patterns found in llm training data, lack of intrinsic knowledge, and different alignment preferences. rather than attempting to correct these biases, we introduce an infilling model that reviews the ambiguous question and initial interpretations, and then generates any missing ones. the infilling model is trained on pairs of default interpretations and missing readings. problematically, existing datasets provide (multiple) sql parses for ambiguous questions without explicitly verbalising the interpretations they correspond to. we create synthetic reference interpretations for ambiqt @xcite , a recently proposed benchmark for parsing ambiguous questions into sql. we exploit the fact that generated interpretations can be converted to sql queries which we execute to verify whether they are correct and to establish which interpretations are missing, i.e., whether there exist gold sql parses for which no interpretation was found. we evaluate our approach on ambrosia @xcite , a dataset different from ambiqt in terms of database content and the types of ambiguity it represents. our contributions are summarized as follows: ## experimental results it is better to disambiguate first and parse later both in in-domain and out-of-domain settings. as can be seen in interpretation generation with infilling further improves single and full interpretation coverage ambiqt ambrosia interpretations single full single full on both datasets. while it does not achieve the highest full coverage on ambiqa, it substantially improves over end-to-end prompting reaching 53%, while delivering the best single interpretation coverage of 92%. it effectively generalises to new domains and ambiguity types, showing the best results on ambrosia among all methods across three out of four evaluation metrics. however, the full coverage on ambrosia is still relatively low (at 19%). ambrosia is a challenging benchmark on its own but it is also possible that some interpretations are missed due to annotation differences between the two datasets. the results in ## conclusion in this work, we present a novel approach for handling ambiguity in text-to-sql semantic parsing. we first disambiguate questions by explicitly verbalising their interpretations and then leverage llm capabilities to predict all valid sql queries. a generator (llm) provides an initial set of default interpretations, which are then augmented with a specialized infilling model. we propose a method for training this model based on automatic annotations obtained by comparing sql query execution results rather than natural language interpretations. our experimental results on ambiqt and ambrosia demonstrate the effectiveness of our approach. our method achieves the highest single interpretation coverage on both datasets and maintains consistent full interpretation coverage in both in-domain and out-of-domain evaluation. however, generating all valid interpretations remains challenging. future work could explore ways to improve ambiguity handling through better processing of database structure or using query execution as a signal for both training and test-time search.