Measuring and Mitigating Constraint Violations of In-Context Learning for Utterance-to- API Semantic Parsing (2023)

Abstract: In executable task-oriented semantic parsing, the system aims to translate users’ utterances in natural language to machine-interpretable programs (API calls) that can be executed according to pre-defined API specifications. With the popularity of Large Language Models (LLMs), in-context learning offers a strong baseline for such scenarios, especially in data-limited regimes. However, LLMs are known to hallucinate and therefore pose a formidable challenge in constraining generated content. Thus, it remains uncertain if LLMs can effectively perform task-oriented utterance-to-API generation, where respecting the API’s structural and task-specific constraints is crucial. In this work, we seek to measure, analyze and mitigate such constraints violations. First, we identify the categories of various constraints in obtaining API-semantics from task-oriented utterances, and define fine-grained metrics that complement traditional ones. Second, we leverage these metrics to conduct a detailed error analysis of constraints violations seen in state-of-the-art LLMs, which motivates us to investigate two popular mitigation strategies– Semantic-Retrieval of Demonstrations (SRD) and API-aware Constrained Decoding (API-CD). Our experiments show that these strategies are effective at reducing constraints violations and improving the quality of the generated API calls, but require careful consideration given their implementation complexity and latency.

## introduction in task-oriented dialog (tod) settings, executable semantic parsers maps natural language utterances from human users to machine-interpretable meaning representations that help achieve users' goals. in order to effectively carry out the user's command, such task-oriented semantic parsers need utterance show me one way options to auditorium by 10 am via 1st ave semantic parse [in:get_directions show me one way options to [sl:destination [in:get_location [sl:category_location auditorium] ] ] [sl:date_time_arrival by 10 am ] via [sl:path 1st ave ] ] tree representation in:get_directions sl:destination sl:date_time_arrival sl:path in:get_location sl:category_location auditorium 1st ave by 10 am api call get_directions ( @xmath0 to be able to interact with the real-world execution environment by translating users' utterances to api calls, that are executable according to predefined api documentation. for example, in figure 1 , given the user's utterance show me one way options to auditorium by 10am via 1st ave, the executable task-oriented semantic parser should not only understand the semantic parse structures from the user's input, but more importantly, be able to generate the api call: where the parsed intents and slots represent functions and arguments in the api calls. existing work has tackled task-oriented seman-tic parsing by using auto-regressive language models @xcite @xcite . recent advances of pre-trained large language models (llms), especially those trained with code, present an exciting opportunity to frame executable semantic parsing tasks as an api call generation problem. in @xcite , @xcite and @xcite , llms prompted with examples are able to generate code-like representations based on user queries. in-context learning (icl) @xcite ) is a particularly appealing approach for semantic parsing tasks as it allows developers to (1) make use of llm's transfer learning abilities in low-resource settings with only a small collection of labelled training data, and (2) easily prototype llm-based systems without needing re-training or fine-tuning. however, it has been shown that outputs from llms are prone to hallucination, lack controllability and have high sensitivity to prompt design @xcite @xcite , even when these models are prompted with human-written instructions and selected exemplars @xcite . these weaknesses are concerning for executable semantic parsing, as generated output needs to respect the task constraints specified by the api documents. in this work, we investigate the question, can llms with in-context learning respect the taskspecific constraints required for task-oriented semantic parsing? to the best of our knowledge, there is no existing work that categorizes and defines fine-grained task-specific constraints for taskoriented semantic parsing or systematically analyzes constraints violations in llm outputs in the icl scenario. we begin by identifying and categorizing the task constraints in section 2, focusing on the top-v2 dataset @xcite , an existing task-oriented semantic parsing dataset, and detecting constraints violations in the output of gpt-neox @xcite , a state-of-the-art llm trained on code. we observe that task-specific constraint violation are ubiquitous when using icl in practice (ranging from 3 -14.8%), while syntactic constraints are lower (≈ 0.6%). in section 3, we show that the proposed constraint violation metrics serve as a useful diagnostic tool, providing a fine-grained view and complementing existing evaluation measures in @xcite . our analysis of errors based on these constraint violation metrics reveal important failure modes, such as the lack of examples pertaining to a particular intent in the icl input prompt, dependence on particular taskspecific keywords, scenarios when hallucination is likely, etc. and inspires us to develop mitigation strategies that reduces constraint violations. we investigate two popular mitigation strategiesretrieval-based augmentation @xcite and constrained decoding @xcite . in section 5, we show that our semantic-retrieval of demonstrations (srd), despite its implementation simplicity, improves the semantic parsing quality in the icl setting and can also reduce constraint violation to an extent. on the other hand, when task-specific apis are available, our api-aware constrained decoding (api-cd) strategy can completely eliminate constraint violations. an amalgamation of these two complementary approaches provides the best results, but incurs some tradeoffs in efficiency. we analyze this trade-off, offering advice for practitioners. finally, we plan to release the modified topv2 dataset to encourage further research in this direction. in summary, our key contributions are: ## defining constraints violations recent tod systems, such as @xcite and @xcite , take (single/multi-turn) utterances of users as input and generate executable programs (api calls) to achieve user goals. such systems are expected to not only generate effective programming codes but also respect the api constraints, relevant to the application domains, specified by developers. to study the task of generating valid api calls, we convert the popular tod benchmark dataset topv2 @xcite to an utterance-to-api format. foot_0 specifically, topv2 consists of pairs of user-utterance and semantic parse representing the users' goals; we process the dataset by converting the semantic parses to api calls (see figure 1 ). note that the topv2 dataset contains many compositional/nested queries, where a particular slot value is the result of another intent. this is reflected under the dialog-to-api framework as nested calls, where the argument value to some function can be the return value from another function. hence, an api call can be represented as a flattened list of functional calls, which we now define as a flattened list representation (examples for flattened list representations are in appendix a.1). definition 2.1. a flattened list representation of an api call y is a list of tuples {(f i , a i )} @xmath1,...n , where each f i is a function (corresponding to the intent) present in the api call and a i is a set of (a, v) pairs (corresponding to the slots and slot values). each function f i is defined to be associated with all arguments a ′ s from a i . for compositional/nested api calls, the values v can be other functions (f ), while for noncompositional apis they are resolvable/grounded values. it follows that in the latter case, the flattened list has only one function call. ## analyzing constraint violation as we noted above, the use of exact match and f1scores @xcite can only provide a bird's eye view of the model's failures. unfortunately, recently employed metrics like bleu and execution match ratio (emr) for such scenarios @xcite are also incapable of providing a magnified view. further, these metrics can reasons for violation constraint violation examples constraint violation analysis misunderstanding task semantics violates c f , ca, c f a [prediction]: get_info_wrecks ( @xmath2 the system simply uses the template get_info_ * that is ubiquitous in the training data (and therefore in the prompt). although it has seen get_info_traffic in the prompts, it fails to make the semantic implicature from wrecks to traffic during generation, thereby resulting in an undefined function call get_info_wrecks and violating all three task-specific constraints. misunderstanding slots/arguments associated with a task violates c f a [utterance]: 10 day forecast please. the system fails to realize that date_time can support 10 day as an argument value. hence, it hallucinates a new argument forecast_days to provide this information. note that its argument definition for date_time is correct, and thus it only violates c f a . lack of relevant examples in the prompt violates c f , c f a [prediction]: show_alarms ( @xmath3 the get_alarm has a small support in the training data. hence, sampling omits it in the example prompt. the model, oblivious to this function call, violates c f by generating an undefined function show_alarms. inability to handle compositionality violates ca, c f a [utterance]: add reminder feed sparky 6pm daily [ground-truth]: create_reminder ( @xmath4, @xmath5, @xmath6 in ## mitigation strategies we study two popular mitigation strategies to reduce constraint violations-(1) a semantic retrieval approach for in-context example selection that improves the prompt design, and (2) an api-aware constrained decoding strategy that constrains token generation based on api information. ## experiments using the constraints defined in the previous section, we now benchmark the performance of incontext learning with large language models on constraint satisfaction. in addition, we use the proposed mitigation strategies, namely retrieval-based and constrained-decoding-based, and show their effectiveness in reducing the constraint violation rates compared to the baselines. finally, we discuss important practical considerations that accompany the proposed mitigation strategies. ## related work generating semantic representation from natural language is a beast with many heads @xcite @xcite . in our work, we look at a subclass of this problem where the input is a task-oriented utterance and the output takes the form of api call representations. our input format aligns us closely to work in task-oriented semantic parsing @xcite , task-oriented dialog @xcite , and intent classification and slot filling @xcite , while our output representation invites challenges similar to ones seen in generating executable semantic parses @xcite , database queries @xcite @xcite @xcite , and aws api calls @xcite . our problem borrows the api-styled output, similar to @xcite , but does not rely on an entire conversation or dialog as the input medium. to this extent, we enhance the existing topv2 dataset @xcite , leveraged in task-oriented settings, with api-styled output representations. with the recent popularity of large language models, adapting them for various tasks with the abilities to use different tools is becoming ubiquitous. @xcite fine-tunes language models to use tools with external apis to execute complex instructions. @xcite demonstrates that language models can learn to use external tools via simple apis in a self-supervised way by in-context learning. similar to our setup, the incontext learning approach @xcite has been considered for intent and slot filing @xcite , semantic parsing @xcite , by predicting english-like canonical representations first @xcite , meaning representations directly @xcite , and dialog-to-api output @xcite . none of these works consider, like us, fine-grained constraint violation metrics that can help to decipher the failure modes of icl in their settings. hence, benefits of in-context example retrieval do not show the shortcomings of these approaches in mitigating constraint violations @xcite @xcite . while lexically constrained decoding has been beneficial in a wide variety of setting such as image captioning, machine translation, semantic parsing, and paraphrase generation @xcite @xcite @xcite , dynamic adaptation of the generation lexicon by leveraging api definitions has not been investigated before. ## conclusion in this paper, we consider the problem of generating an api-call-styled representation given an input task-oriented utterance using in-context learning (icl) approaches with large language models. we propose a set of fine-grained constraint violation metrics that align with the api specifications and show that it lets us diagnose fine-grained failure modes. our analysis leads us to investigate two simple mitigation strategies than can be used alongside the icl approach. a query-based semantic retrieval of demonstration in icl prompts helps obtain gains on traditional metrics while api-aware constraint decoding helps eliminate the proposed constraint violations. coupling the two approaches with icl yields the best of both worlds. future works may focus on speeding up the mitigation approaches, and finding solutions that couple the approaches in a tighter fashion.