Is the Pope Catholic? Yes, the Pope is Catholic. Generative Evaluation of Non-Literal Intent Resolution in LLM s (2024)

Abstract: Humans often express their communicative intents indirectly or non-literally, which requires their interlocutors—human or AI—to understand beyond the literal meaning of words. While most existing work has focused on discriminative evaluations, we present a new approach to generatively evaluate large language models’ (LLMs’) intention understanding by examining their responses to non-literal utterances. Ideally, an LLM should respond in line with the true intention of a non-literal utterance, not its literal interpretation. Our findings show that LLMs struggle to generate contextually relevant responses to non-literal language. We also find that providing oracle intentions substantially improves response appropriateness, but using chain-of-thought to make models spell out intentions before responding improves much less. These findings suggest that LLMs are not yet pragmatic interlocutors, and that explicitly modeling intention could improve LLM responses to non-literal language.

## introduction humans possess the ability to communicate and understand each other even through non-literal utterances and conversational implicatures @xcite @xcite . this is attributed to their ability to make pragmatic inferences arising from contextual factors and conventions in conversation, rather than specific words or phrases @xcite . since humans often use non-literal language in communication, large language models (llms) must also develop pragmatic understanding to facilitate effective and nuanced human-ai interactions. in this work, we introduce a new generative evaluation framework designed to evaluate the ability of llms to understand and resolve intentions through pragmatic response generation. in figure 1 , kelly uses hyperbole to express her desire to read numerous books. a contextually appropriate response would be to ideally echo sentiments like "that sounds like a great plan" rather than interpreting "a million" literally, as seen in responses like "that's quite an ambitious reading list". our framework uses this intuition to compare llms' responses to human-like expectations, enabling a nuanced assessment of their pragmatic understanding and response accuracy. our primary focus on pragmatic response generation marks a departure from prior work @xcite @xcite , which has predominantly measured intention understanding through a discriminative contrastive multiple-choice classification. we show that this setting does not necessarily reflect llms' abilities in generating pragmatic responses, nor does it correspond to the use of llms as conversational agents @xcite . we evaluate the pragmatic understanding of several state-of-the-art open-source llms on various types of non-literal language from @xcite . we observe that llms often struggle with generating contextually appropriate responses and tend to interpret non-literal language literally, with an accuracy of 50-55%. furthermore, we find that llms' ability in detecting intentions does not translate to their pragmatic response generation, highlighting a key distinction between merely detecting intentions and pragmatically acting on them in a generative setting. finally, we explored approaches to improve llms' pragmatic response abilities. using chain-of-thought prompting to make models explicitly spell out intentions before generation has minimal effects in addressing these limitations. while providing the oracle true intentions yielded better performance, models still significantly struggle to effectively utilize these intentions in response generation. overall, our findings indicate a significant gap in current llms' ability in pragmatic understanding. this emphasizes the need for better mechanisms to infer communicative intentions and their effective usage, to enhance pragmatic communication. ## pragmatic response generation we introduce a new framework to evaluate pragmatic generative ability of models-to understand and infer implicit intentions, and use it to generate pragmatic responses to non-literal utterances. setup our evaluation setup (pictured in figure 1 ) measures llms' pragmatic response generation by comparing it to reference dialog chains under the intended true meaning and under a literal misinterpretation. specifically, it requires: we consider four non-literal language phenomena from @xcite al. (2022) 2. irony. speakers use irony to mean the opposite of what they say. irony is not explicitly defined in the context c, but c may include information about characters' emotional states. ## results on pragmatic response generation in this section, we analyze how well llms can generate contextually relevant responses. we compare our proposed generative approach, which evaluates implicit understanding in responses to u n 1 , against a discriminative multiple-choice setup as in @xcite , which evaluates intention detection in u n 1 utterances. results figure 2 indicates that llms exhibit better performance in responding to indirect speech among various non-literal language types, potentially due to conventionalization of responses, or explicit descriptions of requests completed seen during training @xcite . models perform the worst at responding to flouted maxims, performing worse than chance. for instance, models fail to detect the attempt to change the subject in "oh, it's such a pleasant day today" amidst a discussion about a "bad date". llama-2 models exhibit marginally better metaphorical language understanding (metaphors) compared to mistral and zephyr models. in the llama-2 family, we see that models perform better with increasing size. in aggregate, we see that llms perform at or near chance in generating an appropriate response that reflects having inferred the true intent. ## chain-of-thought prompting for pragmatic response generation motivated by the ability of llms to detect intentions in some phenomena, we explore ways to improve their understanding of implicit intentions and, thereby enhancing their capability to generate pragmatic responses using chain-of-thought prompting (cot) @xcite . experiments using chain-of-thought in our experiments with cot, we first generate an inferred intention and then a response (unless otherwise specified). we examine how response generation performance is affected by introducing varying levels of oracle cues at the inferred intention generation step, organized by increasing amounts of "hand-holding": results figure 3 illustrates that specifying the type of non-literal language used along with the speaker's true intent these findings highlight the importance of explicitly modeling intention in llms, indicating that response accuracy to non-literal language can improve with such approaches. overall, there is a clear need for: (a) better learning mechanisms to help models effectively disentangle the linguistic strategies used and communicative intent (e.g., recognizing how exaggeration can create irony to highlight disagreement), and (b) effective utilization of learned intentions during response generation. ## summary we propose a new framework to evaluate how well llms understand intentions and respond to nonliteral language, moving beyond previously employed multiple-choice settings. our results show that llms often struggle to generate contextually relevant responses. while chain-of-thought prompting to spell out inferred intentions offers marginal improvements, explicitly providing oracle intentions and cues, such as for irony, significantly enhances performance. these findings highlight the current limitations of llms in pragmatic understanding, suggesting that improved learning mechanisms to explicitly model intentions and linguistic strategies could significantly enhance conversational abilities.