The G ricean Maxims in NLP - A Survey (2024)

Abstract: In this paper, we provide an in-depth review of how the Gricean maxims have been used to develop and evaluate Natural Language Processing (NLP) systems. Originating from the domain of pragmatics, the Gricean maxims are foundational principles aimed at optimising communicative effectiveness, encompassing the maxims of Quantity, Quality, Relation, and Manner. We explore how these principles are operationalised within NLP through the development of data sets, benchmarks, qualitative evaluation and the formulation of tasks such as Data-to-text, Referring Expressions, Conversational Agents, and Reasoning with a specific focus on Natural Language Generation (NLG). We further present current works on the integration of these maxims in the design and assessment of Large Language Models (LLMs), highlighting their potential influence on enhancing model performance and interaction capabilities. Additionally, this paper identifies and discusses relevant challenges and opportunities, with a special emphasis on the cultural adaptation and contextual applicability of the Gricean maxims. While they have been widely used in different NLP applications, we present the first comprehensive survey of the Gricean maxims’ impact.

## introduction capturing the full nuance of human language requires more than understanding its structure; it necessitates an intricate comprehension of context. this understanding goes beyond the words themselves to grasp the intentions, implications, and subtleties embedded in communication @xcite @xcite . in order to build nlp systems that are able to use language beyond just its literal content, they need to incorporate pragmatic capabilities @xcite @xcite @xcite . a central idea in pragmatics are the gricean maxims, a set of cooperative principles proposed by philosopher grice @xcite . these maxims are descriptions of effective human communication strategies, capturing the implicit expectations and norms that govern human interaction and thereby offering a theoretical framework that has profound implications for the development of nlp technologies. as nlp systems, particularly llms, strive to achieve more human-like understanding and generation of text, the consideration of these pragmatic principles becomes crucial @xcite @xcite . they not only aid in improving the interpretative capabilities of these systems but also enhance their ability to generate coherent, contextually appropriate responses. the gricean maxims consist of four primary directives that guide conversational cooperation. each maxim addresses a different aspect of communication, providing a guideline for what makes a conversation effective and meaningful. these maxims are: ## methodology we compile our list of papers through an exhaustive keyword search on google scholar and the acl anthology database. we combined keywords for the concepts (gricean maxims, cooperative principles, pragmatic principles) with keywords for disciplines (nlp, nlg, conversational ai) in a two-dimensional matrix. after manually filtering out papers that only mention the gricean maxims in their related work or introduction and additions through mentioned related work, we identified 78 relevant papers published between 1990 and 2024. for an overview of all works surveyed, see figure 1 . for a division into the covered maxims see appendix 2. ## data and benchmarks in this section, we show recent advancements in the creation of datasets and benchmarks aimed at evaluating and enhancing the pragmatic reasoning capabilities of nlp systems, particularly llms. @xcite present the grice dataset, a grammar-based dialogue dataset designed to incorporate implicature into pragmatic reasoning within conversations. the dataset aims to bridge the gap in modern open-ended dialogue systems that struggle with understanding the intended meaning beyond the literal statements. grice also addresses other crucial aspects of dialogue modelling, such as coreference, ensuring temporal consistency and intricate implicatures within each dialogue context. the dataset introduces two tasks: implicature recovery and pragmatic reasoning in conversation. experiments reveal a significant gap between the performance of baseline methods (which claim pragmatics reasoning capabilities) and human performance. incorporating ## llms with the widespread use of llms, expectations are emerging for them to have pragmatic abilities: to interpret and generate language in context. in the following, we look at recent approaches that use gricean maxims to evaluate and potentially improve these capabilities. @xcite perform an in-depth evaluation comparing the performance of llms with humans across a spectrum of pragmatic phenomena. their research reveals that top-tier models match humans in terms of accuracy and error tendencies, showing a preference for literal over heuristic interpreta-tions. however, challenges arise with scenarios that demand an understanding of violated social norms. similar gaps are found by pietro et al. (2023) when analysing chatgpt's grasp on pragmatics, identifying its proficiency across various domains but pinpointing deficits in understanding humour, metaphors, and adhering to the quantity maxim. tao et al. (2024) corroborate these findings with a naturalness metric that is based on the cooperative principles and the model again most frequently violating the maxim of quantity. investigating the reverse, @xcite studied if llms can spot maxim violations and implicatures. they find that while the performance of llms did not significantly vary with respect to different conversational maxims, variability existed in the performance among models. @xcite propose an augmented set of conversational maxims to evaluate and guide interactions between humans and llm-driven conversational agents, adding maxims for benevolence (to avoid harm) and transparency (admitting limitations). gricean maxims are also used in critiques of the application of llms like chatgpt in legal environments @xcite , focusing on its tendency to produce unverified content, termed "r-lying." this critique leverages grice's quality maxim to question the reliability and accuracy of responses generated by llms, underlining the imperative for technological advancements to mitigate these issues. in entity description generation, @xcite adapt the maxims of quality and relation into factuality and congruity. their evaluation paradigm disentangles factual errors (nonfactual descriptions) from contextual errors (incongruous descriptions). they find that models struggle with accurate descriptions of less familiar entities, raising concerns about the trustworthiness of language models, as these errors are harder for human readers to detect. in multilingual contexts, park et al. (2024) broaden the scope of assessing llms' pragmatic skills to include korean, utilising diverse question formats to test narrative response capabilities. their study demonstrates gpt-4's strong performance, while cautioning against prompting methods that skew towards literal interpretations, thus limiting pragmatic inference. in the educational domain in german, ai-driven pedagogical agents are evaluated by applying gricean principles extended with a trust maxim @xcite . the findings emphasise trust as a crucial factor in the educational efficacy of chatbots, suggesting that fidelity to gricean norms can significantly impact the utility and dependability of conversational ai in learning contexts. these investigations collectively underscore the ongoing effort to give llms a deeper understanding of pragmatic nuance. while significant progress has been made, the reviewed works highlight the particular relevance of the maxim of quantity in addressing overgeneration issues in llms @xcite and the application of the maxim of quality in improving their expression of uncertainty @xcite . achieving full pragmatic alignment remains a challenge, pointing to future research directions that could bridge the gaps in current capabilities. as argued by @xcite , there is a need to distinguish between the colloquial use of "cooperation" and the use intended by grice, a distinction he terms "cooperation drift." similarly, chen and van deemter (2023) emphasise the need for explicit definitions of over-and under-specifications in referring expressions, noting that these are often loosely aligned with gricean principles without clarifying "required" actually means. neo-griceans @xcite ) simplify grice's maxims into two principles: the q-principle, which encourages providing sufficient information while avoiding unnecessary details, and the i-principle, which emphasizes clarity and informativeness. this approach aims to create a more unified and manageable framework for understanding conversational implicatures additionally, some scholars argue that grice's maxims are vague and oversimplify communication complexities @xcite . others question their universal applicability, noting real-world deviations @xcite , or the dynamic negotiation of meaning that sometimes breaks these maxims to achieve understanding @xcite . power dynamics and politeness strategies, which also influence conversations, are insufficiently addressed by grice's framework @xcite . @xcite culture and language are fused, thereby making a language analysis without looking at the social and cultural aspects of it limited in its insights. this also holds for the gricean maxims. as danziger (2010) documents, while the maxims were intended as universal, certain cultural settings might interpret the maxims differently, indicating a need for cultural adaptation of these principles. a promising way to deal with this is participatory design, where stakeholders affected by ai systems should participate in their design @xcite . an example of an application with relevance to the gricean maxims is the study by medhi thies et al. (2017) who explored chatbot preferences in an exploratory wizard-of-oz study among young, urban indians. @xcite , olaniyi and oyinbo (2021), and @xcite contribute to the discourse by examining the structure of grice's maxims within the indonesian, nigerian and moroccan cultural contexts, respectively. their findings highlight the influence of societal expectations, politeness strategies, and specific linguistic characteristics on conversational implicatures, suggesting that the maxims may require re-formulation or adaptation to align with diverse context dependent cultural norms. ## conclusion the application of gricean maxims in nlp reflects a consistent effort to address the complexities of human communication, spanning from the foundational stages of the field to current advancements. this survey is the first comprehensive review of how these maxims have informed the development and evaluation of nlp systems across a range of tasks, highlighting progress in making systems more pragmatically aligned with human conversation while also pointing out the existing challenges, especially regarding cultural variations and conversational norms. while some papers focus on specific subsets of the maxims, others extend them to capture a broader spectrum of communicative nuances or reinterpret them for their use-case. surveyed work suggests that moving forward, the nlp field can benefit from a more focused integration of pragmatic and cultural considerations, aiming to produce conversational agents that better reflect the intricacies of human communication.