Understand the Implication: Learning to Think for Pragmatic Understanding (2025)

Abstract: Pragmatics, the ability to infer meaning beyond literal interpretation, is crucial for social cognition and communication. While LLMs have been benchmarked for their pragmatic understanding, improving their performance remains underexplored. Existing methods rely on annotated labels but overlook the reasoning process humans naturally use to interpret implicit meaning. To bridge this gap, we introduce a novel pragmatic dataset ImpliedMeaningPreference that includes explicit reasoning (‘thoughts’) for both correct and incorrect interpretations. Through preference-tuning and supervised fine-tuning, we demonstrate that thought-based learning significantly enhances LLMs’ pragmatic understanding, improving accuracy by 11.12% across model families. We further discuss a transfer-learning study where we evaluate the performance of thought -based training for the other tasks of pragmatics (presupposition, deixis) that are not seen during the training time and observe an improvement of 16.10% compared to label trained models.

## introduction human interactions shape relationships through shared understandings, influenced not just by explicit words but by emotional and pragmatic nuances that convey implicit meanings. the ability to interpret beyond the literal meaning of language, known as pragmatics, is essential for social cognition, interpersonal awareness, and emotional intelligence. it allows individuals to navigate conversations fluidly, recognising intentions, cultural contexts, and unspoken implications. recent progress in large language models @xcite @xcite @xcite has advanced the capabilities figure 1 : an example of implicature in a response which demonstrates the effectiveness of thought-based training by capturing the correct pragmatic meaning of conversational ai. these systems exhibit robust performance in natural language generation, reasoning tasks like math word problems, code generation @xcite @xcite , etc., largely due to the exploitation of extensive computational resources and vast language datasets. despite these strengths, current llms struggle with effective communication, specifically in capturing the pragmatic and ambiguous dimensions of user inputs. additionally, conventional training strategies prioritise the production of responses that are safe, objective, and widely acceptable @xcite . this approach, while ensuring reliability, diverges from the goal of replicating truly human-like conversational behaviour, where the subtleties of context, emotion, and cultural nuance are critical. while humans naturally engage in pragmatic reasoning, llms often struggle with this skill, especially the small llms (slms) @xcite , which are often used in practical scenarios due to their lower inference costs, reduced latency, and suitability for local deployment. given the increased interaction between humans and llms, it is very important for the llms to obtain substantial pragmatic understanding of human language and intent. recent work has primarily focused on evaluating llms' pragmatic understanding, yet efforts to enhance their performance on such tasks remain limited @xcite . approaches that try to improve llms in pragmatic reasoning rely on label-based supervision or policy optimisation over annotated datasets @xcite , but these methods do not explicitly incorporate the reasoning process that humans use to grasp implicit meaning. this is mainly due to the absence of training mechanisms which can explicitly incorporate the reasoning process. for instance, as shown in figure 1 , interpreting the response "can birds fly?" as "yes" to the question "can you make a cake?" requires recognising it as a rhetorical question with an obvious affirmative answer-implying that the speaker's answer to the original question is also an obvious "yes". to address this gap, we introduce a novel approach that leverages explicit reasoning, or thoughts, to improve llms' pragmatic comprehension. specifically, we perform thought-based training for the task of implicature recovery, understanding what is implied in a statement even though it is not literally expressed. we then show generalizability on multiple pragmatics domains, which include implicature, presupposition and reference. unlike reasoning tasks such as math word problems or coding challenges, pragmatic reasoning often lacks definitive answers, making it more challenging. the correct interpretation in a given scenario is highly influenced by context, culture, and the individuals involved. this interpretation is often not described in the raw training data explicitly and can not be easily captured during the training process. to mitigate this, an explicit intermediate reasoning process must be provided during the training time along with the correct label, which details the intermediate reasoning process, mimicking how humans derive correct interpretation by deliberate system-2 thinking @xcite . hence, we present a first-ofits-kind pragmatic dataset where each instance includes a thought explaining the reasoning behind the correct label, along with a plausible yet incorrect negative thought justifying the incorrect label. we integrate this thought-based data into both preference-tuning and supervised fine-tuning settings, demonstrating an absolute improvement of 11.12% in accuracy across three model fam-ilies. our findings establish the effectiveness of thought-based learning in advancing llms' ability to interpret implicit meaning in language. our contributions are: ## related work implicature recovery is a central topic in pragmatics, attracting significant attention from linguists and computational researchers alike. one of the most influential theoretical contributions to this field is the formulation of the gricean maxims (grice, 1975), which outline principles governing conversational implicature through quality, quantity, relevance, and manner. various approaches have been proposed to analyse and recover implicatures. for instance, louis et al. (2020); ruis et al. (2023) study indirect answers in polar questions, shedding light on how conversational participants infer unstated meanings. zheng et al. (2021) leverage hierarchical grammar models to interpret both implicatures and deictic references in structured dialogues. additionally, jeretic et al. (2020) explores the role of natural language inference (nli) in understanding scalar implicatures, while deng et al. (2014) integrate implicature-based reasoning into sentiment analysis. further contributions in this domain include corpus-based studies such as @xcite , which provide sentence-level annotations for implicature detection. work by @xcite and @xcite focuses on employing neural networks and linguistic signals to predict scalar inferences, highlighting the potential of machine learning in implicature comprehension. despite these advancements, recent benchmarking efforts @xcite consistently reveal a persistent performance gap between human reasoning and llm capabilities in pragmatics. building upon these findings, @xcite introduces an open-ended evaluation framework to assess llms' pragmatic abilities, showing the superiority of preference-based learning over supervised fine-tuning when label-based data is considered. going forward, our work incorporates the intermediate reasoning steps (thoughts) in fine-tuning and preference optimisation process for pragmatic reasoning. unlike conventional approaches that reward only label accuracy, our method explicitly incorporates thought processes into model training, enabling llms to develop a deeper understanding of pragmatics. in the following sections, we present our datasets, methodology and evaluate the effec-tiveness of structured reasoning in enhancing llm performance in pragmatics tasks like implicature recovery, presupposition and deixis. ## datasets in this section, we discuss the process of generating impliedmeaningpreference data and synthetic qa datasets (syn-circa and syn-ludwig). ## methodology this section discusses our approach in detail. we aim to study the impact of incorporating thought training in two settings: 1) supervised fine-tuning (sft) and 2) direct preference optimization (dpo) @xcite . we formulate the task as follows: given an input consisting of an indirect answer to a question along with a context, output the pragmatic interpretation. let p (x) be the initial prompt which contains the task description t desc and input description i desc where @xmath0, i desc ]. let g be the generated output, which contains a thinking process t thought followed by a predicted label p label . here, @xmath1; p label ] consisting of tokens (g 1 , g 2 , . . . , g t-1 , g t ). in the general supervised fine-tuning process, we aim to maximize the conditional log-likelihood of the output tokens given the input tokens. in the context of our setting, this corresponds to: here l is the total loss (negative log-likelihood of the sequence), |g| is the length of the output sequence g, p θ (g t | p (x), g 1 , g 2 , . . . , g t-1 ) is the model's predicted probability of the token g t at position t, given the input prompt p (x) and all previous tokens g 1 , g 2 , . . . , g t-1 and θ is the model parameters being optimized. contrary to sft, in standard reinforcement learning from human feedback (rlhf) setup, we use the structure of the markov decision process consisting of 4 tuples: (states s, actions a, transition probabilities t p , rewards r). here, we define a function policy π, which maps states to actions (π : @xmath2. the goal is to optimize the policies to maximize the rewards. in our context, given the current state (input prompt), we would like to optimize the policy (language model) to select the actions (which token to predict next) such that the reward function (a function which scores the generated output based on human preferences) yields the maximum value. we aim to study the effects of optimizing policy over the thoughts and labels together. this means that the probability of winning generation (g w ) preferred by humans should be more than the probability of losing generation (g l ), which humans do not prefer. therefore, the bradley terry model for our setup is: in the above equation, π ref is the reference model instantiated with the initial version of the model, π θ is the model obtained after preference tuning, and β is the regularizing parameter used for penalizing the scenario when the resulting model is very far from the base version resulting to the loss of prior knowledge. from a linguistic perspective, our approach is motivated by the need to model pragmatic competence in language understanding. pragmatic reasoning involves interpreting implied meanings that go beyond the literal content of utterances, as theorized in grice's maxims of conversation. traditional models often struggle with implicature resolution because they lack an explicit mechanism for reasoning about contextually inferred meanings. by integrating structured thought processes into both fine-tuning and preference optimization, our method provides a computational analog to human inferential processes in discourse interpretation. this, we hope, should enable llms to better grasp implicatures, handle indirect responses, and align with human-like conversational norms, thereby improving their effectiveness in pragmatic language tasks. ## experimental setup for our experiments, we consider models from three different families: 1) llama-3.2-1b @xcite ) 2) qwen-2.5-1.5b @xcite , 3) gemma-2-2b @xcite . we also report the zero-shot performance of llama3.1 70b for comparison with a large language model. our experiments kept the learning rate at 5e -7 with warmup steps of 500 iterations. we use rmsprop @xcite as our optimiser following @xcite . all models in both settings are trained for one epoch (till convergence), and the greedy decoding mechanism was used throughout the experiments. for qwen-2.5-1.5b and gemma-2-2b, we use the global batch size of 32; for llama-3.2-1b, the global batch size was set to 64. for regularization, we use gradient clipping of 1 in dpo and weight decay of 0.01 in sft. we use 4 nvidia h100 80gb hbm3 gpus for all the experiments in this work, with a total train time of 8 gpu hours. for all other hyper-parameters, we use the default values. we report all the training and evaluation prompts in appendix, sections c and d respectively. we use macro precision (p), recall (r) and f1 scores for evaluation. ## results in we note that training with just labels provides an edge to dpo over sft across models, aligning with @xcite . while training with thought alongside labels provides significantly higher gains for sft when compared to dpo, with thoughtbased sft outperforming thought-based dpo in most cases. the 'thoughts' contain more explicit signals and the interpretation of the reasoning required to reach the right answer, which may be captured in a more straightforward way in the sft setup compared to dpo. intuitively, the thoughtbased training mechanism would require higher updates in the parameters than the scenario when we have to optimise over only label tokens. in general, the optimisation objective for sft does not have any constraints and is more flexible compared to dpo, which requires a regularising parameter β for the kl constraint to prevent divergence from the base model (untrained) during the training. another perspective in the context of this observation was suggested by @xcite shows the gradient of the dpo loss with respect to preferred (winning) response is lower compared to the dispreferred (losing) response which essentially hinders the learning capacity of llms to generate the actual human preferences while introducing the tendency of avoiding human dispreferred responses. this effect may have been further magnified in our setting which has more tokens compared to the only label setting. we also note that our best-performing model, gemma2-2b, supervised-fine-tuned with thoughts, yields comparable performance to llama3.1-70b, which highlights the effectiveness of incorporating thoughts in the training mechanism. in general, the thought-based training mechanism yielded better results compared to the setting, which just incorporates labels, highlighting the importance of learning thought generation. ## analysis in this section, we discuss various insights about the proposed method, which describes the advantages of thought-based learning and some general error cases. ## conclusion and future work in this work, we highlighted the effectiveness of integrating explicit thought processes into two training paradigms: 1) supervised fine-tuning (sft) and 2) direct preference optimization (dpo). our findings indicate that while thought integration benefits both training approaches, thought-based sft consistently outperforms its dpo counterpart in pragmatic reasoning tasks. through a detailed analysis of model predictions, we uncover key patterns in implicature resolution and identify specific failure cases that illuminate areas for further improvement. to reinforce the role of structured reasoning, we investigate the impact of perturbing thought generation, revealing a notable decline in performance when the reasoning process is disrupted. furthermore, our transfer learning experiments demonstrate the adaptability of thought-based training, showing its efficacy in generalizing across previously unseen datasets and pragmatic tasks. in the future, our goal is to refine this approach by developing a process-based reward mechanism that better aligns llms with human pragmatic inference, ultimately bridging the gap between computational and human-like language understanding.