Mimicking How Humans Interpret Out-of-Context Sentences Through Controlled Toxicity Decoding (2025)

Abstract: Interpretations of a single sentence can vary, particularly when its context is lost. This paper aims to simulate how readers perceive content with varying toxicity levels by generating diverse interpretations of out-of-context sentences. By modeling toxicity we can anticipate misunderstandings and reveal hidden toxic meanings. Our proposed decoding strategy explicitly controls toxicity in the set of generated interpretations by (i) aligning interpretation toxicity with the input, (ii) relaxing toxicity constraints for more toxic input sentences, and (iii) promoting diversity in toxicity levels within the set of generated interpretations. Experimental results show that our method improves alignment with human-written interpretations in both syntax and semantics while reducing model prediction uncertainty.

## introduction misunderstandings online can often be traced back to misalignment between the meanings of text intended by the author and those inferred by the readers. this is even further amplified when text is taken out of context -which is commonplace on social media -resulting in frustration and heated discussion. in this paper, we aim to mimic how readers may interpret out-of-context sentences. we do this by modeling and generating for each sentence a diverse set of interpretations @xcite . toxicity is taken as the control factor during generation as we want to simulate human interpretation behavior of sentences that exhibit varying degrees of surface-level toxicity. generating diverse interpretations can help anticipate misunderstandings, explain reactions from readers, and recover underlying toxicity, which is especially beneficial for capturing implicit hostility and harm online @xcite . this paper introduces a novel decoding strategy for interpretation generation that explicitly controls the toxicity level of generated interpretations. our decoding strategy enforces three key objectives that are inspired by toxicity patterns observed in human-written sentence interpretations: align the toxicity level of generated interpretations with that of the input sentence (objective 1); progressively relax toxicity constraints on the interpretations for increasing toxicity in the sentence (objective 2); promote diversity in the toxicity levels across the generated interpretations (objective 3). these objectives are enforced iteratively during the decoding process, enabling fine-grained control over toxicity while maintaining coherence and diversity in generated text. controlling generation in the decoding phase is particularly desirable as it bypasses the need for alterations to model architectures, allowing a plug-and-play integration with existing language models. our results demonstrate the soundness and effectiveness of our decoding strategy. controlling the decoding of interpretations using all three objectives consistently leads to generated interpretations that better align with human-written interpretations in terms of syntax and semantics, compared to when generation is not controlled. our strategy also lowers uncertainty for the base models when predicting the interpretations. ## related work text generation can be controlled using a range of control factors, including text attributes (e.g., sentiment, style) @xcite , syntactic structures @xcite , speaker or reader characteristics @xcite , and structured data (e.g., tables, knowledge graphs) @xcite . a popular approach to condition text generation is in-context learning, where these control factors are integrated into the input @xcite ). an-other method is to control generation during the decoding phase, e.g., by manipulating the output token distributions @xcite @xcite . this paper controls the toxicity of generated interpretations based on the surface-level toxicity of the original sentence during decoding. while much of the existing work on controlling toxicity in text generation focuses on reducing toxicity @xcite @xcite , our work builds on the idea that the toxicity of the original sentence is perceived differently among readers. we aim to capture this variability by constraining generation following three objectives. ## experimental setup dataset we rely on the origamim dataset foot_0 @xcite to evaluate our decoding strategy. origamim is the first dataset that specifically supports the interpretation modeling task @xcite and includes 9,851 humanwritten interpretations of 2,018 sentences from reddit posts. to accommodate the language models for this task, we fine-tune and validate them on the origamim training and validation sets. the test set is used to evaluate our decoding strategy. ## conclusion in this work, we proposed a modular decoding algorithm with three objectives designed to explicitly guide the generation of interpretations of out-ofcontext sentences. we showed that specifically constraining text decoding on toxicity brings generated interpretations closer to those written by humans. however, human interpretation is driven by many factors beyond toxicity like cultural background and personal experiences. we therefore strongly encourage future research to also consider these contextual factors when modeling the diverse ways in which a sentence's meaning is perceived.