C o RE : Condition-based Reasoning for Identifying Outcome Variance in Complex Events (2025)

Abstract: Knowing which latent conditions lead to a particular outcome is useful for critically examining claims made about complex event outcomes. Identifying implied conditions and examining their influence on an outcome is challenging. We handle this by combining and augmenting annotations from two existing datasets consisting of goals and states, and explore the influence of conditions through our research questions and Condition-based Reasoning tasks. We examine open and closed LLMs of varying sizes and intent-alignment on our reasoning tasks and find that conditions are useful when not all context is available. Models differ widely in their ability to generate and identify outcome-variant conditions, which affects their performance on outcome validation, when conditions are used to replace missing context. Larger models like GPT-4o, are more cautious in such less constrained situations.

## introduction knowing which conditions influence a goal's outcome is useful for understanding and planning goaldirected actions seen in complex events @xcite . consider the following 5-sentence short story, also shown in fig. 1 : sam can't sleep at night. sam is afraid of the monsters under the bed. dad tells sam there is no such thing as monsters but it doesn't help. so, dad give sam a blanket and tells sam that it's a magic blanket. sam believes the blanket protects against monsters. while one can reasonably infer that sam's goal (to sleep at night overcoming fears about monsters) was achieved in this story, how stable is this outcome? what states, or conditions, relating to sam or the situation help support the outcome, and if those were changed, would sam's goal still be achieved? some of these conditions, like sam trusting his/her dad or sam being a small child, are relevant to the goal, with a high likelihood of influencing the outcome, while other conditions that @xcite . this type of reasoning is challenging because: (1) conditions that to relate to entity properties and states are not always explicitly stated in a narrative but are implicitly understood through forming a coherent mental representation @xcite and acquiring this implicit knowledge is not easy. inspired by @xcite 's findings that pragmatic knowledge influences how linguistic cues are interpreted, we examine how models use linguistic cues when understanding story outcomes. we explore how implicit conditions influence outcomes and whether they can be used in place of missing context for validating story outcomes. we leverage two previously released datasets pasta @xcite and saga @xcite consisting of participants' goal and state annotations (shown in fig. 2 ) and augment them for exploring conditions and outcomes. we address the following research questions and formulate our condition-based reasoning tasks to explore the capabilities of intent-aligned llms on these tasks. we highlight the usefulness of conditions in determining story outcomes across a variety of alternative narratives, from short five sentence stories to real life newswire articles. rq1: do linguistic cues matter? we explore models' ability to handle different types of outcome descriptions. for the example in fig. 1 , would models consider "sam was sleeping well" to mean the same as "sam slept well?" while these two descriptions do not mean the same according to dowty's imperfective paradox @xcite , @xcite show that meaning agreement is more complex. we examine how models fare on such linguistic differences and find that the state-of-the-art models differ in performance on the imperfective paradox, but understand both ongoing and completed actions, with a slight preference for completed actions. see §5.1 for more details. rq2: can models think about "what-if...?" we examine whether models can reason counterfactually, through generating and identifying outcomevariant contrastive condition pairs. we found that with one exception (mistral-7b-instruct-v0.3), all considered models are better at identifying whether a condition is outcome-related (which does not involve counterfactual thinking), but struggle at identifying outcome variance which requires counterfactual thinking. see §5.2 for details. rq3: when are conditions useful? we examine if models are able to use conditions when context is unavailable or incomplete. we find that while models are able to combine conditions with available context to validate outcomes, some larger models are more cautious when the context is less constrained. see §5.3 for more details. overall, we find that performance on outcome validation improves by 2-6% (macro f1 score) when outcome-variant (as opposed to outcomeinvariant) conditions are used with the story context. since identifying or generating outcomevariant conditions is not easy, performance drops by 3-6% for generated conditions when compared to annotated conditions. for news stories, performance drops by up to 2-18%, when compared to the pasta story generated conditions, likely attributable to a combination of the domain change and models' going beyond the provided context and using their knowledge of past news events from their pretraining. see §6 for more details. our contributions in this work include introducing the concept of conditions and their relationships to an outcome. we artfully combine and bridge two existing datasets to explore intertwined aspects of conditions and outcomes. through our research questions we examine how llms leverage conditions for outcome validation and compare their performance on the practical task of validating outcomes of news stories. our data and code are available at https://github.com/saiumbc/core . ## related work examining the influence of states and actions on goal outcomes is an active research area in the cross-disciplinary fields of cognitive science and psychology @xcite @xcite . the utility of counterfactual conditionals in reasoning @xcite has a long history in linguistics & psychology @xcite . ## data annotations in the following paragraphs, we briefly describe the pasta @xcite and saga @xcite datasets and how these datasets' annotations relate to conditions and outcomes. ## outcome validation through rqs we examine whether an outcome is possible for a condition known about an entity through our research questions. we use both crowd-annotated and llm-generated conditions, where not every condition is supported by the story context or related to the outcome. we formulate reasoning tasks with the aim to compare various models' performance on identifying and generating outcome-variant conditions. we examine the conditions' usefulness in validating outcomes of saga and news stories. models: we examine well-known closed and open, human intent-aligned llms of different sizes: gpt-4o-mini, gpt-4o, flant5-xxl, llama-3.1-8b-instruct (llama-8bi), mistral-7b-instruct-v0.3 (mistral-7bi) and llama-3.1-70b-instruct (llama-70bi). we refer to the models by the shortened names in parentheses. we examine the impact of model type, size and the alignment. inference: for the tasks examined in rq1 and rq3, we use zero-shot prompting to examine models' inherent knowledge. for the tasks examined in rq2 we use few-shot prompting which is required as models do not inherently understand how to generate or identify outcome-variance. inference cost for gpt models was less than $100. ## cross-domain application to news we use the task prompts and setup from rq2 and rq3 to generate conditions for news stories and use them to identify outcome labels. news stories have longer sentences and include more complexities in the number of entities, topics, linguistic cues etc. when compared to saga stories. we examine their affect on model performance, comparing generated conditions from both story types. we compare the settings in rq3, except for partial story context, as this requires identifying story sentence(s) supporting the condition. ## conclusions by defining condition-based reasoning tasks, we have demonstrated the importance of identifying outcome-oriented relationships and their usefulness in validating story outcomes. to do this, we first combined and augmented two existing datasets to obtain outcome-variant and invariant conditions, and then showed that models can effectively use outcome-variant conditions in place of missing context. we also showed how to generate and use outcome-variant conditions on a new domain. we examined and showed models' sensitivity to linguistic cues, which can indirectly affect both condition generation and outcome validation. we hope that our work will help further linguistic examination of model behaviors to achieve a deeper and more robust narrative understanding.