Context-Aware Reasoning On Parametric Knowledge for Inferring Causal Variables (2025)

Abstract: Scientific discovery catalyzes human intellectual advances, driven by the cycle of hypothesis generation, experimental design, evaluation, and assumption refinement. Central to this process is causal inference, uncovering the mechanisms behind observed phenomena. While randomized experiments provide strong inferences, they are often infeasible due to ethical or practical constraints. However, observational studies are prone to confounding or mediating biases. While crucial, identifying such backdoor paths is expensive and heavily depends on scientists’ domain knowledge to generate hypotheses. We introduce a novel benchmark where the objective is to complete a partial causal graph. We design a benchmark with varying difficulty levels with over 4000 queries. We show the strong ability of LLMs to hypothesize the backdoor variables between a cause and its effect. Unlike simple knowledge memorization of fixed associations, our task requires the LLM to reason according to the context of the entire graph.

## introduction scientific discovery has been key to humankind's advances. it is a dynamic process revolving around inquiry and refinement. scientists adhere to a process that involves formulating a hypothesis and then collecting pertinent data @xcite . they then draw inferences from these experiments, modify the hypothesis, formulate sub-questions, and repeat the process until the research question is answered @xcite . central to scientific discovery is formulating hypotheses and identifying relevant variables that drive the underlying causal mechanisms of observed phenomena @xcite . randomized controlled trials are the gold standard for establishing causal relationships, but they are often infeasible due to ethical, financial, or logistical constraints @xcite . in such cases, researchers rely on observational data, where a key challenge lies not only in analyzing relationships but in determining which variables should be observed and included in the analysis, particularly confounders or mediators that influence causal mechanisms underlying the outcomes @xcite . with the recent advancement of large language models (llms), there has been a growing interest in using them for scientific discovery (ai4science and quantum, 2023; @xcite . llms have demonstrated strong performance in internalizing knowledge @xcite and reasoning-based tasks @xcite , including causal discovery, where they infer pairwise causal relationships based on variable semantics @xcite @xcite @xcite . scientific reasoning is fundamentally context-driven; unlike simple factual retrieval, it requires adapting hypotheses based on new evidence and integrating knowledge across varying subpopulations. while recent work has explored the use of llms for causal discovery @xcite @xcite @xcite , much of it assumes a fixed set of variables and focuses on identifying relationships among them. however, a critical and underexplored aspect is determining which variables should be considered in the first place. this demands flexible, context-sensitive reasoning to identify missing causal factors. in our paper, we use the term reasoning operationally to describe the model's ability to generate hypotheses or identify variables that complete partial causal graphs. our usage follows previous work by @xcite in causal discovery and llm research, where "reasoning" often refers to generating plausible hypotheses or prioritizing potential candidates given partial structural information, rather than strict deductive logic. to address this gap, we propose a novel task: given a partial causal graph with missing variables, the llm is prompted to hypothesize what those variables might be, using the structure and known nodes as context. by systematically omitting different variables, we generate diverse test cases to evaluate the robustness of model reasoning. we further decompose the benchmark into subtasks, starting from baseline variable identification to more realistic, open-ended settings where multiple unobserved mediators exist between known treatments and outcomes. our task mirrors real-world scientific workflows, where identifying missing variables-especially confounders and mediators is essential for valid causal inference. this typically demands costly, interdisciplinary effort. llms, trained on diverse knowledge sources, offer a scalable alternative. for example, in a stroke drug study, an llm might suggest socioeconomic status as an unmeasured confounder. while recent works advocate using llms as co-pilots for causal tasks @xcite , systematic evaluations are lacking. our benchmark addresses this gap by assessing llms' ability to infer missing causal variables across domains. our main contributions are: 1) we propose and formalize the novel task of llm-assisted causal variable inference. 2) we propose a benchmark for inferring missing variables across diverse domains of causal graphs. 3) we design experimental tasks with different difficulty levels and knowledge assumptions, such as open-world and closed-world settings, the number of missing variables, etc. 4) our benchmark allows for both grounded evaluations and a reproducible framework to benchmark llms' capabilities in hypothesis generation. ## preliminaries: causal graph a causal relationship can be modeled via a directed acyclic graph (dag). a causal dag represents relationships between a set of n variables defined by @xmath0, ..., v n }. the variables are encoded in a graph @xmath1, e) where e is a set of directed edges between the @xmath2. mathematically, it can be expressed as: we define d(v) as the degree of a node v, representing the total number of edges connected to v. d in (v) is the in-degree, representing the number of incoming edges to v. d out (v) is the out-degree, representing the number of outgoing edges from v. source has no incoming edges; treatment is characterized by nodes that are being intervened upon. outcome is characterized by nodes that are observed for interventions from the treatments. mediator has both incoming and outgoing edges (d in (v) > 0 and d out (v) > 0) as intermediaries in the pathways between treatment and outcome. confounder influences both treatment and outcome, exhibiting edges directed towards the treatment and outcome nodes (d out (v) ≥ 2). hence v is a confounder if it is a parent of both v i and v j . collider has two edges meeting, and d in (v) > 1. i.e., v is a collider if it is a child of both v i and v j . ## inferring causal variables motivated by the challenge of discovering variables that block backdoor paths to ensure unbiased causal inference @xcite , in this work, we leverage language models to infer missing variables in a causal dag. we assume that a part of the graph is already known, and the aim is to find additional variables that can be incorporated into the existing dag to enhance the underlying causal mechanism. this implies that all causal relationships (edges) among variables in v * are known and correctly represented in g * ; i.e., e is fully specified. here, "missing" variables are not latent or hidden by measurement error but known unknowns within the causal graph reflective of the llm's perspective. to systematically assess llms' ability to infer missing causal variables, we construct a multi-stage benchmark with increasing levels of complexity. we begin with a controlled setting, where the model is provided with a partial causal dag and a set of multiple-choice options to identify missing variables. then, the task becomes open-ended, where llms hypothesize missing variables, simulating an open-world paradigm. additionally, as the task escalates, we introduce more complexity by omitting additional nodes, challenging the model to hypothesize multiple missing variables. we evaluate the reasoning capability of llms through prompting. we represent the graph g * using a prompt template p llm (•) which enables llms to parse causal relationships in the dag. ## conclusion most causality research focuses on identifying relationships from observed data, while hypothesizing which variables to observe remains largely reliant on expert knowledge. we propose using llms as proxies for this step and introduce a novel task: hypothesizing missing variables in causal graphs. we formalize this with a benchmark that spans varying levels of difficulty and ground-truth knowledge. our results highlight llms' strengths in inferring backdoor paths, including colliders, confounders, and mediators, which often lead to biased causal inference when unaccounted for. our work llms can serve as useful tools for early-stage hypothesis generation, supporting scientists in formulating plausible causal variables before data collection. by evaluating models across different graph completeness, open-and closed-world settings, we highlight their potential and limitations.