Compositional Causal Reasoning Evaluation in Language Models
Authors: Jacqueline R. M. A. Maasch
Alihan Hüyük
Xinnuo Xu
Aditya V. Nori
Javier Gonzalez
Venue: ICML 2025 poster (2025)
Keywords: language models
causal reasoning
compositional reasoning
reasoning
evaluation
graphical models
causal inference
Primary area: deep_learning->large_language_models
TL;DR: A compositional view enables the systematic evaluation of causal reasoning in language models, revealing taxonomically distinct error patterns.

Abstract: Causal reasoning and compositional reasoning are two core aspirations in AI. Measuring the extent of these behaviors requires principled evaluation methods. We explore a unified perspective that considers both behaviors simultaneously, termed *compositional causal reasoning* (CCR): the ability to infer how causal measures compose and, equivalently, how causal quantities propagate through graphs. We instantiate a framework for the systematic evaluation of CCR for the average treatment effect and the probability of necessity and sufficiency. As proof of concept, we demonstrate CCR evaluation for language models in the Llama, Phi, and GPT families. On a math word problem, our framework revealed a range of taxonomically distinct error patterns. CCR errors increased with the complexity of causal paths for all models except o1.

Compositional Causal Reasoning Evaluation in Language Models
Jacqueline R. M. A. Maasch Alihan Hüyük? ? Xinnuo Xu²
Aditya Nori
Javier Gonzalez
Abstract
GROUND TRUTH A
THE COST OF PATH X
3.5
Causal reasoning and compositional reasoning are two core aspirations in AI. Measuring the ex- tent of these behaviors requires principled eval- uation methods. We explore a unified perspec- tive that considers both behaviors simultaneously, termed compositional causal reasoning (CCR): the ability to infer how causal measures compose and, equivalently, how causal quantities propagate through graphs. We instantiate a framework for the systematic evaluation of CCR for the average treatment effect and the probability of necessity and sufficiency. As proof of concept, we demon- strate CCR evaluation for language models in the Llama, Phi, and GPT families. On a math word problem, our framework revealed a range of tax- onomically distinct error patterns. CCR errors increased with the complexity of causal paths for all models except ol.
1.5
2.0
FORM 1: GLOBAL QUERY
FORM 2: COMPOSITION
@1: What 1a the cost
Q2: What is the sum costa for paths
of path X + 2?
and
A1: 3.5
A2: 3.5
Figure 1. Compositionally consistent responses to formula- tions of a simple (non-causal) query. Reasoning is internally consis- tent if A2==22, and externally valid if Al==A" and A2==A'.
(Frankland & Greene, 2020), underlying both visual per- ception (Schwartenbeck et al., 2023) and language (Lake Baroni, 2023). ! It is both a means of generalization and of coping with complexity: problems can be reformulated as simpler subproblems connected by compositional rules.
1. Introduction
The present work explores causal and compositional reason- ing in tandem. We center our focus on reasoning evaluation in language models (LMs), given increasing interest in LM reasoning emergence (Huang & Chang, 2023; Qiao et al., 2023; Mialon et al., 2023). Following from traditions in causal inference and graphical modeling, we define compo- sitional causal reasoning (CCR) as
Causal reasoning is a defining outcome of human evolution (Goddu Gopnik, 2024). Humans flexibly reason about cause and effect in factual realities that can be observed and intervened on, well as imagined counterfactual worlds. A causal lens enables humans and machines alike to learn generalizable lessons about the mechanics of the universe (Schölkopf et al., 2021). Thus, human-like Al might require reasoning at all three levels of Pearl's Causal Hierarchy: as- sociational, interventional, and counterfactual (Pearl, 2000).
the ability to infer compositions and decompositions of causal measures in factual and counterfactual worlds.
By extension, this requires reasoning over the propagation of causal quantities through graphs. To facilitate CCR eval- uation, we introduce a framework for the exhaustive assess- ment of compositional consistency: correct inference that equivalent compositions are indeed equal. We measure com- positional consistency with respect to ground truth (external validity) and concordance among the LM's responses (in- ternal consistency) (Fig. 1). We empirically demonstrate instantiations of our framework for two causal measures: the average treatment effect (ATE) and the probability of ne- cessity and sufficiency (PNS; Pearl 1999), which coincides with the ATE in certain data generating processes.
Human-like Al might also require compositional reasoning (Lake et al., 2017): the capacity recognize and synthe- size novel combinations of previously observed concepts (Xu et al., 2022). Compositionalliyy is ubiquitous in the physical world, symbolic systems, and human cognition
'Cornell Tech 2Havvard University *Microsoft Re- search Cambridge. Correspondence to: Jacqueline Maasch <maasch @ es. comu. Work done while authors J. Maasch and A. Hüyük were interns at Microsoft Rescarch Cambridge.
Proceedings of the 4nnd International Conference Machine Learning. Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).
'eee Fig. A.1 for examples.

---

Compasitional Causal Reasoning Evaluation in Language Models
1.1. Contributions
to elicit causal, compositional, and mathematical reasoning via fine-tuning (Huyük et al., 2025) or CoTT prompting (Wei et al., 2022; Jin et al., 2023; Press et al., 2023) have shown promising yet limited success. A survey of causal reason- ing benchmarks found that most suffer from critical design flaws (c.g., data contamination and inappropriate evalua- tion metries), highlighting the need for improved evaluation frameworks ( (Yang et al., 2024). To our knowledge, prior frameworks do not explicitly and systematically incorporate causal compositionality. See Appendix A for additional related works. For an in-depth comparison to prior causal reasoning evaluation frameworks, see Table A.I.
$3 A compositional view of causal reasoning in LMs. We formally express CCR as the ability of an LM to in- fer causal measure compositions (inductive reasoning) and decompositions (deductive reasoning).
$4 Taxonomy of reasoners. We propose measures of external validity and internal consistency for composi- tional consistency evaluation. To facilitate error anal- yses, we introduce taxonomy of reasoning patterns: vulid-consistent (VC), validdincconsistent (VI), invalid- consistent (IC), and invalid- inconsistent (II).
$5 Evaluation framework and task generator. We intro- duce a procedure for evaluating inductive CCR for the ATE and PNS in causal graphs with cutpoints (Alg. 1). To facilitate future work, we provide open-source code for randomly generating qualitative and quantitative CCR tasks of scalable graphical complexity. 2
Compositionality & Causality While explicitly combin- ing compositional and causal reasoning evaluation in LMs is new, the intersection of compositionality and causality already enjoys rich mathematical framework: the for- malisms and methods of graphical modeling and causal inference. A central concern of probabilistic and causal graphical modeling is factorization: the expression of com- plex multivariate distributions as products (or compositions) of local distributions, enabling efficient learning and infer- ence (Koller & Friedman, 2009). Graphs provide expres- sive representations for joint distributions, their factors, and the propagation of quantities through systems (Pearl, 1982; Shafer & Shenoy, 1990; Kschischang et al., 2001). In causal inference, the decomposition of causal effects (Avin et al., 2005; Pearl, 2014; VanderWeele, 2014; Singal & Michai- lidis, 2024) plays a central role in mediation analysis (Van- derWeele, 2016), fairness analysis (Plečko & Bareinboim, 2024), and covariate adjustment in the presence of latent variables (Pearl, 1995; Jeong et al., 2022). These traditions offer a convenient mathematical language for evaluating compositional and causal reasoning simultancously.
$6 Empirical demonstration. We deploy Alg. 1 to evalu- ate CCR in seven LM architectures, with and without chain-of- -ftthughhh (CoT) prompting. Even on a simple CCR problem, our framework revealed taxonomically distinct patterns of inconsistent and invalid reasoning, ranging from II to VC.
1.2. Related Works
LM Reasoning This work contributes to the theoretical and empirical study of compositional (Hudson & Manning, 2019; Hupkes et al., 2020: Xu et al., 2022; Ito et al., 2022; Hsieh et al., 2023), causal (Wang et al., 2023c; Du et al., 2022), mathematical (Saxton et al., 2019; Lewkowycz et al., 2022; Stolfo et al., 2023; Lu et al., 2023b), inductive (Qiu et al., 2024), and graphical reasoning (Wang et al., 2023a; He al., 2024) in AI. Compositional reasoning has been examined in various neural architectures, including trans- formers (Li et al., 2023; Lu et al.. 2023a; Dziri et al., 2023) and diffusion models (Du et al., 2023; Okawa et al., 2023; Su et al.. 2024). In this work, we focus on transformer-basee autoregressive LMs. This work introduces a new composi- tional viewpoint on consistency, an ongoing concern in LM research (Wang et al., 2023b; Li et al., 2024).
2. Preliminaries
Notation Uppercase denotes univariate random variables (e.g., Y) and bold uppercase denotes sets multivariate random variables (e.g., V), with realizations in lowercase (e.g., scalars æ;; vector values x). Models and graphs are denoted by calligraphic script (e.g., causal graph G). For X E G, we denote the parents of X by pax.
Despite some optimistic results (Kiemman al., 2023), sev- eral recent works hypothesize that current LMs are not ca- pable of true logical reasoning (Mirzadeh et al., 2025) and are merely causal parrots (Zečević et al., 2023). Counter- factual reasoning in LMs can be brittle (González & Nori, 2024), and performance on formal causal reasoning tasks can decline monotonically with task difficulty (Jin et al., 2023). Similarly, multiple works find that models struggle with compositional reasoning in vision and language, espe- cially for complex compositions (Agrawal et al., 2017; Ma et al., 2023; Ray et al., 2023; Press et al., 2023). Efforts
2.1. Causal Models
Structural causal models (SCMs; Pearl 2009) provide a con- venient coupling for our framework: (1) rich mathematical language for expressing the compositionality of causal mea- sures and (2) an intuitive visual language in the form of directed acyclic graphs (DAGs).
Definition 2.1 (Structural causal model (SCM), Pearl 2001). An SCM is a tuple M = (V, U, U.F, p(u)), where U = {U.)r is a set of exogenous variables determined by fac- tors outside M, V = [ is a set of observed en-
Project page: https: / jmaasch.a...............a..aaa -i io/cor/
2

---

Compasitional Causal Reasoning Evaluation in Language Models
dogenous variables determined by variables in U U V, F {f:)/=1 is set of structural functions such that 0/ = fi u.), and p(u) is the distribution over U.
Definition 2.5 (Identification of the PNS, Tian & Pearl 2000). Given a positive-Markovian SCM (Def. 2.2) for which Y is monotonic in X, the PNS is given as
We restrict attention to Dasitive-Markovianr SCMs whose graphical representations are DAGs.
((yx) P(yz') = PPy do(x)) - P(y do(x'))
(4)
where effects (V)x) and P(Y=)) are identifiable by Eq. 1. Note that Eq. 4 is equivalent to the ATE (Proposition B.4). We will leverage this fact for CCR evaluation in Section 5.
Definition 2.2 (Positive-Markovian SCM, Pearl 1999). An SCM is Markovian if its graphical representation is acyclic
and exogenous variables U; are mutually independent. An
SCM is positiv--Markovinn if is Markovian and ((v)) > 0 for every realization V E V
3. Compositional Causal Reasoning
Let X,Y Y E M be binary random variables. Let pax denote the realization of a parent of X. Under Def. 2.2, the causal effect of X on Y is identifiable as (Pearl, 1999)
Compositionality has been variously defined in linguisties (Haugeland, 1979; Wittgenstein, 2009), category theory (Fong & Spivak, 2019), quantum theory (Coecke, 2023), and other domains. For example, while a Schrödinger com- positional theory dictates that compositions are "greater than the sum of their parts" (Coecke, 2023), such emergent effects are not relevant in our setting. We highlight this variation to clarify that measuring compositional reasoning in Al is contingent on chosen definition of compositional- ity. We select a function based viewpoint that draws from probabilistic graphical modeling and causal inference.
P(Y do(X z))
Σ
P(y z. P(ppa) (1) pax
2.2. Causal Measures
Average Treatment Effect The most widely studied causal estimand (Imbens, 2004), the ATE measures the ef- fect of receiving treatment versus no treatment on the mean outcome over the population.
Definition 3.1. Compositionality exists when measure f can be expressed as function of measures {g. 1n22
Definition 2.3 (Average treatment effect (ATE)). Let X denote binary treatment variable and Y an outcome. We express the ATE as the following difference of expectations:
This definition is intentionally lax to capture a wide range of mathematical behavior. It allows for any function to serve as the compositional rule (e.g., addition, multiplication, etc.). As in probabilistic graphical modeling, we refer to compo- sitions/ ff as global measures and to constituent gi as local measures. These are terms of relative scale: local measure in one setting may also admit decompositions at finer granu- larity. Thus, compositionality implies scale of interest (as exemplified by the physics example in Fig. A. Def. 3.1 arises frequently in causal inference, e.g.:
ATE: == E|y do(X- = 1)] - Y do(X =)].
(2)
Probability of Necessity and Sufficiency In proposi- tional logic, we say that X is necessary for Y when Y + X, X is sufficient for Y when X = Y, and X is necessary and sufficient for Y when X < Y. Pearl (1999) introduced a probabilistic framework for reasoning over necessity and sufficiency with the probabilities of cau- sation (PrC): the probabilities of necessity (PN), sufficiency (PS), and necessity and sufficiency (PNS). In the present work, we focus the PNS.
Example 3.2 (Decomposition of total causal effects in linear SCMs, Pearl 2001). Let TE be the total effect, NDE the natural direct effect, and NIE the natural indirect effect. When causal functions are linear,
Let X and Y denote binary random variables, where X is a cause of Y. Let x and y denote the propositions or events that X = TRUE and Y = TRUE, respectively, while x' and y denote that X = FALSE and Y = FAL.SE
TE NDE + NNFE. global
(5)
local
local
Following from Definition 3.1, we define compositional
Definition 2.4 (Probability of necessity and sufficiency (PNS), Pearl 1999). The probability that x is necessary and sufficient to produce y is given as
interpretation of causal reasoning in AL.
Definition 3.3 (Compositional causal reasoning (CCR)). The ability to correctly infer (1) how local causal measures compose into global causal measures and (2) how global causal measures decompose into local causal measures, in both factual and counterfactual worlds.
PNS == P(ye, v/2)) = P(z, PN + PPS.
(3)
The PNS is point identifiable from causal effects when Y is monotonic in X: changing X from FALSE to TRUE does not induce Y to change from TRUE to FALSE.
Our definition is similar to decomposabiliiy as given by Def. 3.5 in Plečko & Bareinhoim (2024).
3

---

Compasitional Causal Reasoning Evaluation in Language Models
go
gos
formulation of case (2) defines 2px as series of questions, where f is obtained by transformation the vector of responses. We demonstrate the latter formulation in Sec. 6.
9
A. INFER g o f FROM f.g
INFER FROM g f.g
4.2. Measuring Compositional Consistency
Figure 2. (A) Inductive and (B) deductive CCR.
In this work, evaluation is performed w.n.t task-metric- model triple (T, A, for CCR task T, some error metric 6, and model A (Schaeffer et al., 2023).
Definition 3.3 encompasses reasoning over both composi- tions and decompositions, which we can disaggregate into inductive and deductive reasoning (Fig. 2).
Definition 4.1 (CCR task). Let M denote the SCM repre- senting the problem structure and y the estimand of interest. Let Q = { where causal queries {px } cor- respond to the n cause-effect pairs of interest in M. We define the corresponding CCR task as the tuple
Definition 3.4 (Inductive CCR). The ability to reason about the composition of causal measures. With respect to Def. 3.1, this corresponds to inferring composition f given knowledge of local measures {g:]
T == (p,, M, 2).
(8)
Definition 3.5 (Deductive CCR). The ability to reason about the decomposition of causal measures. With respect to Def. 3.1, this corresponds to inferring local measure 9j given knowledge of composition f and measures {9: )== 9y-
Note that multiple sp may be identifiable for M, in- finitely many Q can map (e., M), and success of A on (y, M, Q) does not imply success on (, Q*), Mp,M', M', @³), M,, Q"), etc. The appropriate choice of A will be context-dependent (e.g., mean squared error, etc.).
With respect to Example 3.2, inductive CCR entails infer- ring TE from NDE and NIE, while deductive CCR entails inferring NDE from TE and NIE (etc.).
We now explore a prime benefit of compositional perspec- tives on reasoning: the ease of introducing notions of consis- tency. The following concept of consistency can be applied to any form of reasoning, not only causal.
4. Compositional Consistency Evaluation
4.1. Obtaining Causal Estimates
Definition 4.2 (Compositional consistency). Reasoning is compositionally consistent when theoretically equivalent compositions are inferred to be equal.
Let A denote a model (e.g., an LMM, & the set of all causal measures, and o E & measure of interest that we seek to evaluate for variables V in SCM M (e.g., the ATE). Let fx be a causal query about the value of &p with respect to (W.r.t.) X V. We denote the true value of Px by Fx
Under the umbrella of compositional consistency, we can quantify CCR in many ways. For example, model that succeeds at task T should not be prone to false negatives (i.e., failing to infer compositionality when it exists) nor false positives (i.c.. hallucinating compositionality when it does not exist). In this work, we emphasize the external validity and internal consistency of CCR.
Causal estimates Fx can be obtained by various means, including (I) explicitly querying A for the value of yxx requiring A to directly perform formal causal inference (as in Jin et al. 2023); or (2) implicitly querying A for the value of Pxx where responses are used to perform formal causal inference downstream of A (as in González & Nori 2024; Hüyük et al. 2025). In either case, each causal query is encoded in a question template (or series of templates)
Definition 4.3 (External validity). Reasoning is externally valid when estimates are equivalent to ground truth, up to some error ó:
Qpp == S)
(6)
0(p2t, Px) S 5.
(9)
where S is some surface form that expresses accessory de- tails (c.g.. the background of math word problem) (Stolfo et al., 2023). 2px is expressed in form comprehensible to A (e.g., text, image, etc.) and can either directly state px (as in case 1) or logically imply it (as in case 2). Causal estimates are then obtained by
In Example 3.2, externally valid reasoning for cause-effect pair {X, YJ entails that the following are below some threshold: 0 TExy), O(NDEXY, NDExy), @(TExy, NDExx + NIExy), etc.
Definition 4.4 (Internal consistency). Reasoning is inter- nally consistent when quantities that are theoretically equiv- alent are inferred to be equivalent, up to some error ò:
xx = ((A(2))))
where f is some transformation of the raw response from A. In case (1), f might be the identity function. One possible
Px xx = 0(2x. Fx) (x)st
S 6.
(10)
4

---

Compasitional Causal Reasoning Evaluation in Language Models
CONSISTENT
INCONSISTENT
INVALID VALID
(2 12) ( 12)
A: TRUE
DIVISIBILITY RULE:
Q: (6 12)
A: TRUE.
a number re is divisible by and
3, then it is divisible by 6. That is, (2 | n)) A (3 n) = (6 n).
Q: (2 A 12)
A: FALSE.
Q: (2 12) A (3 12)
A: FALSE.
Q: (6 12)?
A: FALSE.
Q: (6 12)
A: TRUE
Figure VC, VI, IC, and II reasoning for a logic problem applying divisibility rules. For such a prohlem, we see that VT reasoning impossible. However, all four types of reasoners arise in the probabilistic setting in which evaluate LMs.
Here, estimates are compared each other rather than ground truth. In Example 3.2, internally consistent rea- soning requires that ((TExy, NN + -xy) s 8. Definition 4.5 (Completeness). Given threshold 8, reason- ing is complete w.nt. (e, M, 2) when the following holds:
Section 5 suggests one possible procedure for assessing com- positional consistency in causal reasoning (Alg. 1), based compositional properties of the ATE and PNS in graphs with cutpoints. Section 6 provides an empirical illustration in LMs, demonstrating proof of viability.
V fxx : 0(ppx: Pxx S and
(11)
5. Inductive CCR Evaluation for the PrC
V (xxxPxx : Fx = Ppx 0(px, xx) 8.
(12)
5.1. PNS Composition Across Graph Components
The PNS boasts several convenient properties for reasoning evaluation: (1) variables of interest are binary and probabili- ties are bounded by 0 and 1; (2) translating PrC queries to text prompts designed to elicit logical, mathematical, proba- bilistic, and/or causal reasoning is relatively straightforward (González & Nori, 2024; Hüyük et al., 2025); and (3) the PNS and ATE coineide under certain conditions (Proposition B.4), and thus share convenient compositional properties.
Taxonomy of Reasoners Following from Defs. 4.3 and 4.4, we delineate four categories of reasoners for task T:
1. Valid-consistent (VC).
3. Invalid-consistent (IC).
2.
(VI).
4. innaiid-icconsistentt (II).
Following from this taxonomy, there are three distinct pro- files of incomplete CCR (IC, VI, II) but only profile that can achieve completeness (VC). To illustrate, consider the logic problem given by Fig. 3:
Assumptions: DAGs with Cutpoints In this work, derive compositional forms for the PrC in graphs with cut- points. A cutpoint, cut vertex. or articulation point is any node contained in multiple biconnected components (BCCs): maximal biconnected subgraphs induced by a partition of edges, such that two edges are the same partition if and only if they share a common simple cycle (Westbrook & Tar- jan, 1992). Thus, removing cutpoint disconnects the graph. For ease of exposition, we demonstrate our framework on SCMs whose causal DAGs GxY contain the following: (A1) only one root node X (i.e., the cause of interest), (A2) only one leaf node Y (i.c., the effect of interest), (A3) at least one cutpoint, and (A4) no unobserved confounders for cause- effect pairs of interest. Thus, we assume models with a single "source" node whose causal influence follows mul- tiple indirect pathways to single "sink" node. See Figs, 4, B. and D. for DAGs that satisfy A1-A4. Note that alternative formulations of this framework could relax these assumptions (e.g., Appendix C explores violations of A3).
Example 4.6. Consider a logic problem applying the divisi- bility rule [22 n) [3 m) = (6 n)] where 12. A reasoner can respond TRUE or FALSSE each query.
1. VC: Each individual query is answered correctly (ex- ternally valid) and the logical implication TRUE = TRUE is correct (internally consistent).
2. IC: Individual queries are answered incorrectly (exter-
nally invalid), but the logical implication FALSE = FALSE is correct (internally consistent).
3. II: One individual response is wrong (externalyy in- valid) and the logical implication FALSE = TRUB is incorrect (internally inconsistent).
We see that only VC, IC, and II arise in this logic setting. However, LM evaluation is a probabilistic setting where errors are thresholded. Thus, all four types of reasoners can arise LM evaluation, illustrated in Section 6.
PNS Compositionality Though the PN and PS have proven useful for AI reasoning evaluation (González & Nori, 2024; Hüyük et al., 2025), we prove in Appendix B.2 that their composition across BCCs is complex. However, both
This conceptual introduction has left implementation details intentionally vague. In theory, the framework proposed here can be implemented for any com- positional formulae causal inference (e.g., Example 3.2).
Implementation
5

---

Compasitional Causal Reasoning Evaluation in Language Models
Algorithm 1 Inductive CCR eval. in graphs with cutpoints
E
Input: CCT Cxy; estimates {G-}, true values } for M, º) metric (e.g., relative absolute error)
Output: Reasoning errors 7); Y
B
F
Assumptions: & composes according an associative function over the BCCs of causal graph Gxy.
ORIGINAL DAG Gxy
Compute quantity-wiss errors.
hog
1: for all node pairs [Ra, Lizi} in Cxy do
9
h
2: MR,L, + L, Fn,L,
Di External validity,
D
Y
Compute inductive reasoning errors.
gof
3: for all paths i from X to Y in Cxy do
hagof
4: Get composition 6i for path i from knowledge of edges j E
B. CCT Cxy
Figure 4. A running example for our framework. (A) DAG with BCCs (violet, pink, maroon) and cutpoints C, D. (B) CCT Cxy modeling all commutative CCR paths from X to Y. Here, f = PNSxc. g == PNScD, = PNSDY, g f == PNSxD. gof = PNSxy, etc., and composition is multiplicative.
5: 6, - 0--x(p
D External validity.
6: + f(xxy, f f
Di Internal consistency.
return 17, 7
the PNS and ATE display simple compositional form der the following conditions. When the DAG Gxy of linear SCM satisfies A A1-A4, the ATE for the root and leaf of 9xv is a product of the ATE values for the root and leaf of each BCC. This follows from the product-of-coefficee heuristic used classical path-tracing and mediation anal- ysis (Alwin& Hauser 1975; see Appendix for a worked example). This is not guaranteed for the ATE in nonlin- ear data generating processes. Conveniently, we prove in Appendix B.2 that this property extends to the PNS under monotonicity even when causal functions are nonlinear.
Global
PNSxy
Local
PNSxe, PNSXD. PNScD, PNScY, PNSDY
Composition
PNSxc PNSXDPNSDY. PNSxcPNScnPNSox
Table 1. Quantities of interest for inductive CCR over the PNS for Fig. 4. All compositions are equivalent to the global quantity.
Construct a causal chain with nodes XUSUY, where S is topological ordering of the cutpoints in GxY (e.g., X - C D Y for Fig. 4A).
Theorem 5.1 (PNS composition across BCCs). Given an SCM where DAG Gxy satisfies AI-A4 and Y is monotonic in X, the PNS for root X and leaf Y composes as
2. Add a directed edge between any non-adjacent nodes in the chain to yield a complete graph where all directed paths point from root X to leaf Y (e.g., Fig. 4B).
PNSXY
II
PNS
(13)
ERC
CCT' abstract away complexity in our original causal graph by collapsing BCCs into single edges. This allows for evalu- ation on arbitrarily complex DAGs with cutpoints as if they were simply directed chains. This abstraction simplifies the problem representation by (1) marginalizing out variables that are unnecessary for valid causal inference in our setting and (2) visualizing pathways of composition. As demon- strated in Section 5.2, CCTs can be leveraged as design tool for formulating reasoning tasks. As illustrated in Sec- tion 6.2, they can also serve as an interpretable visualization tool for graphically representing CCR errors.
where C is the set of all BCCs in Gxv and RG, Li are the root and leaf of BCC C respectively.
Adjacent BCCs in GxY can be treated as single component and Theorem 5.1 still holds (Fig. B.I). In Appendix E, we illustrate Theorem 5.1 with simulations of inductive and deductive CCR for the ATE and PNS. To facilitate evaluation for compositional forms similar to Thm. 5.1, we introduce new graphical tool for visualizing the flow of causal information through BCCs: the commutative cut tree.
Definition 5.2 (Commutative cut tree (CCT)), Let GXy be causal graph satisfying A A1-A4 and let y be causal measure that composes according to an associative function over BCCs (e.g., multiplication as in Theorem 5.1). CCT Cxy a transformation of 9x that models all CCR pathways from root X to leaf Y for measure sp. C is obtained by a two-step transformation of 9xY:
5.2. Inductive CCR as Commutative Reasoning
We now propose means of systematically evaluating CCR that leverages CCTs and Theorem 5.1 (Alg. 1). applicable to the ATE under lincarity and the PNS under monotonicity. We can view CCR as reasoning that Cxy is commutative:
6

---

Compasitional Causal Reasoning Evaluation in Language Models
1.00
Internal consistency RAE
0.75
0.50
0.25
1.3
2.0
External validity RAE
+
Phi-3
Llama 2
Llama
Llama 3.1
Llama 3.1 CoT
Llama Math
Llama 3.1 Math CoT
* CPr--o
CoT
Xol
Xol ol CoT
Figure 5. For PNS compositions (n 1000 estimates per quantity model), we compare RAE w.r.t ground truth (external validity) and PNSxy (internal consistency) to visualize our four reasoning quadrants (VI/C in yellow; VC in green; II in white). Dotted lines are error thresholds =RAE=0.] 0.1). Models listed by increasing size (Table G.I). The x-axis is truncated; see Fig. G.11 for the full distribution.
every possible composition (corresponding to the paths from X Y in Cxy) should be equivalent each other and to ground truth, up some error. If ground truth values are unavailable, Alg. 1 can be run for internal consistency only. See Appendix D for time complexity.
Factual prompt
Xinyu, Ara, Becca, Celine, Daphne, Emma, Fox, party,
and Yasmin is going to distribute candies. Xinyu will be happy if she gets at least candies. Ara will be happy if Xinyu is happy if he gats at least candies. Becca will be happy if... After distributing
going
Running Example: Intuition for Algorithm For illus- tration, we walk through Alg. where &o is the PNS and DAG GxY is structured according to Fig. 4A. We continue to employ the notation defined in Section 4. In Section 6, implement this same walk-through for LM evaluation.
candies, Xinyu gets 4, Ara gets 6, Becca gets 5, Celine gets 10, Daphne gats Emma gets 1, Fox gets 4, and Yasmin gots 3. Is Coline happy? Be as concise possible.
Counterfactual prompt
The assumption stated in Alg. 1 is satisfied, the PNS composes over BCCs by multiplication (Theorem 5.1). To begin, we must determine the quantities of interest for as- sessing CCR over 9XY (Table 1). To do this, we first obtain the corresponding CCT Cxy. The global quantity inter- will always be the PNS for the root and leaf (PNSxy) ),),, corresponding to edge X + Y in CxY. Local quantities of interest will be the PNS for every remaining pair of nodes in CxY, resulting in (*) quantities (e.g., PNScy) ). Fi- nally, we obtain compositions by considering every distinct indirect path from X to Y in CxY. For each such path, the composition of interest is the product of the PNS val- ues for each cause-effect pair on that path. this case, there are three indirect paths from X to Y: X + C Y, X + D Y. and X + C + D + Y. Thus, our com- positions of interest are and PNS (Fig. G.1).
Now, suppose that Xinyu happy regardless the candy distribution. With this assumption, Celine happy?
possible.
Figure 6. Factual and counterfactual prompt excerpts. For example, we can obtain PNSxc x by simulating potential outcomes = TRUE, X FALSE (Xinyu is or is happy) and then querying for the value of C (Celine is or is not happy). Analogously, we obtain PNSpy with interventions on D (Daphne's happiness) and queries on (Yasmin's 's happiness), etc.
and internal consistency of each composition. Each G7 corresponds one distinct path from X to Y in Cxv, and is obtained by taking the product of PNS estimates for each cause-effect pair on the path. For example, we estimate PNSxcPNNSc PNSxc times PNScy. Finally, Alg. I returns all external validity and internal consistency errors.
PNSxpPNSDy.
Input to Alg. 1 is the set of estimates PNS- f(A(( for each quantity enumerated in Table 1, as well as their ground truth values (if available), Lines 1-2 compute 0 for the external validity of the PNS for each cause-effect pair in Cxy. Lines 3-6 compute 9 for the external validity
Automated Task Generation We provide open-source code for generating random CCR tasks analogous to our running example. To facilitate future benchmarking, genera- tors allow the user to control aspects of task complexity (e.g.. DAG size). See Appendix F for additional functionality.
7

---

Compasitional Causal Reasoning Evaluation in Language Models
Llama 3.1
Llama 3.1 Math
GPT-4o
o1
100
Percent valid
15
KOCDaD
xracy XDaD
XD
XDOY
x0
xy SOCDaD
X0x3
XDDY
16
10
xx
CoT
No CoT
Figure 7. Percent of PNS estimates 1000) that are extemally valid for CoT vs non-Col prompting. PNS are denoted by cause-effect
pair (e.g.. PNSxccPNSer as XC CY). Reasoning was externally valid if 2 90% of estimates had RAE & 0.1 (red dashed line).
6. Empirical Demonstration in LMs
corresponding Boolean values using Llama 3. The resulting Boolean vectors were treated as samples from observational and interventional distributions, which were then used to compute PNS estimates. For each quantity of interest in T, 1000 PNS estimates were obtained per model. Reasoning was considered externaly valid or internally consistent for quantity if 2 90% of the 1000 estimates had RAE I 0.1 (threshold chosen prior to analysis). Reasoning was deemed near-valid if > 75% of estimates met the threshold.
We demonstrate Alg. 1 for inductive CCR for the PNS with an illustrative math problem graphically represented by Fig- 4. See Appendix G for full experimental details. Experi- ments were designed to demonstrate the correct implemen- tation of this framework, not to thoroughly benchmark CCR in current LMs. Thus, we focus deeply on one toy problem to highlight the insights that our framework can provide.
6.1. Experimental Design
6.2. Experimental Results & Discussion
Models Inference was performed using seven architectures:
General Results Fig. G.12 plots all PNS distributions. Figs. 5 and G.11 jointly represent internal consistency and external validity RAE for our three compositions of inter- est, with quadrants representing disparate reasoning profiles: VC, VI, IC, II. Figs. 7, G.13, and G. 14 show the percent of PNS estimates that were externally valid for each model. Er- rors did not decrease monotonically with increasing model size nor Llama version (Figs. G.16, G.17). As demonstrated in Fig. 8, CCTs allow us to see compositional reasoning errors in graphical representation. Only the responses from ol correctly implied that CCT CXY was commutative.
Llama (Touvron et al., 2023), 3 (Dubey et al., 2024), 3.1 (Dubey et al., 2024), and 3.1 Math (Toshniwal al., 2024); Phi-3-Mini (Abdin et al., 2024): GPT-4o (Achiam et al., 2023); and ol (Jaech et al., 2024) (Table G.I). All LMs were fine-tuned for dialogue. Llama 3.1 Math was also fine-tuned for math, and ol is a designated reasoning model.
CCR Task Let M be an SCM represented by the DAG in Fig. 4. Variables V, U E M are binary and causal func- tions are logical or (Eq. G. Quantities of interest are in Table 1. We defined CCR task
(PNS, M, Q) where prompts in Q were based on the CandyParty math word problem (Figs. 6, G.2; González & Nori 2024). Approxi- mation errors were obtained by computing relative absolute errors (RAE; Eq. G.3). Thus, we define task-metric-model triple (T, RAE, A) for each model A. Llama 3.1, Llama 3.1 Math, GPT-4o, and ol were also presented with a CoT wrapper for Q to assess impacts of prompt formulation on CCR elicitation. The CoT wrapper for our original prompt template presented the model with two worked examples: one factual and one counterfactual (Fig. G.3).
Taxonomy of Reasoners Results for task T revealed three taxonomically distinet error patterns: VC (ol), near-VI (GPT-40 with CoT), and II (all others). Only ol placed mass in the VC reasoning quadrant in Fig. 5. All mod- els besides ol were 0% valid for global quantity PNSxY, with and without CoT (Figs. 7, G.13, G.14). Without COT,, Phi-3, Llama 2, Llama 3, and GPT-4o failed to exceed 3% validity for any quantity. II reasoners often displayed high variance (Fig. 5). All error distributions for CoT were sig- nificantly different from non-CoT, except for ol on PNSxY (Wilcoxon test, a 0.05). GPT-4o displayed the greatest sensitivity to prompt formulation, going from II to near- VI. With COT, 75.1% 1% of estimates from GPT-4o were near-valid and mean external validity on local quantities and compo- sitions exceeded 77% and 95%, respectively, but failure on PNSxy x revealed internally inconsistent reasoning.
LMs as Counterfactual Data Simulators To assess CCR at the counterfactual rung of Pearl's Causal Hierarchy, we treat LMs as counterfactual data simulators (González & Nori, 2024; Hüyüü al., 2025). Instead of directly prompt- ing the LM to perform formal causal inference (as in Jin et al. 2023), series of factual and counterfactual "yes/no" ques- tions were submitted to the model (e.g., Figs. 6, G.4, G.5). Natural language responses were then converted to their
CCR Failure Modes Preliminary error analyses included manual assessment of LM text responses. For Llama mod-
8

---

Compasitional Causal Reasoning Evaluation in Language Models
els. poor numeracy explained some errors (Figs. G.4, G.5). GPT-4o with CoT displayed several kinds of faulty reason- ing for Xr (Figs. G.7. G.8) and PNSpy (Figs. G.9, G. These included (1) failure correctly extract causal relations (e.g., true causal parents were missing from the stated parent set; Fig. G.10), (2) incorrect logic despite cor- rect relation extraction (c.g., a clause did not logically follow from the preceding clause; Fig. G.9), and (3) a truncated reasoning process, where the model expressed its logic up to a certain point but ignored the remainder of the underlying causal graph without verbal explanation (e.g., Fig. G.8).
9
Rogoz
hogar
A. LLAMA 3.1 MATH
LLAMA 3.1 MATH CUT
hog o
GPT-40
D. GPT-40 COT
Errors Increase With Mediation For all models except GPT-40 with CoT and ol, mean RAE increased monoton- ically with shortest path distance and total mediating vari- ables between cause and effect (Figs. 9, G.18). Nevertheless, mean RAE for GPT-4o with CoT was > 10x higher for six mediators than for three mediators. Increasing RAE may be explained by error propagation. Additionally, preliminary analyses suggest that models can "lose the train of thought" along long causal paths (Fig. G.6). These results are in line with prior findings that reasoning performance can diminish with task complexity (Agrawal et al., 2017; Ma et al., 2023; Ray et al., 2023; Press et al., 2023; Jin et al., 2023).
god
n naga!
h hogof !
ol
n
NEAR-VI
VC (COMPLETE)
Figure 8, Visualizing (in)complete CCR, where only ol reasons that Cxy commutes. Black edges are externally valid global and local quantities, while gray are invalid. Each path from X to Y that is highlighted in blue represents externaly valid composition. Nodes are black when all paths passing through them are valid.
The Value of Internal Consistency What does internal consistency tell us that external validity alone cannot? Mea- suring both reveals not only if the reasoner is wrong, but also offers insights into how it is wrong. We take VI and IC reasoning as cases in point. First consider GPT-4o with CoT, where external validity alone suggests a degree of successful causal reasoning on task T. However, compositional incon- sistencies paint a fuller picture: (1) only two of four paths from X to Y in Cxy commuted (Fig. 8) and (2) GPT-4o was more performant when asked about proximal relation- ships than distant cause-effect pairs (e.g.. {X, YJ), where the latter required greater degree of compositional reason- ing over indirect relations that were never directly stated in the context prompt (Figs. 9; G. G.7-G.10). This failure mode is akin to a student who passes math quiz by relying more on memorization than true synthesis, and fails recog- nize equivalence between multiple formulations of the same problem. Now consider the IC reasoner. This case can arise when a model is consistently biased. Say model A incor- rectly infers that PNŠXY = 2 PNSxY, but compositions are equally biased: P.N PNScy = 2 x PNSXY, etc. Though these estimates are externally invalid, this overlooks the insight that A correctly reasoned that the quantities of interest are equivalent. Thus, lumping with IC reasoners and VI with VC reasoners limits informativeness.
Phi-3
Llama
-Laaaa Mati Call' Xo]
15
E.ama
Limma 31 CoT
*GPT-40
Xol CvT
Libma Math GPT-46 GST
Mean RAE
10:
Total mediators between cause and effect
Figure 9. Red dashed line denotes the external validity cutoff (RAE
= 0.1). with standard deviations in shaded regions.
and PNS in causal graphs with cutpoints (Alg. 1). Empiries are limited to one illustrative task proof of viability. Re- sults demonstrate how this framework can provide richer picture of causal reasoning than measuring quantity-wise external validity alone, as is conventionally done. Complete reasoning by ol validates the correctness of our implemen- tation, though future work should emphasize significantly greater task complexity. Open source and smaller models did notably worse on this task than GPT-4o with CoT and ol, suggesting that even low-complexity CCR is an area on which significant progress can still be made for an impor- tant class of LMs (smaller and/or open). As this work only considers the ATE and PNS under Theorem 5.1, extensions could consider other estimands and compositional forms.
6.3. Limitations& & Future Directions
This work presents a conceptual foundation for CCR eval- uation in LMs. We instantiate this framework for the ATE

---

Compasitional Causal Reasoning Evaluation in Language Models
Impact Statement
Cai, H., Wang, Y., Jordan, M., and Song, R. On Leam- ing Necessary and Sufficient Causal Graphs, Novem- ber 2023. URL http: htpp:/aaae /arxiv. org/ www./rrg//////o/eeeeooo//eeeee// abs/2301. 12389. urivv:2301.23389 [cs, stat].
The potential for reasoning emergence in Al has broad sci- entific, economic, and social implications for global society. These implications include matters of safety and fairness. This work aims promote the rigorous measurement of reasoning behaviors in LMs. Success on CCR tasks such as those described in this work is necessary but not sufficient for demonstrating that LMs can reason. We encourage cau- tious interpretation of results, particularly with respect to claims of reasoning in models that are deployed safety- critical contexts.
Coecke, B. Compositionality as we see it, everywhere around us. In The Quantum-Like Revolution: Festschrift for Andrei Khrennikov, pp. 247-267. Springer, 2023.
De Toffoli, S. 'Chasing' the diagram - the use of visual- izations in algebraic reasoning. The Review of Symbolic Logic, 10(1):158-186) 2017.
Acknowledgments
Do, Q. Chan, Y. S., and Roth, D. Minimally super- vised event causality identification. In Barzilay, R. and Johnson, M. (eds.), Proceedings of the 2011 Confer- ence on Empirical Methods in Natural Language Pro- cessing, pp. 294-303, Edinburgh, Scotland, UK., July 2011. Association for Computational Linguistics. URL https: /acl:////cclee acl ant ogy wttg
Author J. Maasch acknowledges the US National Science Foundation Graduate Research Fellowship under Grant No.
DGE-2139899.
References
Abdin, M., Jacobs, S. A., Awan, A. A., Aneja, J., Awadallah, A., Awadalla, H., Bach, N., Bahree, A., Bakhtiari, A., Behl, H., al. Phi-3 technical report: A highly capable language model locally on your phone. arXiv preprint arXiv:2404. 14219, 2024.
Du, L.,, Ding, X., Xiong, K., Liu, T., and Qin, B. e-care: a new dataset for exploring explainable causal reasoning. Proceedings of the 60th Annual Meeting of the Asso- ciation for Computational Linguistics (Volume 1: Long Papers), pp. 432-446, 2022.
Achiam, J., Adler, S., Agarwal, S., Ahmad, L., Akkaya, I., Aleman, F. L., Almeida, D., Altenschmidt, J., Altman, S., Anadkat, S., et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08774, 2023.
Du, Y., Durkan, C., Strudel, R., Tenenbaum, J. B., Diele- man, S., Fergus, R., Sohl-Dickstein, J., Doucet, A., and Grathwohl, W. S. Reduce, reuse, recycle: Compositional generation with energy-based diffusion models and meme. In International Conference on Machine Learning, pp. 8489 8510. PMLR, 2023.
Agrawal, A., Kembhavi, A., Batra, D., and Parikh, D. C-vqa: compositional split of the visual question answering (vqa) vl. dataset. arXiv preprint arXiv: 1eo:::704..2444 2017.
Dubey, A., Jauhri, A., Pandey, A., Kadian, A., Al-Dahle, A., Letman, A., Mathur, A., Schelten, A., Yang, A., Fan, A., et al. The llama 3 herd of models. arXiv preprint arXi::2407.21788 2477.21787 2024.
Alwin, D. F. and Hauser, R. M. The decomposition of effects in path analysis. American sociological review, pp. 37-47, 1975.
Dunietz, J., Levin, L., and Carbonell, J. The BECauSE corpus 2.0: Annotating causality and overlapping rela- tions. In Schneider, N. and Xue, N. (eds.), Proceedings of the 11th Linguistic Annotation Workshop, pp. 104, Valencia, Spain, April 2017. Association for Computa- tional Linguistics. doi: 10. 10.18653//1/W17-008 URL https://aclaaaeeeeeeee http:///aaaaathoooacooooo
Avin, C., Shpitser, I., and Pearl, J. Identifiability of path- specific effects. In ISCAI International Joint Conference on Artificial Intelligence, pp. 357-363, 2005.
Bareinboim, E., Correa, J. D., lbeling, D., and Icard, T. On pearl's hierarchy and the foundations of causal inference. In Probabilistic and Causal Inference: The Works of Judea Pearl, pp. 507-556. Association for Computing Machinery, 2022.
org -
Dziri, N., Lu, X., Sclar, M., Li, X. L., Jiang, L., Lin, B. Y., Welleck, S., West, P., Bhagavatula, C., Le Bras, R., et al. Faith and fate: Limits of transformers on compositionality. Advances in Neural Information Processing Systems, 36. 2023.
Bhagavatula, C., Le Bras, R., Malaviya, C., Sakaguchi, K., Holtzman, A., Rashkin, H., Downey, D., Yih, W.-t,, and Choi, Y. Abductive commonsense reasoning. In International Conference on Learning Representations, 2020.
Fong, B. and Spivak, D. I. An invitation to applied category theory: seven sketches in compositionallit. Cambridge University Press, 2019.
10

---

Compasitional Causal Reasoning Evaluation in Language Models
Frankland, S. M. and Greene, J. D. Concepts and compo- sitionality: in search of the brain's language of thought. Annual review of psychology, 71(1):273-303, 2020.
Hsieh, C.-Y., Zhang, J., Ma, Z., Kembhavi, A., and Kr- ishna, R. Sugarcrepe: Fixing hackable benchmarks for vision-language compositionality. Advances in neural information processing systems, 37, 2023.
Frohberg. J. and Binder, CRASS: novel data set and benchmark to test counterfactual reasoning of large language models. In Calzolari, N., Béchet, F., Blache, P., Choukri, K., Cieri, C., Declerck, T., Goggi, S., Isa- hara, H., Maegaard, B., Mariani, J., Mazo, H., Odijk, J., and Piperidis, S. (eds.), Proceedings of the Thir- teenth Language Resources and Evaluation Conference, pp. 2126-2140, Marseille, France, June 2022. Euro- pean Language Resources Association. URL https: //aclanthology wrg. 1rec- 2229/.
Huang. J. and Chang, K. C.-C. Towards reasoning in large language models: A survey. In The 61st Annual Meeting Of The Association For Computational Linguistics, 2023,
Hudson, D. A. and Manning, C. D. Gqa: A new dataset for real-world visual reasoning and compositional question answering. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pp. 6700- 6709, 2019.
Hupkes, D., Dankers, V., Mul, M., and Bruni, Compo- sitionality decomposed: How do neural networks gen- eralise? Journal of Artificial Intelligence Research, 67: 757-795, 2020.
Goddu, M. K. and Gopnik, A. The development of human causal learning and reasoning. Nature Reviews Psychol- ogy, pp. 1-21, 2024.
González, and Nori, A. V. Does reasoning emerge? ex- amining the probabilities of causation in large language models. In Advances in Neural Information Processing Systems, 2024.
Hüyük, A., Xu, X., Maasch, J., Nori, A. V., and González, J. Reasoning elicitation in language models via counterfac- tual feedback. In Proceedings the 13th International Conference Learning Representations (ICLR), 2025. URL wwt.https:/// awlaaaiivvv... abs 241 0 03767.
the
Gordon, A., Kozareva, Z., and Roemmele, M. SemEval- 2012 task 7: Choice of plausible alternatives: An eval- uation of commonsense causal reasoning. In Agirre, E., Bos, J., Diab, M., Manandhar, S., Marton, Y., and Yuret, D. (eds.), *SEM 2012: The First Joint Confer- ence on Lexical and Computational Semantics Vol- ume 1: Proceedings of the main conference and the shared task, and Volume 2: Proceedings of the Sixth International Workshop on Semantic Evaluation (Se- mEval 2012), pp. 394-398, Montréal, Canada, 7-8 June 2012. Association for Computational Linguisties. URL https: / aaclannnannnooyooc.caaca orr.or//////. 1t2-/112112222
Imai, K., Keele, L., and Tingley, D. A general approach to causal mediation analysis. Psychological methods, 15(4): 309, 2010.
Imbens, G. W. Nonparametric estimation of average treat- ment effects under exogeneity: A review. Review of Economics and statistics, 86(1):4-29) 2004.
Ito, T. Klinger, T., Schultz, D., Murray, J., Cole, M., and Rigotti, M. Compositional generalization through ab- stract representations in human and artificial neural net- works. Advances in Neural Information Processing Sys- tems, 35:32225-322322 2022.
Haugeland, Understanding natural language. The Journal of Philosophy, 76(1 (6(11):619-632. 1979.
Jaech, A., Kalai, A., Lerer, A., Richardson, A., El-Kishky, A., Low, A., Helyar, A., Madry, A., Beutel, A., Car- ney, A., et Openai ol system card. arXiv preprint 24 2024.
He, X., Tian, Y., Sun, Y., Chawla, N. V., Laurent, T., LeCun, Y., Bresson, X., and Hooi, B. G-retriever: Retrieval augmented generation for textual graph un- derstanding and question answering. arXiv preprint 0oXi::2402.066300. 2024.
Jeong, H., Tian, J., and Bareinboim, E. Finding and listing front-door adjustment sets. Advances in Neural Informa- tion Processing Systems, 171-3337533, 2022.
Hendrickx, I., Kim, S. N., Kozareva, Z., Nakov, P., Ó Séaghdha, D., Padó, S., Pennacchiotti, M.. Romano, L., and Szpakowiez, S. SemEval-2010 task 8: Multi- way classification of semantic relations between pairs of nominals. In Erk, K. and Strapparava, C. (eds.), Proceedings of the 5th International Workshop on Se- mantic Evaluation, pp. 33-38, Uppsala, Sweden, July 2010. Association for Computational Linguistics. URL https: http///claaaaaa anthhalootoyooo wwgg 00006//
Jin, Z., Chen, Y., Leeb, F., Gresele, L., Kamal, O.,, Zhiheng, L., Blin, K., Adauto, F. G., Kleiman Kleiman-Weiner, M., Sachan, M., et al. Cladder: Assessing causal reasoning lan- guage models. In Thirty-sevnnth conference on neural information processing systems, 2023.
Jin, Z., Liu, J., Lyu, Z., Poff, S., Sachan, M., Mihalcea, R., Diab, M. T., and Schölkopf. B. Can large language models infer causation from correlation? In ICLR, 2024.

---

Compasitional Causal Reasoning Evaluation in Language Models
Johnson, J., Hariharan, B., Van Der Maaten, L.., Fei-Fei, L.,, Lawrence Zitnick, C.,, and Girshick, R. Clevr: A diagnostic dataset for compositional language and ele- mentary visual reasoning. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 2901-2910, 2017.
Li, Y., Sreenivasan, K., Giannou, A., Papailiopoulos, D., and Oymak, S. Dissecting chain-of-thought Compositional ity through in-context filtering and learning. Advances in Neural Information Processing Systems, 36, 2023.
Lu, P., Peng, B., Cheng, H., Galley, M., Chang, K.-W., Wu, Y. N., Zhu, S.-C., and Gao, J. Chameleon: Plug-and- play compositional reasoning with large language models. Advances in Neural Information Processing Systems, 36, 2023a.
KIcıman, E., Ness, R., Sharma, A., and Tan, C. Causal reasoning and large language models: Opening a new frontier for causality. arXiv preprint arXiv:2305.00050, 2023.
Lu, P., Qiu, L., Yu, W., Welleck, S., and Chang, K.- W. A survey of deep learning for mathematical rea- soning. In Rogers, A., Boyd-Grabee J., and Okazaki, N. (eds.), Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 14605-14631, Toronto, Canada, July 2023b. Association for Computational Linguistics. URL https:
Koller, D. and Friedman, N. Probabilistic graphical models: principles and techniques. MIT press, 2009.
Kschischang, F. R., Frey, B. J., and Loeliger, H.-A Factor graphs and the sum-product algorithm. IEEE Transac- fions information theory, 47(2))498-515 2001.
doi: 10. 10.16655w112223..d1-oon..1.11 / //aclanthooooeeeoooooooooo wwt.:/oooooyyyy.o22o3/./ wwt.oo/gg/////o acl- -ong. 817.
Lake, B. M. and Baroni, M. Human- systematic general- ization through a meta-learning neural network. Nature, 623(7985): (11-985):111-1221 121,,111,22122 2023.
Ma, Z., Hong, J., Gul, M. O., Gandhi, M., Gao, I., and Kr- ishna, R. Crepe: Can Dision-languee foundation models reason compositionally? In Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition. pp. 10910 10910-10921, 2023.
Lake, B. M., Ullman, T. D.. Tenenbaum, J. B., and Gersh- man, S. J. Building machines that learn and think like people. Behavioral and brain sciences, 40:e253, 2017.
MacKinnon, D. Introduction to statistical mediation analy-
Lal, Y. K., Chambers, N., Mooney, R., and Balasubrama- nian, N. TellMeWhy: A dataset for answering why- questions in narratives. In Zong, C., Xia, F., Li, W., and Navigli, R. (eds.), Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pp. 596- 610, Online, August 2021. Association for Computa- tional Linguistics. doi:
sis. Routledge, 2012.
Maiti, A., Plecko, D., and Bareinboim, E. Counterfactual identification under monotonicity constraints. In The 39th Annual AAAI Conference Artificial Intelligence, 2025.
Mialon, G., Dessi, R., Lomeli, M., Nalmpantis, C., Pa- sunuru, R., Raileanu, R., Rozière, B.. Schick, T., Dwivedi- Yu, J., Celikyilmaz, A., et al. Augmented language mod- els: survey. arXiv preprint arXiv:2302.0788. 2023.
53. URL https: // htt///aaaaaalllaaeeeeeeeeaa orgp://y..r.g/////. findings-acl.533
Mirza, P., Sprugnoli, R., Tonelli, S., and Speranza, M. Annotating causality in the TempEval-3 corpus. In Kolomiyets, O., Moens, M.-F., Palmer, M., Pustejovsky, J., and Bethard, S. (eds.), Proceedings of the EACL 2014 Workshop on Computational Approaches to Causality in Language (CAtoCL), pp. 10-19, Gothenburg, Swe- den, April 2014. Association for Computational Lin- guistics. doi: 10.3115/w1/W14-07000 URL https: //aclanthology.commm . wwtp:///////w///00o0
Lewkowycz, A., Andreassen, A., Dohan, D., Dyer, E., Michalewski, H., Ramasesh, V., Slone, A., Anil, C., Schlag, I., Gutman-Solo, T., et al. Solving quantitative reasoning problems with language models. Advances in Neural Information Processing Systems, 35:3843-3857, 2022.
Li, A. and Pearl, J. Probabilities of Causation with Nonbi- nary Treatment and Effect. Proceedings of the AAAI Con- ference on Artificial Intelligence, (8:8))2004552044777 March 2024. ISSN 2374-3468, 2159-5339. doi: 10.16 13ai.v38 URL htttttttt///sss..... index.
Mirzadeh, S. I., Alizadeh, K., Shahrokhi, H., Tuzel, O., Bengio, S., and Farajtabar, M. Gsm-symbolic: Under- standing the limitations of mathematical reasoning in large language models. In The Thirteenth International Conference on Learning Representations, 2025.
Li, X. L., Shrivastava, V., Li, S., Hashimoto, T., and Liang, P. Benchmarking and improving generator-validator con- sistency of language models. In The Twelfth International Conference on Learning Representations, 2024.
Mostafazadeh, N., Grealish, A., Chambers, N., Allen, J., and Vanderwende, L. CaTeRS: Causal and tempo- ral relation scheme for semantic annotation of event
12

---

Compasitional Causal Reasoning Evaluation in Language Models
structures. In Palmer, M., Hovy, E., Mitamura, T., and O'Gorman, T. (eds.), Proceedings of the Fourth Workshop Events, pp. 51-61, San Diego, Califor- nia, June 2016. Association for Computational Linguis- URL https://
Qiao, S., Ou, Y., Zhang, N., Chen, X., Yao, Y., Deng, S., Tan, C., Huang, F., and Chen, H. Reasoning with language model prompting: A survey. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume I: Long Papers), pp. 5368-5393, 2023.
tics. doi: 10. 10.18653/v1/W11--0000 aclanthology org/ / 1116-101777
Mueller, S., Li, A., and Pearl, J. Causes of effects: Learning individual responses from population data. Proceed- ings of the Thirty-Fing International Joint Conference on Artificial Intelligence (IJCAI-22), 2022.
Qin, L.,, Bosselut, A., Holtzman, A., Bhagavatula, C., Clark, E., and Choi, Y. Counterfactual story reasoning and generation. In Inui, K., Jiang, J., Ng. V., and Wan, X. (eds.), Proceedings of the 2019 Conference on Empir- ical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pp. 5043-5053, Hong Kong, China, November 2019. Association for Compu- tational Linguistics. doi: 10.18653/ URL https: httcps://aclarrtho...oooo www.hhhooooo......oog.e
Okawa, M., Lubana, E. S., Dick, R., and Tanaka, H. Com- positional abilities emerge multiplicatively: Exploring diffusion models synthetic task. Advances in Neural Information Processing Systems, 36, 2023.
Pearl, J. Reverend bayes on inference engines: A dis- tributed hierarchical approach. Proceedings, AAA1-82, pp. 133.136,10 1982.
Qiu, L., Jiang, L., Lu, X., Sclar, M., Pyatkin, V., Bhaga- vatula, C., Wang. B., Kim, Y., Choi, Y.. Dziri, N., et al. Phenomenal yet puzzling: Testing inductive reasoning ca- pabilities of language models with hypothesis refinement. The Twelfth International Conference on Learning Representations, 2024.
Pearl, J. Causal diagrams for empirical research. Biometrika, 82(4):669-68) 1995.
Pearl, J. Probabilities of Causation: Three Counterfactual Interpretations and Their Identification. Synthese, 121: 93-149, 1999.
Rashkin, H., Sap, M., Allaway, E., Smith, N. A., and Choi, Y. Event2Mind: Commonsense inference on events, intents, and reactions. In Gurevych, I. and Miyao, Y. (eds.), Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Vol- ume 1: Long Papers), pp. 463-473, Melbourne, Aus- tralia, July 2018. Association for Computational Lin- guistics. doi: 10.18653//1//11--1000 URL https: // /aaclanthologyy/o..o. .wwp:gg..orrrgg... wwt/
Pearl, Causality: Models, reasoning, and inference. Cam- bridge, UK: Cambridge University Press, 19(2):3, 2000.
Pearl, J. Direct and indirect effects. In Proceedings of
the Seventeenth Conference on Uncertainy in Artificial Intelligence, 2001.
Pearl, J. Causal inference in statistics: An overview. Statis- lics Surveys, 3:96, 2009.
Pearl, J.
The mediation formula: A guide to the assess- ment of causal pathways in nonlinear models. Causality: Statistical perspectives and applications, pp. 151-179, 2012.
Ray, A., Radenovic, F., Dubey, A., Plummer, B. A., Kr- ishna, R., and Saenko, K. Cola: A benchmark for com- positional text-to-image retrieval. Advances in Neural Information Processing Systems, 2023. URL https: / //arxiv.org/ abs////////// at///305../3.....// 2305. 03689.
Pearl, J. Linear Models: A Useful "Microscope" for Causal Analysis. Journal of Causal Inference, 1(1):155-170, 2013.
Sani. N. and Mastakouri, A. A. Tightening Bounds on Probabilities of Causation By Merging Datasets, Octo- ber 2023. URL http: arxiv. org/ abs/2310. 2310. 08406. arXiv:2310.08406 [cs, math, stat).
Pearl, J. Interpretation and identification of causal mediation.
Psychological methods, 19(4):459, 2014.
Plečko, D. and Bareinboim, Causal fairness analysis: A causal toolkit for fair machine learning. Foundations and Trends@ in Machine Learning, 2024. ISSN 1935-8237, 1935-8244 8245. doi: 10. 1561/22000001666
Sani. N., Mastakouri, A. A., and Janzing, D. Bounding prob- abilities of causation through the causal marginal prob- lem, April 2023, URL http: / arxiv. wwwg..rg/aaaaa /abs/ 2304. 02023. arXiv:2304.02023 [cs, math, stat].
Press, O., Zhang, M., Min, S., Schmidt, I.., Smith, N. A., and Lewis, M. Measuring and narrowing the composi- tionality gap in language models. In Findings of the As- sociation for Computational Linguistics: EMNLP 2023, pp. 5575677711,,, 2023. 2023.
Sap, M., Le Bras, R., Allaway, E., Bhagavatula, C., Lourie, N., Rashkin, H., Roof, B., Smith, N. A., and Choi, Y. Atomic: An atlas of machine commonsense for if-then reasoning. In Proceedings of the AAAI conference on artificial intelligence, 2019a.
13

---

Compositional Causal Reasoning Evaluation in Language Models
Sap, M., Rashkin, H., Chen, D., LeBras, R., and Choi, Y. Socialiqa: Commonsense reasoning about social interac- tions. In Conference Empirical Methods in Natural Language Processing, 2019b.
Stone, A., Wang, H., Stark, M., Liu, Y., Scott Phoenix, D., and George, D. Teaching compositionality to cnns. In Proceedings of the IEEE conference computer vision and pattern recognition, pp. 5058-5067, 2017.
Saxton, D., Grefenstette, E., Hill, F., and Kohli, P. Analysing mathematical reasoning abilities of neural models. In International Conference Learning Representations, 2019.
Su, J., Liu, N., Wang, Y., Tenenbaum, J. B., and Du, Y. Compositional image decomposition with diffusion mod- els. Proceedings of the 4 International Conference on Machine Learning, 2024.
Schaeffer, R., Miranda, B., and Koyejo, S. Are emergent abilities of large language models mirage? Proceedings of the 37th Conference on Neural Information Processing Systems (NeurIPS 2023). 2023.
Suhr, A., Lewis, M., Yeh, J., and Artzi, Y. A corpus of natural language for visual reasoning. In Proceedings of the 55th Annual Meeting of the Association for Computa- tional Linguistics (Volume 2: Short Papers). 2017.
Suhr, A., Zhou, S., Zhang. A., Zhang, I., Bai, H., and Artzi, Y. corpus for reasoning about natural language grounded in photographs. arXiv preprint arXiv:181 1.00499 2018.
Schölkopf, B., Locatello, F., Bauer, S., Ke, N. R., Kalch- brenner, N., Goyal, A., and Bengio, Y. Toward causal representation learning. Proceedings of the IEEE, 109(5): 612-634, 2021.
Thrush, T., Jiang, R., Bartolo, M., Singh, A., Williams, A., Kiela, D., and Ross, C. Winoground: Probing vision and language models for visio-linguistic compositionality. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition, pp. 5238-5248, 2022.
Schwartenbeck, P., Baram, A., Liu, Y. Mark, S., Muller, T., Dolan, R., Botvinick, M., Kurth-Nelson, Z., and Behrens, T. Generative replay underlies compositional inference in the satpoccapl--prefrooot circuit. Cell, 186(22):4885-
4897, 2023.
Tian, J. and Pearl, J. Probabilities of causation: Bounds and identification, Annals of Mathematics and Artificial Intelligence, 28(1):287-313, 2000.
Shafer, G. R. and Shenoy, P. P. Probability propagation. Annals of Mathematics and Artificial Intelligence, 2:327- 351, 1990.
Toshniwal, S., Du, W., Moshkov, I., Kisacanin, B.,
Shi, Z.. Liu, H., Min, M. R., Malon, C., Li, L. E., and Zhu,
Ayrapetyan, A., and Gitman, OpenMathInstruct-22 Accelerating AI for math with massive open-source in- struction data, 2024. URL abs / 22122410..02 01560.
X. Retrieval, analogy, and composition: A framework for compositional generalization in image captioning. In Findings of the Association for Computational Linguis-
tics: EMNLP 2021, pp. 1990-2000, 2021.
Touvron, H., Martin, L.., Stone, K., Albert, P., Almahairi, A., Babaei, Y., Bashlykov, N., Batra, S., Bhargava, P., Bhosale, S., et al. Llama 2: Open foundation and fine- tuned chat models. arXiv preprint arXiv:2307.09285 2023.
Singal, and Michailidis, G. Axiomatic effect propagation in structural causal models. Journal of Machine Learning Research, 25(52):1-71, 2024.
Singh, S., Wen, N., Hou, Y., Alipoormolabashi, P., Wu, T.-L. Ma. X., and Peng. N. COM2SENSE: A com- monsense reasoning benchmark with complementary sentences. In Zong, C., Xia, F., Li, W., and Nav- igli, R. (eds.), Findings of the Association for Com- putational Linguistics: ACL-IJCNLP 2021, pp. 883- 898, Online, August 2021. Association for Computa- tional Linguistics. doi: 10.18653/v1/20211 http://////21...mnngg-aaaaad 78. URL https: /aa/aaaaaaantooyo//o/ org 2021. findings siiggs-ac..85 ac1.78/.
WanderWeele, A unification of mediation and inter- action: 4-way decomposition. Epidemiology. 25(5): 749-761, 2014.
VanderWeele, T. J. Mediation analysis: practitioner's guide. Annual review of public health, 37:17-32, 2016.
Wang, H., Feng, S., He, T., Tan, Z., Han, X., and Tsvetkov, Y. Can language models solve graph problems in natural language? Advances in Neural Information Processing Systems, 36, 2023a.
Stolfo, A., Jin, Z., Shridhar, K., Schoelkopf, B., and Sachan, M. A causal framework to quantify the robustness of mathematical reasoning with language models. In Pro- ceedings of the 6lst Annual Meeting of the Association for Computational Linguistics ( (Volume 1: Long Papers), pp. 545 545-561, 2023.
Wang. X., Wei, J., Schuurmans, D., Le, Q. V., Chi, E. H., Narang, S., Chowdhery, A., and Zhou, D. Self- consistency improves chain of thought reasoning in lan- guage models. In The Eleventh International Conference on Learning Representations, 2023b.
14

---

Compasitional Causal Reasoning Evaluation in Language Models
Wang, Y. and Jordan, M. I. Desiderata for Rep- resentation Learning: A Causal Perspective, Febru- ary 2022. URL http:/ /tt////rxxi 03795. arXiv:2109.03795 [es, stat].
in Natural Language Processing (EMNLP), pp. 4630-
4639, Online, November 2020. Association for Computa- tional Linguistics. doi: 10.18633/w1/2000emmmppp 374. URL https: emn emn monlp-maiin mam.m.in...7..
Wang, Z., Do, Q. V., Zhang, H., Zhang, J., Wang, W., Fang, T., Song, Y., Wong, G., and See, S. Cola: Contextualized commonsense causal reasoning from the causal inference perspective. In Proceedings of the 6lst Annual Meeting of the Association for Computational Linguistics ( Volume
Zhang, W., Wu, T., Wang, Y., Cai, Y., and Cai, H. To- wards trustworthy explanation: On causal rationalization. In International Conference on Machine Learning, PP- 41715-41736, PMLR, 2023.
I: Long Papers). pp. 5253-5271, 2023c.
Wei. J., Wang, X.. Schuurmans, D.,, Bosma, M., Xia, F., Chi, E., Le, Q. V., Zhou, D., et al. Chain-of -f-ouhghhttthtt prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824-2488 2022.
Westbrook, J. and Tarjan, R. E. Maintaining bridge- connected and biconnected components on-line. Algo- rithmica, 7(1):433-464, 1992.
Wittgenstein, L. Philosophical investigations. John Wiley & Sons, 2009.
Wright, S. Correlation and causation. Journal of agricul- tural research, 20(7):557, 1921.
Wüst, A.. Stammer, W., Delfosse, Q., Dhami, D. S., and Kersting, K. Pix2code: Learning to compose neural visual concepts as programs. In The 40th Conference on Uncertainty in Artificial Intelligence. 2024.
Xu, Z., Niethammer, M., and Raffel, C. A. Compositional generalization in unsupervised compositional representa- tion learning: A study on disentanglement and emergent language. Advances in Neural Information Processing Systems, 35:25074-250025 2022.
Yang, L.., Shirvaikar, V., Clivio, O., and Falck, F. A crit- ical review of causal reasoning benchmarks for large language models. In AAAI 2024 Workshop on "Are Large Language Models Simply Causal Parrots?", 2024.
Zečević, M., Willig, M., Dhami, D. S., and Kersting, K. Causal parrots: Large language models may talk causality but are not causal. Transactions on Machine Learning Research, 2023.
Zhang, J. and Bareinboim, E. Non-parametric path anal- ysis in structural causal models. In Proceedings of the 34th Conference on Uncertainty Artificial Intelligence, 2018.
Zhang, L., Lyu, Q., and Callison-Burch C. Reasoning about goals, steps, and temporal ordering with WikiHow. In Webber, B., Cohn, T., He, Y., and Liu, Y. (eds.), Pro- ceedings of the 2020 Conference on Empirical Methods
15

---

Compositional Causal Reasoning Evaluation in Language Models
APPENDIX
A. EXTENDED BACKGROUND
Algebra.
Let (() 2r, (x)) o9, M(x) = + Solve (( (=)))-
Physica.
Subatomic particles.
Carbon-18 18 atom.
Graphite structure.
Graphicat modeling.
Local subgraphes.
Global graph.
Figure I. Composition is ubiquitous in physical and symbolic systems, operating at multiple scales of granularity: atoms + molecules materials; cells tissues organs; words sentences; subroutines programs; graphical substructures global graphs; etc.
Compositional Reasoning in AI Compositional reasoning has been explored for diverse tasks in AI. These include problem-solving in math, logic, and programming (Saxton et al., 2019; Dziri et al., 2023; Lu et al., 2023a); image generation (Du et al., 2023; Okawa et al., 2023), decomposition (Su et al., 2024), and captioning (Shi et al., 2021); visual concept learning (Wüst et al., 2024); object recognition (Stone et al., 2017); representation learning (Xu et al., 2022); chain-of-thought (Li et al., 2023); visual question answering (Agrawal et al., 2017); and text-to-image retrieval (Ray et al., 2023). Models have been shown to struggle with compositional reasoning in vision and language, especially for complex compositions (Agrawal al., 2017; Ma et al., 2023; Ray et al., 2023). Saxton al. (2019) address compositional math reasoning with an emphasis compositions of functions and manipulation of intermediate results, though causal measures not explored. Many works center the development of benchmark datasets for vision and/or language models (e.g., Johnson et al. 2017; Agrawal et al. 2017; Suhr et al. 2017; 2018; Thrush et al. 2022; Ma et al. 2023; Hsieh et al. 2023; Ray et al. 2023). This work introduces new core concepts CCR evaluation for LMs, on which future benchmarks can be based.
Applications of the PrC in AI The majority of works on the PrC have centered on identifiability and the derivation of bounds under various conditions (Tian & Pearl, 2000; Mueller et al., 2022: Sani et al., 2023; Sani & Mastakouri, 2023; Li & Pearl, 2024; Maiti et al., 2025). The PrC have found applications representation learning (Wang & Jordan, 2022), causal discovery (Cai et al., 2023), and explainable Al and rationalization (Zhang al., 2023). LMs, the PrC have been used to evaluate causal reasoning (González Nori, 2024) and to facilitate fine-tuning for counterfactual reasoning (Hüyük et al., 2025). To our knowledge, ours is the first work to explore the utility of PrC compositionality for measuring compositional reasoning in AI.
Commutative Diagrams in Reasoning Evaluation Classically, commutative diagrams have been used as tools for theorem proving (see diagram chasing, De Toffoli 2017). In this work, introduce special form of commutative diagram for modeling reasoning. González & Nori (2024) use commutative diagrams for causal reasoning evaluation in LMs, where one pathway corresponds to the true problem solution for math problem and the other corresponds to the reasoning pathway of the I.M. The present work shows how commutative diagrams in the form of CCTs can provide compact and exhaustive representations for (1) visualizing CCR pathways and (2) systematizing CCR evaluation.
Causal Reasoning Evaluation See Table A. for an extensive comparison to prior works in this area. To further clarify our contributions, we compare to the CLADDER benchmark for formal causal reasoning (Jin et al., 2023). Most prior causal reasoning evaluation centers on commonsense knowledge and relation extraction (Table A. Like CLADDER, our framework and task generator can be used to (1) probe reasoning at the associational, interventional, and counterfactual
16

---

Compasitional Causal Reasoning Evaluation in Language Models
levels; (2) apply formal causal inference methods; and (3) evaluate causal relation extraction (implicitly explicitly). Our method notably departs from CLADDER in the following ways: (1) we evaluate compositional reasoning, another dimension of causal reasoning that has received little attention in LMs; (2) while CLADDER uses simple graphs (3-4 nodes), our task generator allows the user to freely scale the total number of nodes and our empirical results demonstrate on graph with eight nodes; (3) we introduce new metrics for the internal consistency of causal reasoning; and (4) we do not require ground truth quantities, as the user can opt to evaluate on internal consistency only. While CLADDER prompts the model to perform causal inference within single text response, the implementation demonstrated here takes an alternative approach: series of questions are posed to the model, and responses are treated as samples from observational and interventional data distributions. These samples are then used to perform formal causal inference, with the resulting counterfactual quantities (e.g., the PNS) representing the causal inferences implied by the model's 's reasoning at the associational, interventional, and counterfactual levels. Thus, the current implementation applies formal causal inference, but does not directly ask the I.M to formalize the causal query (as in Jin et al. 2023). However, the approach proposed by Jin et al. (2023) could be adapted to measure CCR under alternative prompt formulations.
QUESTIONS
SKILLS
R1
R2
R3
CI
Formal
CRE
QR
Comp
Scale
Causality as Knowledge (Commonsense)
COPA (Gordon et al., 2012)
X
/
X
X
X
X
X
X
X
Even12Mind (Rashkin et al., 2018)
X
/
X
X
X
X
X
X
X
ATOMIC (Sap et al., 2019a)
X
V
X
X
X
X
X
X
X
SocialIQA (Sap et al., 2019b)
X
V
X
X
XXXXX
XX
X
X
X
X
TimeTravel (Qin et al., 2019)
X
V
X
X
X
X
X
X
Goal-Step (Zhang et al., 2020)
X
X XXXXI
X
XXX
Abductive (ART) (Bhagavatula et al., 2020)
X
X
Com2Sense (Singh et al., 2021)
X
X
X
CRASS (Frohberg & Binder, 2022)
X
X
X
X
Causality as Language Comprehension (CRE)
SemEva12021 Task8 (Hendrickx et al., 2010)
XXP
X
X
KW
X
XX
EventCausality (Do al., 2011)
X
(Mirza et al., 2014)
CaTeRS (Mostafazadeh et al., 2016)
BECauSE (Dunietz et al., 2017)
X
X
X
Thl (Lal et al., 2021)
X
X
X
X
X
J
X
X
Formal Causal Reasoning
Corr2Cause (Jin et al., 2024)
CLADDER (Jin et al., 2023)
CCR framework and task generator (ours)
Table A.I. Comparison of causal reasoning evaluation frameworks for LMs (adapted directly from Table 9 in Jin et al. 2023). Question types concern the three rungs of Pearl's Causal Hierarchy (Bareinboim et al., 2022): associational (RZ), interventional (R2), and counterfactual (R3). Skill types entail the application of causal inference methods (CT). formalization of causal queries (Formal), causal relation extraction from text (CRE), qualitative reasoning (QR), systematic composition of causal measures (Comp), and scaling the graphical complexity of the causal DAG (Scale). In our automated task generator (attps: /t///jjmm/ jmaasch. github. vo//er// /), we implement causal prompt templates that require either quantitative reasoning (i.c., numeracy) or qualitative reasoning (c.g., reasoning over colors instead of numbers). Note that compare to CCR at the conceptual framework level and the task instantiation level. Unlike Corr2Cause and CLADDER, the instantiation presented in Section requires the responses to multiple prompts to be aggregated, which somewhat complicates comparison. While Cor22Cause and CLADDER prompt for causal query formalization and application of causal inference methods directly within single LM response, the experiments in this work treat the LM as a counterfactual data simulator whose responses about interventional outcomes logically imply the formal counterfactual quantity, which is then formally computed using the vector of LM responses. However, our conceptual framework (as presented in Section 4) is fully compatible with the direct strategy under alternative prompt formulations, such as application of CLADDER'S CausalCoT (an area for future work).
17

---

Compositional Causal Reasoning Evaluation in Language Models
B. PROBABILITIES OF CAUSATION: COMPOSITIONALITY
B.I. PN & PS
The PS considers the presence of a causal process that can produce a given effect, while the PN considers the absence of alternative explanatory processes (Tian & Pearl, 2000). Like the PNS, the PN and PS are point identifialle when Y is monotonic in X.
Definition B.1 (Probability of Necessity (PN), Pearl 1999). The probability that event y would not have occurred in the absence of event æ; given that and did jointly oceur, is given as
PN = P(ys v. = =P = FALSE| X = TRUE, Y TRUF).
(B.1)
Definition B.2 (Probability of Sufficiency (PS), Pearl 1999). The probability that event would produce event y given that and did not in fact occur is given as
PS = (y 2²,1').
(B.2)
Definition B.3 (Point identification the PN and PS under monotonicity, Tian & Pearl 2000). Given a positive-Markovian SCM for which Y is monotonic in X and the causal effects P(yz) and P(yr)) ) are identifiable by Equation 1, the probabilities of causation are point identifiable by the following expressions.
PN = P(y) P(
P(y) - Pl do(x'))
(B.3)
P( v)
P(x, y)
PS = PP(+) - P(y)
Ply do(x)) P(y)
(B.4)
P(r',0))
P(r', y²)
B.2. PrC Compositionality; Proofs
Proof. Here we show that the PN and PS do not compose according to Theorem 5.1, while the PNS does. Consider a causal model X + Y + Z with binary variables X, Y, Z. Assume that all relations in the causal graph are monotomic such that
Ply': Uwr) = IP (2 2ny)
(B.5)
(B.6)
We start by expressing PNxY, PSxy in terms of probabilities over potential outcomes Yz, Ygr:
IP v/2( (z:)
PPyz: V/rr) P(ye, y2r) + Py V/y PPy wzy))
PNxy = PP (x,
(B.7)
P(y, yz' (z) + Ply (x)
P(yz, y/ |xx'
PSxy Pyy,l ./) Ply. 3' (2) ) + P(y's y'(x") P(y V.r) + PP v'r)
(B.8)
Using these expressions, together with the monotonicity assumption PlyL yer) O and the simple fact that
P(yes y + P(y. vler) + PP'' v'e.) + P(V.. y = 1, we can fully describe the distribution of potential outcomes Y,, Y. (by solving the resulting system of equations):
(B.9)
1/PNx--1 -1
PP Ue.) = 1/ 1/PNxy + 1/PSxv
(B.10)
PP(y, w/z.)
(B.11)
1/PNXY + 1/PSxx
1/PSXy -
P(V/ w/2.)
(B.12)
1/PNxy + 1/PSxx
PV Wer 0.
(B.13)
18

---

Compasitional Causal Reasoning Evaluation in Language Models
Similarly, given 2 2, PSyz, 2, describe the distribution of potential outcomes Zy Zyy
1/PNyz -1 1/PNyz + +1/P8z--
P((yy Fxy)
(B.14)
P(zy: =//) 1/PNyz + +1/PSy2
(B.15)
1/PSyz -1
P(2y) 2//) 1/PNy2 2 + +\PPSy2
(B.16)
P(zy, =
(B.17)
Since we have now fully characterized our causal model, we can easily derive formulas for PNxz or PSxz. Starting with PNxz, we first observe that
P(Za, z're) PNxz= P(za, Exd) P( ('d))
(B.18)
as we have done with PN earlier. Then, we note that
P((aa
(y Ux (P 2' Myas U2.) + P(yz. Voa (((z z'e (M V'z) P(y'e Vol (P We. V'ae
(B.19)
Pyee Ye' (P 244) + P(yer b/z (P(z 2'y) + P( 3/2 (P(z 2ly))
(B.20)
P(ye V (P(zy: 2(y))
(B.21)
(B.22)
I/PNxy + 1/PSxy 1/PNyz + 1/PSyz
Note that we obtain Equation B.21 from B.20 by monotonicity: P(v'.) = 0 and ((22)) 0 (as outcome cannot be false when treatment is true). Then,
P(Zza Rad) ) = Fyy Yo M P( Rnd
( Me. Mer) ) + PMyas w/2 )I 22' Me Vor) + P( W' PP zx 1M
(B.23)
Pl Yu' P((y 2ny + (y 2 )P zy zy) + Ply Va (Zyy)
(B.24)
P( Yuu )P(22) (V Via ([a 2v)) P(U/. Var PP(y))
(B.25)
Plu Yu ) (IP zwr) + Play, 2/y)) + Pyy. v/ (P zx) + P( w2z (1P(2)) 2ag) + P( zy))
(B.26)
1/PNxy + 1/P8xx- 1 x 1/PNyz + 1/PSrz -1
PNxx (
(
PNyz (PNZZ-1) + (PSx
(
(PNN---))
+
(B.27)
Therefore,
1
+ PSxx (PSxy~) -1
PNNy"* (PNzZ-1):
1
(B.28)
PNxz PNXY PNyz
which cannot be reduced further to function of PNxY and PNyz only. This is because our causal model has enough degrees of freedom to fix PN and PNyz but still vary PSXY resulting in different values of PNxz.
Following a similar derivation, we can also write
(PA
1) (1))))())))))))
PSyy -1)
(B.29)
PSxz PSXY PSyz
Thus, while individual PN and PS values compose in a rather complicated way, PNS values happen to compose multiplicatively. We have already proven this statement along the way in (B.21):
PNSx2 P(zr, 2//7) P(yz. Vaet Pl 24y) = PNSxx PNSyz
(B.30)
19

---

Compositional Causal Reasoning Evaluation in Language Models
A
D
H
PNSAH PNSAC: PNSce
PNScH
PNSAG PNSGH
Figure B.I. Multiplicaiive composition of the PNS BCCs when cause-effect pairs satisfy the monotonocity constraint, Nodes with dashed outlines are cutpoints.
Thus, the PNS composes according to Theorem 5,1 under monotonicity. To obtain (B.21), we used monotonicity in (B.19). Otherwise, there would have been fourth term in (B.19) related to P P(y' Mu") and we would have been left with
P(Zz, 2 = Pyyas V )P z'y)) + P(yar Uo )TP( Zgy).
(B.3 1)
This is intuitive: if X - Z, then either X O YAY = X = Y'AY" = Z. Then, you can argue that PNSx2 cannot be pure function of PNSxy and PNSy2: we can fix PNSXY = P(yz: Var)) and PNSyz = P(zy, z/y)) and still easily vary (V/// yx') or (2(2)), Zy/ resulting in different values of SNSxz.
B.3. Extended Discussion of the PNS
B.3.1. THE PNS COINCIDES WITH THE ATE UNDER MONOTONICITY
We highlight the relationship of the PNS to another causal quantity: the average treatment effect (ATE). Under monotonicity, PNS is identifiable from causal effects. the binary setting, the causal effect E[Y do(X = x)] can be expressed as
E[Y do(X = =)] = 11y = do(X = =)) + P(y 0 do(X = x))
(B.32)
P = do = =))).
(B.33)
Proposition B.4. Following from Eq. B.33, we can express the ATE as a difference of probabilities that is equal to the PNS.
ATE = E[ do(X = z)] EE do([ = x')]
(B.34)
=P(y do(X = x)) Ppy do x²))
(B.35)
= PNS.
(B.36)
20

---

Compositional Causal Reasoning Evaluation in Language Models
C. ATE COMPOSITION IN LINEAR SCMs
X5
X
X1
a
X2
b
X3
X4
d
Y
Figure C.I. DAG 9x,y contains subgraph that can be decomposed into two BCCs sharing cutpoint X3. 9x, is the BCC induced by edges in orange, Ix,y by edges in periwinkle. Assume a linear SCM. the dotted edge from X. X6 does not exist, ATEx, X X If the dotted edge does exist, then this product is summed with additional term corresponding to the path-specific effect path X Xs -> X6 + Y, which does not pass through X3.
C.I. Worked Example
In linear SCMs, composition of the ATE over the BCCs of DAG shares the same form as Theorem 5.1. This follows from the product-of-coeffcient heuristic used in classical path- tracing and linear mediation analysis (Wright, 1921; Alwin & Hauser, 1975; MacKinnon, 2012; Imai et al., 2010; Pearl, 2012; 2013; Singal & Michailidis, 2024). Linear path-tracine rules are not guaranteed to apply in nonlinear data generating processes, as addressed in nonparametric causal mediation analysis (Pearl, 2001; 2012) and nonparametric path-tracing (Zhang & Bareinboim, 2018).
We provide an illustrative example of composition for the ATE in linear SCMs for two settings: (1) when the DAG contains at least one cutpoint and (2) when the DAG does not contain a cutpoint, but does contain a subgraph with at least one cutpoint. Finite sample results are in Figure C.2.
DAGs With Cutpoints Take Figure C.1 as an example. First, we assume that the dotted edge from X5 to Xg does not exist. this case, the DAG contains cutpoint X3 and two BCCs. The ATE for (X1, Y) can then expressed as a product of the ATE values corresponding to the root and leaf of each BCC.
ATEx, X, = ab + es
(C.1)
ATEx,y = cd + gh
(C.2)
ATEx, abed abgh + efed + efgh
(C.3)
ATEx,x, ATEx,,y
gh) ATEx,
(C.4)
We can see that the ground truth ATEx,, expressed in Equation C.3 is equivalent to the product expressed in Equation C.4.
DAGs Without Cutpoints Next, assume that the edge from X5 to Xg does exist.
ATEx,y = abed + abgh + efed + efgh + eih
(C.5)
= (ab + eef+ + gh) + eih
(C.6)
-ATEx ATEx,Y + PSE
(C.7)
where PSE is the path-specific effect for X, - X5 + X - Y, the only causal path for (X1, Y) not passing through cut vertex X3.
Finite Sample Simulation In Figure C.2, we demonstrate finite sample results for the case where X5 - X and X5 A Xg. Exogenous variables are drawn from the standard normal distribution. All edge coefficients in Figure C. are set 1.5. Thus, we have the following true ATE values when X; A Xs:
ATEx,3 = 1.5 + 1.52 4.5
(C.8)
ATEx, 1.52 1152 4.5
(C,9)
ATEx, 4(1.5*) 20.25.
(C.10)
21

---

Compositional Causal Reasoning Evaluation in Language Models
X5 does not cause X6
causes X6
23.07
25
23-3
2011
23.
2022
ATE estimate
500
1000
5000
10000
500
1000
5000
10000
Sample size
Compostion Global
Figure C.2. ATE composition for a linear SCM whose graphical representation is given by Figure C.I. Composition = ATEx, A and global ATEx,y. solid red line represents true ATEx,y when Xs A Xo. the dotted line represents A when Xs Xe. Estimates were obtained by linear regression (htpss//sciktttlam..rr)
When X3 Xo, we have the additional path-specific effect for X1 X3 X6 + Y, which equal 1.5 3.375. Thus,
ATEx, 20.25 3.375 23.625.
(C.11)
As shown in Figure C.2, finite sample results approach the true values with small errors. Note that estimation of ATEx,y when X5 X6 requires covariate adjustment for Xs, which acts confounder for X3, Y in this case.
22

---

Compositional Causal Reasoning Evaluation in Language Models
D. ALGORITHM 1: INDUCTIVE CCR EVALUATION
B
E
H
K
A
D
M
N
R
A. DAG 9AM
B. DAG 9NR
ioke
h
toh
9
h
gof
h hogof
johogaf
C. CCT
Figure D.I. (A) DAG GaMz, whose undirccted skeleton is cactus graph with three cutpoints (dashed nodes). For GaMs the global PNS PNSAM is equivalent to the product ANSAAPNNSnGPNN PNSJs while PNS AG decomposes PNS etc. (B) DAG GNR.& directed chain. (C) The CCT shared by both Gam and GNR, which models all CCR pathways from root leaf.
D.1. Time Complexity of Algorithm 1
Let be the number of nodes in the original causal DAG. In the worst the number of nodes in the corresponding CCT will be n (e.g., Figure D. IB). The first for-loop in Algorithm I (Lines 2-3) requires errors to be computed for every pair of nodes in the CCT, resulting in (2) errors. Thus, the first for-loop requires the calculation of Onn²) errors.
Let be the number of unique paths from root to leaf in the CCT. The second for-loop in Algorithm (Lines 3-6) requires two errors be computed for all (k 1) compositions: one for internal consistency and one for external validity. The total number of unique paths from root leaf is k 2n-2, where 2 the number of cutpoints in the original DAG. Thus, O(2") errors must be calculated.
Therefore, the total number of errors to be calculated will be O(2"). In Table D.1 and Figures D.2 and D.3, we empirically demonstrate that the total number of unique paths from root to leaf in a CCT is 2=-2
NODES (n) CUTPOINTS 2) EDGES (?)
PATHS (2n-2)
3
3
4
6
5
10
6
15
16
7
5
21
32
8
6
28
64
9
7
36
128
10
8
45
256
11
55
512
Table D.I. Total cutpoints, edges, and unique paths from root leaf in CCTss total nodes scales.
23

---

Compositional Causal Reasoning Evaluation in Language Models
n = 3
T1 = 4
T 7=
n = 8
12 = 9
= 10
= 11
Figure D.2, CCTs nodes.
Total distinct paths
Total distinct paths
Total unique paths from root leaf
107
Linear
Total unique paths from root leaf
10
Linear
10
Quadratic
Quadratic
Exponential (base 2)
10
Exponential (base 2)
10
Factorial
Factorial
10a
10
10a
10a
10a
10
10
10a
109
10
2
4
Total nodes
Total cutpoints
Figure D.3. Total unique paths from root to leaf a CCT with respect to total nodes (left) and total cutpoints (right).
24

---

Compositional Causal Reasoning Evaluation in Language Models
E. SIMULATIONS
E.1. Numerical Behavior Finite Samples
The following experiments demonstrate the numerical behavior of inductive and deductive CCR for (1) the PNS when all cause-effect pairs satisfy the monotonicty assumption and (2) the ATE in linear SCMs. All simulations were performed on MacBook Pro (Apple M2 Pro).
0.105
PN PTA.CC//PNNNN/)PPPCCCC PNSOX PNNSSC Y)
PNS(X,D) P/
0.100
PNSIX.Y)
0.085
PNS
0.090
0.085
0.080
1000
5000
10000
1e+05
5e+05
Sample size
Figure E. Finite sample simulations internally consistent inductive CCR for the PNS. Causal functions were logical or. For CCT in Figure 4, all compositions converged to the same value sample size increased. See Figure E.3 for analogous results for the ATE in linear-Gausee Gaussian SCMs.
0.275
PNSOCY/PNSXXD)
PNS(X.) PPPSSCC))
0.250
PNS
0.225
0.200
0.175
100
500
1000
5000
10000
1e+05
Se+05
Sample size
Figure E.2. Finite sample simulations of internally consistent deductive CCR for the PNS. Causal functions were logical and. For the CCT in Figure 4, points the same color were expected to converge. See Figure E,4 for analogous results for the ATE in linearGGaussia SCMs.
25

---

Compositional Causal Reasoning Evaluation in Language Models
6.6
ATE(X.C) ATE AT D)
ATE(X CJ ATTTE Y)
ATE(X.D) ATEED.Y) ATE(XY)
100
500
1000
5000
10000
16-05
5e405
Sample size
Figure E.3. Finite sample simulations of internally consistent inductive CCR for the ATE in linear-Gaussian SCMs. For the CCT in Figure 4, compositions converged as sample size increased, Estimates were obtained by lincar regression (https//ssikit-camm.or))
Cutpoint
Cutpoint D
50-
1.150-
ATE
t
ATEJO
1.075-
ATEOCOI YTATA
50
500
1000
5000
10000
1ev05
Sevos
500
1000
5000
10000
1eH05
58405
Sample size
Figure E.4. Finite sample simulations of internally consistent deductive CCR for the ATE in GaussiG SCMs. For the CCT in Figure compositions converged sample size inereased. Estimates were obtained by linear regression (https://sikkttllam..co
26

---

Compositional Causal Reasoning Evaluation in Language Models
F. AUTOMATED TASK GENERATOR
To facilitate future benchmarking, we release open-source code for automated CCR task generation (https:/ // jmaasch. wwesschhgittiiin github. io/cez/). Tasks are of the same form the running example used in this work. Code provides the following functionality:
Generate causal DAG.
Generate its corresponding CCT.
Enumerate quantities of interest for CCR evaluation: global, local, and compositions.
Generate factual and counterfactual text prompts corresponding to the generated SCM and chosen theme. Variables are assigned random human names. Themes are CandyParts y (which requires quantitative reasoning; Figure F.1) and Flower rerGGardee reen (which does not require quantitative reasoning, Figure F.2).
Sample observational and interventional data from the SCM.
Compute the PNS, PN, PS, and ATE from interventional data samples.
The user can control the following sources variation in the SCM:
Total biconnected components in the causal DAG.
Nodes per biconnected component.
Graph type biconnected component (cycle or wheel graph), which impacts connectivity.
The following elements are assigned at random:
Parameters Bernoulli distributions (chosen uniformly random).
Causal functions (a mix of logical or and logical and, chosen uniformly at random).
Each task is constructed as follows. See Figures F.1 and F.2 for concrete examples.
1. Causal world model. First, we define a fictional world corresponding randomly generated causal graph. This will be the causal world model for the LM to reason over. The structural causal model defining our fictitious world is comprised of binary exogenous noise variables, binary endogenous variables, and nonlinear causal functions (monotonic logical operators and, or).
2. Causal context prompt. Second, we construct verbal description of the world model. This verbal description our "causal context prompt" - contains all pertinent details needed for the LM to infer the world model, well extraneous details needed to solve the CCR task. The causal context centers on user defined theme (e.g.. CandyParty, Fl PlowerGarden. etc.).
3. Sampling. Third, we randomly sample exogenous variables and extraneous variables and compute true endogenous variable values. Sampled values are then used to construct the "sample context" in natural language, which is concatenated to our causal context prompt. Each causal context will be copied many times, where each copy paired with a new sample context.
4. Factual query prompts. Next, we construct factual queries by treating the causal context sample context as observational data. All queries phrased as yes/no questions. The factual query is then concatenated to copy of the causal context + sample context.
5. Counterfactual query prompts. Finally, we pair each factual prompt with an interventional query corresponding to the appropriate counterfactual (do(X TRUE) or do(X FALSE)). The resulting counterfactual query is then concatenated to copy of the causal context sample context. As with factual queries, all counterfactual queries are phrased as yes/no questions.
27

---

Compositional Causal Reasoning Evaluation in Language Models
Causal graph
barnice
ebba
crogenous variables
carri
endogenous variables
Causal context
A group of friends is going to a party where candies will be randomly distribured. Carri will be happy if she gets at least 4 candies. Bernice will be happy if she gets at loust 7 candics and Carri is happy. Tiarra will be happy if she gets ut Carri is happy and Bernice is happy. Ebba she gets at least 6 candies or Tiarra happy. Edrle will be happy if she gets az least candles or Tlarra is happy or Ebba happy.
least candies
happy
Sample context
After distributing the candies, Carri gets candies, Bernice gets 4 candies, Tiarra gets candies, Ebba gets candies, and Edrie gets 2 candies.
Factual query
Is Edrie happy?
Intervention (0)
Now, suppose that Tiarra is not happy regardless of all other circumstances. With this new ussumption, is Edric huppy?
Intervention (1)
Now, suppose that Tiarra is happy zegardless of all other circumstanoes. With this new assumption, is Edrie happy?
Figure F.I. A simple CandyParty prompt, which requires quantitative reasoning.
Causal graph
neveah
erogenous variables
angel
mazie
gladys
twila
endogenous variables
Causal context
A group of friends is planting bed of flowers, but the seed packets are not lebeled. Angel will be happy if the flowers she planted are red. Masie will be happy if the flowers she pleated are yellow and Angel is happy. Neveah will be happy if the flowers she planted are purple or if Mazie is happy. Gladys will be happy if the flowers she planted are pink or if Mazie is happy or if Neveah is happy- Twila will be happy if the flowers she planted are yellow and Gladys is happy.
Sample context
Once the flowers bloom, Angel's flowers are blood red, Mazie's flowers are hot pink, Neveah's flowers are lilac purple, Gladys' flowers are lemon yellow, and Twila's flowers are pastel yellow.
Factual query
Is Gladys happy?
Intervention (0)
Now, suppose tha: Masie is not happy regardless of all other cireumstancess With this new assumption, is Gladys happy?
Intervention (1)
Now, suppose that Mazie is happy regardless of all other cireumstances. With
this new assumption, is Gladys happy?
Figure F.2. A simple FlowerGarden prompt, which requires qualitative reasoning.
28

---

Compositional Causal Reasoning Evaluation in Language Models
G. LM EXPERIMENTS
heg
bog
hog
gof
aof
gof
gof
hoge!
hagej
PNSXY
PNSxcPNScY
PNSxpPNNDY
PNSxcPNSeDPNSDx
Figure G.I. All paths from X to in the CCT corresponding to the DAG Figure 4, where go P, h gogo PNSxy, ctc., and composition is multiplicative.
PNSxc. PNScD. = PNSNY.
MODEL
PARAMETERS
LINK
Phi-3-Mini- 3--Miii-28KK Instructt (Abdin et al., 2024)
3.82B
https//huggeinffcc.cooiiii
wwt../ccooooo///iii-3n.... 128k -isttiit---iitttueett
Jlama-2-70 HF (Touvron et al., 2023)
6.74B
Llama-3-8B- 3-nn-s--B-Instrrctt (Dubey et al., 2024)
8.03B
Llama-3. 1-0---3.---B-Instrue (Dubey al., 2024)
8.03B
OpenMath2-Llama3. (Toshniwal et al., 2024)
8.03B
GPT-4o
175B
http:///ppnaaiccccmm/nnxg/ggooop hutps//openaiicoom/00
https://openai.ccom
> 175B
Table G.J. Large language models used for inference. The exact number of parameters in GPT-4o and ol is public knowledge, so we note the size of GPT-3 as lower bound (B denotes billions).
G.1. Experimental Design
Model Inference We used a single A100 GPU for all experiments. Models used for inference were LMs fine-tuned for dialogue, as reported in Table G.I. Llama 3.1 Math was also fine-tuned for math reasoning. OpenAl's ol model marketed as a "high intelligence reasoning model."* We did not perform any additional fine-tuning. All models used default hyperparameters for response generation except when greedy decoding was the default generation strategy, in which case default Hugging Face hyperparameters were used for sampling. For Boolean extraction, greedy decoding was used.
Translating Causal Queries to Text We translated the data generating process represented by the DAG in Figure 4 to a mathematical word problem, as expressed the prompt in Figure G.2. This prompt is based on the CandyParz problem described in González & Nori (2024) and Hüylik et al. (2025). Additionally, we implement CoT wrapper for our original prompt template that uses two examples demonstrate CoT for the model: one factual and one counterfactual (Figure G.3). sampled and calculated identically to the non-CoT experiments.
Questions and answers used in the CoT scenario
We define SCM M = (V, U,F,F, where V are binary variables representing the nodes in Figure 4, causal functions E F are logical or (V), and distribution p(u) is Bernoulli. Logical or is monotone Boolean function, satisfying the monotonicity condition for point identifiability of the PNS from causal effects (Definition 2.5).
Each node the ground truth graph is treated as person in our word problem: = Xinyu, A Ara, Becca, Celine, D = Daphne, Emma, F Fox, Y Yasmin. Values for all V; are given by
U; p01, V... pay V Ber vv:))
(G.1)
where (pa,) 1 are the k parents of V, All (T.) in the context prompt take value of 7, such that all exogenous variables are drawn from Bernoulli distributions parameterized by 0.7.
Examples of factual and counterfactual questions and responses are given in Figures G.4 and G.5 (see also: Figures F.I, F.2). For counterfactual questions, a new assumption was introduced about the variable acting as the cause. The model was then asked question about the variable acting as the effect.
29

---

Compositional Causal Reasoning Evaluation in Language Models
Extracting Boolean Values from Text To compute the PNS, model text responses were first translated to the corresponding Boolean answer. The corresponding Boolean extracted using Llama 3 with greedy decoding, given the following prompt: "I will give you question and its answer. Determine whether the meaning of the answer is 'TRUE' or 'FALSE', An answer is 'TRUE' if it contains phrases like 'yes', 'it holds', 'correct', 'true', or similar affirmations. An answer is 'FALSE' it contains phrases like 'no", 'it does not hold', 'incorrect', 'false,, or similar negations. Respond only with one word: 'TRUE" or 'FALSE'. Question: "q" Answer: the meaning 'TRUE' or 'FALSE'?"
Computing PNS Estimates For each cause-effect pair, we sampled 1000 sets of exogenous variable values, For cach exogenous value set, we generated one factual and one counterfactual question about the effect of interest using the prompt template context (Figure G.2). Five replicate LM responses were then sampled for each question. From these 10,000 total responses (5000 factual and 5000 counterfactual), 1000 factual responses and 1000 counterfactual responses subsampled (one per set of five replicate responses). The subsample of factual and counterfactual responses was then used to compute the PNS. This subsampling procedure was repeated 1000 times, resulting in a distribution of 1000 PNS estimates per cause-effect pair per model.
randomly
Approximation Errors Approximation errors were computed as the relative absolute error (RAE) for each individual PNS estimate. The RAE is the absolute error (AE) normalized by the true PNS. When comparing errors across different quantities (e.g., PNSXx versus PNSxc). normalization needed. Thus, we generally only report the RAE. For external validity, these metries computed with respect to ground truth (PNS*)).
A AEexternal
:= PNS PNS.
(G.2)
PNS PNS.
R.E
(G.3)
PNS
For internal consistency, these metrics are computed using estimates for two equivalent quantities (PNS. and PN5.)).
Aiitermal : PNS. - PNS.
(G.4)
PNS- - PNS.
R.Eeettmaal
(G.5)
PNS.
Prompt
Xinyu, Ara, Becca, Celine, Daphne, Emma, Fox, and Yasmin are going to a party, where the host is going to distributo candics. Xinyu will be happy if she gets at least (TX) candies. Ara will be happy if Xinyu is happy or if he gets at least (TA) candies. Becca will be happy if Xinyu is happy
if she gets Celine will be happy Xinyu is happy or if Ara is happy or if Becca is happy or she gets at least (TC) es. Daphne will be happy if Celine is happy or i she gets at Least [TD) candies. Erma will be happy it Daphne is happy or if she gets at least (TE) candies. Fox will be happy if Daphne is happy if Emma is happy he gets at least (TF) candies. Yasmin will be happy if Emna is happy or if Fox is happy or if she gets at least (TY) candies. After distributing the candies, Xinyu gets (NX), Ara gets (NA), Becca gets (NB), Celine gets (NC), Daphne gets (ND), Emna gets (NE), Fex gets (NF), and Yaamin gets (NY).
least (TB) candies.
if
Figure G.2, Context prompt template for inductive CCR evaluation. For experiments reported in this work, (T-) =
30

---

Compositional Causal Reasoning Evaluation in Language Models
Prompt
Xinyu, Ara, Becca, Celine, Daphne, Emna, Fox, and Yasmin going party, where the host is going to distribute candies. Xinyu will be happy is she gets at least 3 candies. Ara will be happy if Xinyu is happy or if he gets at least 4 candles. Becca will be happy if Xinyu is happy or if she gets at least 10 candies. Celine will be happy if Xinyu is happy
QUESTION:
if Ara is happy or if Becca is happy or if she gets least candies. Daphne will be happy if Celine is happy or if she gets at least candies. Emma will be happy Daphne is happy or if she gets at least 6 candies. Fox will be happy if Daphne is happy or if Emma is happy or if he gets at least 10 candies. Yasmin will be happy if Enna is happy or if Fox is happy or if she gets least candles. After distributing the candies, Xinyu gets 6, Ara gets 7, Becсa gets 5, Celine gets 1, Daphne gets 3, Emma gets 4, Fox gets 9, and Yasmin gets B. Is Becca happy? Be as concise as possible.
ANSWER: Since Xinyu gets candies, which is more than 3, she is happy. In that case, Becca is happy matter how many candies she got, because she will be happy if Xinyu happy. Therefore, yes, Веcca is happy!
QUESTION: Xinyu, Ara, Becca, Celine, Daphne, Enna, Fox, and Yasmir are going to party, where the host is going to distribute candies. Xinyu will be happy is she gets at least candies. Ara will be happy if Xinyu is happy or if he gets at least 4 candles. Becca will be happy if Xinyu is happy
if she gets at least 10 candies.
Celine will be happy Xinyu is happy or if Ara is happy or if Becca is happy or she gots at least 7 candies. Daphne will be happy if celine is happy or if she gets at least 5 candies. Emma will be happy if Daphne is happy or if she gets at least 6 candies. Fox will be happy if Daphne is happy or if Emma is happy or if he gets at least 10 candies. Yasmin will be happy if Enma is happy or Fox is happy or if she gets at least 1 candies. After distributing the candies, Xinyu gets 6, Ara gets 1, Becca gets 5, Celine gets 1, Daphne gets 3, Emma gets 4, Fox gets 9, and Yasmin gets 8. New, suppose that Xinya is not happy regardless of the candy distribution. is Becca happy? Be concise possible.
ANSWER: Since Xinyu gets candies, which is more than 3, she happy. Hovever, we are asked to assume Xinyu is not happy regardless. In that case, Becca is happy only if she gets at least 10 candies. Becca gets 5 candies, which is less than 10. Cherefore, no, Becca is not happy!
QUESTION: Xinyu, Ara, Becca, Celine, Daphre, Enna, Fox, and Yasmin are going to party, where the host is going to distribute candies. Xinyu will be happy is she gets at least 7 candies. Ara will be happy if Xinyu is happy or if he gets at least 7 candies. Beсca will be happy if Xinyu is happy or if she gets at least 7 candies. Celine will be happy if Xinyu is happy or if Ara is happy or if Becca is happy or if she gets
least 7 candies. Daphne will be happy if Celine is happy or if she gets at least 7 candies. Rmma will be happy
Daphne is happy or if she gets at least candies. Fox will be happy if Daphne is happy or if Enma is happy or if he gets at least candies. Yasmin will be happy if Emma is happy
if Fox is happy or is she gets least candies. After distributing the candies, Xinyu gets 10, Ara gets 5, Becca gets 8, Celine gets 5, Daphne gets 4, Enma gets Fox gets 2, and Yasmin gets 3. Now, suppose that Xinyu is not happy regardless of the candy distribution. With this assumpticn, is Yasmin happy? Be as concise as possible.
ANSWER:
Figure G.3. Example CoT formulation the original CandyParty prompt, where two examples that demonstrate CoT are provided the model: one factual and one counterfactual.
31

---

Compositional Causal Reasoning Evaluation in Language Models
Prompt
After distributing the candies, Xyn gets 4, gets 6, Recca gets Celine gets 10, Daphne gets 1, Emma gets 1, Fox gets 4, and Yasmin gets 3. Is Celine happy? Be as concise as possible.
Response
No, Celine is not happy.
She wants
least candies but got only 10.
Figure G.4. Factual question and answer from Llama 2, displaying incorrect numerical reasoning.
Prompt
Now, suppose that Xinyu is happy regardless of the candy distribution. With this assumption, is Celine happy? Be as concise possible.
Response
Under the assurption that Xinyu is happy regardless of the candy distribution, Celine is not happy.
Reason: Since Xinyu happy regardless of the candy distribution, it that she got at Least candies. However, Celine only got 10 candies, which is less than 7. Therefore, Celine is not happy.
Figure G.5. Counterfactual question and answer from Llama 2, displaying incorrect numerical reasoning.
Prompt
After distributing the candies, Xinyu gets 10, Ara gets 6, Becсa gets 7, Celine gets 6, Daphne gets 1, Rmma gets 10, Fox gets 10, and Vasmin gets 2. 1s Daphne happy? He as concise as possible.
Response
Daphne will happy Celine is happy or if she gets at least candies. Since Daphne gets less than candies (1), she won't t be happy based on the candies she gets. Celine gets 6 candies, which is less than 7. since neither of Daphne's conditions is met, Daphne is not happy.
Figure G.6. A case where Llama 3.1 failed to follow the full chain of logic. The correct line of logic is as follows: Daphnee will be happy Celine is happy or she gets at least 7 candies. Celine will be happy if Xinyu is happy Ara is happy Becca happy she gets at least condies. Xinyu will be happy if she gets at least 7 candies. Since Xinyu received 10 candies, she is happy, and so should be Celine. Thus, Daphne should also be happy.
32

---

Compositional Causal Reasoning Evaluation in Language Models
Prompt
Xinyu, Ara, Becca, Celine, Daphne, Emma, Fox, and Yasmin are going to a party, where the host is geing to distribute candies. Xinyu will be happy if she gets at least 7 candies. Ara will be happy if Xinyu is happy Becca will be happy if Xinyu is happy or if she gets at least cardies. Celine will happy if Xinyu is happy or if Ara is happy or Becca happy or if she gets at least candies. Daphne will be happy if Celine is happy or if she gets at least candies. Emma will be happy if Daphne is happy or if she gets at least 7 candies, Fox will be happy if Daphne is happy least candies. Yasmin will be happy if Emma happy or Fox happy or she gets at least candies. After distributing the candies, Xinyu gets 6, Ara gets 9, Recca gets 7, Celine gets 7, Daphne gets 5, Rmma gets 5, Fox gets 4, and Yasmin gets :. la Yaamin happy? Be concise possible.
if he gets
least candies.
if Emma is happy
if he gets
Response
Xinyu is not happy because she gets candies, which is less than 7. Ara happy because he gets 9 candies, which is more than 7. Becca is happy because she gets 7 candies, which is exactly 7. Celine is happy because Ara is happy. Daphne is not happy because she gets 5 candies, which is less than 7, and Celine is happy. Emma not happy because she gets 5 candies, which less than 7, and Daphne not happy. Fox is not happy because he gets candies, which is less than 7, and neither Daphne nor Emma is happy. Yasmin is not happy because she gets 1 candy, which is less than and neither Emma nor Fox is happy. Therefore, no, Yasmin is not happy!
Figure G.7. A case where GPT-4o with CoT failed to follow the full chain of logic given a factual prompt used to compute PNSxy- The first reasoning error is in bold. In reality, Daphne happy if her causal parent Celine happy or if she receives least 7 candies. The model correctly reasons that the number of candies given to Daphne does not satisfy these conditions (5 is less than 7). Despite the fact that the model acknowledges that Celine happy twice, contradicts itself and incorrectly reasons that Daphne is unhappy. This erro then propagates through the remaining logic.
33

---

least candies. Compositional Causal Reasoning Evaluation in Language Models
Prompt
Xinyu, Ara, Becca, Celine, Daphne, Emma, Fox, and Yasmin are going to party, where the host is geing to distribute candies. Xinyu will be happy if she gets least " candies. Ara will be happy is Xinyu Becca will be happy if Xinyu is happy or if she gets at least 7 candies. Celine will be happy if Xinyu is happy or if Ara is happy or if Becca is happy or if she gets least candies. Daphne will be happy if Celine is happy gets at least 7
happy or he gets
if she gets
least candies. Emma will happy candies. Fox will be happy Daphne is happy or if Emma is happy or he gets at least candies. Yasmin will be happy if Emma is happy or if Fox is happy or is she gets at least 7 candies. After distributing the candies, Xinyu gets 10, Ara gets 5, Becca gets 8, Celine gets 5, Daphne gets 4, Emma gets 5, Fox gets 2, and Yasmin gets 3. Now, suppose that Xinyu not happy regardless the candy distribution. With this assumption, is Yasmin happy? Be as concise as possible.
Daphne
happy or
Response
If Xinyu is not happy regardless of the candy distribution, then she is not happy even though she gets candies. In this case, Yasmin will be happy if Emma is happy, Fox is happy, or if she gets at least candies. Yasmin gets 3 candies, which is less thar. 7, so she is not happy based on her own candies.
Now, let's check if Emma Fox is happy. Emna will be happy if Daphne is happy or if she gets at least candles. Emma gets 5 candies, which less than 7, and Daphne is not happy because she gets candies, which is less than 7, and Celine is not happy either. Therefore, Emma is not happy.
Fox will happy if Daphne is happy, Emma is happy, candies. Fox gets 2 sandies, which less than 7, and since neither Daphne nor Emma is happy, Fox is not happy.
gets least 7
Since neither Enna Fox is happy, Yasmin is not happy. Therefore, no, Yasmin is not happy.
Figure G.8. A case where GPT-4o with CoT failed to follow the full chain of logic given a counterfactual prompt used to compute PNSxy- An intervention is placed Xinyu such that is unhappy despite receiving enough candies happy. The reasoning error bold. While the model correctly extracts the causal parents of Yasmin, Emma. and Fox, it does not express understanding of the causal structure upstream. In reality, Becea happy hecause she received greater than candies, which is sufficient to make Celine happy. In turn, this is sufficient to make Daphne happy. However, the model reasons that both Daphne and Celine are unhappy, without providing further explanation. This error then propagates through the remaining logic.
34

---

least candies. Compasitional Causal Reasoning Evaluation in Language Models
Prompt
Xinyu, Ara, Becca, Celine, Daphne, Emma, Fox, and Yasmin are going te a party, where the host is going to distribute candies. Xinyu will be happy if she gets at least ? candies. Ara will be happy if Xinyu is happy
if he gets
happy or if she gets at least cardies. Celine sill be happy if Xinyu is happy or if Ara is happy or if Becca is happy or if she gets at least 7 candies. Dapane will be happy if Celine is happy or if she gets at least candies. Emma will парру if Daphne is happy
Becca will be happy if Xinyu
if she gets at least 7 if he gets at
candies. Fex will be happy if Daphne is happy least candies. Yasmin will be happy if Rmma is happy or Fox is happy or she gets at least 7 candies. After distributing the candies, xinyu gets 6, Ara gets 8, Becca gets 6, Celine gets 7, Daphne gets 2, Enma gets 4, Fox gets 3, and Yasmin gets 2. Is Yasmin happy? Be concise possible.
if Rmma is happy
Response
Xinyu is not happy because she gets 6 candies, which is less than 7. Ara is happy because he gets 8 candies, which is more than 7. Celine is happy because Ara is happy. Daphne is not happy because she gets 2 candies, which is less than 7, and Celine is the only one who can make her happy. Erma is not happy because she gets candies, which is less than 7, and Daphne is not happy. Fox is not happy because he gets 3 candies, which is less than 7, and neither Daphne nor Emma is happy. Yasmin is not happy because she gets 2 candies, which is less than 7, and neither Emma nor happy.
Fox is happy. Therefore, Yasmin
Figure G.9. A case where GPT-4o with CoT failed to reason correctly despite extracting the correct causal relations from factual prompt used to compute PNSny. The first reasoning error is in bold. As in Figure G.7, the model correctly implies that Celine (who is happy) is the only causal parent Daphne underlying DAG, and yet still incorrectly concludes that Daphne is unhappy. This error then propagates through the remaining logic. This same error was observed multiple times from GPT-4o with CoT.
Prompt
Xinyu, Ara, Becca, Celine, Daphne, Emma, Fox, and Yasmin are going to party, where the host is going to distribute candies. Xinyu will be happy if she gets least 7 candies. Ara will be happy is Xinyu
happy or he gets
least candies.
Becca will be happy if Xinyu is happy or if she gets at least 7 cardies. Celine will be happy if Xinyu is happy or if Ara is happy or if Becca is happy or if she gets at least candies. Dapane will be happy if Celine is happy or if she gets at least candies. Emma will be happy
Daphne is happy or if she gets at least 7 candies. Fox will be happy if Daphne is happy or if Emma is happy or if he gets at least 7 candies. Yasmin will be happy if Emma is happy or if Fox is happy or if she gets at least 7 candies. After distributing the candies, Xinyu gets 1, Ara gets Becca gets 8, Celine gets 6, Daphne gets 8, Emma gets 4, Fox gets 10, and Yasmin gets New, suppose that Dapnne is not happy regardless of the candy distribution. With this assumption, is Yasmin happy? Be as concise as possible.
Response
Since Daphne is happy regardless, Yasmin can only be happy if Emma is happy or if she gets at least candies. Emma will be happy she gets at least candics, but she only gets 4 candies. Yasmin gets 3 candies, which is less than 7, Cherefore, no, Yasmin is not happy!
Figure G. A case where GPT-4o with CoT failed to extract causal relations correctly when given a counterfactual prompt used to compute PNSpY. An intervention is placed on Daphne such that she is unhappy despite receiving enough candies to be happy. The first reasoning emor is in bold. In reality, Yasmin's causal parents in the underlying DAG Emma and Fox, but the model only acknowledges Emma. Though the model correctly reasons that Emma is unhappy, Fox is happy despite Daphne's unhappiness because he has received 10 7) candies. As the model does not correctly extract the relevant causal relations, ignores the latter fact and incorrectly concludes that Yasmin is unhappy. This same error persisted across multiple replicates.
35

---

Compositional Causal Reasoning Evaluation in Language Models
G.2. Results
1.30
Internal consistency RAE
0.75
0.50
0.25
External validity RAE
Phi-3
Llama 2
Liama Liama 3.1
Liam 3.1 CoT
Liama 3.1 Math
Llena 3.1 Math CoT *GPT-1o CPT Xot Xot CoT
Figure G.II. [Full distribution for Fig. 5.1 For PNS compositions 1000 estimates per quantity per model), compare RAE w.r.t. ground truth (external validity) and PNSxy (internal consistency) to visualize four reasoning quadrants (VI/IC in yellow; VC in green; Il in white). Dotted lines are error thresholds (RAE=0.1) Models are listed by increasing size (Table G.1).
Local
Global
Composition
PNS
0.2
0.0
DY
XY
XC-CD-DY
XO-OY
X
Phi-3 i Llama 2 Llama 3 Liama 3.1 I Llama 3.1 CoT Llama 3.1 Math Llama 3.1 Math CoT GPT-40 GPT-40 CoT not #o1 CoT
Figure G.12. Estimated PNS distributions (n 1000) for the quantities given in Table 1, denoted here by cause-effect pair (e.g.. Bold black line segments represent ground truth values.
XC CD DY denotes
36

---

Compasitional Causal Reasoning Evaluation in Language Models
01
01 CoT
GPT-4o
GPT-4o CoT 100
100
100
100
100
100
100
100
100
100
160
100
100.
160
100
100
100
100
100
100
100
100
100
36.9
800
50
25
XC-CD-DV
XD-DY
XD-DY
XC-CD-DY
XC-CY
XD-DY
XC-CD-DY
XC-CY
XD-DY
KC-CD-DY Xe-oY
x
xo-oY
Llama 3.1
Liama 3.1 CoT
Llama 3.1 Math
Liama 3.1 Math CoT
100
Percent valid (RAE 0.1)
76.7 78.4
75
73.5
50
47.
25
XC-CD-DV
XD-CV
XD-DY
XC-CD-DY
XC-CV
KD-DY
XC-CD-DY
XC-CY
XD-DY
XC-CD-DV
XC-CY
XD-DY
Liama
Liama
Phi-3
75
50
1.7
11
XC-CD-DV
XD-DV
XD-DY
10
XC-GD-DY
XC-CV
XD-DV
40
40
XD-CD-DV
XC-CY
YD+DV
Cause-effect pair
Figure G.1.3. Percent of PNS estimates (n = 1000) that are externally valid. Reasoning was considered externally valid for a quantity 90% of estimates had RAE < 0.1 (represented by red dashed line). PNS are denoted by cause-effect pair (e.g.. XC CD- DY denotes PN PNSxcPNScoPNS5ppp
37

---

Compositional Causal Reasoning Evaluation in Language Models
01 CoT
GPT-4o
GPT-4o CoT
100
100
100
100
100
100
95.87
75
25
0.08
Llama 3.1
Llama 3.1 CoT
Llama 3.1 Math
Llama 3.1 Math CoT
100
Percent valid (RAE 0.1)
75
25
10.04
Llama 3
Llama 2
Phi-3
100
75
25
1.23
Composition
Global
Local
Figure 1.14. Percent of PNS estimates (n 1000) that are externally valid. Reasoning was considered externally valid for quantity if 90% of estimates RAE < 0.1 (represented by the red dashed line).
38

---

Compositional Causal Reasoning Evaluation in Language Models
01
01 CoT
GPT-4o
GPT-4o CoT
0 100
100
2,0608
2.0
0.3025
0.076
0.078
0.0
0.4g8C
000
0.0po8
00000
09645
0
0.1588
0.1501
0.00
0.0191
0.015
5
6
d
8
6
6²
10
6.4664
4.0
8.3
Mean RAE
1-4058
0,7865
0 apaa
0 96333
0.4263
0 epus
11111
0,5057
0.1435
0.3104
0.6198
Llama 3
Liama
Phi-3
0.7007
11:2122
5.0
2.5
2:1486
3 57588
10225
1.4519
0. 4.403
0,7819 0,176
0.5425
0.8522
0,0605
6P
Cause-effect pair
Figure G.15. Mean external validity RAE (n = 1000). Error bars represent standard deviations. An estimate was considered externally valid quantity if RAE < 0.1 (represented by the dashed line).
39

---

Compositional Causal Reasoning Evaluation in Language Models
CD
CY
DY
0.8
2.0
0.6
1.5
0.4
0.2
0.5
0.0
0.0
XC
XD
XY
4
Relative absolute error
1.5
15
1.0
10
0.5
0.0
XC-CD-DY
XC-CY
XD-DY
2.0
1.5
4
1.0
2
0.5
0.0
Phi-3
Liama 2
COT
Math COT
Math
GPT-40 -T-40
CoT
01 CoT
07
Phi-3
Liama 2
1
COT
Math COT
Math
GPT-40
GPT-40 40 COT
01 CoT
01
Phi-3
Llama 2
1
Math
GPT-T4
CoT
01 CoT
01
Liama
Liama 3.1
Liama
Llama 3.
1
GPT-40
Liama
Llama 3
Llama
3.
1
Liama
Liama
Llama 3.
Math
GPT-40Co
Liama
Liama
Llama 3.1
Llama 3.1
Llama 3.1
Figure G.16. RAE distributions for all quantities. Red lines represent the external validity cutoff (RAE = 0.1). PNS are denoted by
cause-effect pair (e.g.. XC CD- DY denotes PNSxx
40

---

Compasitional Causal Reasoning Evaluation in Language Models
Mean relative absolute error
10
32
128
Total model parameters
Composition
Global
Local
15
Mean relative absolute
10
Liama
Liama 3
Llama 3.1
Liama 3.1 Math
Llama version
Composition
Global
Local
Figure 1. Mean RAE (without CoT) does not consistently decrease with increasing model size (log2 scale; left) nor Llama version (right). However, errors for global quantitics (PNSxy) do monotonically decrease with increasing Llama version. Values were averaged separately for global, local, and composed quantities (Table I). Error bars represent standard deviations. Parameter count for GPT-4o set the GPT-3 count (175B) due to information availability.
Mean RAE
10
Length of shortest path from cause to effect
+ Phi-3
Liama
Liama
Liama 3.1
Liama 3.1 CoT Llama 3.1 Math Llama 3.1 Math CoT GGP--40 GPT-40 CoT
Xot Xot CoT
Figure G.18. RAE generally inereases with length of shortest path from cause effect. Red dashed line denotes external validity cutoff
(RAE 0.1), with standard deviations shaded regions.


Reviews:
[{"review_id": "hFwg73IFFv", "reviewer": "Reviewer_Xrrd", "summary": "The paper proposed a unified framework that evaluates compositional causal reasoning ability in large language models by measuring the average treatment effect and the probability of necessity and sufficiency.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their useful feedback. We hope that the following addresses your concerns.\n\n1. **Limited results / no dataset.**\n   - **Revision 1:** CandyParty experiments now include **results for o1 with and without CoT.** For o1, CCR reasoning was “complete,” enriching our discussion by (1) validating the correct formulation of the task and (2) empirically demonstrating VC reasoning. Revised Figs 5/7 are at https://tinyurl.com/bdddu2vx under `/figures_and_tables/updated_figures/`. \n   - **Revision 2:** “Results for task T revealed three taxonomically distinct error patterns: VC (o1), near-VI (GPT-4o with CoT), and II (all remaining models). Only o1 placed mass in the VC reasoning quadrant (Fig. 5)... However, internal inconsistency made GPT-4o with CoT a near-VI reasoner: 75.1% of all estimates were near-valid,... and yet failure to correctly infer $\\mathrm{PNS}_{XY}$ revealed internally inconsistent reasoning.” See our **response to Reviewer doGN** for further discussion and error analysis for GPT-4o. We hope these updates clear confusion on Fig 5, where each quadrant represents a reasoning type.\n   - **Revision 3:** To facilitate future benchmarking, we offer **open source Python code for automated task generation** for Algorithm 1 / Theorem 1. The user can scale the graphical complexity and choose from 4 themes: candy party, flu vaccination, football, and flower gardening. Further unit tests will be available at camera ready:  https://tinyurl.com/5be5ehyp and https://tinyurl.com/4um56dab\n2. **Figure 7 / Task not challenging for GPT-4o with CoT.**\n   - As noted, this work is not meant as a benchmarking study but a validation of our framework. As further emphasized by our new results and discussion for o1, we chose a low complexity SCM to evaluate a range of models that we anticipated to **reflect a spectrum of CCR behavior.** The fact that our experiments demonstrate partial success (GPT-4o CoT), full success (o1), and failure (all other models) is important, as it validates that our framework is both correctly formulated and capable of disentangling distinct error patterns. \n   - **Revision:**  Added to concluding remarks: “Open source and smaller models did notably worse on this task than GPT-4o (CoT) and o1, suggesting that even low-complexity CCR is an area on which significant progress can still be made for an important class of LMs (smaller and/or open).”\n4. **Motivation, usefulness, comparisons.**\n   - Novelty: (1) compositional consistency, (2) CCR evaluation in LMs, (3) metrics, (4) taxonomy of distinct error patterns, applicable beyond the causal setting, (5) practical algorithm.\n   - **Revision 1:** Added: “Comparison to Prior Methods: See Table A.1 for an extensive comparison. To further clarify our contributions, we compare to the CLADDER benchmark for formal causal reasoning (Jin et al., 2023). Most prior causal reasoning evaluation centers on commonsense knowledge and relation extraction (Table A.1). Like CLADDER, our framework and task generator (1) probe reasoning at the associational, interventional, and counterfactual levels; (2) apply causal inference methods; and (3) evaluate causal relation extraction. Our method notably departs from CLADDER in the following ways: (1) we evaluate compositional reasoning, another dimension of causal reasoning that has received little attention in LMs; (2) while CLADDER uses simple graphs (3-4 nodes), our task generator allows the user to scale graphical complexity; (3) we introduce new metrics for internal compositional consistency of causal reasoning; and (3) we do not require ground truth quantities, as the user can opt to evaluate on internal consistency only.”\n   - **Revision 2:** See Table A.1: https://tinyurl.com/4ctw6bfx.\n5. **Required access to the true graph.**\n   - This is **standard in evaluation (see Corr2Cause, CLADDER, or arXiv:2402.11821)**. Evaluation innately requires reference values. The identifiability conditions for valid causal inference are extensively discussed in prior works. In general, these require **at least partial structural knowledge** (e.g., knowledge of confounders that block backdoor paths; of a valid instrumental variable when latent confounding is present; of mediators for the frontdoor adjustment; etc). \n   - This work is **explicitly rooted in causal graphical modeling** (rather than potential outcomes). We explicitly measure reasoning over graphical components (Theorem 1). Graphical reasoning is an important area itself. Thus, the **causal graph is centrally relevant.** \n   - **Revision:** Added to Sec 5.2: “When ground truth values are unavailable, Alg.1 can be run for internal consistency only.”\n6. **“Metrics / quantities should be accompanied by examples.”**\n   - Each definition is accompanied by concrete examples related to Example 3.2, Fig 2, Fig 3 (e.g. \"internally consistent reasoning entails that $\\theta($...\"). If the reviewer has specific confusions, we are happy to clarify further.", "claims_and_evidence": "The paper claims to introduce a framework for a comprehensive assessment of compositional consistency and demonstrates its utility through experimental results. However, the experimental results and setup explanation do not clearly illustrate the framework's usefulness or the specific insights it provides. Additionally, Figure 5 is quite confusing.", "methods_and_evaluation_criteria": "The paper proposes evaluation metrics for assessing compositional causal reasoning, but it does not effectively explain the motivation behind such a framework. It is unclear why existing reasoning evaluation tasks are insufficient for evaluating causal reasoning ability.\n\nAdditionally, the explanation of the proposed method could be improved. The paper relies heavily on notations and definitions rather than concrete examples, making it difficult to follow. A rigorous mathematical definition of the metrics and quantities should be accompanied by examples to clarify their significance.\n\nFrom my understanding, utilizing this framework requires access to the ground truth causal graph, which is often unavailable and difficult to obtain. Moreover, the paper presents only a framework without an accompanying dataset, which could significantly hinder its practical applicability.", "theoretical_claims": "Please see Methods And Evaluation Criteria.", "experimental_designs_or_analyses": "The experimental results presented in the main paper are limited. While additional results are provided in the supplementary material, the discussion and analysis are insufficient to convincingly demonstrate the usefulness of the proposed framework.\n\nIn particular, I found Figure 5 very confusing, and Figure 7 suggests that the task is not particularly challenging for GPT-4o with CoT. However, these results are presented without sufficient explanation or discussion of their implications.\n\nMoreover, in the last paragraph of Section 6.2, the paper notes that errors increase as the complexity of paths increases—a rather obvious statement. If this is the main conclusion drawn from the experiment, it does not seem to add much value as a key result.", "relation_to_broader_scientific_literature": "The paper claims that the proposed framework explicitly and systematically incorporates causal compositionally. However, it does not clearly state the specific gap it addresses or aims to close.", "essential_references_not_discussed": "NA", "other_strengths_and_weaknesses": "The paper includes extensive background content in the supplementary materials, which helps readers understand the terminology and derivations.", "other_comments_or_suggestions": "NA", "questions_for_authors": "NA", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "a6DyicgnSo", "reviewer": "Reviewer_qJVr", "summary": "Regarding the combination of causal and combinatorial reasoning in generative AI, this paper presents a unified perspective called combinatorial causal reasoning (CCR), which is the ability to infer how causal measures are combined and how causal quantities are propagated in a graph. A framework for systematic evaluation of CCR is developed for average treatment effect (ATE) and probability of necessity and sufficiency (PNS). As a proof of concept, we demonstrate the design of CCR tasks on language models from the LLama, Phi, and GPT families, and experimentally show that CCR errors increase as the causal path complexity increases.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their attention to the correctness of our proofs, the powerful abstraction provided by CCTs, and the gap in the literature that we attempt to close: the explicit and systematic evaluation of compositional consistency in causal reasoning. We address questions below.\n\n1. **Low problem complexity.** \n    - As further emphasized by our **new results and discussion for o1** (see response to Reviewer Xrrd), we chose a relatively low complexity SCM to evaluate a range of models that we anticipated to reflect a spectrum of CCR behavior. The fact that our revised experiments demonstrate partial success (GPT-4o CoT), full success (o1), and failure (all other models) is important, as it validates that our framework is both correctly formulated and capable of disentangling distinct error patterns. \n   - We have **released automated task design code** that enables the user to generate CCR prompts based on casual graphs of scaling graphical complexity. An important future direction will be to benchmark on CCR tasks that simulate real-world problems (e.g., in healthcare), which the \"vaccination\" setting in our new task generation code is a first step toward.\n2. **Parametric assumptions / estimands (ATE w. linearity and PNS w. monotonicity).** Though we instantiate our framework under select conditions in Sec 5, the framework presented in Sec 4 is highly flexible and is not limited to any estimand nor parametric assumptions. As stated in Sec 4.2, “Sec 5 suggests **one possible** procedure for assessing compositional consistency in causal reasoning (Alg. 1).” Under Limitations, we state “As this work only considers the ATE and PNS under Theorem 5.1, extensions could consider other estimands and compositional forms.” \n   - We provide Example 3.2 as one hint for future work: the decomposition of the TE under linearity. However, Pearl introduced a nonparametric mediation formula similar to Example 3.2 that could just as easily be exploited for CCR. Any nonparametric formula will be particularly suited to “realistic/real-world” task design. The possibilities are only limited by the extent of known compositional formulae in causal inference.\n   - **Revision:** Under Limitations: “Potential extensions could implement CCR tasks based on Example 3.2 or for Pearl’s nonparametric mediation formula, which extends Example 3.2 to nonlinear settings.”\n3. **Unobserved confounding.** Sec 4 does not require that all confounders are observed, rather that valid causal inference can be performed. Sec 5 assumes this only to simplify task design.\n   - **Revision:** \"Future work could accommodate latent confounding by exploiting known identifiability results for such cases.\" (See Pearl for many related works.)\n4. **Inference hyperparameters.** Added to a new table in Appendix F.1.\n5. **Elaboration on CCTs.** We thank the reviewer for acknowledging the utility of CCTs. We direct them to Figure D.1 of the original submission for more examples. \n   - **Revision 1:** Added to discussion w/ link to Fig D.1: \"This allows for evaluation on arbitrarily complex DAGs with cutpoints as if they were simply directed chains (Fig. D.1)... As demonstrated in Sec 5.2, CCTs can be leveraged as a design tool for formulating reasoning tasks. As illustrated in Sec 6.2, they can also serve as an interpretable, intuitive visualization tool for graphically representing CCR successes and failures (Fig. 9).\"\n   - **Revision 2:** New Fig 9 -- CCTs as a visualization tool for error analysis: https://tinyurl.com/yc5cf9ev\n   - **Revision 3:** We provide open source **code to generate CCTs from random DAGs** under sufficient conditions, along with our new automated task generation method: https://tinyurl.com/5be5ehyp and https://tinyurl.com/4um56dab. \n6. **Mechanisms underlying failures / CoT and internal consistency.** Mechanistic explanations (e.g., limitations in attention mechanisms), additional prompting strategies, architectural modifications, or novel fine-tuning regimes (e.g., Huyuk et al 2025) are all potentially important areas for future work.\n   - **Revision:** While most of the above is out of scope for this work, we expand our error analysis for further insight. See our **response to Reviewer doGN.** Examples: https://tinyurl.com/bdddu2vx under `/figures_and_tables/new_figures/error_analysis_figures/`.\n8. **Data and code release.**  All code provided in the repo above will be public, as will code used for **simulations** in Appendices C/E. Notably, this includes our **automated task generation codebase,** which enables customized prompt dataset generation. Code and data related to model inference will at minimum be available upon reasonable request after publication.\n8. **Significance tests: CoT vs non-CoT.** See https://tinyurl.com/yck6bd28 for results. All error distributions for CoT were significantly different than non-CoT except for o1 on PNS(X,Y) (Wilcoxon, alpha = 0.05).", "claims_and_evidence": "The main idea of this paper is to put forward a unified evaluation framework for combinatorial causal reasoning (CCR), and to prove the effectiveness of this framework through experiments.\n\nClaim:\n1. Combining causal reasoning and combinatorial reasoning through structural causal model (SCM) and combinatorial rules.\n2. External validity (consistency with the true value) and internal consistency (consistency of the model itself) can measure the CCR capability.\n3.  With the increase of causal path complexity, model error rate increases, and CoT hints are helpful for some models.\n\nEvidence supports:\n1. The authors demonstrate the composability of PNS in double-connected components (BCCs) through mathematical definitions and theorems (e.g. Theorem 5.1), providing a theoretical basis for the framework.\n2.  Experiments on LLaMA, Phi, and GPT series models show a significant decline in the model's performance on complex causal pathways, and CoT suggests some help for GPT-4O.", "methods_and_evaluation_criteria": "Methods:\n1. The use of structural causal model (SCM) and directed acyclic graph (DAG) is a standard method in the field of causal reasoning, suitable for the decomposition and synthesis of combinatorial reasoning.\n2.  The definition of composition as the functional relationship between global and local measures conforms to the tradition of probability graph model, but it may be too broad and lack the pertinence of specific tasks.\n3.  Simplifying complex DAGs into chain structures through exchange tree cutting (CCT) is helpful for systematic evaluation, but some structural features of the original drawings may be ignored.\n\nEvaluation criteria:\n1. External validity and internal consistency can measure the accuracy and reasoning consistency of the model respectively, and are reasonable evaluation dimensions.\n2.  Mathematical problems based on CandyParty can test counterfactual reasoning, but the problem complexity is low, and may not fully reflect the CCR ability in real scenarios.\n3. The evaluation is only for ATE and PNS, and the causal function is assumed to be linear or monotonic, which may not be applicable to more complex causal relationships.", "theoretical_claims": "1. The proof in Appendix B.2 correctly shows that PNS composes multiplicatively across BCCs under monotonicity. The derivation leverages the monotonicity assumption to simplify joint probabilities, leading to the product formula. However, the proof assumes no unobserved confounders (Assumption A4), which may limit its applicability to real-world DAGs with latent variables. \n2. The equivalence between PNS and ATE under monotonicity is derived correctly in Appendix B.3.1. By expressing ATE as a difference of probabilities and relating it to PNS via Equation (B.36), the authors establish a crucial link that justifies using ATE and PNS interchangeably for CCR evaluation.", "experimental_designs_or_analyses": "1. Using binary variables and logical \"or\" functions for SCMs is appropriate for monotonicity assumptions required by PNS/ATE compositionality (Theorem 5.1).    However, the choice of a single toy problem (CandyParty) limits generalizability to real-world causal scenarios with non-monotonic relationships or continuous variables.\n2. Sampling 1000 exogenous variable sets per quantity is sufficient for distributional analysis, but the lack of statistical significance tests (e.g., t-tests for CoT improvements) weakens claims about CoT’s effectiveness.", "relation_to_broader_scientific_literature": "Aiming at the problem that causal compositionality is not clearly and systematically combined in previous work, this paper proposes a formal expression of CCR, which defines combinatorial causal reasoning as the ability of model induction and deductive reasoning, and compensates for this problem through the establishment of a theoretical framework.", "essential_references_not_discussed": "na", "other_strengths_and_weaknesses": "na", "other_comments_or_suggestions": "The commutative cut tree (CCT) transformation (Section 5.1) is a powerful abstraction but requires more detailed explanation to ensure readers grasp its role in simplifying complex DAGs.", "questions_for_authors": "1. The error analysis suggests that models struggle with longer causal paths due to \"error propagation\" and \"losing the train of thought.\" Could you elaborate on the mechanisms underlying these failures? For example, are they due to limitations in attention mechanisms, parameter capacity, or task-specific prompt design?\n2. The CoT prompting improved external validity for GPT-4o but not internal consistency. Do you have hypotheses about why CoT aids in local quantity estimation but fails to ensure compositional consistency? Could this be addressed through more sophisticated prompting strategies or architectural modifications?\n3. The paper mentions releasing data and code upon publication. Could you provide a timeline or additional details about the reproducibility package (e.g., which components will be included, dependencies, etc.)?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "tuaN9LomQK", "reviewer": "Reviewer_doGN", "summary": "In their paper, the authors formalize ways to measure the ability of reasoning systems to consistently reason over compositional causal quantities. The authors propose the general task of compositional causal reasoning (CCR) which is then defined in terms of external validity (- the adherence to ground truth quantities) and internal consistency (- answer consistency over different compositions of the same quantity). The authors define metrics to qualitatively detect the presence of both quantities and create a taxonomy that feature validity and consistency (or any subset of those) to assess the ability of models to perform CCR reasoning.\n\nThe authors specifically measure reasoning compositionality in terms of combination of average treatment effect (ATE) and necessity and sufficiency (PNS). Here, the special case of multiplicative compositionality is considered and (given particular monotonicity assumptions), the authors proof alignment of ATE and PNS. To test multiplicative compositionality, commutative cut trees (CCT) are introduced which result in chains of graphical components. The correct working of the proposed metrics is evaluated on a synthetic graph.\n\nThe authors propose an algorithm to test validity and consistency under all possible combinations of compositional ATEs using the proposed CCT. Experiments are carried out over a simple synthetic graph and constructed real-world example using several Llama versions, Phi-3-mini and GPT-4o. Experiments indicate better CCR performance for more recent models, with performance degrading with the increasing number of composed components. None of the tested models seems to feature valid and consistent behavior.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their thorough feedback and for acknowledging the soundness of our approach. We hope that the comments below address all concerns.\n\n1. **“Outliers” for GPT-4o (CoT).**\n   - Results for $PNS_{XY}$ and $PNS_{DY}$ are not extreme outliers, though Fig 7 might imply such (we update to clarify this possible misinterpretation in-text). Please see original Figs F.7, F10: though the model did not achieve external validity according to our pre-defined threshold, predicted PNS distributions were not extremely far from truth.\n   - **Revision 1:** Added to Sec 6.2 (new figures https://tinyurl.com/bdddu2vx under `figures_and_tables/new_figures/error_analysis_figures/`): \"Preliminary analyses indicate that GPT-4o with CoT displayed several kinds of faulty reasoning for both $PNS_{XY}$ (Figs. F.6, F.7) and $PNS_{DY}$ (Figs. F.8, F.9). These include (1) **failure to correctly extract causal relations** (e.g., true causal parents are missing from the stated parent set; Fig. F.9), (2) **incorrect logic despite correct causal relation extraction** (e.g., a clause will not logically follow from the preceding clause; Fig. F.8), and (3) a **truncated reasoning process**, where the model expresses its logic up to a certain point in the underlying causal graph but ignores the remainder of the graph without explanation (e.g., Fig. F.7).\"\n2. **CoT formulation.**\n   - The reviewer notes that “proper working of the evaluation framework can only be shown within a valid setup.” This is true, and **new results for o1** (which show complete VC reasoning) verifies that this setup is **valid and achievable by a sufficiently capable model.** See response to Xrrd. \n   - Potential influence of prompt engineering indeed motivated our comparison of CoT and non-CoT prompting. Our results for GPT-4o do indicate sensitivity to this. Though it is out-of-scope to perform additional prompt engineering experiments, this is an important area for future inquiry.\n   - **Revision**: Added under Limitations: \"Future work should explore sensitivity to prompt engineering, including alternative formulations of CoT.\"\n3. **Is VI reasoning a valid category?**\n   - **Revision:** As the VI reasoner is a counterintuitive yet informative category, we added the following to Sec 6.2: \"This framework not only reveals *if* the reasoner is wrong, but offers insight into *how* it is wrong. We take VI reasoning as a stark and perhaps counterintuitive example, where external validity alone (as typically measured in reasoning evaluation) provides an incomplete picture. Consider GPT-4o with CoT: the model was near-valid to valid on the majority of quantities, implying some measure of success. Upon closer inspection, it was (near-)valid on most local estimates (e.g., $\\widehat{\\mathrm{PNS}}\\_{CD}$) and their products (e.g., $\\widehat{\\mathrm{PNS}}\\_{XC}\\widehat{\\mathrm{PNS}}\\_{CY}$) but not for $\\widehat{\\mathrm{PNS}}\\_{XY}$. While it might appear reasonable to assume that invalid global estimates stem from poor local reasoning or relation extraction abilities, the potential for alternative explanations is a key insight of our work. Here, GPT-4o displayed compositional inconsistency despite local relation extraction: it was more performant when asked directly about relatively proximal local relationships than when asked about distal cause-effect pairs where the answer relied on correct compositional reasoning over multiple direct and indirect relationships. VI reasoners highlight this potential failure mode: the reasoner does not learn to compose over multiple strands of logic yet can answer local questions when asked directly (akin to a student passing a math quiz simply by memorizing specific answers, instead of synthesizing; see incorrect logic despite relation extraction in new Fig F.8). Uncovering this behavior is precisely the benefit of measuring both external validity and internal consistency.\"\n4. **Proof – B.20 $\\to$ B.21.** \n   - We thank the reviewer for reading our proof line-by-line! This is due to monotonicity: $P(y′\\_{x}) = 0$ and $P(z′\\_{y}) = 0$ (outcome cannot be false when treatment is true). So, $P(y′\\_{x}, y′\\_{x′} )P(z\\_{y′}, z′\\_{y′})$ and $P(y\\_{x},y\\_{x′})P(z_{y},z′\\_{y})$ zero out.\n   - **Revision:** Added to line in proof: “By monotonicity.”\n5. **Potential outcomes vs graphical models.** \n   - This is not a problem, and going between the two is common (as they each provide their own usefulness). Note: “Formally, the two frameworks are logically equivalent; a theorem in one is a theorem in the other, and every assumption in one can be translated into an equivalent assumption in the other.” — Judea Pearl\n6. **Complexity analysis.** We thank the reviewer for their attention and agree that this analysis can be improved.\n   - **Revision:** Notebook validating the improved complexity analysis: https://tinyurl.com/3tnut7tj\n7. **Quotation marks in Fig F.2.** This was a copy-paste error. Updated figure: https://tinyurl.com/nhctxkat", "claims_and_evidence": "The authors claim to propose novel metrics, which are able to assess the compositional causal reasoning (CCR) abilities of reasoning systems. The proposed external validity and internal consistency metrics are generally suited to measure the intended effects.\n\nThe authors leverage and proof the compositionality of ATE and PNS within a restricted setting (monotonicity, no confounding, existence of cut points, single source/leaf node). The claimed compositionality is furthermore practically validated on a synthetic graphical model.\n\nCategorization of the tested LLMs under the proposed taxonomy seem to indicate that no model yields valid and consistent CCR behavior. Here, strong outliers within model individual performance might draw the particular conclusion with respect to the prompting method into question. However, the prior proposed evaluation framework and algorithm are unaffected.", "methods_and_evaluation_criteria": "General approach on inspecting compositional causal reasoning abilities of reasoners is well motivated. The CCR aspects of validity and consistency (Def. 4.2 and 4.3) are soundly defined and natural candidates for the specific setup. The choice of ATE is common and the relation to PNS is explained and proven, making it a valid target metric. The proven PNS compositionality is shown over a simple yet sufficient graph.\n\nExperiments on LLM where, again, tested over the synthetic graph (assumption due to Fig. 7 only featuring variables X,C,D and Y) and a constructed 'CandyParty' example, where variables in a chain depend on a given, fixed constant and previous variable value(s). While both setups are synthetic, they are sufficient to assess the proposed evaluation framework.\n\nAlthough the paper only considers compositionality in a purely mathematical sense (e.g., requiring the full disentanglement of components and exclusion of compositional emergent properties), both setups are suited to show proper working of the proposed evaluation framework and the CCR task in general. The shown percentages of PNS estimates for individual components and the evaluation of validity, consistency and number of mediators in terms of (relative) absolute errors seems to be a natural choice.", "theoretical_claims": "I intuitively assume Theorem 5.1 in Appendix B.2 to be sound under the made assumption of monotonicity. However, I am not particularly familiar with the implications of applying potential outcome perspectives onto graphical models. The transformations within the presented formulas seem mostly to be consistent. Upon closer consideration it was unclear to me, how formulas B.20 -> B.21 simplify. E.g. why dropping the first and last terms of the addition is permitted (or rather why the respective terms have zero probability).\n\nThe introduced of commutative cut trees (CCT; Def. 5.2) is well defined and poses a practical aid to iterate the possible compositional combinations within the proposed algorithm.\n\nWhile technically correct, the analysis of time complexity of the algorithm in Appendix D.1 might be expressed slightly more explicitly in the number of nodes only. From my understanding the considered setup reduces to paths between single source and sink nodes along a chain with all subpaths existing due to compositionality. Here, the problem might be formalized as selecting individual cut points from the superset of all cut points. By 'excluding' source and sink (but adding the fully compositional path), the number of compositional paths $k$ should be upper bounded by (or rater exactly be)  $2^{n-2} + 1$ for all $n>2$.", "experimental_designs_or_analyses": "The validity of Theorem 5.1 is empirically demonstrated via simulation in Appendix E. The experiment is suited to show and confirms the desired effects. The proposed \"inductive CCR evaluation\" algorithm follows naturally from the proposed definitions and metrics under the given multiplicative compositionality assumption.\n\nThe evaluation contains six LLMs of recent and moderate age, including of Llama 2/3.X, Phi-3-mini and GPT-4o. LLMs are prompted with and without CoT reasoning, showing expected differences in reasoning performance. CoT reasoning is realized via the provision of two example inferences. The provided CoT prompt wrapper (Fig. F.2) contains a strange number of quotation marks, which, however, shouldn't throw of newer models.\n\nHowever, and to the best of my knowledge, realization of CoT via the provision of examples is rather uncommon for recent instruction-tuned 'chat' style models (e.g. GPT-4o or particular Llama-3.1-XX-Instruct variants) and better results are obtained with writing explicit instructions (which might again include example prompts). The 'incorrect' use of CoT prompting might explain the stark decreases in performance of GPT-4o in Fig. 7 for some scenarios (DY and XY). While authors explicitly do not aim to assess performance of the inspected LLM, they might still want to investigate the causes for the strong degradation of GPT-4o performance from 100% to 0% (or 2.5%, respectively), and confirm the correct prompting of instruction instruction-tuned 'chat' style LLMs.\n\nThe further discussion of evaluations and plots in Sec. 6.2 demonstrate the proper working of the proposed evaluation framework. Particularly, the increase in reasoning errors with an increasing number of mediator nodes aligns with findings of prior works.", "relation_to_broader_scientific_literature": "The authors soundly relate their approach to general (causal) LLM reasoning. The particular aspect of consistent compositional reasoning can be seen as a precondition for general causal reasoning. Most prior works in the field only considered the overall performance of causal LLM reasoning. The newly proposed metrics, therefore, allow for a more nuanced inspection on reasoning abilities in the often required joint reasoning over composed causal quantities. The measured LLM performances generally align with the findings of prior work.\n\nAspects on the probability of necessity and sufficiency and the relation of potential outcomes to ATE seem to be sufficiently discussed and proven.", "essential_references_not_discussed": "The paper is well embedded into current literature and comprehensively cites related works on causal reasoning abilities of LLM. Aspects on general reasoning of LLM and the role of compositionality within general deep learning models are discussed and cited accordingly. The probability of necessity and sufficiency and relations to potential outcomes seem to be sufficiently discussed and cited.", "other_strengths_and_weaknesses": "**Strengths**\n\n1) The proposed metrics allow to assess the causal compositional reasoning abilities of LLM in a more fine-grained fashion than previous works that only consider the overall reasoning performance of LLM in terms of accuracy. The problem formulation and resulting metrics are well motivated and clearly defined.\n2) The focus on the special case of multiplicative compositionality allows for a natural definition of the metrics and lets the proposed algorithm follows naturally. The validation of validity and consistency aspects is clearly separated and formalized under the given setting.\n3) The experimental results are generally well presented and discussed. The experiments and simulations indicate the sound setup of the evaluation framework and the algorithm. Up to sparse outliers, the gained results indicate alignment with other findings of prior work.\n\n\n\n**Weaknesses**\n\n1) As already mentioned in the experimental designs section, I find the particularly strongly outliers within the results to be rather unusual. While the detailed evaluation LLM performance is mentioned as to not constitute the main focus of the paper, the proper working of the evaluation framework can only be shown within a valid setup. As mentioned before, the authors might improve on this topic by further investigating on the particular failure cases of the model and validate the correct application of CoT prompting for instruction-tuned chat-style LLM.\n2) Within their taxonomy, the authors consider all four possible combinations of valid and consistent reasoners. While the authors argue that valid inconsistent reasoners should exist, I would consider the validity of ground truth to force answers to be consistent. Specifically, validity bounds every prediction $\\hat\\varphi_x$ to lie within $\\pm\\delta$ of the ground truth, such that valid but inconsistent predictions might, at most, lie $2\\delta$ apart. While such samples might technically violate the imposed $\\delta$-threshold, I would rather consider this a insufficiency of the proposed definition, than an inconsistency of the reasoner. This aspect might simply be improved by slightly extending the discussion in the paper.\n3) The authors place their framework into the setting of multiplicative compositionality. Here, the forced reduction of CCT onto chains prevents the framework from considering the compositional (additive) effects of parallel components, (as, e.g., mentioned in the initial example of direct and indirect causal effects).", "other_comments_or_suggestions": "1) Following Definition 5.1, the authors talk about the evaluation of \"arbitrarily complex DAGs with cutpoints as if they were simply directed chains\". While this statement is technically true in terms of the algorithm, the complexity now lies in computing the CCT edges. Due to the assumed compositionality, the authors might want to consider to mention the use of dynamic programming (or similar) techniques for computing the effects of individual CCT edges for more complex scenarios than the demonstrated ones.\n2) I would assume consistency to generally imply a sort of systematicity within the model predictions, meaning that the predictions are 'consistently wrong'. Such behavior might usually defined as a (reasoning) bias. Depending on whether the authors agree on this view, they might consider mentioning this aspect in the paper.", "questions_for_authors": "My questions mainly regard the prompting and existence of valid inconsistent reasoners as mentioned before:\n\n1) **Prompting.** Could the authors provide further insights on the particular cause of the reported outliers with regard to the proper application of CoT prompting?\n2) **VI Reasoner.** Under which scenarios do valid, but inconsistent reasoners exist? Is the classification only due to technicalities of the definition or can they appear as a qualitative class of reasoners?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "HjO1BeUhlz", "reviewer": "Reviewer_GPYQ", "summary": "This paper applies the concept of compositional reasoning to causal inference, thereby introducing the notion of compositional causal reasoning (CCR). It further analyzes the relationships among different tiers of causal measures. Building on these findings, the authors propose an evaluation framework for compositional reasoning along two principal dimensions: validity and consistency. Validity represents the correctness of the answer, and consistency represents whether the model answers the global query and the local combination to get the same result. The evaluation of these two dimensions reveals the shortcomings of existing models in causal reasoning tasks.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for acknowledging the clarity and value of our framework, as well as the importance of validity and consistency. We appreciate the suggestion that this framework has potential utility for AI interpretability and safety, areas where we see increasing crossover with causal theory. We address additional comments below.\n\n1. **Comment: Small-scale experiments & presenting results more clearly.**\n   - **Revision 1:** To address this comment, we have extended the CandyParty experiments to include **results for o1 with and without CoT.** See revised Figures 5 and 7 at https://tinyurl.com/bdddu2vx under `/figures_and_tables/updated_figures/` and new Fig 9 (CCTs as a visualization tool for error analysis): https://tinyurl.com/yc5cf9ev\n   - **Revision 2**: As further emphasized by our new results and discussion for o1, we evaluated on a task structure that we anticipated to elicit a spectrum of CCR behavior as a means of highlighting the validity and utility of our framework (not as a benchmarking study). The updated discussion emphasizes the implications of this range of behavior, e.g.:  “Results for task T revealed three taxonomically distinct error patterns: VC (o1), near-VI (GPT-4o with CoT), and II (all remaining models). Only o1 placed mass in the VC reasoning quadrant (Fig. 5)... However, internal inconsistency made GPT-4o with CoT a near-VI reasoner: 75.1% of all estimates were near-valid, mean external validity on local quantities and compositions exceeded 77% and 95%, respectively, and yet failure to correctly infer $\\mathrm{PNS}_{XY}$ revealed internally inconsistent reasoning.” See our **response to Reviewer doGN** for further discussion of VI reasoning and error analysis for GPT-4o. \n   - **Revision 3:** To facilitate future benchmarking works, we release open source **Python code for automated CCR task generation** that can be evaluated according to Algorithm 1 for the PNS, where the user can scale the graphical complexity of the causal DAG and choose from 4 themes: candy party, flu vaccination, football games, and flower gardening. The code can be viewed here, and further unit tests will be available at camera ready: https://tinyurl.com/5be5ehyp and https://tinyurl.com/4um56dab\n2. **Comment: Sensitivity to prompt engineering.**\n   - Indeed, the potential for sensitivity to prompt engineering was the motivation for comparing CoT and non-CoT prompting in our work. Results from GPT-4o in particular demonstrate such sensitivity. While it is out of scope to consider additional prompting strategies in this work, we agree that this would be an important future direction, especially for large-scale CCR benchmarking.\n3. **Comment: Taxonomy of reasoners.**\n   - The reviewer asks whether these reasoning categories define a “single large language model’s overarching behavior, or does it represent the performance of such a model on particular sets of tasks?” In the present work, we limit such labels to **task-dependent behavior,** though in the future more extensive benchmarking might (cautiously) enable broader claims of overarching behavior. In Section 4 we cite the prior notion of a task-metric-model triple and state that “We tether evaluation to a task-metric-model triple”, clarifying that “success on $\\langle \\varphi, \\mathcal{M}, \\mathcal{Q} \\rangle$ does not imply success on $\\langle \\varphi, \\mathcal{M}, \\mathcal{Q'} \\rangle$, $\\langle \\varphi, \\mathcal{M'}, \\mathcal{Q'} \\rangle$, $\\langle \\varphi', \\mathcal{M}, \\mathcal{Q'} \\rangle$, etc.” and “success on CCR tasks such as this is necessary but not sufficient for demonstrating that LMs can reason.” \n   - **Revision 1:** To further clarify, we have added the following to Section 6.3 Limitations & Future Directions: “This proof-of-concept limits application of our reasoning taxonomy to task-dependent observations on model behavior. However, extensive benchmarking could potentially reveal model behaviors that persist across categories of tasks. It remains to be seen if and how different LM architectures might display significant tendencies toward certain categorizations (VC, VI, IC, II).”\n   - **Revision 2:** Please see our response to Reviewer doGN for further clarification on the VI reasoning category.", "claims_and_evidence": "The authors contend that more fine-grained evaluation metrics are needed to assess whether a model genuinely engages in causal reasoning. Relying solely on accuracy (i.e., the proportion of correct answers) does not suffice to determine whether reasoning processes are authentically at work.", "methods_and_evaluation_criteria": "The paper defines a causal reasoning task composed of multiple sub-questions and employs external validity and internal consistency as the key metrics for model evaluation. Through the “CandyParty” example, it illustrates how to estimate two causal measures—Average Treatment Effect (ATE) and Probability of Necessity and Sufficiency (PNS)—under the assumption of monotonicity and specific network structures. The authors also derive formulas that connect these metrics under those assumptions.", "theoretical_claims": "Under the assumptions of monotonicity and particular structural constraints, the paper presents transformation formulas for ATE and PNS. These formulas are then used to construct and assess the proposed evaluation tasks.", "experimental_designs_or_analyses": "The authors deploy a range of model sizes to estimate PNS in both factual and counterfactual scenarios. The findings suggest that task complexity (e.g., longer causal paths) generally increases the error rate. While incorporating chain-of-thought (CoT) prompts can improve the external validity of the model’s estimates, internal consistency remains an open challenge.", "relation_to_broader_scientific_literature": "This work introduces a potentially valuable method for evaluating whether large language models (LLMs) possess genuine causal reasoning capabilities. The proposed framework may also be beneficial for interpretability and model safety considerations.", "essential_references_not_discussed": "None identified.", "other_strengths_and_weaknesses": "Strengths:\n\n* The theoretical framework is clearly formulated.\n* Experimental results show notable patterns, underscoring the importance of both validity and consistency.\n\nWeaknesses:\n\n* The current evaluation relies on small-scale, toy examples, thus necessitating more extensive, large-scale tests.\n* The model may be overly sensitive to prompt engineering. Future work could explore advanced prompting strategies such as ReAct.", "other_comments_or_suggestions": "The evaluation results could be presented more clearly, for instance by providing a direct comparison or mapping between Figures 3 and 5 to highlight their relationship and implications.", "questions_for_authors": "Could you clarify how the “Taxonomy of Reasoners” (i.e., valid-consistent, valid-inconsistent, invalid-consistent, invalid-inconsistent) is defined? Specifically, does it refer to a single large language model’s overarching behavior, or does it represent the performance of such a model on particular sets of tasks?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: The paper introduces a framework for evaluating compositional causal reasoning (CCR) in large language models (LLMs), focusing on metrics like average treatment effect (ATE) and probability of necessity and sufficiency (PNS). It claims to provide a systematic assessment of compositional consistency, validated through experiments on models such as LLaMA, Phi, and GPT-4o, showing increased errors with causal path complexity and partial benefits from chain-of-thought (CoT) prompting. Strengths include a clear theoretical foundation, rigorous proofs, and a taxonomy of reasoner types. Weaknesses noted by reviewers include limited experimental scope (small-scale, toy problems like CandyParty), unclear result presentation, reliance on ground truth causal graphs, and sensitivity to prompt engineering. 

Following the authors’ rebuttal, reviewers raised concerns about motivation clarity, experimental generalizability, CoT prompt design, and the validity of the “valid-inconsistent” reasoner category. The authors addressed these by adding results for the o1 model, clarifying motivation through comparisons (e.g., to CLADDER), releasing open-source task generation code, and expanding error analyses (e.g., GPT-4o’s inconsistencies). In the AC-reviewer discussion, one of the reviewer increased scores after motivation and scope clarifications but sought better Figure 5 explanations. Another reviewer appreciating o1 results and framework validation, though noting low problem complexity. In my recommendation, I weighed the addressed novelty and theoretical rigor heavily, viewing the framework’s conceptual contributions as significant despite experimental limitations. The open-source code mitigates dataset concerns. The authors would need to improve the representation of some of the paper details and see if this can be applied to more complicated tasks.
Decision: Accept (poster)