Failure Modes of LLMs for Causal Reasoning on Narratives

## Abstract

Abstract In this work, we investigate the causal reasoning abilities of large language
models (LLMs) through the representative problem of inferring causal
relationships from narratives.
We find that even state-of-the-art language models rely
on unreliable shortcuts,
both in terms of the narrative presentation and their parametric knowledge.
For example, LLMs tend to determine causal relationships based on the
topological ordering of events (i.e., earlier events cause later ones),
resulting in lower performance whenever events are not narrated in their
exact causal order.
Similarly, we demonstrate that LLMs struggle with long-term
causal reasoning and often fail when the narratives
are long and contain many events.
Additionally, we show LLMs appear to rely heavily on their parametric
knowledge at the expense of reasoning over the provided narrative.
This degrades their abilities whenever the narrative opposes parametric knowledge.
We extensively validate these failure modes through carefully controlled synthetic experiments, as well as evaluations
on real-world narratives.
Finally, we observe that explicitly generating a causal graph generally
improves performance
while naive chain-of-thought is ineffective.
Collectively, our results distill precise failure modes of current
state-of-the-art models and can pave the way for future techniques to enhance causal reasoning in LLMs.

## 1 Introduction

Causal reasoning is a core component
of intelligence and decision-making,
enabling an agent to move beyond associations
(i.e., events that are observed to occur together)
to determine the consequences of their actions (i.e., interventions)
and answering *what-if* questions (counterfactuals).
Causal models have been widely studied in
machine learning and artificial intelligence (Peters et al., ; Pearl, )
as well as in the context of human cognition (Waldmann, ).
The advent of Large Language Models (LLMs)
has led to opportunities for leveraging
large-scale textual data for improving
causal inference (Liu et al., ; Zhang et al., ).

While many works have shown that LLMs
are capable of memorizing and recalling
causal knowledge,
they can fail at reliably leveraging that
knowledge for reasoning (Jin et al., ; Zečević et al., ).
When reading a news article or a story,
understanding *why* a particular event occurred
requires reasoning over the causal structure of the events
implied by the given narrative.
Similarly, determining which events in the narrative
will be affected by certain actions requires
reasoning about which events are causally downstream
of that action.
In such cases, simply
recalling parametric causal knowledge
(that some event typically causes another event)
may not suffice as the agent must
leverage the causal structure
inherent to the specific narrative at hand.

In this work, we aim to understand the causal reasoning abilities
of LLMs from textual narratives.
We consider settings where there is an (unknown)
underlying causal chain graph of the form $V_{1}\rightarrow V_{2}\rightarrow\ldots\rightarrow V_{N}$, where each node $V_{i}$ has some semantic meaning (e.g., smoking),
that is verbalized in the form of a (realistic) narrative $N$.
For a given narrative $N$,
we consider two causal reasoning tasks:
(1) Does $V_{i}$ have a causal effect on $V_{j}$? and
(2) Given the node identities $(V_{1},\ldots V_{N})$,
construct a causal chain graph faithful to the narrative.
While these tasks do not encompass all aspects of causal reasoning
(e.g., counterfactual reasoning),
they are important primitives for successful causal reasoning.

Our primary contribution is to shed light on
the limitations and unreliable shortcuts that
LLMs use for these two causal reasoning tasks.
We focus on the following failure modes.
Firstly, we show that LLMs heavily rely on the
order in which the causal relationships are verbalized
in the narrative.
We observe that when the narrative is constructed
in the reverse topological order of the causal chain
(i.e., the edge $V_{i}\rightarrow V_{i+1}$ is narrated
*before* $V_{i-1}\rightarrow V_{i}$),
the performance of the LLM suffers
as it often assigns the cause to an earlier event
and the effect to a later event in the narrative.
Secondly, we show that LLMs use their
parametric causal knowledge
(i.e., if an event typically causes another event)
as a shortcut
to answer causal questions.
Thus, when the cause-and-effect pairs
implied by the narrative conflict
with the parametric knowledge,
the LLM often ignores the specifics of the narrative and
defaults to its parametric knowledge.
This occurs even when the narratives are written
to explicitly disagree with parametric knowledge
and the LLM is prompted to ignore outside information.
Thirdly, we find that LLMs fail more often as the
narratives become longer.

Perhaps surprisingly,
we also find that the LLM’s answers to cause-and-effect queries,
including with Chain of Thought (CoT) (Wei et al., )
and In-Context Learning,
do not have a high degree of consistency with the
causal graph that the LLM extracts from the narrative.
On the other hand, we observe that just using the extracted graph to make decisions
often seems to alleviate the above failure modes. This finding is notable as it suggests that the specific formatting of the causal graph extraction task can more reliably elicit the language model’s causal reasoning capabilities relative to the baseline of CoT prompting.

We first validate our findings through a series of
carefully controlled experiments
(^2^22The code and data is at [https://github.com/shantanu95/llm_causal_reasoning](https://github.com/shantanu95/llm_causal_reasoning))
with synthetic causal events and narratives (Sec. 3).
We use the LLM to generate a causal chain of events that
form the ground truth causal graph.
We then prompt the LLM to generate a narrative-based
on that causal graph.
Next, we conduct experiments on narratives generated
from real-world causal chain graphs (Sec. 4)
extracted from *CauseNet* (Heindorf et al., ),
a dataset of (claimed) causal relationships between real-world concepts.
We generate narratives for
these chain graphs
using the LLM as well as from sentences in *CauseNet*,
ensuring that the narrative as a whole remains coherent.

Leveraging the LLM to verbalize causal relationships
leads to more natural-sounding narratives
(which is difficult with formulaic templates).
Depending on the specific causal relationship,
causality might be indicated using a phrase like
‘*causes*’, ‘*leads to*’, ‘*resulted in*’, etc.
We focus on simple chain graphs to isolate the simplest
cases of the failure modes,
demonstrating that the LLMs take unreliable shortcuts
on these simple graphs,
both when the narratives are generated by the LLM itself
as well as narratives with real-world sentences.

Taken together, our work makes the following contributions:

- 1.
We demonstrate that even *state-of-the-art* large language models are unreliable at extracting causal relationships from realistic narratives expressing simple chain graphs.
- 2.
Focusing on two key aspects of causal reasoning,
we distill concrete failure modes to account for this unreliability: reliance on positional shortcuts, parametric knowledge, and narrative length.
- 3.
We investigate the impact of various prompting strategies on reasoning ability,
finding that *incorporating the estimated causal graph structure* is most helpful.

## 2 Related Works

##### Reasoning in Large Language Models

Recently, there has been significant interest in evaluating and improving the reasoning capabilities of large language models. Prior works have examined reasoning in diverse settings including mathematics (Cobbe et al., ; Hendrycks et al., ), social interactions (Sap et al., ), and common-sense tasks (Zellers et al., ). Through this extensive investigation, large language models have demonstrated unreliable reasoning capabilities, often failing unexpectedly on relatively straightforward queries (Wan et al., ; Nezhurina et al., ). Extensive prior work has also studied methods for improving the reasoning capabilities of large language models such as by eliciting step-by-step explanations (Wei et al., ), finetuning on reasoning traces (Zelikman et al., ), and training models to find and correct errors (Kumar et al., ). In this work, we perform a comprehensive study of the causal reasoning capabilities of large-language models, identifying the existence of key failure modes in state-of-the-art models.

##### Causal Reasoning in Large Language Models

Prior works have also examined the causal reasoning capabilities of large-language models. Jin et al. () develop a benchmark for testing causal reasoning in LLMs given causal graphs, finding that language models can struggle with the task. However, the queries examined in Jin et al. () require probability calculations, potentially conflating causal reasoning and arithmetic failures. Zečević et al. () hypothesizes that large language model may only be “imitating” causal reasoning abilities from their pretraining corpus. Their work additionally investigates what they call “natural word chains,” or how well the LLM is able to accurately identify causation of events when they are linked together in chain graphs. Our paper builds on this by performing various tests on narratives that correspond to graphs. Kıcıman et al. () shows that LLMs have strong abilities to generate causal texts. We make use of this generation for narrative creation in our synthetic experiments. Tan et al. () shows the capability of a neural network trained on news data to label causal structures in individual sentences.

##### Parametric Knowledge Conflicts

Large language models have been shown to memorize factual knowledge present in their pretraining corpora, resulting in *parametric knowledge* (Petroni et al., ; Jiang et al., ). Prior works have observed that language models can often fail to use information provided in their context as a result of such memorized knowledge (Krishna et al., ; Longpre et al., ). This has been identified as a significant hurdle in the reliability of summarization and retrieval augmented-based systems (Rehman et al., ; Jin et al., ). Xie et al. () find that the coherence of the provided context can control whether a model chooses to use its parametric knowledge over contextual information. In this work, we examine the distinct setting of causal reasoning and demonstrate that it can also be harmed by over-reliance on parametric knowledge.

## 3 Experiments with Synthetic Data

### 3.1 Data Generation Process

In our synthetic experiments, we use two leading LLMs:
OpenAI’s GPT-4 (OpenAI et al., ) and Anthropic’s Claude
3.5 Sonnet (Anthropic, ).
The purpose of our synthetic setup is to carefully control the conditions under which the LLMs are tested.
In terms of the general setup of our fully synthetic experiments,
as we summarize in Figure 1,
we first use the LLM to generate events
(which are real world phenomena like *rain* or *plants growing*).
Then these events are linked together into a chain graph $G$
that acts as the causal ground truth.
The LLM is given $G$ and asked to create a narrative that stays faithful to the causal relationships in $G$.

Providing only the narrative as input (and not $G$),
we then ask the LLM to find $G^{\prime}$,
the prediction of the underlying causal structure
expressed by the narrative.
Next, a series of causal questions is created
by randomly sampling $10$ tuples of events from $G$
and asking the LLM whether an event in
the tuple causes the other based on the narrative and/or $G^{\prime}$.
We also explore many mediums of prompting including Chain of Thought (COT) (Wei et al., ) and In-Context Learning (Dong et al., )
(see Appendix A for the prompts).

Figure: Figure 1: Summary of Synthetic Experiment Setup
Refer to caption: extracted/5968664/images/flow3.png

Figure: Figure 2: GPT-4 Test of the LLM’s ability to reason on narratives written in the Forward and Reverse topological orientations. Chain size is the number of nodes in $G$. Forward/Reverse Chain - COT, Forward/Reverse Mean -simple prompting strategy, Forward/Reverse Graph Mean - using only $G^{\prime}$, Forward/Reverse Graph/Narr Mean - prompting the LLM to consider both the narrative and $G^{\prime}$, Forward/Reverse Context Mean - In-Context Learning. Accuracy measures LLM answer agreement with $G$, consistency measures agreement with $G^{\prime}$. Shaded zone around lines is 95 $\%$ CI.
Refer to caption: x1.png

### 3.2 Reliance on Topological Ordering

Our experiments show that LLMs rely on the
ordering in which the events are verbalized
in a narrative when determining causal relationships.
To investigate this,
we started with randomly generated events
that were used to make a ground truth graph $G$.
During the creation of the narrative,
we specified that the LLM either place the events
in
(1) the order that matches the topological causal ordering of the graph
(e.g., if event $A$ causes $B$,
then event $A$ is mentioned before $B$ in the narrative),
or
(2) a way that runs opposite to the causal ordering
(event $B$ would be mentioned before event $A$ in the narrative
even though $A$ causes $B$).
We refer to these as the *Forward*
and a *Reverse* topological ordering, respectively.
Upon inspection of the generated narratives, human evaluators (paper authors who were blinded to the causal graph) were able to properly extract the ground truth causal graph (94$\%$ of the time out of 50 randomly sampled narratives).
As an example, the following is a GPT-4
generated *Reverse* topological narrative
for the causal chain
*Film festival* $\rightarrow$ *Food truck rally* $\rightarrow$ *Trampoline park party*:

> In a *trampoline park party*, guests bounced and laughed under the neon lights. The high-flying event was made possible by the success of the *food truck rally* that took place earlier in the day. Food trucks lined the streets, offering a variety of delicious treats that fueled the fun and energy of the park party attendees. *The film festival* set the stage for the rest of the events, creating a desire for cultural experiences and community gatherings that ultimately led to the *trampoline park party*.

Each edge in the narrative is verbalized
in the opposite order to its place in the causal chain.
All narratives can be found in the linked code.

As shown in Figure 2 (left),
in the *Forward* direction,
a simple question prompting strategy asking causal questions (line labeled Forward Mean), naive CoT prompting (line labeled Forward-Chain Mean) where we ask the LLM to think step-by-step, and an In-Context Learning prompt (line labeled Forward-Context Mean) where we provide example narratives and causal answering all perform fairly well. This is in contrast to the *Reverse* orientation when we look at the performance of the simple prompt (line labeled Reverse Mean), naive CoT prompt (line labeled Reverse-Chain Mean), and In-Context Learning prompt (line labeled Reverse-Context Mean). From this plot, we can note that naive COT prompting and In-Context Learning prompts do not seem to boost accuracy under our conditions.
Perhaps more interestingly, we find that the way the LLM answers questions using the narratives is not consistent with the causal graph $G^{\prime}$ that the LLM builds when asked to predict the underlying graph structure (see consistency plot in bottom of Figure 2,
where consistency measures agreement between the answers of the LLM and $G^{\prime}$).
Additionally, the trend of those prompting strategies on the consistency plot for the *Forward* orientation narratives (comparing performance to $G^{\prime}$)
mirrored their trend on the accuracy plot which compares performance to ground truth $G$ (left side).

This led us to try using only $G^{\prime}$ to answer causal questions
and we found that it did significantly
better in the *Forward* direction
than the other prompting strategies
(line labeled Forward Graph Mean),
while doing no better in the *Reverse* direction
(line labeled Reverse Graph Mean).
In this case, once $G^{\prime}$ is extracted by the LLM it is
not given to the LLM again
to answer questions (but rather used directly).

When prompted to use the narrative and $G^{\prime}$ (the LLM is given $G^{\prime}$ in this case in the prompt) to answer questions for *Forward* direction narratives, the accuracy again increases. This technique could be thought of as a type of CoT prompting strategy. In the Claude 3.5 Sonnet (Fig. 4), The LLM also does far better with the *Forward* topological ordering than it does for the *Reverse* ordering. For the *Forward* ordering, the best performance again comes from using the extracted graph in concert with the narrative in the prompt.

Figure: Figure 3: GPT-4 Test of the LLM’s ability to reason on narratives that agree with parametric knowledge (Causal) and disagree with parametric knowledge (Anti-Causal). Label descriptions match those of Figure 2. Example illustration is of how $G$, the ground truth causality, is set up. Shaded zone around lines is 95 $\%$ Confidence Interval
Refer to caption: x3.png

Figure: Figure 4: Claude 3.5 Sonnet Experiments: Left side plot mirrors the left side plot of Fig. 2 and the right side mirrors the plot in Fig. 3. Descriptions of prompt strategies match the before-mentioned figures.
Refer to caption: x4.png

### 3.3 Parametric Knowledge Conflict

We also find that LLMs tend to rely on parametric knowledge
when it is present,
and can fail when narratives run counter
to their parametric knowledge.
To test this, instead of picking random events,
we have to draw out the LLM’s pre-existing parametric knowledge.
To do so, we prompt the LLM to
pick a series of events such that
each event has some relation to the subsequent event–either
the event is *Causal* to the next event
(e.g., disease causes shorter lives)
or the event is *Anti-Causal* (e.g., disease causes longer lives).
Let’s call this graph of parametric knowledge $P$.
We then take the odd indexed events (1st, 3rd etc) from $P$ and place them in the first half of the causal ground truth graph $G$ and the even indexed events (2nd, 4th etc) from $P$ in the second half of $G$.
As we can see in the example illustration in Figure 3, each node in the first half of the causal graph $G$ now has a parametric relation to a specific node in the second half of $G$.
For example we might know that node $1$ is *Anti-Causal* to node $2$ from parametric knowledge. Thus when we make the causal ground truth graph ($1\rightarrow 3\rightarrow$ 2), create a narrative from it, and then ask the LLM if node $1$ causes $2$ based on the narrative, it should say yes even though that disagrees with its parametric knowledge. After the ground truth graph is created, we generate the narrative in the *Forward* topological orientation to avoid confounding failure modes.

We find that, in synthetic experiments,
this kind of occurrence is generally rare and
the LLM finds the correct causal relation
generally only when that relation agrees
with its parametric knowledge.
This is exemplified in the plot in Figure 3 where
we observe that when certain pairs of events are *Causal* in the parametric knowledge,
the LLM also generally finds that the pairs of events in question are *Causal* given the narrative.
However, when the pairs of events are *Anti-Causal* (like nodes $1$ and $2$ in our previous example), even though the LLM should find them to be *Causal* given the narrative it generally does not. For the *Causal* parametric knowledge case, prompting with just the extracted graph $G^{\prime}$ is usually slightly better at most chain sizes than solely using the narrative (in simple prompts, COT or In-Context Learning), and this is only improved in most instances when using the narrative and extracted graph in the prompt.

We also notice an interesting phenomenon for the *Anti-Causal* parametric case where although using the extracted graph and narrative is better than any prompting strategy with just the narrative, using just the extracted graph provides massive improvements over all of these modalities and is comparable in performance to when the parametric knowledge is *Causal*.
As an even more stark finding, the extracted graph for the Claude 3.5 Sonnet experiment in the right side of 4 has near perfect performance for the Anti-Causal case. It seems the narrative may only serve to distract the LLM when parametric knowledge disagrees with the narrative. When the narrative is provided, the LLM will often say things (in its COT explanation) like “Although this relationship is displayed in the chain of events in the narrative, it is not logical and counter-intuitive” and answer incorrectly. This happens despite explicit instruction to ignore outside information, allow for ill-logical relationships, and answer solely based on the hypothetical narrative.

### 3.4 Failure of Long-Term Causal Reasoning

The performance of LLMs tends to decay as the size of the narrative and the number of events in the narrative increases. As we can see in Figures 2 and 3, generally all prompts that solely rely on the narrative without the extracted graph to answer causal questions have a performance that decays with the number of events in the narrative (chain size). The exception to this occurs in the *Reverse* topological orientation in 2 as the accuracy is psuedo-random at all chain sizes. This finding is supported by the well known phenomenon that LLMs often fail to reason about longer form contexts especially as they “get lost in the middle” (Liu et al., ). What perhaps is a surprising result that is the extracted graph tends to maintain a consistent level of accuracy for causal answering across narrative sizes even when only using the narrative in the prompt leads to severe dips in accuracy.

## 4 Experiments with Real World Causal Graphs

In this section, we test our methods on
narratives constructed using
real-world causal graphs from *CauseNet* (Heindorf et al., ),
a large-scale knowledge graph of (claimed) causal relationships
between real-world concepts.
We use *GPT-4o* (OpenAI et al., ) for our experiments.

### 4.1 Generating the narratives

The *CauseNet* dataset can be represented as
a collection of $D$ tuples $\{(C_{i},E_{i},\mathbf{S}_{i}\}_{i=1}^{D}$,
where $C_{i}$ denotes the cause (e.g., fatigue),
$E_{i}$ denotes the effect (e.g., accidents),
and $\mathbf{S}_{i}$ is a set of sentences
(extracted from Wikipedia and ClueWeb12 (Callan, ))
that entail a causal relationship from $C_{i}$ to $E_{i}$.
We retrieve causal chain graphs
$V_{1}\rightarrow V_{2}\rightarrow\ldots\rightarrow V_{N}$
of various lengths,
where each causal relation
$V_{i}\rightarrow V_{i+1}$ is from *CauseNet*.
Below, we describe the strategy for generating
narratives for a given chain graph
(see Appendix B.1 for
the prompt templates).

##### Semi-synthetic narratives.

Each chain graph is verbalized into a narrative
by prompting the LLM to generate a sentence
one edge at a time
such that the sentence for $V_{i}\rightarrow V_{i+1}$
follows the sentence for the previously verbalized edge,
ensuring that the narrative as a whole is sensible.
We generate the narratives
by enumerating the edges in the topological order
of the graph and in the reverse order.
We call this semi-synthetic because
the nodes in the graph represent real-world causal
relationships but the narratives
are synthetic.
For example, the following is the generated narrative
in the forward direction for the causal chain
*fatigue* $\rightarrow$ *accidents* $\rightarrow$ *injury*:

> *Fatigue* can cloud judgment and slow reaction times, leading to an increase in *accidents* on the road. As a result, these *accidents* often lead to serious *injury* for those involved, highlighting the dangerous consequences of driving while fatigued.

##### Real-world narratives.

For the real-world narratives,
the sentence for each edge is chosen from the *CauseNet*
dataset.
To ensure that the narrative as a whole remains coherent,
we prompt the LLM to ensure that
the sentences for every pair of adjacent edges
logically follow each other.
For example, the following is the narrative for the causal chain
*fatigue* $\rightarrow$ *accidents* $\rightarrow$ *injury*:

> Workers work long hours in mines and factories where *fatigue* and a lack of concentration can easily cause *accidents*. These *accidents* are the leading cause of *injury* in this country for people ages 1-34.

Additional examples of semi-synthetic and real-world
narratives are presented in Appendix B.2
(the entire set of narratives used for our experiments
is available in the linked code).

### 4.2 Effect of Narrative Topological Ordering and Chain Length

As described in the previous section,
we verbalize each causal chain graph
$V_{1}\rightarrow V_{2}\rightarrow\ldots\rightarrow V_{N}$
from *CauseNet*
into a narrative in the forward and
reverse topological order.
We give the narrative as input to the LLM and
evaluate its causal reasoning abilities using
the following prompting strategies (see Appendix B.3
for the prompt templates):
(1) (Direct prompting) For every pair of nodes $(V_{i},V_{j})$,
we ask does $V_{i}$ have a direct or indirect causal effect on $V_{j}$;
(2) (CoT prompting) Same as (1), but we now use CoT
and ask the LLM to think step-by-step; and
(3) (Graph Extraction) Generate
a causal chain graph faithful to the narrative
when given the node identities
$(V_{1},V_{2},\ldots,V_{N})$ in a random order.

We evaluate the accuracy for each pair of nodes $(V_{i},V_{j})$
for the three prompting strategies on the semi-synthetic and
real-world narratives (see Fig. 5).
We denote the Direct prompting strategy (1) using
the labels *Forward* and *Reverse*;
the CoT prompting strategy (2) using
the labels *Forward_COT* and *Reverse_COT*;
and the accuracy using the extracted graph from strategy (3)
using the labels *Forward_Graph* and *Reverse_Graph*.

Figure: (a) Semi-synthetic narratives
Refer to caption: x6.png

In both the semi-synthetic (Fig. 5(a))
and real-world narratives (Fig. 5(b)),
the *Forward_Graph* strategy performs the best,
with its accuracy remaining stable even
as the chain length increases.
By contrast, *Reverse_Graph* has
relatively lower accuracy, with the accuracy declining
as the chain length increases
(this effect is particularly significant in the real-world narratives).
We also observe that *Forward*(*_COT*)
outperforms *Reverse*(*_COT*),
with the *Reverse*(*_COT*) accuracy
declining substantially as the chain size gets large.
Additionally, unlike the forward direction,
the graph extraction does not improve accuracy
substantially over the Direct and CoT-based prompting strategies. Altogether, these results demonstrate that LLM’s ability
to extract the causal graph
as well as determine causality between two nodes declines
when the narrative is verbalized in the reverse order,
with the impact of the increasing chain lengths being
particularly pronounced in the reverse direction.

### 4.3 Effect of Parametric Knowledge Conflicts.

**Table 1: The average accuracy across different narratives with the three prompting strategies partitioned by whether the cause-effect pairs conflict with the LLM’s parametric knowledge (we omit the 95% CIs as they are smaller than $0.3$). The accuracy (with and without CoT) is substantially lower when knowledge conflicts exist, suggesting that LLM relies on this knowledge rather than the narrative. By contrast, the graph extraction is more robust to knowledge conflicts.**
|  | Forward (w CoT) | Reverse (w CoT) | Forward Graph | Reverse Graph |
| --- | --- | --- | --- | --- |
| Semi-synthetic |  |  |  |  |
| Without Conflict | 97.86 (97.94) | 95.85 (96.47) | 98.54 | 87.99 |
| With Conflict | 65.47 (80.54) | 45.63 (67.83) | 97.70 | 84.32 |
| Real-world |  |  |  |  |
| Without Conflict | 95.54 (96.06) | 92.38 (91.63) | 87.74 | 57.93 |
| With Conflict | 48.03 (59.57) | 27.40 (49.43) | 85.67 | 53.09 |

Next, we analyze the extent to which the
LLM relies on its parametric knowledge to answer the causal
query as opposed to the specific causal effects
expressed in the narrative.
For every pair of nodes $(V_{i},V_{j})$ in the chain graphs,
we extract the parametric knowledge of the LLM
by prompting the LLM to answer
“Does $V_{i}$ typically have a causal effect on $V_{j}$?”.
This parametric knowledge represents some average case
notion of causality learned by the LLM
from the pretraining corpora.

Successful causal reasoning often requires
going beyond the parametric causal knowledge
as it can differ from the causality between events
in the context of the narrative at hand.
For example, in a chain graph from the our dataset,
there is a causal path from
*streambank erosion* to *higher prices*,
but this contradicts the LLM’s parametric knowledge
since this causal effect may not typically exist in
the real-world.
In the chain graphs we evaluate,
roughly $39\%$ of cause-and-effect pairs
conflict with the LLM’s parametric knowledge.

We evaluate the three prompting strategies
separately on the subsets of cause-and-effect pairs
that are in agreement and in conflict with the
parametric knowledge (see Table 1).
We observe that when there is no conflict
(i.e., the parametric knowledge agrees with the
causality expressed in the narrative),
the accuracies in both the *Forward* and *Reverse*
directions (with and without CoT)
are greater than $90\%$.
However, when the parametric knowledge conflicts with
the narrative’s causality,
the accuracy of *Forward* and *Reverse* is
significantly lower:
even with CoT,
the LLM has below-random accuracy
in the reverse direction.
This suggests that when asked to reason about
cause and effect in a narrative,
the LLM seems to rely heavily on
its parametric knowledge and
is unable to grasp the specific causal chains
expressed in the narrative itself
(despite the causal chains as a whole being realistic).

Interestingly, when using extracted graph
for performing causal reasoning
(*Forward_Graph* and *Reverse_Graph*),
the performance in both cases seems to be comparable.
This is likely because when asked to extract the graph
from the narrative with the *given* set of nodes,
the LLM pays more attention to the entire narrative
as opposed to when directly queried on a cause-and-effect pair
(where the LLM defaults to its parametric knowledge).
These results show that even when
the LLM constructs a reasonably
good causal chain graph
in the *Forward* direction,
the LLM does not leverage this graph
when queried directly about the causal effects
in the narrative (even with CoT),
further highlighting the advantage of
extracting the causal graph directly.

Finally, we also observe that
the accuracy of *Forward* and *Reverse*
prompting strategies in Fig. 5
were largely driven by the parametric knowledge.
To disentangle this effect,
we evaluate the three prompting strategies across
different chain lengths *only* on the subset
of cause-and-effect pairs where a knowledge
conflict exists (see Fig. 6).
We observe that the accuracy of all prompting
strategies, except *Forward_Graph* and *Reverse_Graph*,
decline substantially relative to Fig. 5,
with *Reverse* falling well below
the random baseline of $50\%$ as
the chain length gets larger.
In the real-world narratives (Fig. 6(b)),
even with CoT,
the accuracy does not improve over the
random baseline of $50\%$
in the reverse direction,
demonstrating that the LLM ignores the causal
chain in the narrative.
Qualitatively, we observe that in the case of
knowledge conflicts,
the CoT explanation justifies the incorrect
(according to the narrative)
causal relationship by injecting
its parametric knowledge into the chain-of-thought.
Additionally, we also evaluate the three
prompting strategies on the subset of cause-and-effect pairs
in agreement with parametric knowledge
(see Appendix B.4),
showing high accuracy of the Direct and CoT-based
prompting strategies on this subset.

Figure: (a) Semi-synthetic narratives
Refer to caption: x9.png

## 5 Discussion

Our work takes some initial strides
towards examining the reasons behind
the success and failure of LLMs to reason causally on narratives that express a chain of causal events.
We focus on two questions of key importance
in causality:
(1) Does one event cause another? and
(2) Can the LLM extract the causal graph from the narrative.
We find three significant failure modes of LLM reasoning by conducting experiments in carefully controlled synthetic, semi-synthetic and real-world settings:

- 1.
Topological Ordering: LLMs tend to perform well when the ordering of events in the narratives matches that of the ordering of the underlying graph. For example if B is mentioned first in the narrative and then A, then the LLM would would expect $B\rightarrow A$ even if $A\rightarrow B$ is the truly underlying causal graph. As such, performance breaks down when the ordering of events in the narratives does not match that of the underlying graph.
- 2.
Parametric Knowledge: LLMs rely on their parametric pre-training knowledge as a shortcut
to infer causal relations.
When the narrative suggests one causal relation
(e.g., $A\rightarrow B$) and
the parametric knowledge suggests another ($B\rightarrow A$), the LLM will often answer using its parametric knowledge,
ignoring the narrative.
- 3.
Narrative Size: LLM accuracy degrades as the chain length
increasing, resulting in poor reasoning on longer narratives.

We also find that asking the LLM to extract what it predicts to be the underlying graph from the narrative, and using just that extracted graph or the extracted graph in concert with the narrative to answer can lead to mitigation of these failure modes.

Our work builds upon past work that examines causal reasoning in LLMs.
Various works including Jin et al. ()
suggest that LLMs may have difficulty
performing causal reasoning in tasks involving
numerical probabilistic computations.
Previous work by Zečević et al. ()
conjectured that LLMs simply
imitate causal relations from their training data.
Our work aims to assess causal reasoning from textual narratives
(not just on individual sentences)
that is required to understand why certain events occurred
and how they would impacted by certain interventions.
Like previous works, we find a strong reliance
on pre-training knowledge.
However, we also suggest a way to mitigate this by using the LLM to extract the entire causal graph from the narrative at once.
When doing this, the graph remains generally accurate even in the presence of conflicts with parametric pre-training knowledge.

### 5.1 Limitations and Future Works

One limitation of our work is that
there are other forms of causal reasoning that
we did not test for in the narratives.
This motivates many potential directions for future work.
For example, it could be interesting to ask the LLM to reason about counterfactual cases.
Another limitation of our work is that we only deal with chain graphs.
An interesting direction for future work would
be to generate narratives from complex graph structures.
This could potentially yield novel failure modes
and insights into how LLMs reason.
Another limitation of our work is
the amount of insight we provide behind the phenomenon we observe.
When we notice the graphs doing better
than the prompts that only use the narrative itself,
one conjecture is that this is because the LLM is
looking at the entire narrative and
reasoning about it to make the extracted graph–however we don’t formally prove this.
Future work could theoretically or empirically examine explanations for such LLM behavior and others noted in the paper.

Our analysis also has implications for algorithmic interventions to improve causal reasoning. The failure modes we identify in this paper could inform the design of targeted synthetic tasks to use in finetuning for improved causal reasoning. Additionally, our findings on the benefits of extracting a causal graph can inform prompt engineering efforts to elicit reliable causal reasoning from language models. We believe investigating both directions represents an exciting direction for future work.

## Acknowledgments

SG and ZL gratefully acknowledge the NSF (FAI 2040929 and IIS2211955), UPMC, Highmark Health, Abridge, Ford Research, Mozilla, the PwC Center, Amazon AI, JP Morgan Chase, the Block Center, the Center for Machine Learning and Health, and the CMU Software Engineering Institute (SEI) via Department of Defense contract FA8702-15-D-0002, for their generous support of ACMI Lab’s research.
The authors are grateful to David Childers for
helpful discussions and feedback.

## Appendix A Synthetic Data Experiments

### A.1 Selected Synthetic Prompts

We use an LLM to generate the events E.
From the events, we create a ground truth causal graph G which is used to structure and inform the narrative sequence and causality. N is the corresponding narrative created by the LLM from G. To evaluate the LLM’s performance, we extract a causal graph, G’, from the narrative N as produced by the LLM, and compare it with the ground truth causal graph G. In this context, n refers to the number of events to generate, while A and B represent pairs of events queried for causal relationships. The task then becomes assessing whether event A causes event B . All prompts, data processing steps, and results are included in the attached code.

#### A.1.1 Topological Experiment - Generating Random Events ( E )

“generate *n* random distinct events”

#### A.1.2 Parametric Experiment -Generating Causal Events ( E )

“generate *n* events that cause each other (for example the first event causes the second and the second causes the third)”

#### A.1.3 Parametric Experiment - Generating Anti-Causal Events ( E )

“generate *n* events that are anticausal, for example the first event could be cancer and the second event could be a longer life
because in reality, cancer causes a shorter life. Make it so each of the *n* events are anticausal in this way such that the next event is actually the opposite of what it should be. For 3 events we might have 1.Rain 2.Plants Die 3.Increased Oxygen”

#### A.1.4 Forward Topological Narrative ( N )

“ context is *G*, generate a hypothetical narrative from this causal chain graph and make causal relations explicit, even when the causal relations do not make sense, keep the causal relations as they were in the context. The events in the story should occur in the same order as in the chain graph (eg first item in chain graph should appear in narrative before the second item)”

#### A.1.5 Reverse Topological Narrative ( N )

“context is *G*, generate a hypothetical narrative from this causal chain graph and make causal relations explicit, even when the causal relations do not make sense, keep the causal relations as they were in the context. The events in the story should occur in the opposite order as in the chain graph (eg last item in chain graph should appear first in the narrative etc)”

#### A.1.6 Standard Prompt

“Use this narrative *N* as context. Did *A* cause *B*? Output your answer with $<answer>Yes/No</answer>$. The cause can be direct or indirect.”

#### A.1.7 Chain of Thought Prompt

“Use this narrative *N* as context. Did *A* cause *B*? Do step by step reasoning. Then output your answer with $<answer>Yes/No</answer>$. The cause can be direct or indirect.”

#### A.1.8 In-Context Prompt

“Use this narrative *N* as context. Did *A* cause *B*? Output your answer with $<answer>Yes/No</answer>$. The cause can be direct or indirect.
An example narrative would be: Rains leads to plants growing. This then causes increased oxygen in the atmosphere. A potential question would be does rain cause increased oxygen in the atmosphere? The answer would be Yes.
Another example narrative would be: Increased oxygen in the atmosphere is because of plants growing. Plants grow because rain provides them essential nutrients. A potential question would be does rain cause increased oxygen in the atmosphere? The answer would be Yes.
Another example narrative would be: Rain leads plants to grow. Plants growing causes less oxygen in the atmosphere. A potential question would be does rain cause less oxygen in the atmosphere? The answer would be Yes.”

#### A.1.9 Narrative + Graph Prompt

“Use this narrative *N* and this causal ordering *G’* ((such that each item is a cause of every item after it, for example the first list item is a cause of the third, fourth, fifth items etc)) as context. Did *A* cause *B*? Output your answer with $<answer>Yes/No</answer>$. The cause can be direct or indirect.”

## Appendix B Real-world Causal Graphs

### B.1 Prompt templates for narrative generation

Recall that we have a ground truth causal chain graph of
the form $V_{1}\rightarrow V_{2}\rightarrow\ldots\rightarrow V_{N}$
from *CauseNet* that we need to verbalize into a coherent narrative.
For the semi-synthetic narratives, we use the LLM (GPT-4o)
to do so one edge at a time,
while ensuring that the newly verbalized edge logically
follows the previous one.
The following is the prompt template for generating the
narratives in the topological order of the graph:

> Output a short narrative (use one or two sentences) that expresses the causal link [$V_{i}\rightarrow V_{i+1}$] and logically follows this narrative: { Narrative for the previous edge $V_{i-1}\rightarrow V_{i}$}. Ensure that the combined sentences convey the causal chain [$V_{i-1}\rightarrow V_{i}\rightarrow V_{i+1}$] and that the words [$V_{i}$, $V_{i+1}$] are present. Only output the newly generated narrative.

Similarly, we generate narratives in the reverse topological
order of the graph by verbalizing edges in the reverse direction
with the following prompt template:

> Output a short narrative (use one or two sentences) that expresses the causal link [$V_{i}\rightarrow V_{i+1}$] and logically follows this narrative: { Narrative for the previous edge $V_{i+1}\rightarrow V_{i+2}$}. Ensure that the combined sentences convey the causal chain [$V_{i}\rightarrow V_{i+1}\rightarrow V_{i+2}$] and that the words [$V_{i}$, $V_{i+1}$] are present. Only output the newly generated narrative.

For generating real-world narratives, for each edge $V_{i}\rightarrow V_{j}$,
we use the set of sentences from *CauseNet*.
Each edge in *CauseNet* is linked
to multiple sentences from various sources.
Picking a sentence for each edge at random and concatenating
them does not always lead to sensible narratives.
To improve the quality of narratives, we use the following prompt
to concatenate sentences for adjacent edges:

> Consider the following sentences. { Sentence for edge $V_{i}\rightarrow V_{i+1}$ }. { Sentence for edge $V_{i+1}\rightarrow V_{i+2}$ }. Do the sentences logically follow each other and express the causal chain [$V_{i}\rightarrow V_{i+1}\rightarrow V_{i+2}$]? Answer with Yes or No.

For verbalizing narratives in the topological order,
for a given graph $V_{1}\rightarrow V_{2}\rightarrow\ldots\rightarrow V_{N}$,
we only use sentences such that the above prompt returns *Yes*
for every pair of adjacent edges $V_{i}\rightarrow V_{i+1}\rightarrow V_{i+2}$.
This ensures that the narrative as a whole remains coherent
and conveys the entire causal chain graph.
We use a similar prompting strategy to verbalize narratives
in the reverse topological order.

### B.2 Additional examples of the generated narratives

#### B.2.1 Semi-synthetic narratives

Below, we present some examples of
semi-synthetic narratives in the
forward and reverse directions.

The narrative in the forward direction for the chain
*higher prices* $\rightarrow$ *reduced demand* $\rightarrow$ *lower prices*:

> As *higher prices* swept through the market, consumers began to tighten their budgets, leading to a noticeable *reduction in demand* for many goods. As a result of the *reduced demand*, suppliers were forced to *lower prices* in order to attract buyers back to the market.

The narrative in the reverse order for the causal chain
*bankruptcy* $\rightarrow$ *bad credit* $\rightarrow$ *rejection* $\rightarrow$ *anger*:

> The sting of rejection ignited a fire within her, transforming her hurt into a seething anger that demanded to be felt. Her bad credit had led to the rejection she never saw coming, and now that sting of rejection ignited a fire within her, transforming her hurt into a seething anger that demanded to be felt. Her bankruptcy had left her with bad credit, a shadow that loomed over her every application, and now that sting of rejection ignited a fire within her, transforming her hurt into a seething anger that demanded to be felt.

The narrative in the reverse order for the causal chain
*pollution* $\rightarrow$ *climate change* $\rightarrow$ *extreme weather events* $\rightarrow$ *natural disasters*:

> As extreme weather events become more frequent and severe, they increasingly lead to devastating natural disasters that disrupt communities and ecosystems alike. Climate change is driving the rise in extreme weather events, which in turn are causing unprecedented natural disasters that threaten the stability of communities and the health of ecosystems. Pollution is a major contributor to climate change, which is driving the rise in extreme weather events that threaten the stability of communities and the health of ecosystems.

#### B.2.2 Real-world narratives

Below, we present some examples of
real-world narratives in the
forward and reverse directions.

The narrative in the forward direction for the chain
*higher prices* $\rightarrow$ *reduced demand* $\rightarrow$ *lower prices*:

> *Higher prices* generally lead to reduced demand. *Lower prices*, caused by *reduced demand* and increased competition for soybeans and corn, largely contributed to the overall bulk export decline.

The narrative in the reverse order for the causal chain
*bankruptcy* $\rightarrow$ *bad credit* $\rightarrow$ *rejection* $\rightarrow$ *anger*:

> Embittered by an abusive upbringing, seething with resentment, irritated by others’ failure to fulfill his or her superior sense of entitlement, and fuelled by anger resulting from rejection, the serial bully displays an obsessive, compulsive and self-gratifying urge to displace their uncontrolled aggression onto others whilst exhibiting an apparent lack of insight into their behavior and its effect on people around them. Bad credit normally leads to rejection but now with bad credit secured loan, you can avail the loan of your choice. For example, if you are applying for a loan, the lender may reject your application on the basis of bad credit caused by bankruptcy.

The narrative in the reverse order for the causal chain
*pollution* $\rightarrow$ *climate change* $\rightarrow$ *extreme weather events* $\rightarrow$ *natural disasters*:

> In addition to forced migrations from rising seas, climate change is also increasing extreme weather events causing natural disasters such as cyclonic storms (hurricanes or typhoons), floods and droughts. This is worsened by extreme weather events caused by climate change. This landmark bill would jump start the economy by creating millions of new clean energy jobs, increase national security by reducing dependence on foreign oil, and preserve the planet by reducing the pollution that causes climate change.

Figure: (a) Semi-synthetic narratives
Refer to caption: x12.png

### B.3 Prompt templates for assessing causal reasoning

We use the following template for the Direct prompting strategy:

> Consider the following hypothetical narrative. {narrative} According to the hypothetical narrative, does {cause} have a (direct or indirect) causal effect on {effect}? Answer in Yes/No.

We use the following template for the Chain-of-Though (CoT) prompting strategy:

> Consider the following hypothetical narrative. {narrative} According to the hypothetical narrative, does {cause} have a (direct or indirect) causal effect on {effect}? Think step-by-step and end your answer with <answer>Yes/No</answer>.

We use the following template to extract a chain graph from the narrative:

> Consider the following hypothetical narrative. {narrative} According to the hypothetical narrative, construct a causal chain graph using the following nodes: { nodes in random order }. Ensure that the graph contains all the given nodes and only output a single chain graph of the form <graph>node1 $\rightarrow$ node2 $\rightarrow$ node3 </graph>. Only output the graph between the <graph></graph>tags.

### B.4 Additional results

In Fig. 7,
we present the accuracy of the three prompting strategies
discussed in Sec. 4
on the subset of cause-and-effect pairs
that are in agreement to the LLM’s parametric knowledge
(this is a counterpart to Fig. 6).
We observe that when the parametric knowledge agrees,
the Direct and CoT-based prompting leads to high
accuracy.

## Peer Reviews

confidence: 4
contribution: 3
initial_rating: 6
presentation: 3
questions: 1) I wonder your takes on the issues I point out above. 

2) Moreover, you use  GPT-4o only in real world experiments, why is that? 

3) Why does the overall  accuracy scores decrease in Figure 5, in real world narratives compared to Semi-synthetic narratives?
soundness: 3
strengths: - Very relevant line of research, with high impact. (spotting the failure modes of LLMs on causal reasoning ).  
-Good writing and. exposition in general and easy to follow. 
- The proposed setting of approach is interesting, and the depth and multitude of the analysis is sufficient.
-Overall, results are interesting, intuitive and worth considering. I find the results expected but not trivial.  I am particularly pleased to see the positive effect of the causal graph, which can help the CoT practice as well, as they put it in the limitations sections.
summary: The paper investigates the causal reasoning capabilities of recent LLMs; through various settings; via synthetic, semi-synthetic and real world narratives, comparing their consistency in forward and reverse causal narrative, against their parametric knowledge (pre-trained model knowledge) via  direct prompting, CoT prompting,  and graph extracting, with varying length of causal (anti-causal) chains. Results suggest that LLMs show considerable  bias towards their parametric knowledge, and are more successful in queries that align with them. They do in general perform pretty weak in causal reasoning task. The  extracted graph mostly leverages their strength since it helps them to spend more attention to the whole of the narrative text.
weaknesses: My main concern  is that the experiments/results can be misinterpreted and therefore may not be strongly conclusive for the intended tasks,  due to the followings:  

1) Vagueness in expressions: This is mainly due to vagueness in expression formulation sespecially in the reverse part. : Take the given example given in the paper: "The film festival set the stage for the rest of the events, creating a desire for cultural experiences and community gatherings that ultimately led to the trampoline park party."  Here the "The film festival set the stage for the rest of the events" sounds as if it took place afterwards due to "rest" of the events, and the part (desire and community gatherings) of the sentence mentioned before the "trampoline park party" races against the "film festival" and distracts the attention, hence the overall degrading in the causal reasoning estimation is likely.  

2) Conformity to human communication: Another aspect is that LLMs can mis-read your intention, since they can falsely "assume" or "expect" you do typos or miswrite/communicate. This is a strategic choice in their design (and perhaps even their nature due to their training corpora) to not make them brittle, and work with real human conversations. 


3) Referral mismatch. You say: “Although this relationship is displayed in the chain of events in the narrative, it is not logical and counter-intuitive” and answer incorrectly." Here, although you told the LLMs to ignore the parametric knowledge (or as you put it "explicit instruction to ignore outside information, allow for ill-logical relationships, and answer solely based on the hypothetical narrative"), it may still refer to the "meta-explanation" when it says it is not logical or counter-intuitive since it "informs" the user.  So there might be a spurious mismatch between the task you expect vs.  the setting LLM assume itself in. 

4) On conflicts (unknown vs. opposite): You say: "streambank erosion to higher prices, but this contradicts the LLM’s parametric knowledge since this causal effect may not typically exist"  Here again I find this rather vague conflict, and LLM might likely to conform the given example, when it cannot find counter examples, and can simply hallucinate. I would expect rather a hard conflict e.g.,  inverse causation, something like, rooster cause the sun rise. 

I appreciate the paper's work, but the above concerns is due to inherent complexity of dealing with natural language, and its discrepancy against the intended task (hence we should be ourself causally-careful.)

Typos: 

-  would would
-the our dataset

---

confidence: 3
contribution: 2
initial_rating: 5
presentation: 3
questions: - In Figure 4, Claude showcases a more significant divide between the forward and reverse settings for the narratives compared to ChatGPT, any insight as to why that is the case?
- What would happen if the narrative doesn’t explicitly state the causal relations using phrases like “results-in”, “causes”?
- What was the accuracy of LLM recovering the correct chain graph in the settings “Forward/Reverse-Graph”, “Causal/Anti-Causal Graph”?
- Why not combine both chain of thought reasoning and G’ to see if there is any improvement in performance?
-clearly prompt based solutions fails across many reasoning tasks as reported heavily by past works, and causal discovery is no exception.   Several past works suggest to combine the data driven method with LLM for causal discovery. Relying entirely on inherent knowledge of LLM have proven to be problematic in reasoning tasks and not sure this paper puts significant lights on as how we devise better methods for causal reasoning,
soundness: 2
strengths: - The paper provides a formal characterization of the failure modes of an LLM towards causal reasoning tasks.
- Some of these failure modes are quite interesting, for example, the one where the narrative introduces the causal variables in reverse order leads to poor performance.
- The paper is well presented and written, making it easy to follow.
summary: In this paper, the authors study the effectiveness of using LLMs to learn the cause effect relationships in a chain graph in both synthetic and real-world settings. The authors report the various modes through which the LLMs fails in this task. These failure modes can be summarized as follows:
1. LLMs rely heavily on the order in which the causal relationships are verbalized in the narrative.
2. LLMs use their parametric knowledge as a shortcut to answer causal questions.
3. LLMs fail more often when the narrative becomes larger.
The authors also investigate various prompting strategies that help avoid some of these pitfalls.
weaknesses: - In all the experiments, the narratives provided to the LLM convey significant insight on the causal structure of the chain graph under consideration. Using phrases like “results in”, “leads to”, “causes” in the narrative directly places a causal link between the variables. This approach would fail in scenario where such information is not already available. This makes the role of LLM quite straightforward (at least in the forward narrative setting).
- There was no explicit characterization of the error in the chain graph G’ that is produced by the LLM, without this it makes it unclear as to where the error stem in this setting stems from.
- It would help to provide more insight in to why there are inconsistencies between the answers to the causal reasoning task and the chain graph G’ predicted by the LLM
-in light of existing works in use of LLM for causal discovery, some of these failures have been observed by past works and hence advocated hybrid approaches by combining with data driven methods.

---

confidence: 4
contribution: 1
initial_rating: 3
presentation: 1
questions: Please refer to the 'Weaknesses' section.
soundness: 2
strengths: 1. The paper tackles the crucial issue of LLM's causal reasoning ability, a topic of significant current interest.
2. The experiment includes both synthetic and real-world data, which provides a more comprehensive assessment.
summary: This paper investigates the causal reasoning abilities of Large Language Models (LLMs) in narratives. It identifies several limitations, including reliance on event topological order, struggles with long context, and over-reliance on parametric knowledge. Furthermore, the authors demonstrate that explicitly prompting LLMs to generate causal graphs can improve performance.
weaknesses: 1. The core contributions and findings of this paper appear to heavily overlap with the previously published work *LLMs Are Prone to Fallacies in Causal Inference*, which **appeared on arXiv in June 2024** and was subsequently **published at EMNLP 2024**. However, this highly relevant and similar prior work is not cited or discussed. Moreover, the technical details and experimental validation are less comprehensive than in the aforementioned paper. Ensuring proper citation of relevant prior work is crucial for advancing academic discourse and providing a transparent foundation for new contributions.
2. The evaluation uses yes/no questions, meaning the random guess baseline is 0.5.  It is concerning that many results for GPT-4, GPT-4o, and Claude-3.5-Sonnet are below this baseline. While this might be theoretically possible under certain circumstances, the paper does not provide sufficient explanation or analysis of this phenomenon.  This lack of explanation weakens the validity of the experimental findings and necessitates further investigation and a detailed error analysis.
3. The paper lacks essential details about the evaluation dataset, making it difficult to assess the validity of the results.  Crucially, the number of samples tested, the number of nodes in each graph, and the specific types of events present in the narratives are not clearly specified. While line 149 mentions that "events are real-world phenomena", this description is too broad and lacks necessary citations.  Furthermore, the inconsistency between Figure 5(b) (maximum chain length 9) and Figure 6(b) (maximum chain length 10) raises questions about whether different datasets were employed for the synthetic and real-world narratives, or is this an oversight? Clarification on these points is essential.
4. The paper does not adequately address how the quality of the generated narratives is ensured. How are the narratives verified to ensure they conform to the specified chain structures? What mechanisms are in place to check and assess the data quality? Furthermore, given the identified issue of parametric knowledge conflicts within the models, how does the generation process prevent the introduction of factual errors or inconsistencies into the narratives? This lack of detail raises concerns about the reliability of the experimental data.
5. While the paper's focus on chain causal relationships provides a starting point, the scope remains relatively narrow. Exploring more complex scenarios is crucial for a more comprehensive understanding of LLM's causal reasoing ability.
6. The model selection is quite limited, using only GPT-4, GPT-4o, and Claude-3.5-Sonnet for both data generation and evaluation. This raises concerns about the generalizability of the findings. Including leading open-access models, such as Llama 3.1, would strengthen the analysis and provide insights into whether the observed phenomena are specific to limited-access models or hold across a wider range of LLMs.
7. The paper's logical structure could be improved to enhance clarity and readability. Additionally, the writing also requires significant revision to ensure the work is accessible and understandable to the reader.
8. The poor quality of the figures hinders understanding of the presented work. For example, Figure 3 is blurry and difficult to interpret. 
9. The referencing of both articles and figures needs substantial improvement. Several errors were noted, including incorrect article citations on lines 156-157 and incorrect figure references on lines 248, 304, and 320.

---

confidence: 3
contribution: 2
initial_rating: 3
presentation: 2
questions: My questions mainly regard the points mentioned in the weaknesses above. In particular, I would like to ask the authors to comment on the following questions:

1. I would like to ask the authors to comment on point 1 of the above weaknesses. In particular, I would like to know whether the generated narratives adhere to the underlying causal structure and ordering. Are there differences in predictive performance of the LLM for samples that deviate from the desired ordering?
2. Regarding point 3 of the above weaknesses: Does the order of the provided graph information cohere to the ordering of the causal chain and would this be suited to leak information to the model, possibly improving its performance? Could the authors please elaborate and provide evidence for or against this scenario?
3. I would like to ask the authors elaborate on the necessity of reordering events in the experiments of section 3.3. How can the effects of reordering and anti-causal edges be better differentiated?
soundness: 2
strengths: The authors provide a series of evaluations on generated, semi-synthetic and real-world causal relations, that sufficiently support the claims of the paper. The paper provides a more detailed analysis on causal reasoning capabilities than previous works and experiments are generally well described.

By reasoning over counterfactual observations, the authors successfully show that LLM have trouble incorporating contextual information of the prompt that stands in contrast to learned knowledge. While previous work already identified general shortcomings in causal reasoning capabilities of LLM, the authors provide a more detailed breakdown of particular factors. The found confounding between predictive performance and order of narrative might be particular important to overcome.

While the authors test LLMs in a rather restricted settings, the presented results provide clear evidence towards the claimed effects. Limitations on the causal graph structure and types of inspected effects are discussed sufficiently, and one would expect results to only worsen when applied to more complex graphs.
summary: In their work, the author aim to inspect the causal reasoning capabilities of LLMs in natural language texts. The authors study effects on predictive performance in long-term reasoning and investigate performance degradation when reason over contextual information that is counterfactual to their learned 'parametric' knowledge.

The authors utilize LLMs to generate (semi-synthetic) narratives about events resembling causal chains from CauseNet. In that, 'anti-causal' relations are purposefully injected, which stand in contrast to factual observations of the real-world. Finally, reverse topological narratives are created in which items of the causal chain appear in reverse order.

The authors measure LLM performance, by tasking the LLMs to restore the causal graph G from the narrative, using CoT and In-Context learning. In summary, the authors find that performance degradates when presenting narratives in reverse causal order, for 'anti-causal' relations and with longer chains of events. CoT and in-context learning do not boost performance, while providing graph information does seem to help even for longer chains.
weaknesses: While I agree on many interpretations of the presented results, I have several questions regarding the soundness of some experimental setups and the provision of graph information:

1) In section 3.2, the authors give an example of a "Film festival → Food truck rally → Trampoline park party" narrative. While the goal of generating a reversed narrative was pursued, the term 'Trampoline park party' was again mentioned after the 'Film festival' in the last sentence, establishing a relation in the correct relation order. Enforcing items of the causal chain to appear in reversed order, might pose a challenging task, as item positions can get switched due to the syntax of natural language. It would be interesting to know, how often such generation 'errors' occur within the data. The paper could be improved by analyzing to which extend LLM generations obey the desired ordering of events.
2) Throughout the paper, the authors refer to 'anti-causal' effects to indicate inversion of the causal effect *strength*, e.g. "cancer->longer life". The term 'anti-causal', is commonly reserved to indicate a reversed causal *direction* e.g. "cancer<-longer life". This is quite confusing, and given that it appears in the prompt presented in A.1.3, might have an impact on narrative generation. I would like to suggest to consider an alternative wording, e.g. counterfactual effects.
3) The authors provide causal graph information to the LLM during several experiments. While this seems to improve LLM performance, I could not find any details on the exact format of the provided graph information. It could be, that graph information is provided in the correct causal order, which would again improve LLM performance due to ordering bias and not due to reasoning capabilities of the LLM. The paper could be improved by further specifying and, if needed, randomizing the order of the provided causal effects in the experiments.
4) The reordering of relations in the experimental setup of section 3.3 might conceal the true influence of the anti-causal edges on the predictive performance of LLMs. As reordering was previously shown to reduce performance, it becomes unclear whether anti-causality or reordering of relation is the true cause for the performance degradation. I believe that effects of anti-causal relations might be better demonstrated when keeping the ordering intact and simply varying the amount of anti-causal relations within the chains.
5) To the best of my understanding, the problem of identifying and reasoning about causal relations in texts is commonly known as event causality identification (ECI; in the field of NL) or simply event causality (when moving closer to classical Pearlian causality). While the authors are able to produce new insights on the failure modes of LLM, several prior works derived similar results and are not discussed. This regards general reasoning with counterfactual contexts (e.g. [1,2,3]), as well as general causal text understanding (e.g. [4,5]). While these works certainly form a niche within the bigger field of causality and LLMs, I would recommend the authors to consider these relations.


**Minor**

* Unnecessary brackets are added for citations in Sec. 3.1 (l156/157) and in Sec. A.1.9.
* The resolution of figure 3 is quite low, such that the legend is unpleasant to read. I would like to suggest to either embed the figure as a vector graphic/PDF or increase its resolution.
* Some words seem to be missing in line 248.


[1] Li, Jiaxuan, Lang Yu, and Allyson Ettinger. "Counterfactual reasoning: Do language models need world knowledge for causal inference?." NeurIPS 2022 Workshop on Neuro Causal and Symbolic AI (nCSI). 2022.  
[2] Li, Jiaxuan, Lang Yu, and Allyson Ettinger. "Counterfactual reasoning: Testing language models' understanding of hypothetical scenarios." arXiv preprint arXiv:2305.16572 (2023).  
[3] Frohberg, Jörg, and Frank Binder. "Crass: A novel data set and benchmark to test counterfactual reasoning of large language models." arXiv preprint arXiv:2112.11941 (2021).  
[4] Gao, Jinglong, et al. "Is chatgpt a good causal reasoner? a comprehensive evaluation." arXiv preprint arXiv:2305.07375 (2023). 
[5] Ashwani, Swagata, et al. "Cause and Effect: Can Large Language Models Truly Understand Causality?." arXiv preprint arXiv:2402.18139 (2024).