Title: Discovery of the Hidden World with Large Language Models
Venue: NeurIPS 2024
Accepted: yes
Keywords: Causal Discovery

Large Language Models

Causal Representation Learning
Abstract: Revealing the underlying causal mechanisms in the real world is the key to the development of science. Despite the progress in the past decades, traditional causal discovery approaches (CDs) mainly rely on high-quality measured variables, usually given by human experts, to find causal relations. The lack of well-defined high-level variables in many real-world applications has already been a longstanding roadblock to a broader application of CDs. To this end, this paper presents Causal representatiOn AssistanT (COAT) that introduces large language models (LLMs) to bridge the gap. LLMs are trained on massive observations of the world and have demonstrated great capability in extracting key information from unstructured data. Therefore, it is natural to employ LLMs to assist with proposing useful high-level factors and crafting their measurements. Meanwhile, COAT also adopts CDs to find causal relations among the identified variables as well as to provide feedback to LLMs to iteratively refine the proposed factors. We show that LLMs and CDs are mutually beneficial and the constructed feedback provably also helps with the factor proposal. We construct and curate several synthetic and real-world benchmarks including analysis of human reviews and diagnosis of neuropathic and brain tumors, to comprehensively evaluate COAT. Extensive empirical results confirm the effectiveness and reliability of COAT with significant improvements.

Discovery of the Hidden World with Large Language Models

Abstract

Revealing the underlying causal mechanisms in the real world is the key to the development of science. Despite the progress in the past decades, traditional causal discovery approaches (CDs) mainly rely on high-quality measured variables, usually given by human experts, to find causal relations. The lack of well-defined high-level variables in many real-world applications has already been a longstanding roadblock to a broader application of CDs. To this end, this paper presents C ausal representati O n A ssistan T (COAT) that introduces large language models (LLMs) to bridge the gap. LLMs are trained on massive observations of the world and have demonstrated great capability in extracting key information from unstructured data. Therefore, it is natural to employ LLMs to assist with proposing useful high-level factors and crafting their measurements. Meanwhile, COAT also adopts CDs to find causal relations among the identified variables as well as to provide feedback to LLMs to iteratively refine the proposed factors. We show that LLMs and CDs are mutually beneficial and the constructed feedback provably also helps with the factor proposal. We construct and curate several synthetic and real-world benchmarks including analysis of human reviews and diagnosis of neuropathic and brain tumors, to comprehensively evaluate COAT. Extensive empirical results confirm the effectiveness and reliability of COAT with significant improvements.

1 Introduction

Science originates along with identifying important variables and revealing their causal relations [1, 2]. Despite the progress in the past decades, existing causal discovery approaches (CDs) mainly rely on high-quality measured variables, which are usually given by human experts [3, 4, 5]. However, the desired variables and their measurements are usually unavailable in a wide range of real-world applications. For example, Amazon sellers who want to analyze the factors related to user ratings only have raw user reviews, which are written according to the underlying user preferences for certain product characteristics. Therefore, the lack of high-quality high-level variables has been a longstanding impediment to broader real-world applications of CDs or causality-inspired methods [6].

Score 4:

This

small

apple,

appealing

in its compact form …

Score 1:

The

aroma, understated

and forgettable …

'What are the high-level factors

associated with the Score?'

After examining the reviews, I

decide to use the following

factors:

1. Size appeal

:

1:[small size]

-1:[Large or medium size]

0:[Neutral/not mentioned]

...

Score 4:

This

small

apple,

appealing

in its compact form …

Score 1:

The

aroma, understated

and forgettable …

'Please help me annotate the

data according to …'

Reviewer, Size, …, Score

0,    1, …,     4

1,    0, …,     1

...

(b) Factor Annotation

🤖

<External Resource>

Causal-learn

Score 3:

and its

taste

, … ,

misses the zestful balance of

tartness …

Score 1: A bite of the

apple

feels the bug…

'What are the high-level factors

associated with the scores

other

than

Size

(c) Causal Discovery & Feedback Construction

The recently emerged Large Language Models (LLMs) [7, 8, 9, 10] offer a new opportunity to mitigate the gap [11]. Trained from massive observations of the world, LLMs demonstrate impressive capabilities in comprehending unstructured inputs , and leveraging the learned rich knowledge to resolve a variety of general tasks [12]. A surge of early tests demonstrates promising results that LLMs can effectively leverage the learned knowledge to answer commonsense causal questions [11, 13, 14]. Nevertheless, existing approaches mainly focus on incorporating LLMs as a straightforward reasoner with respect to the given causal variables . The reliability of LLMs in directly reasoning the true causal structure behind any specific data-generating process remains a debate [13, 15, 16, 17] due to a series of drawbacks of LLMs [18, 19, 20]. In addition, all of the existing combinations of LLMs and causal discovery have surprisingly overlooked the identifiability of causal structure, which plays an important role in classic causal discovery literature [3, 4, 5]. Hence, a challenging question comes:

How can LLMs reliably assist in revealing the causal mechanisms behind the real world?

In this work, we answer the question with a focus on local causal discovery with respect to a target variable that poses high value such as customer ratings and medical diagnosis, and introduce C ausal representati O n A ssistan T (COAT). Specifically, given the target variable Y , COAT aims to identify a Markov blanket to Y from raw observation and also produce the theoretical-guaranteed causal structure. To achieve the goal, as illustrated in Fig. 1, COAT employs two mutually beneficial components: LLMs and CDs. Iteratively, at step (a), COAT leverages LLMs to look into a set of unstructured observations (e.g., customer comments) and propose potentially useful high-level factors. The proposed factors contain both the definition and the annotation criteria. Therefore, at step (b), another LLM is employed to give concrete values following the criteria. Then in step (c), CDs is used to reveal the structure among the identified factors. To ensure the reliability of the factor identification, COAT constructs feedback from the intermediate causal discovery results from step (c) to further inspire LLMs to improve further factors. The feedback includes sampling important observations that can not be well explained by the existing identified results. We show that the feedback provably helps with identifying the desired Markov Blanket and the structure (Proposition 2.2). We present a comprehensive analysis of COAT on both synthetic simulations and

real-world case studies, ranging from analysis of human reviews to diagnosis of neuropathic and brain tumors (Sec. 4). Our contributions are summarized as follows:

To the best of our knowledge, we are the first to leverage LLMs to propose high-level variables, thereby extending the scope of CDs to unstructured data (Sec. 2.1).

We establish the first benchmarks with real-world data including AppleGastronome and Neuropathic to examine the unstructured causal discovery (Sec. 4).

We propose the first framework COAT that combines the best of LLMs and CDs to find theoretically grounded causal results (Sec. 2.2), which are verified with extensive empirical studies (Sec. 4).

Additionally, the analysis of COAT also derives the first metrics that measure the causal representation learning capabilities of various LLMs (Definition 2.3).

2 Representation Assistant for Causal Discovery

In this section, we present the formulation of leveraging a language model to serve as a representation assistant for causal discovery on unstructured data. The representation assistant needs to extract useful factors that capture sufficient information for an interested target variable.

2.1 Problem Definition

Data To begin with, we are given a target variable of interest Y , e.g., stars rated by a customer or the tumor type of a patient. We treat Y as a scalar random variable without loss of generality. The unstructured data , or raw observations, e.g., customer review of a certain product or images of tumors, are denoted as X . The dataset D consists of n paired samples { ( x i , y i ) } n i =1 that are independently drawn from the distribution over ( X , Y ) . Note that the target variable Y serves as a guider, and no prior assumption on the relations between X and Y is assumed.

Objective We seek a mapping h : X ↦→ Z which elicits the structured representation Z = h ( X ) such that Y ⊥ ⊥ X | Z . In other words, Z serves as a Markov Blanket of Y for the unstructured raw observations. Built upon the structure representation, then, downstream causal discovery methods can be applied on Z ∪ { Y } . The revealed causal structure can provide insights about the target variable Y [21, 22], such as what factors of the product would be most satisfactory to customers. Furthermore, the framework can be easily extended to discover a complete causal graph by shifting the target variable to the other identified factors or the other additionally available variables. Formal assumptions are discussed in Sec. 2.4.

LLM as a representation assistant We aim to make the most use of the rich knowledge of LLMs to assist in extracting the relevant information from the raw observations X . To this end, the mapping h is decomposed as a collection of factors W = { w 1 , w 2 , ..., w k } , each of which is a function w i : X ↦→C that maps the raw observation x to a predefined value space C . In other words, the structured representation is composed of multiple factors: h ( X ) = ( w 1 ( X ) , w 2 ( X ) , · · · , w k ( X )) . Throughout this work, for the notation of factors, we use the symbol w i to denote the factor itself like sweetness , size , or scent , and w i ( · ) to denote the function that maps from raw observation space X to the predefined value space C .

Descriptions of the factors Without loss of generality, in this work, we consider the descriptions of the factors in natural language, which can be divided into two categories: i) Implicit factors, which need to be discovered and elaborated by LLMs. To obtain the values of the implicit factors, one could feed the factor descriptions and the unstructured input x i to a suitable LLM for value annotation; For example, given a customer review on an apple x i , a discovered implicit factor C = {-1 , 0 , 1 } : w 1 ( x i ) = 1 could mean that the customer appreciates the sweetness of the apple; w 1 ( x ) = -1 means that the customer is disappointed about the sweetness; w 1 ( x ) = 0 means the sweetness has not been mentioned. ii) Explicit factors such as heart rate, whose descriptions are already available. The measure of the explicit factors usually requires some external tools.

2.2 The COAT Framework

We approach the aforementioned problem via a new framework called C ausal representati O n A ssistan T (COAT) (Algorithm 1). COAT aims to extract useful factors through multiple rounds of

You are an excellently helpful AI assistant for analyzing, abstracting, and processing data. Now try your best.

Data

group with 'score' = -1.0

-

This apple variety impresses with its robust

nutritional profile, packing a hearty dose of

dietary fiber and vitamin C...

group with 'score' = 0.0

-

Despite its large size and intense aroma,

this apple offers disappointing nutritional...

group with 'score' = 1.0

-

Despite its large size, which may appeal to

those looking for a sizable fruit option, this

...

Your output should contain parts described as follows.

Part 1: Consideration.

In this part, feel free to write down the process of your considerations. Hint: You need to abstract, identify, and design suitable factors with corresponding criteria, and each factor is only allowed to take value from  [-1, 0, 1] .

Part 2: Factor filtration.

You shoud decide whether to use each of the proposed factors by following criteria: - Each new factor should be helpful to distinguish the groups.

Each Factor should focus on one concrete aspect and try to avoid overlape.

Part 3: Final Output.

In this part, you are required to report the factors that you decided to use.

Each factors' value should from [-1, 0, 1]

You should give specific criterion for each value of each factor.

Keep the criterion for value 0 fixed as "Otherwise; or not mentioned".

Report the factors in following template :

```

Factor Name

-

1: [Positive Criterion]

-

0: [Otherwise; or not mentioned].

-

-1: [Negative Criterion]

```

iteration. We use the superscript t to refer to the input and output of LLM at the t -th round. We also denote the union of results from the first t rounds using the superscript ' ≤ t '.

Factor proposal To induce useful highlevel factors from the rich world knowledge of LLMs, COAT employs a prompt p and a few samples ̂ D ⊆ D .

Specifically, in Fig. 2, p contains three components: samples , instructions , and format control . To encourage LLM to focus on the information related to the target variable Y , samples are grouped by the value of Y . The instruction requires Ψ to give each proposed factor w i a concrete description of the mapping w i ( · ) , like how to decide the factor values. In addition, the metadata about the task such as the task description and context can also be incorporated if available. The prompt p essentially imitates human experts [23] in selecting and

Algorithm 1 The COAT Framework

1: Required: Dataset D = { ( x i , y i ) } n i =1 ; LLM for factor proposal Ψ ; Model for factor parsing Ψ s ; causal discovery algorithm A ; Feedback construc- tor F ; Maximal rounds T ;

2: Random sampling ̂ D 1 ;

3: Constructing p 1 ;

4: while not converge and current round t < T do

5: W t ← Ψ( p t , ̂ D t ) ; //factor proposal

6: Z t ← Ψ s ( D , W t , p p ) ; //factor parsing

7: G t ←A ( Z ≤ t ∪ { Y } ) ; //causal discovery

8: ( ̂ D t +1 , p t +1 ) ←F ( G t , D , p t ) ; //feedback

9: end while

10: return G

T

defining high-level variables. The set of resulting factors in the t -th round, defined with natural language by the LLM Ψ , is denoted as W t = Ψ( p t , ̂ D t ) . We merge them with factors proposed in the previous rounds to update the set of all factors W ≤ t = W 1 ∪ · · · ∪ W t .

Factor parsing Once we obtain the candidate factors, we then collect the values of the factors from the unstructured observations. In prior works, they are usually collected from human experts according to the given factors [3]. To do so, another LLM Ψ s is instructed to read the annotated guidelines of each variable w i and parse the unstructured observations into structured or tabular data:

where p p refers to the additional instruction to prompt Ψ s to parse the observed data, and z i refers to the parsed values for the corresponding factor ̂ w i . We define Z ≤ t := Concat ( { z i for w i ∈ W ≤ t } ) .

When the data curation of the proposed factors requires additional domain-specific knowledge/skills (e.g., intervening on the external environments) that the LLMs do not acquire, we could fetch z i through some external process [24, 25]. For example, studying the causes of a disease requires annotating relevant symptoms from diagnosis records and conducting additional medical checks [26]. Our experiments show that COAT can effectively extract the hidden factors under both schemes.

Causal discovery With the given values Z ≤ t associated with the candidate factors W ≤ t , a CD algorithm A (e.g., FCI [27]) is used to reason about the causal structure based on the parsed data:

where G t is the discovered causal structure. In general, the inputs in each round to A may contain noises as well as latent confounders, any CDs with suitable theoretical assumptions could be used for A . The noises injected through LLM-based parsing may be of independent interest to the literature of causal discovery [5, 28]. In this work, we demonstrate the idea of COAT via the FCI algorithm [4] as it is flexible with respect to different functional classes of the underlying generation process, allows for the existence of latent confounders [27], which aligns well with our objective.

Improving factor proposal with causal feedback LLMs require proper prompts to fully unlock their capabilities [29, 30, 31]. When it comes to factor proposing, it is also hard for LLMs to propose all factors at once. Nevertheless, from the causal discovery results, we could find useful information and thus provide feedback to further improve the factor proposal:

where F samples specific examples from D and constructs new prompts according to the results of A for the next round of factor proposal. For example, FCI is able to imply the existence of latent confounders, from which we could refine p to prompt Ψ to focus on the corresponding factors.

2.3 Causal Feedback

Let X be the high-dimensional random variable for the raw data. After t rounds, COAT identifies h ≤ t ( X ) as the Markov Blanket for Y w.r.t. to { w i ( X ) | w i ∈ W ≤ t } . If Y ̸ ⊥ ⊥ X | h ≤ t ( X ) , which means it cannot serve as a Markov Blanket [32] for Y w.r.t. X , then there exists a potential factor ̂ w : X ↦→C such that:

where H ( · ) refers to the entropy. If the LLM Ψ can not find the desired ̂ w , it means that h ≤ t ( X ) is already sufficient to separate Y from X . Therefore, for the next ( t +1) -th iteration with sample ̂ D t +1 , Ψ is expected to propose new factor ̂ w that also satisfies the similar property:

y

y

̂

w

Y

Y

Y

Y

W

̂

w

̂

w

̂

w

W

Figure 3: Illustration of variables that could be discovered with COAT. Let W ∈ h ≤ t ( X ) be an identified variable, and assume there exists a latent variable ̂ w to be discovered. When ̂ w is the direct parent or child of Y , finding hard-to-explain samples can help uncover it. When ̂ w is the direct parent and also a child of W , or the spouse of Y with W as the common child of Y and ̂ w , conditioning on W facilitates the discovery of ̂ w .

where H ̂ D t ( ·|· ) refers to the conditional entropy measured on ̂ D t . As shown in Fig. 3, finding factors satisfying Eq. 4 progressively expands the discovered factors and pushes h ≤ t ( X ) to a valid Markov Blanket. Also, it implies conditioning on the identified factors would further strengthen the correlation between ̂ w and Y . Therefore, to find the desired factor, we are motivated to select suitable ̂ D t +1 for the next iteration such that ̂ D t +1 = arg max ̂ D⊆D H ̂ D ( Y | h ≤ t ( X )) , where h ≤ t ( X )

can not well explain Y . This problem can be converted into a classification problem in which ̂ D ∗ are the samples that the fitted classifier yields a large prediction error. In our experiments, we implement the classification via clustering with respect to h ≤ t ( X ) . The clustering elicits C groups ̂ D c := { x i for i ∈ I c } : I 1 , · · · , I C = K-Means ( h ≤ t ( X ) ) . We then take the group of samples with the largest conditional entropy to construct the feedback.

In practice, many factors, such as the LLM capabilities, data faithfulness, and prompt templates, could affect the satisfaction of Eq 5. Therefore, in the next section, we will establish a theoretical framework to discuss the influence of the factors above to the satisfaction of Eq 5.

2.4 Theoretical Analysis

We then theoretically analyze some critical steps in COAT, including the feedback and identifiability.

Feedback analysis Given a new factor w k +1 , with the current representation as h [ k ] ( X ) = ( w 1 ( X ) , w 2 ( X ) , · · · , w k ( X )) , and COAT tests:

COAT also requires the following usual condition about distribution and causal graph:

Assumption 2.1 (Faithful and Markov conditions, adjusted from [33]) . For any disjoint non-empty subsets A,B,C ⊂ W ≤ t ∪ { Y } , A and B are d-separated by C on the causal graph iif A ⊥ ⊥ B | C on the factors' distribution. All conditional independencies are preserved after factor parsing.

The annotation from a poor model could introduce an additional 'error term' on the true factor values, disturbing the true distribution, as one can observe in Fig. 4(a) and 4(b). If Assumption 2.1 holds, the conditional mutual information between Y and X given the desired factors decreases:

Proposition 2.2. Under assumption 2.1, if condition 6 holds, then for Markov Blanket S ⊆ [ k +1] of Y , i.e., Y ⊥ ⊥ h [ k +1] \S ( X ) | h S ( X ) , we have the following about conditional mutual information:

Factor identification We provide an initial exploration under what conditions LLMs can identify target-related factors and how the ability of an LLM influences this procedure.

Definition 2.3 (Ability of LLMs) . Given a suitable prompt about current factors and data, the LLM Ψ has non-zero probability p Ψ > 0 to propose a new factor w k +1 that satisfies condition 6 and

for some positive constant C Ψ whenever I ( Y ; X | h [ k ] ( X ) ) > 0 . Note that h [0] ( X ) = ϕ , hence we also use I ( Y ; X | h [0] ( X ) ) to refer I ( Y ; X ) . We use p instead of p Ψ when the context is clear.

We further explain the intuition behind Def 2.3: the Perception Score p captures the LLM's responsiveness to the given prompts and the feedback; the Capacity Score C Ψ captures the quality of the factors proposed by the LLM. Empirically, the two scores are used to estimate the abilities of the predominant LLMs (Sec. 3.2). Theoretically, we use them to characterize the influence of prompt templates, the LLM responsiveness, and the quality of factors on the performance of COAT:

Proposition 2.4 (Characterization for Factor Identification Process) . With assumption 2.1, for any small number ϵ, δ ∈ (0 , 1 2 ) , perception score p > 0 , capacity score C Ψ > 0 , with t COAT rounds that

where z δ is the δ -quantile of the standard normal distribution, we have

The proof is given in Appendix D.2. Prop. 2.4 gives a guarantee on identifying a Markov Blanket. Intuitively, Prop. 2.4 also characterizes the influence of prompt templates, the LLM responsiveness, and the quality of factors on the performance of COAT via the two proposed measures: p and C Ψ . When both of them are positive, COAT would converge exponentially:

Proposition 2.5 (Rate of Convergence) . With assumption 2.1, for any small number ϵ, δ ∈ (0 , 1 2 ) , perception score p > 0 , capacity score C Ψ > 0 , after t COAT rounds, the following inequality holds with probability at least 1 -δ :

Causal structure identification It is clear that LLMs are not involved in the causal discovery process, which is mainly executed by causal discovery methods such as FCI. Therefore, the CD guarantees the identifiability of the final causal graph over the LLM-proposed factors. The concrete

p

Ψ

Size

Smell

Taste

0

20

40

60

80

100

Random

GPT-3.5

(a) Apple attributes Acc.

Size

Smell

Taste

0

20

40

60

80

Random

GPT-3.5

(b) Preference matchness Acc.

0.18

0.19

0.20

0.21

0.22

0.23

0.24

0.25

0.26

C

0.35

0.40

0.45

0.50

0.55

0.60

0.65

0.70

0.75

GPT 4o

GPT 4

GPT 3.5

Mistral-Large

Mistral-Medium

llama-3-70b

llama-2-70b

Qwen-1.5-110B

DeepSeek-V2

Claude-3-Opus

(c) LLM Ability Scatters

assumptions required for identifiability depend on the specific CD used in COAT. For instance, the FCI algorithm requires faithfulness of the data distribution with respect to the true causal graph [27]. In our experiments, we also verify that the structured data annotated by LLMs has a high accuracy and little noise, which is friendly to the CD assumption. In general, one could switch to another CD in COAT, while using different CDs may require different assumptions. For example, the LiNGAM algorithm requires the relations among variables to be linear and non-Gaussian models. Empirically, we find that COAT with LiNGAM works very well (Appendix F).

2.5 Practical Discussions

Prompt template The instruction following capability and the context window of LLMs may affect the satisfaction to the constraints of the prompt template. Including more data samples or background knowledge may improve the p and C Ψ , but it is more challenging for the LLM.

Modern causal discovery We use FCI in this paper in order to illustrate the idea. To attain identifiability better than the Markov equivalent class, one can choose more advanced Causal Discovery methods under different assumptions, see Appendix C if interested.

We also discuss some cases where we need to handle them properly in practice with LLMs.

Factor filtering LLMs may output several factors with similar semantics or exhibit multicollinearity in the annotated data, which will hinder the causal discovery process. To mitigate the issue, one could do factor filtering that adopts PCA or early conditional independence tests given the currently discovered variables in the Markov Blanket to detect and eliminate these variables.

Factor pool LLMs may discover useful factors in early rounds while being discarded. For example, the underlying spouse variables of the target label y may be independent with y without conditioning on their common children variables. To resolve the issue, we could introduce a factor pool that stores the candidate variables proposed in the past, and replay the variables that have not been passed by conditional independence tests with existing variables in the Markov Blanket for a double check.

3 Empirical Analysis of COAT

We evaluate whether COAT can propose and identify a set of high-level factors belonging to the Markov Blanket of the target variable Y . We construct the first benchmark, called AppleGastronome, to verify the effectiveness of COAT in finding useful causal information, and compare COAT to previous methods [11] that merely leverage LLMs to perform causal discovery. Specifically, we use AppleGastronome to examine the capabilities of 10 predominant LLMs including GPT 4o [34], Claude-3-Opus [35], LLaMA3-70b [36], and Mistral-Large [37] in realizing COAT. Due to the space limit, we report only the results of the popular LLMs and present full results in Appendix E.4.

3.1 Experimental Setup

In AppleGastronome benchmark, we consider the target variable as a rating score of the apple by astronomers. We prepare different high-level factors: 3 parents of Y, one child of Y, and one spouse of Y. These factors form a Markov blanket of Y. In addition, we also prepared one disturbing factor

score

market potential

size

aroma

taste

juiciness

nutrition

(a) Ground truth

Score

Market Potential

Size

Taste

Juicy

Nutrition

(b) GPT-4 META

score

market potential

nutritional value

size

taste profile

(c) GPT 3.5 COAT

score

market potential

nutrient content

size

sweetness level

flavor balance

aroma

(d) Claude-3-Opus COAT

related to Y but not a part of this blanket. A good method is expected to propose the five high-level factors (up to semantical meanings) and exclude the disturbing factor.

Benchmark construction Each apple has its own attributes, including size, smell, and taste (or sweetness). Each gastronomy pays unique attention to a subset of the above three attributes. They will write a review according to their preference and give the rating score. We generate the review using GPT 4 by feeding GPT 4 the preferences and the apple attributes. We generated 200 samples for LLMs' analysis and annotation. More details are left in Appendix E.1.

Baselines For factor proposal, we mainly employ two different uses of LLMs as the baselines: META is the zero-shot factor proposal given only the context to LLMs; and DATA additionally gives some samples of raw observations, which is an ablation of COAT without the feedback module, i.e., only one COAT round. For causal relation inference, we follow Kiciman et al. [11] that prompt LLMs to reason for the causal direction of each pair of the discovered variables by DATA .

Metrics We evaluate the ability on factor proposal based on three metrics: MB , NMB , and OT . MB means the desired factor forming the Markov Blanket of Y. NMB means the undesired factor relevant to data but not in MB . OT means the unexpected factors irrelevant to data. We also present the corresponding recall, precision, and F1 with respect to MB ( Y ) .

3.2 Analysis with AppleGastronome Benchmark

Key findings Empirically, LLMs with CoT can be aware of high-level factors behind data (lower OT than META ) but still struggle to distinguish the desired factors in Markov Blanket (higher NMB than COAT). COAT is more resistant to the 'disturbing' factor, which is supported by the lower NMB column. COAT filters out irrelevant factors from LLMs' prior knowledge that are not reflected by the data, which is

supported by the lower OT column. COAT robustly encourages LLM to find more expected factors through the feedback, which is supported by the higher MB column.

Can LLMs be an effective factor proposer? As discussed in Sec. 2.4, there are two crucial abilities for LLMs in identifying potential high-level factors. The first one is to be aware of the existence of potential factors, and the second is to synthesize and describe these factors. Inspired

R shoulder impingement

R C4 Radiculopathy

R C5 Radiculopathy

DLI C4-C5

R C6 Radiculopathy

DLI C5-C6

(a) Faithful ground truth

R shoulder impingement

DLI C5-C6

R C6 Radiculopathy

L C6 Radiculopathy

R C4 Radiculopathy

R C7 Radiculopathy

L C5 Radiculopathy

DLI C4-C5

R C5 Radiculopathy

(b) GPT-4 COAT

R shoulder impingement

R C5 Radiculopathy

R C6 Radiculopathy

(c) GPT-4 reasoning

by this observation, we propose two novel metrics to quantify LLMs' causal ability: a) Perception that quantifies the ratio of valid factors (satisfying Prop. 2.2) proposed by LLMs in each round; b) Capacity that measures the effective mutual information drop in Assumption 2.3. As shown in Fig. 4(c), LLMs differ largely on the perception score while comparably on the capacity score.

Can LLMs be an effective factor annotator? As shown in Fig. 4, both GPT 3.5 and GPT 4 annotate subjective attributes well. Regarding objective human preferences, the performances are still relatively high. Empirically, LLMs will not introduce new confounders, see Appendix E.3.

Can COAT reliably recover the causal relationships? We present quantitative and qualitative results in Table 3 (in Appendix E.1) and Fig. 5 (on page 8), respectively. Compared to directly adopting LLMs to reason the causal relations, COAT significantly boosts the causal relation recovery. Meanwhile, COAT maintains high performances based on various LLMs, which further demonstrates the effectiveness of the causal feedback in COAT to improve the robustness of this system. In fact, the causal feedback focuses on making maximal use of the rich knowledge of LLMs, and reducing the reliance on the reasoning capabilities of different LLMs, to assist with causal discovery. We provide the full results of 10 LLMs in Appendix E.3.

4 Empirical Study with Realistic Benchmarks

After examining the capabilities of COAT in AppleGastronome, we are further motivated to challenge COAT in a more complex setting from neuropathic panic diagnosis [26], brain tumor detection with MRI images [38], three-years news summary about one stock from the New York Times [39], and climatic reanalysis data with fine-grained time and space coverage [40]. We refer to Appendix H for a complete summary of all five benchmarks.

4.1 Experimental Setup

Benchmark construction In the Neuropathic benchmark, we convert the dataset into a clinical diagnosis task. In the original dataset, there are three levels of causal variables, including the symptom level, radiculopathy level, and the pathophysiology level. In the experiments, we mainly consider the target variable of right shoulder impingement. When generating the clinical diagnosis notes as x using GPT 4, we avoid any mentioning of variables other than symptoms. We generated 100 samples for LLMs' analysis; since the number of possible factors is finite, we generate 1000 tabular data for CI tests.

Table 2: Factor proposal results in Neuropathic. PA, AN, and OT refer to the parents, ancestors, and others, respectively. Accuracy and F1 measure the recovery of the causal ancestors.

As we intend to leverage the Neuropathic benchmark to simulate the real-world diagnosis, after the factor proposal stage, we directly incorporate external tools to measure the values of the candidate factors. More details about the construction of the Neuropathic are given in Appendix E.6.

Momentum

Flux, V-

Component

Air

Temperature

Volumetric

Soil

Moisture

Surface

Roughness

Convective

Precipitation

Rate

Net Flux

Soil

Temperature

Sensible

Heat Net

Flux

SST Change

Downward

Longwave

Radiation

Flux

Zonal Wind

Momentum

Flux, U-

Component

Sea Level

Pressure

Equatorial Pacific Region

Nino3 Region

South American Coastal Region

Target Variable

Proposed Factor (stationary)

Proposed Factor (non-stationary)

Evaluation and baselines In Neuropathic, we adopt a similar evaluation protocol and the baselines as in AppleGastronome. Nevertheless, due to the faithfulness issue of the original dataset [26], for the evaluation of causal relation discovery, we mainly conduct a qualitative comparison between the ground truth that is faithful to the data, against the baselines and COAT.

4.2 Empirical Results on Neuropathic Benchmark

Factor proposal The quantitative results on Neuropathic benchmark are given in Table 2. Similarly, we can find that COAT consistently outperforms all of the baselines regardless of which LLMs are incorporated. In particular, even with the weakest backbone model, i.e.,LLaMA2-7b, COAT can still effectively leverage the intrinsic rich knowledge and beat the baselines with more powerful LLMs.

Causal relation recovery Fig. 6(a) shows the causal graph obtained by FCI running on the original data, where we can find that several causal relations cannot hold on the data. As shown in Fig. 6, when using LLMs to perform the reasoning, LLMs cannot identify the faithfulness issues. In contrast, COAT can imply faithful causal insights.

4.3 More Real-world Results

El Ni˜ no-Southern Oscillation (ENSO) case study El Ni˜ no-Southern Oscillation (ENSO) is a climatic phenomenon in the Pacific Ocean that influences global weather patterns profoundly. To understand its mechanism, we apply COAT on NOAA dataset [40]. There are 13 factors identified by COAT, and their instantaneous causal relations are visualized in Fig 7. The target variable is the future change in monthly SST in the Nino3 region, which could be an important indicator of ENSO events. Each factor is a time series about a certain climate measurement above a specific level averaged over a specific region. The paths about Sea level Pressure , Momentum Flux , and Cloud Cover matches the existing understanding from literature [41, 42, 43, 44]. It also suggests several inserting hypotheses that are less explored in literature, like the path from Soil Temperature in South American Coastal Region. We refer details in Appendix K.

More real-world empirical studies We also report concrete results on real-world problems involving MRI, time series, and NetCDF data in Appendix I, J, and K.

5 Conclusions

In this paper, we proposed a new paradigm COAT to incorporate the rich knowledge of LLMs into the CD pipeline. We found that COAT effectively extends the scope of CDs to unstructured data by identifying useful high-level variables from raw observations for CD methods. COAT suggests a new pathway towards building a causal foundation model for discovery. We leave more detailed discussions about future studies in Appendix B.

Acknowledgments and Disclosure of Funding

We thank the reviewers for their valuable comments. This material is based upon work supported by NSF Award No. 2229881, AI Institute for Societal Decision Making (AI-SDM), the National Institutes of Health (NIH) under Contract R01HL159805, and grants from Salesforce, Apple Inc., Quris AI, and Florin Court Capital. CXL and BH were supported by NSFC General Program No. 62376235, Guangdong Basic and Applied Basic Research Foundation Nos. 2022A1515011652 and 2024A1515012399, HKBU Faculty Niche Research Areas No. RC-FNRA-IG/22-23/SCI/04, and HKBU CSD Departmental Incentive Scheme. MMG was partially supported by the following Australian Research Council projects: DE210101624 and DP240102088. JC was supported by CUHK direct grant 4055146.

References

Norwood Russell Hanson. Patterns of discovery : an inquiry into the conceptual foundations of science . Cambridge University Press, 1958. (Cited on page 1)

Thomas S. Kuhn and David Hawkins. The structure of scientific revolutions. American Journal of Physics , 31:554-555, 1963. (Cited on page 1)

Peter Spirtes, Clark Glymour, and Richard Scheines. Causation, Prediction, and Search, Second Edition . Adaptive computation and machine learning. MIT Press, 2000. ISBN 978-0-262-19440-2. (Cited on pages 1, 2, 4, 21 and 22)

Peter Spirtes, Clark Glymour, Richard Scheines, and Robert Tillman. Automated Search for Causal Relations: Theory and Practice , 2018. (Cited on pages 1, 2, 5, 21 and 22)

Matthew J. Vowels, Necati Cihan Camgoz, and Richard Bowden. D'ya like dags? a survey on structure learning and causal discovery. ACM Computing Survey , 55(4), 2022. ISSN 0360-0300. (Cited on pages 1, 2, 5, 21 and 22)

Bernhard Sch¨ olkopf, Francesco Locatello, Stefan Bauer, Nan Rosemary Ke, Nal Kalchbrenner, Anirudh Goyal, and Yoshua Bengio. Towards causal representation learning. arXiv preprint , arXiv:2102.11107, 2021. (Cited on pages 1, 21 and 22)

Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. Language models are few-shot learners. In Advances in Neural Information Processing Systems , 2020. (Cited on pages 2, 21 and 22)

OpenAI. Chatgpt. https://chat.openai.com/chat/ , 2022. (Cited on pages 2, 21 and 22)

Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timoth´ ee Lacroix, Baptiste Rozi` ere, Naman Goyal, Eric Hambro, Faisal Azhar, Aur´ elien Rodriguez, Armand Joulin, Edouard Grave, and Guillaume Lample. Llama: Open and efficient foundation language models. arXiv preprint , arXiv:2302.13971, 2023. (Cited on pages 2 and 21)

OpenAI. Gpt-4 technical report, 2023. (Cited on pages 2, 21 and 22)

Emre Kiciman, Robert Ness, Amit Sharma, and Chenhao Tan. Causal reasoning and large language models: Opening a new frontier for causality. arXiv preprint , arXiv:2305.00050, 2023. (Cited on pages 2, 7, 8, 21 and 22)

S´ ebastien Bubeck, Varun Chandrasekaran, Ronen Eldan, Johannes Gehrke, Eric Horvitz, Ece Kamar, Peter Lee, Yin Tat Lee, Yuanzhi Li, Scott M. Lundberg, Harsha Nori, Hamid Palangi, Marco T´ ulio Ribeiro, and Yi Zhang. Sparks of artificial general intelligence: Early experiments with GPT-4. arXiv preprint , arXiv:2303.12712, 2023. (Cited on pages 2, 21 and 22)

Cheng Zhang, Stefan Bauer, Paul Bennett, Jiangfeng Gao, Wenbo Gong, Agrin Hilmkil, Joel Jennings, Chao Ma, Tom Minka, Nick Pawlowski, and James Vaughan. Understanding causality with large language models: Feasibility and opportunities. arXiv preprint , arXiv:2304.05524, 2023. (Cited on pages 2, 21 and 22)

Ahmed Abdulaal, adamos hadjivasiliou, Nina Montana-Brown, Tiantian He, Ayodeji Ijishakin, Ivana Drobnjak, Daniel C. Castro, and Daniel C. Alexander. Causal modelling agents: Causal graph discovery through synergising metadata- and data-driven reasoning. In The Twelfth International Conference on Learning Representations , 2024. (Cited on pages 2, 21, 22 and 23)

Matej Zeˇ cevi´ c, Moritz Willig, Devendra Singh Dhami, and Kristian Kersting. Causal parrots: Large language models may talk causality but are not causal. Transactions on Machine Learning Research , 2023. ISSN 2835-8856. (Cited on pages 2, 21 and 22)

Zhijing Jin, Yuen Chen, Felix Leeb, Luigi Gresele, Ojasv Kamal, Zhiheng LYU, Kevin Blin, Fernando Gonzalez Adauto, Max Kleiman-Weiner, Mrinmaya Sachan, and Bernhard Sch¨ olkopf. CLadder: A benchmark to assess causal reasoning capabilities of language models. In Thirty-seventh Conference on Neural Information Processing Systems , 2023. (Cited on pages 2, 21 and 22)

Zhijing Jin, Jiarui Liu, Zhiheng Lyu, Spencer Poff, Mrinmaya Sachan, Rada Mihalcea, Mona T. Diab, and Bernhard Sch¨ olkopf. Can large language models infer causation from correlation? arXiv preprint , arXiv:2306.05836, 2023. (Cited on pages 2, 21 and 22)

Yue Zhang, Yafu Li, Leyang Cui, Deng Cai, Lemao Liu, Tingchen Fu, Xinting Huang, Enbo Zhao, Yu Zhang, Yulong Chen, Longyue Wang, Anh Tuan Luu, Wei Bi, Freda Shi, and Shuming Shi. Siren's song in the AI ocean: A survey on hallucination in large language models. arXiv preprint , arXiv:2309.01219, 2023. (Cited on pages 2, 21 and 22)

Chenhang Cui, Yiyang Zhou, Xinyu Yang, Shirley Wu, Linjun Zhang, James Zou, and Huaxiu Yao. Holistic analysis of hallucination in gpt-4v(ision): Bias and interference challenges. arXiv preprint , arXiv:2311.03287, 2023. (Cited on pages 2, 21 and 22)

Lukas Berglund, Meg Tong, Max Kaufmann, Mikita Balesni, Asa Cooper Stickland, Tomasz Korbak, and Owain Evans. The reversal curse: Llms trained on 'a is b' fail to learn 'b is a'. arXiv preprint , arXiv:2309.12288, 2023. (Cited on pages 2, 21 and 22)

Constantin F. Aliferis, Alexander Statnikov, Ioannis Tsamardinos, Subramani Mani, and Xenofon D. Koutsoukos. Local causal and markov blanket induction for causal discovery and feature selection for classification part i: Algorithms and empirical evaluation. Journal of Machine Learning Research , 11(7):171-234, 2010. (Cited on page 3)

Shantanu Gupta, David Childers, and Zachary Chase Lipton. Local causal discovery for estimating causal effects. In Conference on Causal Learning and Reasoning , volume 213, pages 408-447, 2023. (Cited on page 3)

Judea Pearl and Dana Mackenzie. The Book of Why: The New Science of Cause and Effect . Basic Books, Inc., USA, 1st edition, 2018. ISBN 046509760X. (Cited on pages 4 and 22)

Timo Schick, Jane Dwivedi-Yu, Roberto Dess` ı, Roberta Raileanu, Maria Lomeli, Luke Zettlemoyer, Nicola Cancedda, and Thomas Scialom. Toolformer: Language models can teach themselves to use tools. arXiv preprint , arXiv:2302.04761, 2023. (Cited on page 4)

Zhiheng Xi, Wenxiang Chen, Xin Guo, Wei He, Yiwen Ding, Boyang Hong, Ming Zhang, Junzhe Wang, Senjie Jin, Enyu Zhou, Rui Zheng, Xiaoran Fan, Xiao Wang, Limao Xiong, Yuhao Zhou, Weiran Wang, Changhao Jiang, Yicheng Zou, Xiangyang Liu, Zhangyue Yin, Shihan Dou, Rongxiang Weng, Wensen Cheng, Qi Zhang, Wenjuan Qin, Yongyan Zheng, Xipeng Qiu, Xuanjing Huan, and Tao Gui. The rise and potential of large language model based agents: A survey. arXiv preprint , arXiv:2309.07864, 2023. (Cited on page 4)

Ruibo Tu, Kun Zhang, Bo C. Bertilson, Hedvig Kjellstr¨ om, and Cheng Zhang. Neuropathic pain diagnosis simulator for causal discovery algorithm evaluation. In Advances in Neural Information Processing Systems , pages 12773-12784, 2019. (Cited on pages 4, 9, 10 and 43)

Peter Spirtes, Christopher Meek, and Thomas Richardson. Causal inference in the presence of latent variables and selection bias. In Uncertainty in Artificial Intelligence , page 499-506, 1995. (Cited on pages 5, 7 and 22)

Clark Glymour, Kun Zhang, and Peter Spirtes. Review of causal discovery methods based on graphical models. Frontiers in Genetics , 10, 2019. (Cited on pages 5 and 22)

Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, brian ichter, Fei Xia, Ed H. Chi, Quoc V Le, and Denny Zhou. Chain of thought prompting elicits reasoning in large language models. In Alice H. Oh, Alekh Agarwal, Danielle Belgrave, and Kyunghyun Cho, editors, Advances in Neural Information Processing Systems , 2022. (Cited on pages 5 and 22)

Jiaxin Huang, Shixiang Gu, Le Hou, Yuexin Wu, Xuezhi Wang, Hongkun Yu, and Jiawei Han. Large language models can self-improve. In Conference on Empirical Methods in Natural Language Processing , pages 1051-1068, 2023. (Cited on page 5)

Shuofei Qiao, Yixin Ou, Ningyu Zhang, Xiang Chen, Yunzhi Yao, Shumin Deng, Chuanqi Tan, Fei Huang, and Huajun Chen. Reasoning with language model prompting: A survey. In Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , July 2023. (Cited on page 5)

Judea Pearl. Probabilistic reasoning in intelligent systems - networks of plausible inference. In Morgan Kaufmann series in representation and reasoning , 1991. (Cited on page 5)

Jonas Peters, Dominik Janzing, and Bernhard Schlkopf. Elements of Causal Inference: Foundations and Learning Algorithms . The MIT Press, 2017. ISBN 0262037319. (Cited on page 6)

OpenAI. Hello, gpt-4o! https://openai.com/index/hello-gpt-4o/ , 2024. Accessed: 2024-05-20. (Cited on page 7)

Anthropic. Claude 3 family. https://www.anthropic.com/news/claude-3-family , 2024. Accessed: 2024-05-20. (Cited on page 7)

Meta AI. Meta llama 3. https://ai.meta.com/blog/meta-llama-3/ , 2024. Accessed: 2024-05-20. (Cited on page 7)

Albert Q. Jiang, Alexandre Sablayrolles, Antoine Roux, Arthur Mensch, Blanche Savary, Chris Bamford, Devendra Singh Chaplot, Diego de Las Casas, Emma Bou Hanna, Florian Bressand, Gianna Lengyel, Guillaume Bour, Guillaume Lample, L´ elio Renard Lavaud, Lucile Saulnier, Marie-Anne Lachaux, Pierre Stock, Sandeep Subramanian, Sophia Yang, Szymon Antoniak, Teven Le Scao, Th´ eophile Gervet, Thibaut Lavril, Thomas Wang, Timoth´ ee Lacroix, and William El Sayed. Mixtral of experts. arXiv preprint , arXiv:2401.04088, 2024. (Cited on page 7)

Sartaj Bhuvaji, Ankita Kadam, Prajakta Bhumkar, and Sameer Dedge. Brain tumor classification using deep learning algorithms. https://github.com/SartajBhuvaji/ Brain-Tumor-Classification-Using-Deep-Learning-Algorithms , 2024. Accessed: 2024-05-19. (Cited on pages 9 and 43)

Bidec Innovations. Stock price and news related to it, 2023. URL https://www.kaggle. com/datasets/BidecInnovations/stock-price-and-news-realted-to-it . Accessed: 2023-10-01. (Cited on pages 9 and 43)

Gilbert P Compo, Jeffrey S Whitaker, Prashant D Sardeshmukh, Nobuki Matsui, Robert J Allan, Xungang Yin, Byron E Gleason, Russell S Vose, Glenn Rutledge, Pierre Bessemoulin, et al. The twentieth century reanalysis project. Quarterly Journal of the Royal Meteorological Society , 137(654):1-28, 2011. (Cited on pages 9, 10, 43 and 49)

Jakob Bjerknes. Atmospheric teleconnections from the equatorial pacific. Monthly weather review , 97(3):163-172, 1969. (Cited on pages 10 and 51)

Chunzai Wang. Enso, atlantic climate variability, and the walker and hadley circulations. In The Hadley circulation: Present, past and future , pages 173-202. Springer, 2004. (Cited on pages 10 and 51)

Yinge Liu, Ninglian Wang, Lingang Wang, Zhongming Guo, and Xiaobo Wu. Variation of cloud amount over china and the relationship with enso from 1951 to 2014. International Journal of Climatology , 36(8):2931-2941, 2016. (Cited on pages 10 and 51)

Anoop Kumar Mishra. Investigating changes in cloud cover using the long-term record of precipitation extremes. Meteorological Applications , 26(1):108-116, 2019. (Cited on pages 10 and 51)

Peter Lee, Sebastien Bubeck, and Joseph Petro. Benefits, limits, and risks of gpt-4 as an ai chatbot for medicine. New England Journal of Medicine , 388(13):1233-1239, 2023. (Cited on page 21)

Ruibo Tu, Chao Ma, and Cheng Zhang. Causal-discovery performance of chatgpt in the context of neuropathic pain diagnosis. arXiv preprint , arXiv:2301.13819, 2023. (Cited on pages 21 and 23)

Judea Pearl and James M. Robins. Causal diagrams for epidemiologic research. Epidemiology , 10 1:37-48, 1999. (Cited on page 22)

Shohei Shimizu, Patrik O Hoyer, Aapo Hyv¨ arinen, Antti Kerminen, and Michael Jordan. A linear non-gaussian acyclic model for causal discovery. Journal of Machine Learning Research , 7(10), 2006. (Cited on page 22)

Kun Zhang and Aapo Hyvarinen. On the identifiability of the post-nonlinear causal model. arXiv preprint arXiv:1205.2599 , 2012. (Cited on page 22)

Patrik Hoyer, Dominik Janzing, Joris M Mooij, Jonas Peters, and Bernhard Sch¨ olkopf. Nonlinear causal discovery with additive noise models. Advances in neural information processing systems , 2008. (Cited on page 22)

Biwei Huang, Kun Zhang, Jiji Zhang, Joseph Ramsey, Ruben Sanchez-Romero, Clark Glymour, and Bernhard Sch¨ olkopf. Causal discovery from heterogeneous/nonstationary data. Journal of Machine Learning Research , 21(89):1-53, 2020. (Cited on pages 22 and 50)

Karren Yang, Abigail Katcoff, and Caroline Uhler. Characterizing and learning equivalence classes of causal DAGs under interventions. In International Conference on Machine Learning , pages 5541-5550, 2018. (Cited on page 22)

Philippe Brouillard, S´ ebastien Lachapelle, Alexandre Lacoste, Simon Lacoste-Julien, and Alexandre Drouin. Differentiable causal discovery from interventional data. In Advances in Neural Information Processing Systems , pages 21865-21877, 2020. (Cited on page 22)

Joris M. Mooij, Sara Magliacane, and Tom Claassen. Joint causal inference from multiple contexts. Journal of Machine Learning Research , 21(99):1-108, 2020. (Cited on page 22)

Ronan Perry, Julius von K¨ ugelgen, and Bernhard Sch¨ olkopf. Causal discovery in heterogeneous environments under the sparse mechanism shift hypothesis. In Advances in Neural Information Processing Systems , pages 10904-10917, 2022. (Cited on page 22)

Daniel Malinsky and Peter Spirtes. Learning the structure of a nonstationary vector autoregression. In International Conference on Artificial Intelligence and Statistics , pages 2986-2994, 2019. (Cited on page 22)

Biwei Huang, Kun Zhang, Mingming Gong, and Clark Glymour. Causal discovery and forecasting in nonstationary environments with state-space models. In International Conference on Machine Learning , pages 2901-2910, 2019. (Cited on page 22)

Chenxi Liu and Kun Kuang. Causal structure learning for latent intervened non-stationary data. In International Conference on Machine Learning , pages 21756-21777, 2023. (Cited on page 22)

Zhengming Chen, Feng Xie, Jie Qiao, Zhifeng Hao, Kun Zhang, and Ruichu Cai. Identification of linear latent variable model with arbitrary distribution. In AAAI Conference on Artificial Intelligence , pages 6350-6357, 2022. (Cited on page 22)

Anpeng Wu, Haoxuan Li, Kun Kuang, Zhang Keli, and Fei Wu. Learning causal relations from subsampled time series with two time-slices. In International Conference on Machine Learning , 2024. (Cited on page 22)

Biwei Huang, Charles Jia Han Low, Feng Xie, Clark Glymour, and Kun Zhang. Latent hierarchical causal structure discovery with rank constraints. Advances in neural information processing systems , pages 5549-5561, 2022. (Cited on page 22)

Xinshuai Dong, Biwei Huang, Ignavier Ng, Xiangchen Song, Yujia Zheng, Songyao Jin, Roberto Legaspi, Peter Spirtes, and Kun Zhang. A versatile causal discovery framework to allow causally-related hidden variables. arXiv preprint , arXiv:2312.11001, 2023. (Cited on page 22)

Yibo Jiang and Bryon Aragam. Learning nonparametric latent causal graphs with unknown interventions. In Advances in Neural Information Processing Systems , volume 36, pages 60468-60513, 2023. (Cited on page 22)

Xiu-Chuan Li, Kun Zhang, and Tongliang Liu. Causal structure recovery with latent variables under milder distributional and graphical assumptions. In International Conference on Learning Representations , 2024. (Cited on page 22)

Aapo Hyvarinen, Hiroaki Sasaki, and Richard Turner. Nonlinear ica using auxiliary variables and generalized contrastive learning. In International Conference on Artificial Intelligence and Statistics , pages 859-868, 2019. (Cited on page 22)

Ilyes Khemakhem, Diederik Kingma, Ricardo Monti, and Aapo Hyvarinen. Variational autoencoders and nonlinear ica: A unifying framework. In International Conference on Artificial Intelligence and Statistics , pages 2207-2217, 2020. (Cited on page 22)

Francesco Locatello, Ben Poole, Gunnar Raetsch, Bernhard Sch¨ olkopf, Olivier Bachem, and Michael Tschannen. Weakly-supervised disentanglement without compromises. In International Conference on Machine Learning , pages 6348-6359, 2020. (Cited on page 22)

David Klindt, Lukas Schott, Yash Sharma, Ivan Ustyuzhaninov, Wieland Brendel, Matthias Bethge, and Dylan Paiton. Towards nonlinear disentanglement in natural data with temporal sparse coding, 2021. (Cited on page 22)

Sebastien Lachapelle, Pau Rodriguez, Yash Sharma, Katie E Everett, R´ emi LE PRIOL, Alexandre Lacoste, and Simon Lacoste-Julien. Disentanglement via mechanism sparsity regularization: A new principle for nonlinear ICA. In Conference on Causal Learning and Reasoning , pages 428-484, 2022. (Cited on page 22)

Kartik Ahuja, Jason S Hartford, and Yoshua Bengio. Weakly supervised representation learning with sparse perturbations. In Advances in Neural Information Processing Systems , pages 15516-15528, 2022. (Cited on page 22)

Kartik Ahuja, Divyat Mahajan, Yixin Wang, and Yoshua Bengio. Interventional causal representation learning. In International conference on machine learning , pages 372-407, 2023. (Cited on page 22)

Jiaqi Zhang, Kristjan Greenewald, Chandler Squires, Akash Srivastava, Karthikeyan Shanmugam, and Caroline Uhler. Identifiability guarantees for causal disentanglement from soft interventions. Advances in Neural Information Processing Systems , 2024. (Cited on page 22)

Julius von K¨ ugelgen, Michel Besserve, Liang Wendong, Luigi Gresele, Armin Keki´ c, Elias Bareinboim, David Blei, and Bernhard Sch¨ olkopf. Nonparametric identifiability of causal representations from unknown interventions. In Advances in Neural Information Processing Systems , pages 48603-48638, 2023. (Cited on page 22)

Chandler Squires, Anna Seigal, Salil S Bhate, and Caroline Uhler. Linear causal disentanglement via interventions. In International Conference on Machine Learning , pages 32540-32560, 2023. (Cited on page 22)

Simon Buchholz, Goutham Rajendran, Elan Rosenfeld, Bryon Aragam, Bernhard Sch¨ olkopf, and Pradeep Ravikumar. Learning linear causal representations from interventions under general nonlinear mixing. Advances in Neural Information Processing Systems , 2024. (Cited on page 22)

Yongqiang Chen, Yonggang Zhang, Yatao Bian, Han Yang, Kaili Ma, Binghui Xie, Tongliang Liu, Bo Han, and James Cheng. Learning causally invariant representations for out-ofdistribution generalization on graphs. In Advances in Neural Information Processing Systems , 2022. (Cited on page 22)

Yongqiang Chen, Yatao Bian, Kaiwen Zhou, Binghui Xie, Bo Han, and James Cheng. Does invariant graph learning via environment augmentation learn invariance? In Advances in Neural Information Processing Systems , 2023. (Cited on page 22)

Yongqiang Chen, Yatao Bian, Bo Han, and James Cheng. Interpretable and generalizable graph learning via subgraph multilinear extension. In ICLR 2024 Workshop on Machine Learning for Genomics Explorations , 2024. (Cited on page 22)

Yongqiang Chen, Kaiwen Zhou, Yatao Bian, Binghui Xie, Kaili Ma, Yonggang Zhang, Han Yang, Bo Han, and James Cheng. Pareto invariant risk minimization. arXiv preprint , arXiv:2206.07766, 2022. (Cited on page 22)

Yongqiang Chen, Wei Huang, Kaiwen Zhou, Yatao Bian, Bo Han, and James Cheng. Understanding and improving feature learning for out-of-distribution generalization. In Advances in Neural Information Processing Systems , 2023. (Cited on page 22)

Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Carroll L Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to follow instructions with human feedback. arXiv preprint arXiv:2203.02155 , 2022. (Cited on page 22)

Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. Training verifiers to solve math word problems. arXiv preprint , arXiv:2110.14168, 2021. (Cited on page 22)

Nouha Dziri, Ximing Lu, Melanie Sclar, Xiang Lorraine Li, Liwei Jiang, Bill Yuchen Lin, Sean Welleck, Peter West, Chandra Bhagavatula, Ronan Le Bras, Jena D. Hwang, Soumya Sanyal, Xiang Ren, Allyson Ettinger, Zaid Harchaoui, and Yejin Choi. Faith and fate: Limits of transformers on compositionality. In Thirty-seventh Conference on Neural Information Processing Systems , 2023. (Cited on page 22)

Bingbin Liu, Jordan T. Ash, Surbhi Goel, Akshay Krishnamurthy, and Cyril Zhang. Transformers learn shortcuts to automata. In The Eleventh International Conference on Learning Representations , 2023. (Cited on page 22)

Xuan Li, Zhanke Zhou, Jianing Zhu, Jiangchao Yao, Tongliang Liu, and Bo Han. Deepinception: Hypnotize large language model to be jailbreaker, 2024. URL https://arxiv.org/ abs/2311.03191 . (Cited on page 22)

Zhanke Zhou, Rong Tao, Jianing Zhu, Yiwen Luo, Zengmao Wang, and Bo Han. Can large language models reason robustly with noisy rationales? In ICLR 2024 Workshop on Reliable and Responsible Foundation Models , 2024. (Cited on page 22)

Brett Drury, Hugo Gonc ¸alo Oliveira, and Alneu de Andrade Lopes. A survey of the extraction and applications of causal relations. Natural Language Engineering , 28:361 - 400, 2022. (Cited on page 22)

Shaobo Cui, Zhijing Jin, Bernhard Sch¨ olkopf, and Boi Faltings. The odyssey of commonsense causality: From foundational benchmarks to cutting-edge reasoning. arXiv preprint arXiv:2406.19307 , 2024. (Cited on page 22)

Pedram Hosseini, David A Broniatowski, and Mona Diab. Predicting directionality in causal relations in text. arXiv preprint arXiv:2103.13606 , 2021. (Cited on page 22)

Lei Gao, Prafulla Kumar Choubey, and Ruihong Huang. Modeling document-level causal structures for event causal relation identification. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) , pages 1808-1817, 2019. (Cited on page 22)

Noah Weber, Rachel Rudinger, and Benjamin Van Durme. Causal inference of script knowledge. arXiv preprint arXiv:2004.01174 , 2020. (Cited on page 22)

Xiao Liu, Da Yin, Yansong Feng, Yuting Wu, and Dongyan Zhao. Everything has a cause: Leveraging causal inference in legal text analysis. arXiv preprint arXiv:2104.09420 , 2021. (Cited on page 22)

Andrew Kyle Lampinen, Stephanie C.Y. Chan, Ishita Dasgupta, Andrew Joo Hun Nam, and Jane X Wang. Passive learning of active causal strategies in agents and language models. In Advances in Neural Information Processing Systems , 2023. (Cited on page 22)

Kristy Choi, Chris Cundy, Sanjari Srivastava, and Stefano Ermon. Lmpriors: Pre-trained language models as task-specific priors. arXiv preprint , arXiv:2210.12530, 2022. (Cited on pages 22 and 23)

Stephanie Long, Alexandre Pich´ e, Valentina Zantedeschi, Tibor Schuster, and Alexandre Drouin. Causal discovery with language models as imperfect experts. arXiv preprint , arXiv:2307.02390, 2023. (Cited on pages 22 and 23)

Taiyu Ban, Lyuzhou Chen, Xiangyu Wang, and Huanhuan Chen. From query tools to causal architects: Harnessing large language models for advanced causal discovery from data. arXiv preprint , arXiv:2306.16902, 2023. (Cited on pages 22 and 23)

Moritz Willig, Matej Zeˇ cevi´ c, Devendra Singh Dhami, and Kristian Kersting. Can foundation models talk causality? In UAI 2022 Workshop on Causal Representation Learning , 2022. (Cited on page 22)

Stephanie Long, Tibor Schuster, and Alexandre Pich´ e. Can large language models build causal graphs? arXiv preprint , arXiv:2303.05279, 2023. (Cited on page 22)

Zhiheng LYU, Zhijing Jin, Rada Mihalcea, Mrinmaya Sachan, and Bernhard Sch¨ olkopf. Can large language models distinguish cause from effect? In UAI 2022 Workshop on Causal Representation Learning , 2022. (Cited on page 23)

Yanming Zhang, Brette Fitzgibbon, Dino Garofolo, Akshith Kota, Eric Papenhausen, and Klaus Mueller. An explainable AI approach to large language model assisted causal model auditing and development. arXiv preprint , arXiv:2312.16211, 2023. (Cited on page 23)

Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric Xing, et al. Judging llm-as-a-judge with mt-bench and chatbot arena. Advances in Neural Information Processing Systems , 36:4659546623, 2023. (Cited on page 40)

Daniel Lyndon, Joseph A Lansley, Jane Evanson, and Anant S Krishnan. Dural masses: meningiomas and their mimics. Insights into imaging , 10(1):11, 2019. (Cited on page 44)

Nisreen Haydar, Khatoun Alyousef, Usama Alanan, Rana Issa, Fawaz Baddour, Zuheir AlShehabi, and Moatasem Hussein Al-Janabi. Role of magnetic resonance imaging (mri) in grading gliomas comparable with pathology: A cross-sectional study from syria. Annals of Medicine and Surgery , 82:104679, 2022. (Cited on page 44)

Birkan Tunc ¸, David A Hormuth II, George Biros, and Thomas E Yankeelov. Modeling of glioma growth with mass effect by longitudinal magnetic resonance imaging. IEEE Transactions on Biomedical Engineering , 68(12):3713-3724, 2021. (Cited on page 44)

J Watts, G Box, A Galvin, P Brotchie, N Trost, and T Sutherland. Magnetic resonance imaging of meningiomas: a pictorial review. Insights into imaging , 5:113-122, 2014. (Cited on page 44)

John Markoff. Competing as software goes to web. The New York Times , 2007. URL https://www.nytimes.com/2007/06/05/technology/05compute.html . Accessed: 202403-14. (Cited on page 47)

Fama Eugene. Efficient capital markets: A review of theory and empirical work. Journal of finance , 25:383-417, 1970. (Cited on page 47)

Jessica A Wachter. Can time-varying risk of rare disasters explain aggregate stock market volatility? The Journal of Finance , 68(3):987-1035, 2013. (Cited on page 47)

Leland Bybee, Bryan T Kelly, Asaf Manela, and Dacheng Xiu. Business news and business cycles. Journal of Finance, Forthcoming , 2023. (Cited on page 47)

Michael J McPhaden. Genesis and evolution of the 1997-98 el ni˜ no. Science , 283(5404): 950-954, 1999. (Cited on pages 49 and 50)

Chester F Ropelewski and Michael S Halpert. Global and regional scale precipitation patterns associated with the el ni˜ no/southern oscillation. Monthly weather review , 115(8):1606-1626, 1987. (Cited on page 49)

Wenju Cai, Simon Borlace, Matthieu Lengaigne, Peter Van Rensch, Mat Collins, Gabriel Vecchi, Axel Timmermann, Agus Santoso, Michael J McPhaden, Lixin Wu, et al. Increasing frequency of extreme el ni˜ no events due to greenhouse warming. Nature climate change , 4(2): 111-116, 2014. (Cited on page 49)

Chunzai Wang, Clara Deser, Jin-Yi Yu, Pedro DiNezio, and Amy Clement. El ni˜ no and southern oscillation (enso): a review. Coral reefs of the eastern tropical Pacific: Persistence and loss in a dynamic environment , pages 85-106, 2017. (Cited on page 49)

Joris M Mooij, Sara Magliacane, and Tom Claassen. Joint causal inference from multiple contexts. Journal of Machine Learning Research , 21(99):1-108, 2020. (Cited on page 50)

Yi Ge Zhang, Mark Pagani, Jorijntje Henderiks, and Haojia Ren. A long history of equatorial deep-water upwelling in the pacific ocean. Earth and Planetary Science Letters , 467:1-9, 2017. ISSN 0012-821X. (Cited on page 50)

J. Tarazona and W. Arntz. The Peruvian Coastal Upwelling System , pages 229-244. Springer Berlin Heidelberg, 2001. (Cited on page 51)

Contents

Appendix of COAT

A Table of Notations

Raw Observations from the Real World

Learning

Treatment Effect

Estimation

……

Task Level

LLMs

Application Level

Marketing

Analysis

Disease

Understanding

Financial Asset

Pricing

……

Causal Foundation Model

Counterfactual

Reasoning

COAT: Causal

Representation Assistant

B Limitation and Future Opportunities

Future Opportunities Despite the progress in the past decades, existing causal discovery algorithms mainly rely on high-quality measured variables given by human experts [3, 4, 5]. However, the causal variables and their measurements are usually available in a wide range of real-world applications. For example, Amazon sellers who want to analyze the factors related to user ratings only have user reviews, generated by the underlying user preferences for certain product characteristics. Therefore, the lack of measured high-quality causal variables has been a major impediment to broader real-world applications of causal or causality-inspired methods [6].

This work establishes and demonstrates a preliminary implementation of the system C ausal representati O n A ssistan T (COAT). We present comprehensive evaluations of COAT and find plentiful evidence that the recent emergence of Large Language Models (LLMs) [7, 8, 9, 10] has a great potential to mitigate the gap. In particular, as shown in Fig. 8, we envision an entangled system towards causal foundation models, which consists of two mutually beneficial components: LLMs and causal discovery methods (CDs). On the one hand, LLMs that learn rich world knowledge about the world can assist with discovering high-level hidden variables from low-level observational data [12], or even certain commonsense causal knowledge [11, 13, 14]. On the other hand, it has also been found that the reliability of LLMs in the reasoning of causality remains a debate [13, 15, 16, 17], due to a series of drawbacks of LLMs [18, 19, 20], as well as the ethical considerations [45, 46]. Nevertheless, CDs can uncover the underlying causal relations between the discovered high-level variables with guarantees. More importantly, CDs can also provide feedback to improve the identification of the variables. The combination of LLMs and CDs iteratively improves the discovery of the hidden causal world, and hence opens up a broader adoption of various causal methods.

Limitations We would like to discuss some technical limitations of the current version of COAT.

Multicollinearity. Sometimes, LLM may output factors with shared similar semantic meanings or overlapped factor-value criteria. The consequence of this phenomenon is that multicollinearity could occur in the resulting structured data, or the data matrix (each sample per row) would not be full column rank. This would hinder the conditional independent tests. In experiments, we adopt numerical methods like PCA to drop similar factors. More sophisticated methods could be investigated in the future.

Toward complete causal graph. In this work, we mainly focus on one single target variable and identify a set of factors to serve as a Markov blanket of this target variable. Indeed, one can explore a more complete causal graph by applying COAT procedures on those identified factors to acquire a more comprehensive causal graph. One may also introduce multiple target variables or let LLM define a suitable target variable (and also parse it out) from pure raw observations. These extensions are also important, and we leave it to future work.

Meanwhile, identifiable causal discovery also requires certain assumptions about the data:

Faithfulness. The empirical distribution of the data reflects the actual data-generating process.

No selection bias. Otherwise, the condition of faithfulness would be violated.

Enough sample size. Our method involves statistical tests; the more, the better.

C Related Work

Causal discovery aims to discover the unknown causal relations from the observational data [3, 4], which is critical to both real-world applications and scientific discoveries [23, 47]. We use FCI [27] in this paper just to illustrate the idea. To attain identifiability better than the Markov equivalent class, one can choose more advanced Causal Discovery methods under different assumptions, like constrained functional [48, 49, 50], multiple domain data [51, 52, 53, 54, 55], and non-stationary data [51, 56, 57, 58]. There are also some works aiming to address realistic challenges, like structure models with arbitrary distribution [59], subsampled time series [60], and latent hierarchical causal structure [61, 62, 63, 64]. Despite recent theoretical and empirical improvements [5, 28], most existing causal discovery approaches rely on well-structured data with human-crafted factors as inputs. They can easily suffer from the low quality of annotated data (e.g., latent confounders).

Causal representation learning aims to establish provable identifiability for latent high-level variables (like location and color of an object) from low-level observations such as images [6]. Such identification can be feasible with certain conditions, like auxiliary information [65, 66, 67], sparsity [68, 69, 70], or interventional data [71, 72, 73, 74, 75]. Recent works also generalize the causal representation learning to graph-structured data [76, 77, 78], and discuss the optimization challenges in realizing causal representation learning [79, 80]. In this work, we show that incorporating LLMs that learn world knowledge from massive training data could effectively relieve the need for artificial causal factor annotation. Meanwhile, COAT also opens up a new line to learn causal representations with rich pre-trained knowledge as well as feedback from causal discovery algorithms.

Reasoning with LLMs has achieved remarkable performance across a variety of tasks with few demonstrations of the samples [7, 8, 10, 81]. The strong capabilities of LLMs show that it is evident that LLMs could acquire and understand commonsense knowledge about the world [12]. The power of LLMs can be further unlocked with suitable context as inputs [29]. Nevertheless, LLMs have also been found to make mistakes in basic algorithmic reasoning [82, 83], easy to hallucinate nonfactual results [18], tend to learn shortcuts or dataset biases [19, 20, 84], vulnerable to adversarial jailbreaks or noisy interruptions [85, 86], and fall short in complex planning and reasoning tasks [12]. The drawbacks of LLMs render it risky to rely on the direct LLM reasoning results to derive any rigorous results. Therefore, we do not directly derive the results from LLMs. Rather, we merely leverage the learned world knowledge in LLMs to find useful causal factors by constructing proper instructions based on causal feedback.

Text mining of causal relations aims to analyze the causal relations implied by the semantics given a text [87, 88, 89]. In particular, some works focus on the identification of causal relations among events specified in documents [90, 91, 92]. Different from these tasks, where factors are entities or events described by text, in this work, COAT emphasizes crafting high-level factors that go beyond the unstructured data like text description and also establish identifiability on the Markov blanket for a given target variable. Note that the target variable is allowed to be not mentioned in the text or other unstructured data. Therefore, this work brings new scope and opportunities for these text-understanding tasks.

Causal learning with LLMs has received muchattention results from the community [11, 13]. Kiciman et al. [11] find that LLMs can recover the pairwise causal relations very well. Lampinen et al. [93] show that transformer-based agents can learn causal strategies passively if intervention is allowed during tests. Abdulaal et al. [14], Choi et al. [94], Long et al. [95], Ban et al. [96] propose to incorporate the causal discovery results by LLMs as a prior or constraint to improve the performance of data-driven causal discovery algorithms. However, Zeˇ cevi´ c et al. [15], Willig et al. [97] find that LLMs can not understand causality but simply retell the causal knowledge contained in the training data. Zhang et al. [13], Jin et al. [16, 17] find that LLMs can hardly provide satisfactory answers for discovering new knowledge or decision-making tasks. Although Long et al. [98] find that LLMs can

build causal graphs with 3-4 nodes, Tu et al. [46] find that the performance of LLMs in more complex causal discovery remains limited as LLMs can hardly understand new concepts and knowledge. The aforementioned debate implies the limitations in directly adopting the causal discovery results by LLMs, which motivates us to incorporate the existing causal discovery algorithms with rigorous guarantees instead of LLMs to learn the causal relations.

LYU et al. [99] find that it is crucial to incorporate prompts aligned with the underlying causal story for LLMs to do pairwise causal relation inference. Zhang et al. [100] propose to leverage LLMs to audit causal discovery results in an explainable way.

The closest works to ours are Abdulaal et al. [14], Choi et al. [94], Long et al. [95], Ban et al. [96] which also incorporates LLMs into the pipeline of causal discovery. Nevertheless, all of the existing combinations of LLMs and causal discovery still focus on artificially curated structured data and rely on the capability of LLMs to infer causal relations, therefore, limited in both the reliability and the utility of LLMs in causal learning.

D Proofs for Theoretical Results

D.1 Proof for Proposition 2.2

Proposition D.1 (Restatement of Proposition 2.2) . Given the current representation as h [ k ] ( X ) = ( w 1 ( X ) , w 2 ( X ) , · · · , w k ( X )) , and a new factor w k +1 satisfying

for Markov Blanket S ⊆ [ k +1] of Y , i.e.,

we have the following result about conditional mutual information:

Proof for Prop. 2.2. From Eq. 12, we have

From Eq. 13, we have

Therefore:

Also note that

therefore:

which is Eq. 14.

D.2 Proof for Proposition 2.4

Proposition D.2 (Restatement of Proposition 2.4) . Given Assumption 2.3, for any small number ϵ, δ ∈ (0 , 1 2 ) , with sufficiently t rounds of COAT, i.e.,

where z δ is the δ -quantile of the standard normal distribution, we have

Proof for Prop. 2.4. Let n s be the number of rounds that LLM proposed at least one usable factor satisfying Assumption 2.3. Since t is large, its Binomial distribution Binom ( t, p ) can be approximated by Gaussian distribution N ( tp, tp (1 -p )) . To enforce

we have

with Gaussian distribution approximation.

√

Isolating t > 0 from the above inequality, we would have the desired result.

E More Details about Experiments

E.1 More Details on Constructing AppleGastronome

In the AppleGastronome benchmark, we consider the target variable as a rating score of the apple by several gastronomes. Each apple has its own attributes, including size, smell, and taste (or sweetness). Each gastronome has a unique preference for some attributes of the apple. They will give and rating as well as write a review according to the matchness of the apple with respect to their preference. We generate the review using GPT 4 by fetching GPT 4 the preferences and the apple attributes.

The prompts for generating the unstructured inputs are given in Fig. 9. The additional results on Relation Extraction are given in Table. 3.

Examples of AppleGastronome are given in Fig. 10.

E.2 More Details on Prompts for AppleGastronome

The prompts for factor proposal are given in Fig. 11.

The prompt for factor annotation is given in Fig. 12.

You are a picky gastronome on apples. You

are ready to

evaluate apples and write

reviews

. Your writing should be

clear,

solid and convincing to suppliers and

customers

.

Task

Please write a short review about the

evaluation results for a given apple.

Evaluation Results:

- a high content of essential nutrients,

including dietary fiber, vitamin C, etc.

- musty or rotten

- more sour than sweet

- dry and lacking Moisture

- might not bring the expected returns and

could even lead to losses

Additional Requirement:

- Combine those evaluation results into

more detailed reasoning.

- Single paragraph; No quotation marks.

- Modern English

- Begin this part by a mark [Review Begin].

- no more than 60 words.

## Score: -3

- The assessed apple falls short in offering the essential nutrients that consumers demand,

presenting a concerning deficiency. Its diminutive stature coupled with a distinct tartness that

overshadows any sweetness detracts from its palate appeal. Moreover, the dry texture fails to

satisfy those seeking a juicy bite. Given these drawbacks, this apple variety may disappoint

buyers, posing a financial risk for suppliers.

- Despite its diminutive stature, this apple fails to meet even basic quality expectations. The

presence of a musty or rotten scent signals underlying issues that can neither be overlooked

by discerning consumers nor excused by vendors seeking profit. Ultimately, stocking this fruit

is a risky venture that's far more likely to incur losses than yield any anticipated returns.

… …

## Score: 3

- This apple variety, rich in essential nutrients like dietary fiber and vitamin C, impresses

with its large size and delightful strong aroma. Its balanced flavor, leaning more towards sweet

than sour, and abundant refreshing moisture guarantee a delectable eating experience. This

apple's significant market potential is evident, and it certainly merits broader acclaim for its

high-quality attributes.

The prompts for constructing feedback are given in Fig. 13.

E.3 More Details of Results on AppleGastronome

The detailed causal graph results are given from Fig. 14 to Fig. 18.

Independent tests about annotation on the Apple Gastronome benchmark are shown in Table 4.

You are an excellently helpful AI

assistant for analyzing, abstracting,

and processing data. Now try your

best.

Rating=0

…

Rating=1

…

Rating=2

…

Tasks: Factor Abstraction.

What are the high-level factors behind

the text that contribute to the

allocation of groups?

•

Propose as many as possible factors

based on your knowledge.

•

Adjust your candidate factors by

your observation on given data.

•

The proposed factors should be

diverse and different semantically.

Their overlapping should be

minimized.

You are an excellently helpful AI assistant

for analyzing, abstracting, and processing

data.

Now try your best.

# Data

The reviewer comments on an apple that are

randomly picked:

{text}

# Tasks

For the given sample, what is the most

appreciate factor value based on the

criterion?

{factor_state}

**Your final output should follow this

template**:

**"The value is: ___."**

# Data

… <groups of samples> …

# Tasks: Factor Abstraction.

**What are the high-level factors behind the text that contribute to the

allocation of groups?**

- Propose your candidate factors based on your observation on given data.

- The proposed factors should be diverse and different semantically. Their

overlapping should be minimized.

# About Output

Your output should contain parts described as follows.

**Part 1**: Consideration.

In this part, feel free to write down the process of your considerations.

Hint: You need to abstract, identify, and design suitable factors with

corresponding criteria, and each factor is only allowed to take value from

[-1, 0, 1] .

**Part 2**: Factor filtration. You should decide whether to use each of the

proposed factors by following criteria:

- Each new factor should be helpful to distinguish the groups.

- Each Factor should focus on one concrete aspect and try to avoid

overlapping.

- Avoid overlapping with

those existing factors:

- size

- aroma

E.4 The Full Result on the Apple Gastronome Benchmark

Full Results on the Apple Gastronome Benchmark are shown in Table 5.

Full Results of Causal Metrics each Round of each LLM on Apple Gastronome Benchmark are shown in Table 6.

E.5 Implementation of the FCI algorithm

We use a third-party open-sourced Python library to perform the FCI algorithm: https://causallearn.readthedocs.io/en/latest/

We set α = 0 . 05 , and independence test method="fisherz" hroughout all experiments. Other parameters are kept as the default.

score

market potential

size

aroma

taste

juiciness

nutrition

(a) Ground truth

score

market potential

nutrition

size

smell

taste

juiciness

(b) FCI results

Score

Market Potential

Size

Taste

Juicy

Nutrition

(a) GPT 4 reasoning

score

market potential

flavor balance

moisture and juiciness

nutrient content

size

(b) GPT 4 COAT

Score

Size

Taste

Market Potential

Nutrition

(a) GPT 3.5 reasoning

score

market potential

nutritional value

size

taste profile

(b) GPT 3.5 COAT

Score

Size

Taste

Juicy

(a) LLaMA2 reasoning

score

texture

taste profile

flavor profile

size

(b) LLaMA2 COAT

Score

Market Potential

Size

Aroma

Taste

Nutrition

(a) Mistral reasoning

score

market potential

sourness

flavor profile

freshness

size

nutrient content

(b) Mistral COAT

Score

Market Potential

Size

Aroma

Taste

Nutrition

(a) Claude-3-Opus reasoning

score

market potential

nutrient content

size

sweetness level

flavor balance

aroma

(b) Claude-3-Opus COAT

Score

Size

Market Potential

Taste

Nutrition

Juicy

(a) DeepSeek-V2 reasoning

score

market appeal

size

nutrient content

moisture content

flavor profile

(b) DeepSeek-V2 COAT

Score

Aroma

Market Potential

Taste

Nutrition

(a) Llama-3-70b reasoning

score

market potential

freshness and quality

sensory appeal

nutrient richness

(b) Llama-3-70b COAT

Score

Market Potential

Size

Aroma

Taste

Nutrition

(a) mistral-Large reasoning

score

market potential

nutrient profile

flavor balance

flavor profile

size

freshness

(b) mistral-Large COAT

Score

Market Potential

Size

Aroma

Taste

Nutrition

(a) qwen-1.5-110B reasoning

score

market potential

taste balance

sweetness dominance

perceived freshness

visual & physical appeal

nutritional content

(b) qwen-1.5-110B COAT

Score

Market Potential

Size

Taste

(a) GPT-4o reasoning

score

economic impact

flavor balance

taste

size

(b) GPT-4o COAT

E.6 More Details on Constructing Neuropathic

In the Neuropathic benchmark, we convert the dataset into a clinical diagnosis task. In the original dataset, there are three levels of causal variables, including the symptom-level, radiculopathy-level and the pathophysiology-level. In experiments, we mainly consider the target variable of right shoulder impingement. When generating the clinical diagnosis notes as x using GPT 4, we will avoid any mentioning of variables other than symptoms.

As we intend to leverage the Neuropathic benchmark to simulate the real-life diagnosis, after the factor proposal stage, we directly incorporate external experts that measure the values of the candidate factors. The prompts to generate the diagnosis records are given in Fig. 25.

Examples of Neuropathic are given in Fig. 26.

E.7 More Details of Results on Neuropathic

The detailed causal graph results are given from Fig. 27 to Fig. 31.

E.8 Discussion on the time complexity

Assume there are m samples with n possible factors.

Symptom Diagnosis Summary: The patient presents with a left shoulder impingement and associated intracapsular problems. The symptoms include limited and painful range of motion with a positive outcome when performing a "thumbs up" test on the left side. Right upper limb assessment also reveals issues; the patient wears an armband suggesting ongoing management for right lower arm disorders. The patient reports lumbago and pain localized to the left thigh. Physical therapy assessment (L pta) is noted, indicating involvement in a therapeutic intervention. Joint assessments reveal a left hip joint condition and right hip arthritis, accompanied by right hamstring discomfort.

R shoulder impingement

R C4 Radiculopathy

R C5 Radiculopathy

R C6 Radiculopathy

DLI C4-C5

DLI C5-C6

(a) Ground truth

R shoulder impingement

R C4 Radiculopathy

R C5 Radiculopathy

DLI C4-C5

R C6 Radiculopathy

DLI C5-C6

(b) FCI results

META only needs to interact once with LLM, so it is O(1). DATA runs one single COAT round, so it is O(1). COAT interacts with LLMs multiple rounds. In each round, at least one new factor should be proposed; otherwise, the loop will stop. So it is O(n) For Factor Annotation:

META: Not applicable. DATA: At most n factors would be proposed in a single round. And each of them needs m times annotations by LLMs for all samples. So it is O(nm). COAT: At most n factors would be proposed during all rounds. And each of them needs m times annotations by LLMs for all samples. So it is O(nm). For Causal Discovery:

Pair-wise reasoning by LLMs: O( n 2 ) COAT: LLM is not involved. The computational cost of it depends on the numeric methods. The FCI algorithm used by COAT has a time complexity that goes exponentially with n. Learning the causal graph over a large number of nodes effectively is still an open problem in causal discovery literature.

E.9 Resources

We utilized a system comprising two Intel Xeon E5-2630v4 processors with 2.2GHz, two NVIDIA Tesla P40 GPUs, and 256 GB of memory. For conversations with large language models (LLMs), we leveraged the poe.com platform, while annotations were facilitated using the OpenAI API and the Mistral API.

R shoulder impingement

R C5 Radiculopathy

R C6 Radiculopathy

(a) GPT 4 reasoning

R shoulder impingement

DLI C5-C6

R C6 Radiculopathy

L C6 Radiculopathy

R C4 Radiculopathy

R C7 Radiculopathy

L C5 Radiculopathy

DLI C4-C5

R C5 Radiculopathy

(b) GPT 4 COAT

R shoulder impingement

DLI C4-C5

L S1 Radiculopathy

L L4 Radiculopathy

L C6 Radiculopathy

R T7 Radiculopathy

R C6 Radiculopathy

R C5 Radiculopathy

L C4 Radiculopathy

R S1 Radiculopathy

R C4 Radiculopathy

DLI C5-C6

L C5 Radiculopathy

R C2 Radiculopathy

L C7 Radiculopathy

R C7 Radiculopathy

L T10 Radiculopathy

R L5 Radiculopathy

L L5 Radiculopathy

L T7 Radiculopathy

R L4 Radiculopathy

R T10 Radiculopathy

(a) GPT 3.5 reasoning

R shoulder impingement

DLI C5-C6

R C6 Radiculopathy

L C6 Radiculopathy

R C4 Radiculopathy

L C7 Radiculopathy

R C7 Radiculopathy

L C5 Radiculopathy

DLI C4-C5

R C5 Radiculopathy

(b) GPT 3.5 COAT

R shoulder impingement

DLI C5-C6

R C6 Radiculopathy

L C6 Radiculopathy

R C4 Radiculopathy

L C7 Radiculopathy

R C7 Radiculopathy

L C5 Radiculopathy

DLI C4-C5

R C5 Radiculopathy

(a) LLaMA-2-70b reasoning

R shoulder impingement

DLI C5-C6

R C6 Radiculopathy

L C6 Radiculopathy

R C4 Radiculopathy

L C7 Radiculopathy

R C7 Radiculopathy

L C5 Radiculopathy

DLI C4-C5

R C5 Radiculopathy

(b) LLaMA-2-70b COAT

R shoulder impingement

R C6 Radiculopathy

R C4 Radiculopathy

L C5 Radiculopathy

R shoulder impingement

R C6 Radiculopathy

L C6 Radiculopathy

DLI C5-C6

R C5 Radiculopathy

L C5 Radiculopathy

DLI C4-C5

L C7 Radiculopathy

R C7 Radiculopathy

R C4 Radiculopathy

(b) Mistral-Medium COAT

F COAT with Different Causal Discovery Algorithm

G Ablation Study

Group Size in Prompt. In the COAT prompt, several samples are given and grouped by the values of the target variable. The samples in each group are randomly selected to a fixed number (like 3 samples per group). Empirically, we keep it to be 3 throughout all experiments (sometimes smaller than 3 if samples are not enough). In practice, it is mainly constrained by the LLM's context length.

The Number of Clusters. When constructing feedback, we first use clustering to separate the dataset and then find the cluster where the target variable is not explained well by current factors (This is a heuristic for the problem in line 191). Empirically, we set the number of clusters to be one plus the number of current factors.

We conduct ablation studies of COAT with GPT-4 using different hyperparameters:

As shown in Table 8, one can observe that COAT is not sensitive to these hyperparameters and performs robustly well than the baselines under different hyperparameter setups.

Prompt Template. We conduct an ablation study with a different prompt template following [101]:

We put the task description (also the format instructions) in the beginning [System] part, and we put samples in the last [Data] part of the prompt.

The markdown grammar is replaced by blankets to represent headings, like [System] , [Data] , and [Groups with Y=1] ...

3 COAT iterations are performed, which is aligned with the original experimental setup.

In Table 9, we observe that COAT is robust to the choice of templates, rejects unexpected factors (zero NMB and OT), and keeps a high precision.

R shoulder impingement

R C4 Radiculopathy

R C5 Radiculopathy

DLI C4-C5

R C6 Radiculopathy

DLI C5-C6

(a) Faithful ground truth

R shoulder impingement

L C5 Radiculopathy

0.28

DLI C4-C5

0.13

DLI C5-C6

0.08

R C6 Radiculopathy

0.24

R C4 Radiculopathy

0.06

0.87

R C5 Radiculopathy

0.92

L C6 Radiculopathy

0.93

0.89

-0.14

0.84

R C7 Radiculopathy

L C7 Radiculopathy

0.87

(b) GPT-4 COAT with LiNGAM

R shoulder impingement

DLI C4-C5

0.13

L C5 Radiculopathy

0.28

R C4 Radiculopathy

0.12

R C6 Radiculopathy

0.24

DLI C5-C6

0.08

R C5 Radiculopathy

0.92

0.87

0.89

L C6 Radiculopathy

-0.14

0.93

R C7 Radiculopathy

L C7 Radiculopathy

0.87

(c) GPT-3.5 COAT with LiNGAM

R shoulder impingement

L C5 Radiculopathy

0.28

R C6 Radiculopathy

0.24

R C4 Radiculopathy

0.06

DLI C5-C6

0.08

DLI C4-C5

0.13

0.87

0.89

L C6 Radiculopathy

-0.14

0.93

R C5 Radiculopathy

0.92

0.84

R C7 Radiculopathy

L C7 Radiculopathy

0.87

(d) llama-2-70b COAT with LiNGAM

R shoulder impingement

L C5 Radiculopathy

0.28

DLI C4-C5

0.13

DLI C5-C6

0.08

R C6 Radiculopathy

0.24

R C4 Radiculopathy

0.06

0.87

R C5 Radiculopathy

0.92

L C6 Radiculopathy

0.93

0.89

-0.14

0.84

(e) Mistral-med COAT with LiNGAM

42

H Summary of Benchmark Data

I Case Study on Brain Tumor

In this section, we utilize the COAT to explore the image dataset.

I.1 The Brain Tumor Dataset

Dataset Magnetic Resonance Imaging (MRI) is an important technique for detecting tumors in the human brain. The images are from an open Kaggle dataset (kaggle/brain-tumor-classification-mri) with an open-sourced project. [38]. Each sample is a scanning MRI of a human brain. In this case study, the interesting variable is the tumor type. We consider three types of MRI images: glioma, meningioma, and no tumor. We include 20 images for each category and the total sample size is 60.

Data Processing We use gpt-4-vision-preview to handle image samples. As the current gpt4 cannot process multiple images simultaneously, we concatenate samples from different categories into one picture, as shown in Fig. 33, and give instructions in the prompt to explain the format, as shown in Fig. 34. For each proposed factor, the LLMs would go through all 60 sample images individually to evaluate the factor value.

Evaluation The ground truth is not directly available to this dataset. Therefore, each factor in the final result will be evaluated by searching the related literature and will be explained by human interpretation.

I.2 Result and Discussion

Result interpretation As shown in Fig. 35, there are two visual factors that appeared in the final causal graph: contrast enhancement and mass effect . The first factor (contrast enhancement) is about whether the bright part is uniform or heterogeneous in the sample image. Another factor (mass effect) is about whether the tumor tissue has caused significant displacement of normal brain structures. One may refer to Fig. 36 to see detailed descriptions of these two factors.

Justification for the final factors We verify the proposed factors by the medical literature [102, 103, 104, 105], as shown Fig. 37. Note that these papers are searched according to keywords in factor descriptions and, therefore, can exclude some relevant studies. Therefore, the conclusions of this preliminary exploration should be considered a reference point for further in-depth validation by domain experts.

The factor description for contrast enhancement directly matches with the related paper (the first two papers displayed in Fig. 37). In addition, this can be visually checked in Fig. 33. Therefore, we believe it is a good factor.

The second factor, mass effect , may not directly match the papers. It is pointed out in the last two papers displayed in Fig. 37 that the two tumors have different axial locations and thus influence the magnitude of the displacement of tissues. Both the keywords 'axial location' and 'displacement' have occurred in its factor description.

Tumor Type

Contrast Enhancement

Mass Effect

J Case Study on Stock News

In this section, we utilize COAT to explore time series data with text.

J.1 The Stock News Dataset

Dataset The dataset consists of the stock value of Microsoft (MSFT) from 2006 to 2009 and its news summary (only include news in the New York Times ). This is a subset of an open Kaggle dataset (kaggle/stock-price-and-news-realted-to-it). Each sample is one trading day with the company's close stock price and news. The target value is the future return rate, and we are curious about factors in the related news. We fed data during the first 200 trading days to COAT, and we used the following 400 trading days for evaluation.

Data processing The future return rate is calculated by close prices:

The target value is a binary variable according to the return rate

where 1 A is the indicator function. News fed to COAT are grouped according to the value of Y t . To keep the whole prompt within the context limit, only 3 samples would be randomly included in each group.

Factor processing Given one proposed factor, denote its annotated value at the t -th trading day as a t ∈ {-1 , 0 , 1 } . Each factor is rolling averaged over the past M days:

'They (meningiomas) almost always demonstrate uniform strong enhancement on post-contrast imaging … ' Lyndon, Daniel, et al. "Dural masses: meningiomas and their mimics." Insights into imaging 10.1 (2019): 11.

'High-grade (gliomas ) tumors were heterogeneous contrast enhancement with other medical imaging aspects of MRI such as bleeding, necrosis, and edema'

Haydar, Nisreen, et al. "Role of Magnetic Resonance Imaging (MRI) in grading gliomas comparable with pathology: A cross-sectional study from Syria." Annals of Medicine and Surgery 82 (2022): 104679.

'Glioblastoma multiforme (GBM) is the most common primary brain tumor and is associated with a poor prognosis. One of its defining characteristics is the significant deformation of surrounding tissue, the so-called 'mass effect'.

Tunç, Birkan, et al. "Modeling of glioma growth with mass effect by longitudinal magnetic resonance imaging." IEEE Transactions on Biomedical Engineering 68.12 (2021): 3713-3724.

'Meningiomas typically appear as lobular, extra axial masses with well-circumscribed margins. They typically have a broad-based dural attachment and, if sufficiently large, inward displacement of the cortical grey matter. '

Watts, J., et al. "Magnetic resonance imaging of meningiomas: a pictorial review." Insights into imaging 5 (2014): 113-122.

Each variable, including return rate, is normalized by the rolling standard deviation over the past M days:

An example of a processed factor can be seen in Fig. 38.

J.2 Result and Discussion

Result Causal Graph As shown in Fig. 39, four factors are identified in the final causal graph to form a Markov blanket of return rate: Product Focus , Legal and Regulatory Issues , Market Strategy , and Innovation and Technology Focus . One can refer to Fig. 40 for more detailed descriptions of these factors. One factor ( Innovation and technology focus ) is identified as a possible cause of the return rate; this matches the nature of the company's type and reflects people's expectation of the company to keep creating innovative computer software. It is also interesting to see that COAT captures the structure between factors and market strategy , where product focus and legal and regulatory issues are identified as potential causes. It also implies the existence of a latent confounder between market strategy and return rate that may not be significantly reflected in news text.

innovation and technologyfocus

innovationandtechnologyfocus

0.6

0.4

FactorValues

0.2

0.0

-0.2

2006-12

2007-01

2007-02

2007-03

2007-04

2007-05

2007-06

2007-07

2007-08

2007-09

2007-10

Date

Return Rate

Market Strategy

Innovation and Technology Focus

Legal and Regulatory Issues

Evaluation by Trading Strategies For each factor { F t } processed in Eq. 27, we establish a trading strategy on it to see the performance in the out-of-sample trading days. At each trading day t , we fit the following model

using samples { r i , F i } t -200 ≤ i<t . Note that r t is the future return rate defined in Eq. 24 and is not available on this day. After the estimation, we make decision based on ˆ r t := ˆ α + ˆ βF t . If ˆ r t > 0 , we go long with 1 unit capital, i.e., purchase that amount of stock, the gain would be gain would be r × 1 units capital. If ˆ r t < 0 , we go short with 1 unit capital, i.e., borrow and sell that amount of stocks immediately and buy them back next time, the gain would be -r × 1 units capital.

To align with the definition in Eq. 24, we make trading decisions every 4 trading day. We introduce an additional Buy and Hold baseline to always go long one unit capital. The trading evaluation is after the 200 -th day and thus has no overlapping with data fed to COAT.

The cumulative return plot is shown in Fig. 41, and the metrics commonly used in economic literature [107, 108, 109] are shown in Table 11. One important metric to see a trading strategy's effectiveness is the sharp ratio: a measure of risk-adjusted return, showing the excess return (over the risk-free rate, we set it to be 2% ) per unit of standard deviation. A higher Sharpe ratio indicates better performance per unit of risk. We see that the Innovation and Technology Focus factor yields the highest sharp ratio and outperforms other non-causal factors.

Out-of-Sample Strategy Performance Comparison

Date

2007-09

2007-11

2008-01

2008-03

2008-05

2008-07

2008-09

2008-11

2009-01

2009-03

2009-05

0.50

0.25

0.00

0.25

0.50

0.75

1.00

1.25

Buy and Hold

product focus

legal/regulatory issues

market strategy

innovation and technology focus

K Case Study on El Ni˜ no-Southern Oscillation (ENSO)

In this section, we use COAT to explore a scientific problem using open datasets in climate science. The case study shows its potential to handle more complex data formats when combined with tools.

The ENSO Background El Ni˜ no-Southern Oscillation (ENSO) is an important phenomenon in the Pacific Ocean and is characterized by irregular fluctuations in sea surface temperatures[110]. ENSO consists of the warm phase (known as El Ni˜ no ) and the cold phase (known as La Ni˜ na ). It profoundly influences global weather patterns, including precipitation, storm development, and temperature anomalies [111]. The prediction of ENSO events involves the complex interplay between oceanic and atmospheric systems and, therefore, is still an open problem [112, 113].

NOAA Dataset The NOAA 20th Century Reanalysis V3 dataset contains high-dimensional information about Earth's atmosphere with fine-grained time and space coverage from the 19th century to the early 21st century [40]. This dataset amalgamates many observational data and uses the ensemble filter data assimilation method to reconstruct the historical state of the global atmosphere. The dataset includes a fine-grained spatial coverage with 360 × 181 grids. We only use monthly data in this case study.

K.1 Setting and Data processing

The Target Variable Construction The focus of this study is the future change in monthly SST in the Nino3 region, which could be an important indicator of ENSO events. The target variable is manually crafted from the dataset using API and is visualized in Fig. 42, where the oscillation pattern can be seen.

monthly change in sea surface temperature (nino3)

1800

1840

1880

1920

1960

2000

1.5

1.0

0.5

0.0

0.5

1.0

1.5

2.0

Prepare Tools for LLMs The dataset is in the NetCDF format (network Common Data Format), so it is not convenient to be directly fed to LLMs. We prepare a function Observation(measurement

, level, region) to help define factors. For example, in Lst. 1, we define the target variable with the help of this function. The measurement specifies the climate variables like precipitation rate or temperature; the level specifies the vertical location, like surface or specific pressure level; the region is a rectangle about the area concerned. This function would output a single time series about the measurement on the specified level averaged within the specified region. Note that the target variable is the only factor defined by humans, all other factors are defined by code from LLMs.

Feedback Construction The dataset is in a special format; we can only draw a tabular subset of data from it in this setting. Therefore, it is difficult to make groups on samples like what we did in the previous cases. Instead, we construct feedback with some summary information. As shown in Fig. 43, two types of information are included in the feedback: intermediate causal graph and OLS regression result of factors in the current estimation of a Markov blanket on the target variable. These two types of information can be drawn naturally from the COAT's intermediate results and are not complex, so LLMs can always access them and easily understand them. Intuitively, these factors would help LLMs better understand existing factors and propose further possible factors based on the provided dataset and its knowledge.

K.2 Causal Graph and Discussion

Result Interpretation In this case study, we are treating non-stationary climate data, while assuming the causal structure is invariant. Under this assumption, the non-stationarity is actually helpful for causal structure learning [51, 114]. To this end, we utilize the CD-NOD algorithm [51] to fully utilize the changing causal modules for better identifiability. CD-NOD would first identify the non-stationary nodes, whose conditional distribution given the causal parents are changing with time, and then use them for better causal structure recovery. Four factors are identified to be non-stationary, as shown in Fig. 44.

We have adjusted the nodes' names, shapes, and colors for better visualization. There are 14 nodes in total, with 13 factors identified by COAT. Each factor is a time series about a certain climate measurement above a specific level averaged over a specific region. For simplicity, we only considered instantaneous causal relations among those time series.

There are three regions identified to be relevant to the ENSO phenomenon:

Equatorial Pacific Region (Orange Nodes). This region (5N-5S, 120W-280W) is one of the most active places about ENSO. It becomes significantly warm during the El Ni˜ no phase and becomes significantly cold during the La Ni˜ na phase [115].

Nino3 Region (Blue Nodes). This region (5N-5S, 150W-90W) is one of the most classical regions to monitor the El Ni˜ no events by scientists. It is also used by humans to construct the target variables [110].

monthly change in sea surface temperature (nino3)

cloud cover (nino3 region)

sea level pressure (equatorial pacific region)

air temperature (nino3 region)

zonal wind (equatorial pacific region)

pseudo node (non-stationary indicator)

Follow the instructions to complete the task.

see the picture

……

## Regression Results on a Markov Blanket

OLS Regression Results

=================================================================================================================

Dep. Variable:

next month shift for nino3 sst (sea surface temperature)

R-squared (uncentered):

0.695

Model:

OLS

Adj. R-squared (uncentered):

0.694

Method

F-statistic:

951.9

Date:

Fri, 12 Apr 2024

Prob (F-statistic):

0.00

Time:

20:22:43

Log-Likelihood: -784.83

No. Observations:

2513

AIC:

1582.

Df Residuals:

2507

BIC:

1617.

Df Model:

6

Covariance Type:

nonrobust

=================================================================================================================

coef

std err

t

[0.025

0.975]

-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

sea surface air temperature gradient (equatorial pacific)

-0.0003

0.001

-0.600

0.548

-0.001

0.001

ocean-atmosphere heat flux (central to eastern equatorial pacific)

0.0023

0.001

2.813

0.005

0.001

0.004

lag_ocean-atmosphere heat flux (central to eastern equatorial pacific)

0.0094

0.001

12.164

0.000

0.008

0.011

lag_albedo variability (eastern equatorial pacific)

-0.9733

0.034

-28.406

0.000

-1.040

-0.906

lag_next month shift for nino3 sst (sea surface temperature)

0.6513

0.014

45.317

0.000

0.623

0.679

……

# Task

What are the new factors that might influence 'monthly change in sea surface temperature (nino3)'?

1. Analyze the results (Causal Graph result & Regression Results) step by step.

2. Guess potential new factors based  on what you have known.

South American Coastal Region (Green Nodes). This region (0N-20S, 80W-60W) includes the Peruvian coastal up-welling system, and is noticed to be relevant to the ENSO cycle [116].

Some insights are delivered by paths in the output causal graph:

Sea level Pressure. This factor is about the sea-level pressure on the equatorial Pacific region. The pressure gradient would influence the movement of warm water, and thus influence the sea surface temperature (SST) change [41]. In addition, pressure can influence the water evaporation and thus regulate through the water circulation. This also matches another indirect path Sea level Pressure → Sensible Heat Net Flux → Volumetric Soil Moisture → Cloud Cover → SST Change.

Momentum Flux, V-compoent. This factor is about the vertical movement of air. It is crucial in driving atmospheric convection [41], and it is related to the Walker Circulation, which is an important component in the ENSO system[42]. Also, it could influence the change in sea level pressure and indirectly influence the SST change.

Cloud Cover. The factor is the fraction of the sky covered by clouds in the NINO3 region. It could influence the SST Change through solar radiation as well as water circulation. It is confirmed to have a significant correlation [43] with ENSO events and plays an important role in the atmospheric circulation and hydrological cycle [44].

Soil Temperature. This might be a novel hypothesis proposed by COAT, since we found no sufficient research to confirm this point to the best of our knowledge. This causal graph has also suggested two possible indirect mechanisms: (1) through Volumetric Soil Moisture

and Cloud Cover ; and (2) through Convective Precipitation Rate and Sea Level Pressure . Therefore, this finding encourages more serious investigations of these hypotheses.

Momentum

Flux, V-

Component

Air

Temperature

Volumetric

Soil

Moisture

Surface

Roughness

Convective

Precipitation

Rate

Net Flux

Soil

Temperature

Sensible

Heat Net

Flux

SST Change

Downward

Longwave

Radiation

Flux

Zonal Wind

Momentum

Flux, U-

Component

Sea Level

Pressure

Equatorial Pacific Region

Nino3 Region

South American Coastal Region

Target Variable

Proposed Factor (stationary)

Proposed Factor (non-stationary)

Discussion on Additional assumption The constructed feedback contains the causal structure among existing factors and their OLS regression analysis on the target variable. By doing so, some additional assumptions are implicitly made. For example, it assumes a linear relation between factors and the target variables. More detailed theoretical analysis is out of the scope of this case study, and we left it for future work.

Discussion on Hallucination Although we have clarified how to use tools in prompts, LLM can sometimes propose unsupported data requests. For example, only certain pressure levels are supported in the dataset. To this end, we include a Python code in the prompt, as listed in Lst. 3. This function can check whether the LLM's request is supported. We ask LLM to run this function in its code interpreter to make sure the proposed factors are valid.

L Broader Impacts

This work focuses on fully leveraging the rich knowledge learned by LLMs during pre-training to facilitate causal discovery from unstructured data, with the hope of empowering broader applications and social benefits. Besides, this paper does not raise any ethical concerns. This study does not involve any human subjects, practices to data set releases, potentially harmful insights, methodologies and applications, potential conflicts of interest and sponsorship, discrimination/bias/fairness concerns, privacy and security issues, legal compliance, and research integrity issues.

NeurIPS Paper Checklist

1. Claims

Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?

Answer: [Yes]

Justification: See the end of the introduction part.

Guidelines:

The answer NA means that the abstract and introduction do not include the claims made in the paper.

The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.

The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.

It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.

2. Limitations

Question: Does the paper discuss the limitations of the work performed by the authors?

Answer: [Yes]

Justification: See Sec. 2.5 and Appendix B.

Guidelines:

The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.

The authors are encouraged to create a separate 'Limitations' section in their paper.

The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.

The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.

The authors should reflect on the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.

The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.

If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.

While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.

3. Theory Assumptions and Proofs

Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?

Answer: [Yes]

Guidelines:

The answer NA means that the paper does not include theoretical results.

All the theorems, formulas, and proofs in the paper should be numbered and crossreferenced.

All assumptions should be clearly stated or referenced in the statement of any theorems.

The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.

Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.

Theorems and Lemmas that the proof relies upon should be properly referenced.

4. Experimental Result Reproducibility

Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?

Answer: [Yes]

Guidelines:

The answer NA means that the paper does not include experiments.

If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.

If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.

Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.

While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example

(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.

(b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.

(c) If the contribution is a new model (e.g., a large language model), then there should either be a way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).

(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.

5. Open access to data and code

Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?

Answer: [Yes]

Justification: https://causalcoat.github.io/

Guidelines:

The answer NA means that paper does not include experiments requiring code.

Please see the NeurIPS code and data submission guidelines ( https://nips.cc/ public/guides/CodeSubmissionPolicy ) for more details.

While we encourage the release of code and data, we understand that this might not be possible, so 'No' is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).

The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines ( https: //nips.cc/public/guides/CodeSubmissionPolicy ) for more details.

The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.

The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.

At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).

Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.

6. Experimental Setting/Details

Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?

Answer: [Yes]

Justification: See Sec. 3.1, Sec. 4.1 and Appendix E.

Guidelines:

The answer NA means that the paper does not include experiments.

The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.

The full details can be provided either with the code, in appendix, or as supplemental material.

7. Experiment Statistical Significance

Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?

Answer: [Yes]

Justification: Standard Deviations are included.

Guidelines:

The answer NA means that the paper does not include experiments.

The authors should answer 'Yes' if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).

The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)

The assumptions made should be given (e.g., Normally distributed errors).

It should be clear whether the error bar is the standard deviation or the standard error of the mean.

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.

For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).

If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.

8. Experiments Compute Resources

Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?

Answer: [Yes]

Justification: See Appendix E.9.

Guidelines:

The answer NA means that the paper does not include experiments.

The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.

The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.

The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).

9. Code Of Ethics

Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines ?

Answer: [Yes]

Justification: Authors follow the NeurIPS Code of Ethics.

Guidelines:

The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.

If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.

The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).

10. Broader Impacts

Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?

Answer: [Yes]

Justification: See the Appendix L.

Guidelines:

The answer NA means that there is no societal impact of the work performed.

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.

Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to

generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.

The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.

If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).

11. Safeguards

Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?

Answer: [NA]

Justification: No sensitive or risky content is included in this paper.

Guidelines:

The answer NA means that the paper poses no such risks.

Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.

Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.

We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.

12. Licenses for existing assets

Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?

Answer: [Yes]

Guidelines:

The answer NA means that the paper does not use existing assets.

The authors should cite the original paper that produced the code package or dataset.

The authors should state which version of the asset is used and, if possible, include a URL.

The name of the license (e.g., CC-BY 4.0) should be included for each asset.

For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.

If this information is not available online, the authors are encouraged to reach out to the asset's creators.

13. New Assets

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?

Answer: [Yes]

Guidelines:

The answer NA means that the paper does not release new assets.

Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.

The paper should discuss whether and how consent was obtained from people whose asset is used.

At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.

14. Crowdsourcing and Research with Human Subjects

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?

Answer: [NA]

Justification: not involve crowdsourcing nor research with human subjects.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.

According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.

15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?

Answer: [NA]

Justification: This paper does not involve crowdsourcing nor research with human subjects.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.

For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.

Peer reviews:
comments: SUMMARY:
This paper presents Causal representatiOn AssistanT (COAT), which introduces large language models (LLMs) to bridge this gap. LLMs are trained on massive observations of the world and have shown great capability in extracting key information from unstructured data. Thus, employing LLMs to propose useful high-level factors and craft their measurements is natural. COAT also uses CDs to find causal relations among the identified variables and provides feedback to LLMs to iteratively refine the proposed factors. This mutual benefit enhances both LLMs and CDs.

STRENGTHS:
1. Interesting topic. Employing LLMs to propose useful high-level representations for causal discovery.
2. Develop two benchmarks for the unstructured causal discovery. AppleGastronome and Neuropathic.
3. Derives the first metrics that measure the causal representation learning capabilities of various LLMs.

WEAKNESSES:
1. ‘We will release an anonymous link during the discussion period.’ I will consider raising my score if the code is reasonable.
2. The contribution of LLM in COAT is a little small. I assume LLM is used as a representation tool to learn the conceptual level attributes, including iterative refining. The causal structural learning can still be considered as the downstream task.
3. COAT will inherit the shortcomings of downstream causal structure learning algorithms.

QUESTIONS:
1. How can we ensure that LLM does not introduce erroneous prior knowledge for reasoning?

LIMITATIONS:
N/A
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2024-07-03

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper tackles the problem of discovering relevant features for recovering the underlying causal graph in the absence and/or in lieu of a human domain expert. The proposed method, COAT, first queries an LLM through a prompt elucidating the task (for eg., discovering relevant features that affect a product review using a few text reviews), then the proposed variables are fed into another LLM that assigns a value to each of these variables, thus outputting structured/tabular data that can be used for causal discovery. Finally, the tabular data is used in conjunction with a traditional causal discovery algorithm (FCI and LiNGAM in this case) to retrieve a causal graph with respect to a target variable (for eg., review score) using the proposed variables. The process repeats until the proposed variables form a markov blanket for the target variable w.r.t the raw unstructured input data (for eg., the text reviews), progressively expanding the markov blanket in each iteration. Additionally, the LLM can receive feedback at the end of each iteration in the form of samples that the proposed variables cannot sufficiently explain. In particular, the authors propose clustering the samples w.r.t the latent variables induced by the LLM and picking the samples in the cluster with the largest conditional entropy.\n\nInitial theoretical analysis of the proposed method implies that the proposed method is able to identify the markov blanket for a target variable using the proposed variables given that enough iterations of COAT are performed.\n\nThe authors evaluate COAT empirically over two synthetic datasets and three real-world datasets. They compare COAT against two simple baselines 1) factors being directly proposed by the LLM based on the prompt without further iterations 2) factors being proposed by LLM when queried using both the prompt and some samples of raw observations. The second baseline is essentially COAT without the LLM receiving any feedback after each iteration. The experiments are conducted using 10 different LLMs and primarily one causal discovery algorithm (FCI), with additional experiments on one dataset using LiNGAM. Additionally, the paper proposes two novel metrics for quantitatively assessing the performance of LLMs for feature proposals to be used for causal discovery.\n\n\nUpdate: I moved my rating up in the hope that teh authors will add the experiments as they promised to the final version. We have no way of enforcing it but hopefully, the authors will follow up on their promise.\n\nSTRENGTHS:\nThe paper addresses the important problem of causal discovery and employs an effective two pronged approach involving LLMs and traditional causal discovery algorithms. This approach leverages the strengths of both the LLMs and the causal discovery algorithms i.e, ability to respond to complex prompts and unstructured data with high-level and possibly noisy information, and robust causal discovery with strong theoretical guarantees although requiring strong assumptions on the faithfulness of the data and causal mechanisms, respectively. Overall, I believe this is a promising direction wherein the two components complement each other effectively.\n\nThe empirical evaluation is sufficient in terms of the large number of LLMs considered and the moderate amount of datasets evaluated. The results, based on the chosen metrics, sufficiently demonstrate the effectiveness of the proposed method over the simple baselines.\n\nFinally, the paper is well-written and clearly explains the steps involved in each iteration. The further explanations provided in the appendix also aid in this.\n\nWEAKNESSES:\nThe theoretical aspects of the proposed algorithm are exaggerated in the introduction. Given the strong assumptions of “sufficiently powerful” LLMs, “sufficiently diverse” examples and further assumptions pertaining to the chosen causal discovery method, the propositions, while appreciated, are rather straightforward. In particular, it would be far more interesting to theoretically analyse the impact of modules involving the LLMs themselves, such as the chosen prompt template, quality of factor annotations and responsiveness of LLMs to feedback regarding causal discovery, even though some of these are evaluated empirically. Also, an analysis on the rate of convergence of COAT would be beneficial.\n\nSecondly, while the modularity of the proposed approach facilitates utilising a cross product of LLMs, causal discovery methods and feedback mechanisms, it also necessitates extensive ablation studies. In particular, the paper would be strengthened by a thorough ablation of the initial prompts and feedback. In particular, a discussion and ablation on the chosen prompt template and its effect on the proposed factors, or lack thereof, is needed. A robust template would allow more seamless adoption of the proposed method. Finally, the chosen baselines are far too simple to make any strong claims on the effectiveness of COAT. Comparing against some of the methods covered in the related work section would help bolster this claim.\n\nQUESTIONS:\nThe paper addresses an important and timely problem and proposes a simple and intuitive solution, leveraging the strengths of LLMs and traditional causal discovery methods. While the experiments demonstrate the effectiveness of the proposed method over two simple baselines, stronger baselines and more ablations on prompts and factor annotations would strengthen this claim. Theoretical analysis is limited to the well-studied causal discovery aspect of the pipeline while making strong assumptions on the powerfulness of the LLMs, diversity and faithfulness of the raw observational data, and the number of iterations being sufficiently large, seems rather unsurprising.\n\nLIMITATIONS:\nSee the weakness and questions above.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2024-07-11"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis work proposes COAT (Causal representation AssistanT), a novel framework to leverage LLMs to assist with causal discovery from unstructured data. COAT aims to combine the advantages of LLMs and causal discovery algorithms. To do so, COAT employs LLMs to identify high-level variables and parse unstructured data into structured data. On the other hand, causal discovery algorithms read the parsed data to identify causal relations. To improve the reliability of the results, COAT also constructs feedback from the causal discovery results to iteratively improve the high-level variable identification. The authors conduct extensive case studies ranging from synthetic data to realistic data, and find COAT effectively helps with discovering meaningful causal structures that well explain the target variable.\n\nSTRENGTHS:\n1. This work identifies a crucial and timely problem for how to advance the causal tasks including causal learning and reasoning with foundation models likes LLMs;\n2. COAT is novel, interesting and well-motivated. The authors also provide theoretical discussion to justify its soundness;\n3. COAT is model-agnostic and robust to the choice of LLMs, and input data modalities;\n4. The authors construct several benchmarks, present comprehensive case studies, and conduct extensive experiments to verify their claims. The improvements over direct prompting LLMs are significant.\n\nWEAKNESSES:\n1. The authors should provide more comparisons with advanced prompting techniques such as CoT.\n2. More discussions should be provided on the hyperparameters used in COAT, such as the group size in feedback.\n3. Model names are inconsistent. The name in Fig 4(c) is not the same with other names.\n4. GPT-4 reasoning in Fig 7(c) is unclear in meaning.\n\nQUESTIONS:\nPlease refer to \"Weaknesses\".\n\nLIMITATIONS:\nNo concerns regarding limitations.", "TITLE": "Official Review", "RECOMMENDATION": 8, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-12"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper combines the power of LLMs with that of causal discovery by proposing a Causal representatiOn AssistanT (COAT) approach. Specifically, it considers datasets with textual descriptions, and tries to identify the Markov blanket with respect to a target variable (such as customer ratings and medical diagnosis). The key contribution is discovery of the causal factors through a pipeline that uses both LLMs and a causal discovery algorithm.\n\nSTRENGTHS:\nI find this an interesting and practical paper that combines the advantages of LLMs – such as the vast amount of knowledge that they encode – and that of causal discovery approaches. The ideas around combination are generally simple but novel, and I believe the approach could potentially be valuable in a suite of applications, although the extent of the value is unclear from the paper.\n\nWEAKNESSES:\nA major limitation of the work is the empirical evaluation, even though it comes across on the surface as being extensive. I sympathize with the authors about benchmarks for causal discovery, but it seems they have used GPT-4 to generate the textual description of the data, and then used LLMs in their COAT procedure. This is clearly a synthetic dataset that can be problematic. Even the “realistic” benchmarks do not come across as sufficiently realistic, based on my understanding and the lack of details in the main paper.\n\nI don’t understand why key aspects of the evaluation were moved to the appendix. I find it impossible to fully evaluate the work based solely on the contents of the main paper. I understand the need to make space and to move things to the appendix, but it’s never suitable to move key aspects such as the description of the benchmarks and the key results that show value of the work. This has impacted my assessment of this work and I have had to decrease my score because of the authors’ choices around appendix content.\n\nA related weakness is the lack of any attempt at describing limitations, of which there are clearly many.\n\nQUESTIONS:\nCould the authors share more about the scope of the work? Are there some other restrictions on the problem setting, besides needing a discrete label y? My assessment is that there is a gap between the scope mentioned in the problem setting and what is described in the experiments, which seems more restricted. Perhaps the authors can clarify.\n\nIdentifiability is mentioned loosely on page 2, with some technical references, but seems to have been used in an imprecise way here. The connection here seems tenuous at best.\n\nThere is a comment on pg. 3: “Note that the target variable Y serves as a guider, and no specific relation between x and Y is assumed.” What is a guider? And what do you mean no relation between x and Y is assumed? I thought the entire point is to do causal discovery: x is a function of z, and Y is a function of z. This line seems incorrect.\n\nSection 3.2 would be much easier to follow with some illustrative examples. The content in Fig. 2 is too abstract to be really useful. I think the authors missed a trick here.\n\nThe meaning of C and p are unclear to me from what is described on pg. 6. How does one assess the significance of Proposition 3.3?\n\nThe details about benchmarks are incredibly important, and it should be easy for anyone to understand at least a high-level sense of a benchmark – basic things like the number of data points, for instance. Please fix Section 4 accordingly.\n\nWhat is OT in Section 4.2? Is it the same as OT in the next section? Define MB, NMB, OT somewhere. I don’t see them mentioned anywhere clearly, although I understand from context that MB means Markov blanket.\n\nAre Table 3 and Fig. 5 in the Appendix? If so, then mention that.\n\nI’m not convinced that the “realistic” benchmarks are realistic. It’s too bad I can’t gauge this from the main text.\n\nPlease add a detailed limitations section. Mention all the limitations around evaluation in particular, as well as the significant risks of relying on LLMs for causal discovery.\n\nMinor comments: line 181: “an potential” should be “a potential”; line 208: “Ability” should be “ability”; lines 211 and 212: seems there is a grammatical error here; line 216: what are “shared notations”?\n\nLIMITATIONS:\nThe authors have not suitably described limitations. I consider this a major weakness; please see my previous comments.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2024-07-13"}