Discovery of the Hidden World with Large Language Models
Authors: Chenxi Liu
Yongqiang Chen
Tongliang Liu
Mingming Gong
James Cheng
Bo Han
Kun Zhang
Venue: NeurIPS 2024 poster (2024)
Keywords: Causal Discovery
Large Language Models
Causal Representation Learning
Primary area: causal_inference
TL;DR: A new framework leveraging large language models to extend the scope of causal discovery to unstructured data.

Abstract: Revealing the underlying causal mechanisms in the real world is the key to the development of science. Despite the progress in the past decades, traditional causal discovery approaches (CDs) mainly rely on high-quality measured variables, usually given by human experts, to find causal relations. The lack of well-defined high-level variables in many real-world applications has already been a longstanding roadblock to a broader application of CDs. To this end, this paper presents Causal representatiOn AssistanT (COAT) that introduces large language models (LLMs) to bridge the gap. LLMs are trained on massive observations of the world and have demonstrated great capability in extracting key information from unstructured data. Therefore, it is natural to employ LLMs to assist with proposing useful high-level factors and crafting their measurements. Meanwhile, COAT also adopts CDs to find causal relations among the identified variables as well as to provide feedback to LLMs to iteratively refine the proposed factors. We show that LLMs and CDs are mutually beneficial and the constructed feedback provably also helps with the factor proposal. We construct and curate several synthetic and real-world benchmarks including analysis of human reviews and diagnosis of neuropathic and brain tumors, to comprehensively evaluate COAT. Extensive empirical results confirm the effectiveness and reliability of COAT with significant improvements.

Discovery of the Hidden World with Large Language Models
Chenxi Liu Yongqiang 2.344 Tongliang Liu
'TMLR Group, Hong Kong Baptist University
Mohamed bin Zayed University of Artificial Intelligence
{cscxliu, busclll,,bannl)eecee hacmmm/le@ccmm.h.ueuuu.... (yqchen, jcheng) jec.enggecee..uukkeeuu @cse.cuhk.edu..k
Mingming 2 James
James Cheng", Bo Han Kun Zhang2.
Carneeiee Mellon University *The Chinese University of Hong Kong 'Sydney Al Centre, The University Sydney "The University of Melbourne tongliang liu@sydneyy.du. mingming gong@unimelb.edu.a au kunz @cmu.edu
https:/ccauaalcontggihhbbbo
Abstract
Revealing the underlying causal mechanisms in the real world the key the development of science. Despite the progress in the past decades, traditional causal discovery approaches (CDs) mainly rely on high-quality measured variables, usu- ally given by human experts, to find causal relations. The lack of well-defined high-level variables many
applications has already been longstand- ing roadblock broader application of CDs. To this end, this paper presents Causal representatiOn AssistanTT (COAT) that introduces large language models (LLMs) to bridge the gap. LLMs are trained on massive observations of the world and have demonstrated great capability extracting key information from unstruc- tured data. Therefore, it is natural to employ LLMs to assist with proposing useful high-level factors and crafting their measurements. Mcanwhile, COAT also adopts CDs to find causal relations among the identified variables as well as to provide feedback to LLMs to iteratively refine the proposed factors. We show that LLMs and CDs are mutually beneficial and the constructed feedback provably also helps with the factor proposal. We construct and curate several synthetic and real-world benchmarks including analysis of human reviews and diagnosis of neuropathic and brain tumors, to comprehensively evaluate COAT. Extensive empirical results confirm the effectiveness and reliability of COAT with significant improvements.
Introduction
Science originates along with identifying important variables and revealing their causal relations [1, 2]. Despite the progress in the past decades, existing causal discovery approaches (CDs) mainly rely on high-quality measured variables, which are usually given by human experts [3, 4, 5]. However, the desired variables and their measurements are usually unavailable in wide range of real-worr world applications. For example, Amazon sellers who want analyze the factors related to user ratings only have raw user reviews, which are written according to the underlying user preferences for certain product characteristics. Therefore, the lack of high-quality high-level variables has been a longstanding impediment to broader real-world applications of CDs or causaliy-inspiied methods [6].
'These authors contributed equally. 'Correspondence to Bo Han (bhanml @ www.nlddmooooooooooiiiooo
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

Factor Proposal
(b) Factor Annctation
Scare_4: 4: 701s seall apple, agpenling Ise corpaot corn doore 1: The areea, underatated and Eocoutiable
Arter exsetning the recters, 1 decide to use Low fellowing
Scare mix srall apple, appealing the conpact Boure d: The azosa, understated and forquttable
facier::
1. sise appeal:
1r [iforall size"
"Mat are the sigh-ieree factore assucraced the acozer"
[Nuutasb/zot rantisned]
Carze of medion sizs]
epleace help ne annotate che data uccoeding to
Large Language Model
Basouraey
2coxe 3: and its Laste,
nizzen the zestiul balance castnean
Dan nevieves, Size, 0,
Scoce 1: bite the apple Loula the boç.
high-levez
asscciated the othor
Causal-learn
Causel Discnvery Feedbback Construction
Figure I: Illustration COAT framework to analyze the rating scores of AppleGastronome, COAT aims to uncover the underlying Markov Blanket with respect to the given ratings of the apples (i.e., factors that fit the preferences of gastronomes). COAT first (a) adopts an LLM to read, comprehend. and relate the rich knowledge about tasting the apples. The LLM needs to propose series of candidate factors such as apple sizes and smells, along with some meta-information such as annotation guidelines. Based the candidate factors, COAT then (b) prompts another LLM to annotate the unstructured review into structured data. (c) The CD algorithm then finds causal relations among the factors, and constructs feedback based samples where the ratings not be well explained by the existing factors. By looking into the new samples, the LLM is expected to associate more related knowledge uncover more desired causal factors.
The recently emerged Large Language Models (LLMMs) [7, 8, 9, 10] offer new opportunity to mitigate the gap [11]. Trained from massive observations of the world, LLMs demonstrate impressive capabilities comprehending unstructured inputs, and leveraging the learned rich knowledge to resolve a variety of general tasks [12]. A surge of early tests demonstrates promising results that LLMs can effectively leverage the learned knowledge answer commonsense causal questions [11, 13, 14]. Nevertheless, existing approaches mainly focus on incorporating LLMs as straightforward reasoner with respect the given causal variables. The reliability of LLMs in directly reasoning the true causal structure behind any specific data- gettagenerating process remains a debate [13, 15, 16, 17] due to a series of drawbacks of L.LMs [18, 19, 20). In addition, all of the existing combinations of LLMs and causal discovery have surprisingly overlooked the identifiability of causal structure, which plays an important role in classic causal discovery literature [3, 4, 5]. Hence, challenging question comes:
How can LLMs reliably assist in revealing the causal mechanisms behind the real world?
In this work, we answer the question with a focus on local causal discovery with respect to a target variable that poses high value such as customer ratings and medical diagnosis, and introduce Causal representatiOn Assistan'T (COAT). Specifically, given the target variable Y, COAT aims to identify Markov blanket to Y from structure. To achieve the goal, as illustrated in Fig. 1, COAT employs two mutually beneficial components: LLMs and CDs. Iteratively, at step (a), COAT leverages LLMs look into a set of unstructured observations (e.g., customer comments) and propose potentially useful high-level factors. The proposed factors contain both the definition and the annotation criteria. Therefore. at step (b), another LLM is employed to give concrete values following the criteria. Then in step (c). CDs is used to reveal the structure among the identified factors. To ensure the reliability of the factor identification, COAT constructs feedback from the intermediate causal discovery results from step (c) to further inspire LLMs to improve further factors. The feedback includes sampling important observations that can not be well explained by the existing identified results. We show that the feedback provably helps with identifying the desired Markov Blanket and the structure (Proposition 2.2). We present comprehensive analysis of COAT on both synthetic simulations and
observation and also produce the aieooetical-guaranteen
causal

---

real-world case studies, ranging from analysis of human reviews to diagnosis of neuropathic and brain tumors (Sec. 4). Our contributions are summarized follows:
To the best of our knowledge, we are the first to leverage LLMs to propose high-level variables, thereby extending the scope of CDs to unstructured data (Sec. 2.1).
We establish the first benchmarks with woal-world data including AppleGastronome and Neuropathic to examine the unstructured causal discovery (Sec. 4).
We propose the first framework COAT that combines the best of LLMs and CDs to find theoretically grounded causal results (Sec. 2.2), which are verified with extensive empirical studies (Sec. 4).
Additionally, the analysis of COAT also derives the first metries that measure the causal representa- tion learning capabilities of various LLMs (Definition 2.3).
2 Representation Assistant for Causal Discovery
In this section, we present the formulation of leveraging a language model to serve as a representation assistant for causal discovery unstructured data. The representation assistant needs to extract useful factors that capture sufficient information for an interested target variable.
2.1 Problem Definition
Data To begin with, we are given a lurget variable of interest Y, e.g., stars rated by a customer or the tumor type of a patient. We treat Y as a scalar random variable without loss of generality. The unstructured data, or raw observations, c.g., customer review of a certain product or images of tumors, are denoted as X. The dataset D consists of n paired samples {(x,, 11) that are independently drawn from the distribution over (X,Y) Note that the target variable Y serves as guider, and no prior assumption on the relations between X and Y is assumed.
Objective We seek a mapping h X + Z which elicits the structured representation 2 = h(X) such that Y I X Z. In other words, serves as Markov Blanket of for the unstructured raw observations. Built upon the structure representation, then, downstream causal discovery methods can be applied on U (Y). The revealed causal structure can provide insights about the target variable Y [21, 22], such as what factors of the product would be most satisfactory to customers. Furthermore, the framework can be easily extended to discover a complete causal graph by shifting the target variable to the other identified factors the other additionally available variables. Formal assumptions are discussed in Sec. 2.4.
LLM as a representation assistant We aim to make the most use of the rich knowledge of LLMs assist in extracting the relevant information from the raw observations X. To this end, the mapping h is decomposed as collection of factors W {w,, W2, .... Wi J. each of which is function w; + C that maps the raw observation æ to a predefined value space C. other words, the struc- tured representation is composed of multiple factors: h(X) (wr, (X) )), ; W22 (X).... we (X)). Throughout this work, for the notation of factors, we use the symbol w, to denote the factor itself like sweetness, size, or scent, and w.(-) to denote the function that maps from raw observation space X to the predefined value space C.
Descriptions of the factors Without loss of generality, in this work, we consider the descriptions of the factors in natural language, which can be divided into two categories: i) Implicit factors, which need to be discovered and elaborated by LLMs. To obtain the values of the implicit factors, one could feed the factor descriptions and the unstructured input æ; to a suitable LLM for value annotation; For example, given a customer review on apple 2;, discovered implicit factor C = {-1,0, 1]: wi (a;) = could mean that the customer appreciates the sweetness of the apple; un (ar) = -1
means that the customer is disappointed about the sweetness; w (a:) = 0 means the sweetness has not been mentioned. ii) Explicit factors such as heart rate, whose descriptions are already available. The measure of the explicit factors usually requires some external tools.
2.2 The COAT Framework
We approach the aforementioned problem via a new framework called Causal representatiOn Assistan'T (COAT) (Algorithm 1). COAT aims to extract useful factors through multiple rounds of
3

---

You are accalistly selpfst arxiatant analyring, abatricting, and processing deta, try
Data
each the factora tolloving critaria: Each new fictor shculd be helptul
thoud deside shethar
with "acuze" -1.0
This apple varsety tepresser with the rabuat nuvertienel piotile, pecking a leccly dose of cietary Libet and vicenin C...
diacinculah the groupa. Fach Factor should fooor on one conerete sapect and Ley aveid pvezlape.
group
Part Zinel Outyut.
apple offers olleggelne mitriciona
requied
Usopi ie its large sise, whuch may appecl to those leoking for aisable tests option, this
with 'acare' 1.0
Each factora' value abouid feor. 1-,, 0, 1; Yon should glve specific criterton for each value of euch fuctot,
factars that desided to use.
Keep the criterion for valos fised *othecviaur not muntisted""
Your exsput ahouis contain parta described az
Past Consideration.
tesplate:
In this part, feal Iree to weite down the prodasa of yeuz conaidraation.. sinci You need to atstract, idencify, and denign auitablu dactora wish cossaspending critaria, and each faotor ie only sctewed to lake value Loox (-k,
Factor Saze
1:
[Pozitive costacton)
0:
fothewwizer oe nut aeationed)
[Segatiive Costerion)
Figure 2: Illustration of the prompt template for factor proposal in COAT.
iteration. We use the superscript / to refer to the input and output of LLM at the 1-th round. We also denote the union of results from the first rounds using the superscript "S 6"
Factor proposal To induce useful high- level factors from the rich world knowledge of LLMs, COAT employs prompt p and a few samples D D.
Algorithm 1 The COAT Framework
1: Required: Dataset D {(x,, (1) M))=i LLM for factor proposal V Model for factor parsing V.; causal discovery algorithm A: Feedback construc- tor F; Maximal rounds T;
Specifically, in Fig. 2. p contains three com- ponents: samples, instructions, and format control. To encourage LLM to focus the information related to the target variable
2: Random sampling D);
3: Constructing p';
Y. samples are grouped by the value of Y. The instruction requires to give each proposed factor w, concrete description of the mapping w.(.), like how to decide the factor values. In addition, the meta- data about the task such as the task descrip- tion and context can also be incorporated if available. The prompt p essentially im- itates human experts [23] in selecting and defining high-level variables. The set of resulting factors in the t-th round, defined with natural language by the LLM V, is denoted as W (ppp DDt. We merge them with factors proposed in the previous rounds to update the set of factors WE Wi U... U..UUW Wt
4: while not converge and current round t < T do
5:
W + V'p', D³); //factor proposal
6:
Z' + t (D, Pp); //factor parsing
7:
gt - (Y)); //causal discovery
8:
(De+1,ptt) +
//feedback
end while
10: return g"
Factor parsing Once we obtain the candidate factors, we then collect the values of the factors from the unstructured observations. In prior works, they are usually collected from human experts according to the given factors [3]. To do so, another LLM Vr is instructed to read the annotated guidelines of each variable w; and parse the unstructured observations into structured or tabular data:
(w:(w)),-.. (,(()))))..,,((((()))) wx)
V (x w. Pp),
(I)
where refers to the additional instruction to prompt V to parse the observed data, and z; refers to the parsed values for the corresponding factor w; We define z = Concat { zz, for w, E WS*)).
When the data curation of the proposed factors requires additional domain- knowledge/skills (e.g., intervening on the external environments) that the LLMS do not acquire, we could fetch 2; through some external process [24, 25]. For example, studying the causes of disease requires annotating relevant symptoms from diagnosis records and conducting additional medical checks [26]. Our experiments show that COAT can effectively extract the hidden factors under both schemes.

---

Causal discovery With the given values 0 associated with the candidate factors WS a CD algorithm A (e.g. FCI (27]) used reason about the causal structure based on the parsed data:
g' = A(Z U (YY),
where G is the discovered causal structure. In general, the inputs in each round to A may contain noises well as latent confounders, any CDs with suitable theoretical assumptions could be used for A. The noises injected through LLM-based parsing may be of independent interest to the literature of causal discovery [5, 28]. In this work, we demonstrate the idea of COAT via the FCI algorithm [4] as it is flexible with respect to different functional classes of the underlying generation process, allows for the existence of latent confounders [27], which aligns well with our objective.
Improving factor proposal with causal feedback LLMs require proper prompts to fully unlock their capabilities [29, 30, 31j. When it comes to factor proposing, it is also hard for LLMs to propose all factors at once. Nevertheless, from the causal discovery results, we could find useful information and thus provide feedback to further improve the factor proposal:
(D'+1, - F(g', F('',
where samples specific examples from D and constructs new prompts according to the results of A for the next round of factor proposal. For example. FCI is able to imply the existence of latent confounders, from which we could refine p to prompt ! to focus on the corresponding factors.
2.3 Causal Feedback
Let X be the high-dimensional random variable for the raw data. After t rounds, COAT identi- fies (X)
W
W
(X) as the Markov Blanket for Y w.r.t. to (X) uy; E WS(). Y Y X (X), which means it cannot serve as a Markov Blan- ket [32] for Y w.r.t. X, then there exists a po-
Figure 3: Illustration of variables that could be discovered with COAT. Let W E hgg(X) be an identified variable, and assume there exists la- tent variable u to be discovered. When w the direct parent child of Y, finding hard-to-explain samples can help uncover it. When w is the direct parent and also a child of W, or the spouse of Y with W as the common child of Y and w, condi- tioning on W facilitates the discovery of w.
tential factor w X + C such that:
(X)) > H(Y))g:(X), (XX))))))) (4) where H--) refers to the entropy. If the LLM can not find the desired w. means that (X)( is already sufficient to separate Y from X. Therefore, for the next (L + 1)-th iteration with sample Be+1, V is expected to propose new factor w that also satisfies the similar property:
HB (Y|h:: - Hau ( 00
(5)
where Hz. (-I-) refers to the conditional entropy measured Dr. As shown in Fig. 3. finding factors satisfying Eq. progressively expands the discovered factors and pushes (x)(()) to valid Markov Blanket. Also, it implies conditioning on the identified factors would further strengthen the correlation between and Y. Therefore, to find the desired factor, we are motivated to select suitable Di+1 for the next iteration such that D arg "a (Y (X)),, where (X)
can not well explain Y. This problem can be converted into a classification problem in which D are the samples that the fitted classifier yields large prediction error. In our experiments, implement the classification via clustering with respect to (X). The clustering elicits groups D. ;= {æ, for i E }: Zi. Ic = MMeans(hce We then take the group of samples with the largest conditional entropy to construct the feedback.
In practice, many factors, such as the LLM capabilities, data faithfulness, and prompt templates, could affect the satisfaction of Eq 5. Therefore, in the next section, we will establish a theoretical framework to discuss the influence of the factors above to the satisfaction of Eq 5.
2.4 Theoretical Analysis
We then theoretically analyze some critical steps in COAT, including the feedback and identifiability.
5

---

Feedback analysis (wi (X), w2 (X), .. w (X)), and COAT tests:
Given new factor w1+1, with the current representation as hxn
=
Y + wx+1(X) (x).
(6)
COAT also requires the following usual condition about distribution and causal graph:
Assumption 2.1 (Faithful and Markov conditions, adjusted from [33]). For any disjoint non-empty subsets A, B, C' C WS U }Y), A and B are d-separated by C on the causal graph iif A 1 B C
on the factors' distribution. All conditional independencies are preserved after factor parsing.
The annotation from poor model could introduce an additional "error term" the true factor values, disturbing the true distribution, as one can observe in Fig. 4(a) and 4(b). Assumption 2.1 holds, the conditional mutual information between Y and X given the desired factors decreases:
Proposition 2.2. Under assumption 2.1, if condition 6 holds, then for Markov Blanket S [ [k+11] +] ofY, i.e,, Y L (14+1),,(()) (Xs(),, we have the following about conditional mutual information:
IY;;X (x)) (Y; (X)) <I (ape)(()
(7)
Factor identification We provide an initial exploration under what conditions LLMs can identify target-related factors and how the ability of LLM influences this procedure.
Definition 2.3 (Ability of LLMs). Given suitable prompt about current factors and data, the LLM has non-zero probability pa > 0 to propose new factor wx+11 that satisfies condition and
1 (Y; (x)+
< CCy
(8)
1 (Y; (x))
for some positive constant Cs whenever I (Y; X (x)( > Note that bug (X) = ④. hence we also use (Y;X) (hoo(()) to refer (Y; X). We use p instead of pv when the context is clear.
We further explain the intuition behind Def 2.3: the Perception Score captures the LLM's respon- siveness the given prompts and the feedback; the Capacity Score C captures the quality of the factors proposed by the LLM. Empirically, the two scores are used to estimate the abilities of the predominant LLMs (Sec. 3.2). Theoretically, we use them to characterize the influence of prompt templates, the LLM responsiveness, and the quality of factors on the performance of COAT:
Proposition 2.4 (Characterization for Factor Identification Process). With assumption 2. I, for any small number e, 8 (0, }), perception score p > capacity score Cy > 0, with i COAT rounds that
4loge e
Vi
2VP
21(1 p) log (1 Cv)
where is the 8-quantile of the standard normal distribution, we have
[X;X] hss(())
Pr
<e) 2 21
(10)
X(Y;X)
The proof is given in Appendix D.2. Prop. 2.4 gives guarantee on identifying Markov Blanket. Intuitively, Prop. 2.4 also characterizes the influence of prompt templates, the LLM responsiveness, and the quality of factors the performance of COAT via the two proposed measures; and Cy. When both of them are positive, COAT would converge exponentially:
Proposition 2.5 (Rate of Convergence). With assumption 2.1, for any small number e, (0, H), perception score > O, capacity score Cy > O, after t COAT rounds, the following inequality holds with probability at least 1 8:
-tp 20- 1o=22ttp((1p 1pp11
T(Y; (x)( & 1- I(Y:))
(11)
Causal structure identification It is clear that LLMs are not involved in the causal discovery process, which mainly executed by causal discovery methods such as FCI. Therefore, the CD guarantees the identifiability of the final causal graph over the LLM-proposed factors. The concrete

---

DeepStekoV2
0.70
Claude-3-Opu
0.40
60
Mistral-Large GPT
GGPT 40
2wan 5-1108
Mistral-Meduum
0.50
Jaana-3-70b
70
9.4
Jama-2-70b
57m
50w
Smell
Vode
03g
(a) Apple attributes Acc.
(b) Preference matchness Acc.
(c) LLM Ability Seatters
Figure 4: Quantitative evaluation of the causal capabilities of LLMs in COAT.
assumptions required for identifiability depend on the specific CD used in COAT. For instance, the FCI algorithm requires faithfulness the data distribution with respect to the true causal graph [27]. also verify that the structured data annotated by LLMs has high accuracy and little noise, which is friendly to the CD assumption. in COAT, while using different CDs may require different assumptions. For example, the LiNGAM algorithm requires the relations among variables to be linear and non-Gaussian models. Empirically, find that COAT with LiNGAM works very well (Appendix F).
In experiments,
general, one could switch to another CD
2.5 Practical Discussions
Prompt template The instruction following capability and the context window of L.LMs may affect the satisfaction to the constraints of the prompt template. Including more data samples or background knowledge may improve the and Cp, but it is more challenging for the LLM.
Modern causal discovery We use FCI in this paper order illustrate the idea. To attain identi- fiability better than the Markov equivalent class, one can choose more advanced Causal Discovery methods under different assumptions, see Appendix C if interested.
We also discuss some cases where we need to handle them properly in practice with LLMs.
Factor filtering L.MSs may output several factors with similar semantics or exhibit multicollinearity in the annotated data, which will hinder the causal discovery process. To mitigate the issue, one could do factor filtering that adopts PCA or early conditional independence tests given the currently discovered variables in the Markov Blanket detect and eliminate these variables.
Factor pool LLMs may discover useful factors in early rounds while being discarded. For example, the underlying spouse variables of the target label y may be independent with y without conditioning on their common children variables. To resolve the issue, we could introduce factor pool that stores the candidate variables proposed in the past, and replay the variables that have not been passed by conditional independence tests with existing variables in the Markov Blanket for double check.
3 Empirical Analysis of COAT
We evaluate whether COAT propose and identify set of high-level factors belonging to the Markov Blanket of the target variable Y. We construct the first benchmark, called AppleGastronome, verify the effectiveness of COAT in finding useful causal information, and compare COAT to previous methods [11] that merely leverage L.MMs to perform causal discovery. Specifically, we AppleGastronome examine the capabilities of 10 predominant LLMs including GPT 4o [34], Claude-3-Opus [35], LLaMA3-70b [36], and Mistral-Large [37] in realizing COAT. Due to the space limit, we report only the results of the popular LLMs and present full results in Appendix E.4.
3.1 Experimental Setup
In AppleGastronome benchmark, we consider the target variable rating score of the apple by astronomers. We prepare different high-level factors: parents Y, one child of Y, and one spouse of Y. These factors form Markov blanket of Y. In addition, also prepared one disturbing factor
7

---

ONES
Farte peierul
(a) Ground truth
(b) GPT-4 META
(c) GPT 3.5 COAT
(d) Claude-3-Opus COAT
Figure 5: The discovered causal graphs in AppleGastronome, Compared to the ground truth results, directly adopting LLMs to reason the causal relations can easily elicit many false positive edges. In contrast, the relations recovered by COAT have high precision and recall. The directed edge between "taste" and "juiciness" can not be recovered by COAT because of the limitations of FCI.
related to Y but not a part of this blanket. A good method is expected to propose the five high-level factors (up to semantical meanings) and exclude the disturbing factor.
Benchmark construction Each apple has its own attributes, including size, smell, and taste (or sweemess). Each gastronomy pays unique attention to a subset of the above three attributes. They will write review according to their preference and give the rating score. We generate the review using GPT 4 by feeding GPT 4 the preferences and the apple attributes. We generated 200 samples for LLMs' analysis and annotation. More details are left in Appendix E.
of LLMs the baselines: META is the zero-shot factor proposal given only the context to LLMs; and DATA additionally gives some samples of raw observations, which is ablation of COAT without the feedbuck module, iee, only one COAT round. For causal relation inference, we follow Kiciman et al. [1 ]] that prompt LLMs reason for the causal direction of each pair of the discovered variables by DATA.
Baselines For factor proposal, mainly employ two different
Metrics We evaluate the ability on factor proposal based on three metrics: MB, NMB, and OT. MB means the desired factor forming the Markov Blanket of Y. NMB means the undesired factor relevant to data but not in MB. OT means the unexpected factors irrelevant to data. We also present the corresponding recall, precision, and F1 with respect to MB(Y).
3.2 Analysis with AppleGastronome Benchmark
Key findings Empiri- cally, LLMs with CoT can be aware of high-level fac- tors behind data (lower OT than META) but still strug- gle to distinguish the de- sired factors in Markov Blanket (higher NMB than COAT). COAT is more re- sistant to the "disturbing" factor, which is supported by the lower NMB column.
Table 1: Factor proposal results in Apple Gastronome benchmark (Full results in Appendix E.4).
LLM
METROD
MR
NMB
OT
PROPUSAL RECALL
PREEISION
F1
META
2.67
0.67.a47
2.33
0.53
D 1.46
0.491921
GPT
DATA
3.00
0. 3.333447
0.00s.00
0.602000
92
0.72=064
0.17:429
0.87
0.8429M
COAT
4.00
0.33
0.0014.0
0.80..0
0.93
0.851927
META
3.33
0.33
4.33
0.6720m
0 1242
2.5
GPT 3.5
DATA
2.675447
0.67
0. 0.00sa.0
0.53-005
0.81
0.64
DATA+COT
3.00s000
00:0000000
1.33asd
1.00000
68.8800
COAT
3.675647
0.00:sa00
0.00:a00
0.73ams9
1,00:t:000
0.84
LLAMA2 -70B
DATA
2.33ss0
2.335a.47
0.67:0047
0.6720447
0.00sa00
0.47a0u9
0.75+s2.0
0.32
0.37
0.57=920
DATA+COT
1.00=1.72
0.67
0. 0.33
0.60=035
0.65
COXT
3. 3.00:200
0.67:047
0.00:aa
0.60
0.83
0.69
COAT filters out irrelevant factors from LLMs' prior knowledge that are not re- flected by the data, which supported by the lower OT column. COAT robustly encourages LLM to find more expected factors through the feedback, which is supported by the higher MB column.
3.00samv
0.67:0047
1.67ax22
0.60
0.59
MEDIUMM
DATA+CUT
4.33
0.67:00.7
1.00
0.67
1877
0.73
0.69:964
0.79
COAT
4.67.8.77
0.00
0. 0.001a
0.93
1.00
0.96
Can LLMs be an effective factor proposer? As discussed in Sec. 2.4, there are two crucial abilities for L.Ms in identifying potential high-level factors. The first one is be aware of the existence of potential factors, and the second is to synthesize and describe these factors. Inspired
8

---

Rimder
(a) Faithful ground truth
(b) GPT-4 COAT
GPT-4 reasoning
Figure 6: The discovered causal graphs in Neuropathic. (c) shows the result based directly prompting LLM to reason for the causal relations among all factors. Disconnected ones are dropped.
by this observation, we propose two novel metrics to quantify LLMs' causal ability: a) Perception that quantifies the ratio of valid factors (satisfying Prop. 2.2) proposed by LLMs in each round; b) Capacity that measures the effective mutual information drop in Assumption 2.3. As shown in Fig. 4(c), LLMs differ largely on the perception score while comparably on the capacity score.
Can LLMs be an effective factor annotator? As shown in Fig. 4, both GPT 3.5 and GPT annotate subjective attributes well. Regarding objective human preferences, the performances are still relatively high. Empirically, LLMs will not introduce new confounders, see Appendix E.3.
Can COAT reliably recover the causal relationships? We present quantitative and qualitative results in Table 3 (in Appendix E.1) and Fig. 5 (on page 8), respectively. Compared to directly adopting LLMs to reason the causal relations, COAT significantly boosts the causal relation recovery. Meanwhile, COAT maintains high performances based on various LLMs, which further demonstrates the effectiveness of the causal feedback in COAT improve the robustness of this system. In fact, the causal feedback focuses on making maximal use of the rich knowledge of LLMs, and reducing the reliance the reasoning capabilities of different LLMs, to assist with causal discovery. We provide the full results of 10 LLMs in Appendix E.3.
4 Empirical Study with Realistic Benchmarks
After examining the capabilities of COAT in AppleGastronome, we are further motivated to chal- lenge COAT in more complex setting from neuro- pathic panic diagnosis [26], brain tumor detection with MRI images [38], three-years news summary about one stock from the New York Times [39], and climatic reanalysis data with fine-grained time and space coverage [40]. We refer to Appendix H for complete summary of all five benchmarks.
Table 2: Factor proposal results in Neuropathic. PA, AN, and OT refer to the parents, ancestors, and others, respec- tively. Accuracy and Fl measure the recovery of the causal ancestors.
FACTOR PROPOSAL
LLM
Acc
META
3
5
6
0.91
0.59
DATA+COT
3
13
0.81
0.35
COAT
6
0.96
0.80
4.1 Experimental Setup
GPT
DATA
META
0.91
0.94
0.67
0.59
DATA+COT
2
0,91
0.36
Benchmark construction In the Neuropathic benchmark, we convert the dataset into clinical diag- nosis task. In the original dataset, there are three lev- els of causal variables, including the symptom level, radiculopathy level, and the pathophysiology level. In the experiments, we mainly consider the target vari- able of right shoulder impingement. When generat- ing the clinical diagnosis notes using GPT 4, we avoid any mentioning of variables other than symp- toms. We generated 100 samples for LLMs' analysis; since the number of possible factors is finite, we gen- erate 1000 tabular data for CI tests.
COAT
0.96
0.77
LLAMA2
META
0,91
0.95
0.60
0.53
-708
DCTT
0.88
0.47
COAT
0.97
0.86
LLAMA2 -138
META
0.88
0.40
DATA
6
0.95
0.75
DATA+COT
COAT
0.97
0.86
LLAMA2
META
17
0.72
0.08
-7n
DATA+COT
DATA
0
10
0.96
0.79
0.80
COAT
0.97
0.86
MISTRAL MEDIUM
META
3
6
0.96
0.80
DATA
0.94
0.66
0.88
0.53
COAT
6
2
0.97
0.86
As we intend to leverage the Neuropathic benchmark
to simulate the real-world diagnosis, after the factor proposal stage, we directly incorporate external tools to measure the values of the candidate factors. More details about the construction of the Neuropathic are given in Appendix E.6.

---

Dewttad
Rodaati
Exeaerie Padti Roges
Suraex
Regio
Targe: Varalee
Frapore Bucse (saacaar))
Progosed Foroo (mee-sxtonar))
Sull Musatand
Na This
Flax
Suil Ienipennne
Pecpphatio Bade
Stoweea Flax V Campenyy
Change
Soy Lovel Pressur
Figure 7: The final causal graph found by COAT in the ENSO case study
Evaluation and baselines In Neuropathic, we adopt a similar evaluation protocol and the baselines as in AppleGastronome. Nevertheless, due to the faithfulness issue of the original dataset [26], for the evaluation of causal relation discovery, we mainly conduct qualitative comparison between the ground truth that is faithful to the data, against the baselines and COAT.
4.2 Empirical Results on Neuropathic Benchmark
Factor proposal The quantitative results Neuropathic benchmark are given in Table 2. Similarly, we can find that COAT consistently outperforms all of the baselines regardless of which LLMs are incorporated. particular, even with the weakest backbone model, Lee.LLaMM2276. COAT can still effectively leverage the intrinsic rich knowledge and beat the baselines with more powerful LLMs.
Causal relation recovery Fig. 6(a) shows the causal graph obtained by FCI running the original data, where we can find that several causal relations cannot hold on the data. As shown in Fig. 6, when using LLMs perform the reasoning. LLMs cannot identify the faithfulness issues. In contrast, COAT can imply faithful causal insights.
4.3 More Real-world Results
El Niño-SSuthern Oscillation (ENSO) case study El Niño-SSuthhrn Oscillation (ENSO) is climatic phenomenon in the Pacific Ocean that influences global weather patterns profoundly. To understand its mechanism, we apply COAT NOAA dataset [40]. There 13 factors identified
by COAT, and their instantaneous causal relations are visualized Fig 7. The target variable is the future change in monthly SST in the Nino3 region, which could be important indicator of ENSO events. Each factor a time series about a certain climate measurement above specific level averaged over specific region. The paths about Sea level Pressure, Momentum Flux, and Cloud Cover matches the existing understanding from literature [41, 42, 43, 44]. It also suggests several inserting hypotheses that are less explored in literature, like the path from Soil Temperature in South American Coastal Region. We refer details in Appendix K.
More real- world empirical studies We also report concrete results real-world problems involving MRI, time series, and NetCDF data in Appendix I, J, and K.
Conclusions
In this paper, we proposed new paradigm COAT incorporate the rich knowledge of LLMs into the CD pipeline. We found that COAT effectively extends the scope of CDs to unstructured data by identifying useful high-level variables from raw observations for CD methods. COAT suggests a new pathway towards building causal foundation model for discovery. We leave more detailed discussions about future studies Appendix B.
10

---

Acknowledgments and Disclosure of Funding
We thank the reviewers for their valuable comments. This material is based upon work supported by NSF Award No. 2229881, AI Institute for Societal Decision Making (AI-SDM), the National Institutes of Health (NIH) under Contract RO1 HL159805, and grants from Salesforce, Apple Inc., Quris AI, and Florin Court Capital. CXL and BH were supported by NSFC General Program No. 62376235, Guangdong Basic and Applied Basic Research Foundation Nos. 2022A151501 1652 and 1224A115511339 HKBU Faculty Niche Research Areas No. FFNRA-IG/22-23//0022A and HKBU CSD Departmental Incentive Scheme. MMG was partially supported by the following Australian Research Council projects: DE210101624 and DP240102088. JC supported by CUHK direct grant 4055146.
References
[1] Norwood Russell Hanson. Patterns of discovery an inquiry into the conceptual foundations of science. Cambridge University Press, 1958. (Cited on page 1)
[2] Thomas S. Kuhn and David Hawkins. The structure of scientific revolutions. American Journal of Physics, 31:554-555, 1963. (Cited on page 1)
[3] Peter Spirtes, Clark Glymour, and Richard Scheines. Causation, Prediction, and Search, Second Edition. Adaptive computation and machine learning. MIT Press, 2000. ISBN 978-0-262-19440--. (Cited on pages 1, 2, 4, 21 and 22)
[4] Peter Spirtes, Clark Glymour, Richard Scheines, and Robert Tillman. Automated Search for Causal Relations: Theory and Practice, 2018. (Cited on pages 1, 2, 5, 21 and 22)
[5] Matthew J. Vowels, Necati Cihan Camgoz, and Richard Bowden. D' like dags? survey on structure learning and causal discovery. ACM Computing Survey, 55(4), 2022. ISSN 0360-0300. (Cited on pages 1, 2, 5, 21 and 22)
[6] Bernhard Schölkopf, Francesco Locatello, Stefan Bauer, Nan Rosemary Kc, Nal Kalchbrenner, Anirudh Goyal, and Yoshua Bengio. Towards causal representation learning. arXiv preprint, arXiv:2102. 11007 2021. (Cited on pages 1, 21 and 22)
[7] Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agar- wal, Ariel Herbert- Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler. Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCan- dlish, Alec Radford, Ilya Sutskever, and Dario Amodei. Language models are few-shot learners. In Advances in Neural Information Processing Systems, 2020. (Cited on pages 2, 21 and 22)
[8] OpenAL. Chatgpt. https: //tptt./chaatttcc openai..pp.i....mm wtt.:ppaaa....ccaat/t.o.oooom//t
(Cited on pages 2, 21 and 22)
[9] Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Tim- othée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, Aurélien Ro- driguez, Armand Joulin, Edouard Grave, and Guillaume Lample. Llama: Open and efficient foundation language models. arXiv preprint, arXiv:2302.13971 2023. (Cited on pages and 21)
[10] OpenAL. Gpt-4 technical report, 2023. (Cited on pages 2, 21 and 22)
[11] Emre Kiciman, Robert Ness, Amit Sharma, and Chenhao Tan. Causal reasoning and large language models:
Opening a new frontier for causality. arXiv preprint, arXiv:2305.00m0 (Cited on pages 2, 7, 8, 21 and 22)
2023.
[12] Sébastien Bubeck, Varun Chandrasekaran, Ronen Eldan, Johannes Gehrke, Eric Horvitz, Ece Kamar, Peter Lee, Yin Tat Lee, Yuanzhi Li, Scott M. Lundberg, Harsha Nori, Hamid Palangi, Marco Túlio Ribeiro, and Yi Zhang. Sparks of artificial general intelligence: Early experiments with GPT-4. arXiv preprint, 1tti::2333.121212 2023. (Cited pages 2, 21 and 22)

---

[13] Cheng Zhang, Stefan Bauer, Paul Bennett, Jiangfeng Gao, Wenbo Gong, Agrin Hilmkil, Joel Jennings, Chao Ma, Tom Minka, Nick Pawlowski, and James Vaughan. Understand- ing causality with large language models: Feasibility and opportunities. arXiv preprint, arXiv:2304.05524, 2023. (Cited on pages 2, 21 and 22)
[14] Ahmed Abdulaal, adamos hadjivasiliou, Nina Montana-Brown Tiantian He, Ayodeji Ijishakin, Ivana Drobnjak, Daniel C. Castro, and Daniel C. Alexander. Causal modelling agents: Causal graph discovery through synergising metadata- and data-driven reasoning. In The Twelfth International Conference on Learning Representations, 2024. (Cited on pages 2, 21, 22 and 23)
[15] Matej Zečević, Moritz Willig, Devendra Singh Dhami, and Kristian Kersting. Causal parrots: Large language models may talk causality but are not causal. Transactions on Machine Learning Research, 2023. ISSN 2835-8856. (Cited on pages 2, 21 and 22)
[16] Zhijing Jin, Yuen Chen, Felix Leeb, Luigi Gresele, Ojasv Kamal, Zhiheng LYU, Kevin Blin, Fernando Gonzalez Adauto, Max Kleiman-Weiner, Mrinmaya Sachan, and Bernhard Schölkopf. CLadder: A benchmark to assess causal reasoning capabilities of language models. In Thirty-seventh Conference on Neural Information Processing Systems, 2023. (Cited on pages 2, 21 and 22)
[17] Zhijing Jin, Jiarui Liu, Zhiheng Lyu, Spencer Poff, Mrinmaya Sachan, Rada Mihalcea, Mona T. Diab, and Bernhard Schölkopf. Can large language models infer causation from correlation? arXiv preprint, arXiv:2306.055. 005366 2023. (Cited on pages 2, 21 and 22)
[18] Yue Zhang, Yafu Li, Leyang Cui, Deng Cai, Lemao Liu, Tingchen Fu, Xinting Huang, Enbo Zhao, Yu Zhang, Yulong Chen, Longyue Wang, Anh Tuan Luu. Wei Bi, Freda Shi, and Shuming Shi. Siren's song in the Al ocean: A survey on hallucination in large language models. arXiv preprint, itti::3990012111 2023. (Cited on pages 2, 21 and 22)
[19] Chenhang Cui, Yiyang Zhou, Xinyu Yang, Shirley Wu, Linjun Zhang, James Zou, and Huaxiu Yao. Holistic analysis of hallucination in gpt-4v(ision): Bias and interference challenges. arXiv preprint, arXiv:2311.00 1.02287. 2023. (Cited on pages 21 and 22)
[20] Lukas Berglund, Meg Tong, Max Kaufmann, Mikita Balesni, Asa Cooper Stickland, Tomasz Korbak, and Owain Evans. The reversal curse: Lims trained on "a is b" fail to learn "b is a". arXiv preprint, arXiv:2309.12288 2023. (Cited pages 2, 21 and 22)
[21] Constantin F. Aliferis, Alexander Statnikov, Ioannis Tsamardinos, Subramani Mani, and Xenofon D. Koutsoukos. Local causal and markov blanket induction for causal discovery and feature selection for classification part i: Algorithms and empirical evaluation. Journal of Machine Learning Research, 11(7):71-234 2010. (Cited on page 3)
[22] Shantanu Gupta, David Childers, and Zachary Chase Lipton. Local causal discovery for estimating causal effects. In Conference on Causal Learning and Reasoning, volume 213, pages 408-447, 2023. (Cited on page 3)
[23] Judea Pearl and Dana Mackenzie. The Book of Why: The New Science of Cause and Effect. Basic Books, Inc., USA, 1st edition, 2018. ISBN 046509760X. (Cited on pages 4 and 22)
[24] Timo Schick, Jane Dwivedi-Yos Yu, Roberto Dessi, Roberta Raileanu, Maria Lomeli, Luke Zettle- moyer, Nicola Cancedda, and Thomas Seialom. Toolformer: Language models can teach themselves to use tools. arXiv preprint, arXiv:2302.04761. 2023. (Cited on page 4)
[25] Zhiheng Xi, Wenxiang Chen, Xin Guo, Wei He, Yiwen Ding, Boyang Hong, Ming Zhang, Junzhe Wang, Senjie Jin, Enyu Zhou, Rui Zheng, Xiaoran Fan, Xiao Wang. Limao Xiong, Yuhao Zhou, Weiran Wang, Changhao Jiang, Yicheng Zou, Xiangyang Liu, Zhangyue Yin, Shihan Dou, Rongxiang Weng, Wensen Cheng, Qi Zhang, Wenjuan Qin, Yongyan Zheng, Xipeng Qiu, Xuanjing Huan, and Tao Gui. The rise and potential of large language model based agents: A survey. arXiv preprint, wtXiv:2309.078666 2023. (Cited page 4)
[26] Ruibo Tu, Kun Zhang, Bo C. Bertilson, Hedvig Kjellström, and Cheng Zhang. Neuropathic pain diagnosis simulator for causal discovery algorithm evaluation. In Advances in Neural Information Processing Systems, pages 22773-12784 2019. (Cited on pages 4, 9, 10 and 43)
12

---

[27] Peter Spirtes, Christopher Meek, and Thomas Richardson. Causal inference in the presence of latent variables and selection bias. In Uncertainty in Artificial Intelligence, page 499-506, 1995. (Cited on pages 5, 7 and 22)
[28] Clark Glymour, Kun Zhang, and Peter Spirtes. Review causal discovery methods based on graphical models. Frontiers in Genetics, 10, 2019. (Cited pages 5 and 22)
[29] Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, brian ichter, Fei Xia, Ed H. Chi, Quoe V Le, and Denny Zhou. Chain of thought prompting elicits reasoning large language models. In Alice H. Oh, Alekh Agarwal, Danielle Belgrave, and Kyunghyun Cho, editors, Advances in Neural Information Processing Systems, 2022. (Cited on pages 5 and 22)
[30] Jiaxin Huang, Shixiang Gu, Le Hou, Yuexin Wu, Xuezhi Wang, Hongkun Yu, and Jiawei Han. Large language models can self-improve. In Conference on Empirical Methods in Natural Language Processing, pages 1051-1068, 2023. (Cited on page 5)
[31] Shuofei Qiao, Yixin Ou, Ningyu Zhang, Xiang Chen, Yunzhi Yao, Shumin Deng, Chuanqi Tan, Fei Huang, and Huajun Chen. Reasoning with language model prompting: survey. In Annual Meeting of the Association for Computational Linguistics (Volume I: Long Papers), July 2023. (Cited on page 5)
[32] Judea Pearl. Probabilistic reasoning in intelligent systems networks of plausible inference. Morgan Kaufmann series representation and reasoning, 1991. (Cited on page 5)
[33] Jonas Peters, Dominik Janzing, and Bernhard Schlkopf. Elements of Causal Inference: Foundations and Learning Algorithms. The MIT Press, 2017. ISBN 01262037399. (Cited on
page 6)
[34] OpenAL. Hello, gpt-4o! httpe://openai.cco//nee///lllloo 2024. Accessed: 2024-05-20. (Cited on page 7)
[35] Anthropic. Claude 3 family. https: //wps://wwww httplyyyhttps/////.aatntirecroco Accessed: 2024-05-20. (Cited on page 7)
2024.
[36] Meta AI. Meta Ilama 3. https://at http:/aa.eaaa.ccommee//////e comm/bloggmmett-llamm-- 2024. Accessed:
mwt..
2024-05-20. (Cited on page 7)
[37] Albert Q. Jiang, Alexandre Sablayrolles, Antoine Roux, Arthur Mensch, Blanche Savary, Chris Bamford, Devendra Singh Chaplot, Diego de Las Casas, Emma Bou Hanna, Florian Bressand, Gianna Lengyel, Guillaume Bour, Guillaume Lample, Lélio Renard Lavaud, Lucile Saulnier, Marie-Anne Lachaux, Pierre Stock, Sandeep Subramanian, Sophia Yang, Szymon Antoniak, Teven Le Scao, Théophile Gervet, Thibaut Lavril, Thomas Wang, Timothée Lacroix, and William El Sayed. Mixtral of experts. arXiv preprint, arXiv:2401.04088 2024. (Cited on page 7)
[38] Sartaj Bhuvaji, Ankita Kadam, Prajakta Bhumkar, and Sameer Dedge. Brain tumor classification using deep learning algorithms. https: / /ttpt/ggttttbbbb com/Sar Satjuuaajjj Brain-Tumee CaaerrCllsiffii wesifffiaaiionni Boro---inn-eee--Lamnnnneeee 2024-05-19. (Cited on pages 9 and 43)
thms, 2024. Accessed:
[39] Bidec Innovations. Stock price and news related to it, 2023. URL https://twwet kaggle. com/datasets detpssetta//iidmmnnvaaaaaaae 2023-10-01. (Cited on pages and 43)
stock-price-ame wwwprriieeaanneess--eeeeeeeeeeee news-real ree-tttttoooite Accessed:
[40] Gilbert P Compo, Jeffrey S Whitaker, Prashant D Sardeshmukh, Nobuki Matsui, Robert J Allan, Xungang Yin, Byron E Gleason, Russell S Vose, Glenn Rutledge, Pierre Bessemoulin, et The twentieth century reanalysis project. Quarterly Journal of the Royal Meteorological Society, 137(654):1-28 2011. (Cited on pages 9, 10, 43 and 49)
[41] Jakob Bjerknes. Atmospheric teleconnections from the equatorial pacific, Monthly weather review, 163-1216 1969. (Cited pages 10 and 51)
13

---

[42] Chunzai Wang. Enso, atlantic climate variability, and the walker and hadley circulations. In The Hadley circulation: Present. past and future, pages 173-202. Springer, 2004. (Cited on pages 10 and 51)
[43] Yinge Liu, Ninglian Wang, Lingang Wang, Zhongming Guo, and Xiaobo Wu, Variation of cloud amount over china and the relationship with enso from 1951 to 2014. International Journal of Climatology, (((8):2931-294)) 2016. (Cited on pages 10 and 51)
[44] Anoop Kumar Mishra. Investigating changes in cloud cover using the long-term record of precipitation extremes. Meteorological Applications, 26(1): 2019. (Cited on pages 10 and 51)
[45] Peter Lee, Sebastien Bubeck, and Joseph Petro. Benefits, limits, and risks of gpt-4 as ai chatbot for medicine. New England Journal of Medicine, 388(13):1233-122) 2023. (Cited on page 21)
[46] Ruibo Tu, Chao Ma, and Cheng Zhang. Causal-discovery performance of chatgpt in the context of neuropathic pain diagnosis. arXiv preprint, arXiv:2301.13819 2023. (Cited on pages 21 and 23)
[47] Judea Pearl and James M. Robins. Causal diagrams for epidemiologic research. Epidemiology. 10 1:37-48, 1999. (Cited on page 22)
[48] Shohei Shimizu, Patrik O Hoyer, Aapo Hyvärinen, Antti Kerminen, and Michael Jordan. A linear non-gaussian acyclic model for causal discovery. Journal of Machine Learning Research, 7(10), 2006. (Cited on page 22)
[49] Kun Zhang and Aapo Hyvarinen. On the identifiability of the post-nonlinear causal model. arXiv preprint 1205. 2012. (Cited on page 22)
[50] Patrik Hoyer, Dominik Janzing, Joris M Mooij, Jonas Peters, and Bernhard Schölkopf. Nonlin- ear causal discovery with additive noise models. Advances in neural information processing systems, 2008 (Cited on page 22)
[51] Biwei Huang, Kun Zhang, Jiji Zhang, Joseph Ramsey, Ruben Sanchez-Romero, Clark Glymour, and Bernhard Schölkopf. Causal discovery from heterogeneous/honstatonnn data. Journal of Machine Learning Research, 21(89): 2020. (Cited on pages 22 and 50)
[52] Karren Yang, Abigail Katcoff, and Caroline Uhler. Characterizing and learning equivalence classes of causal DAGs under interventions. In International Conference on Machine Learning, pages 5541-5550, 2018. (Cited on page 22)
[53] Philippe Brouillard, Sébastien Lachapelle, Alexandre Lacoste, Simon Lacoste-Julien, and Alexandre Drouin. Differentiable causal discovery from interventional data. Advances in Neural Information Processing Systems, pages 21865-21877, 2020. (Cited on page 22)
[54] Joris M. Mooij, Sara Magliacane, and Tom Claassen. Joint causal inference from multiple contexts. Journal of Machine Learning Research, 21(99):1-108, 2020. (Cited on page 22)
[55] Ronan Perry, Julius von Kügelgen, and Bernhard Schölkopf. Causal discovery in heterogeneous environments under the sparse mechanism shift hypothesis. In Advances in Neural Information Processing Systems, pages 10904-10917, 2022. (Cited on page 22)
[56] Daniel Malinsky and Peter Spirtes. Learning the structure of a nonstationary vector autoregres- sion. In International Conference on Artificial Intelligence and Statistics, pages 2986-2994, 2019, (Cited on page 22)
[57] Biwei Huang, Kun Zhang, Mingming Gong, and Clark Glymour. Causal discovery and fore- casting in nonstationary environments with state-space models. International Conference on Machine Learning, pages 2901-29100 2019. (Cited on page 22)
[58] Chenxi Liu and Kun Kuang. Causal structure learning for latent intervened non-stationary data. In International Conference on Machine Learning, pages 21756-21777, 2023. (Cited page 22)
14

---

[59] Zhengming Chen, Feng Xie. Jie Qiao, Zhifeng Hao, Kun Zhang, and Ruichu Cai. Identification of linear latent variable model with arbitrary distribution. AAAI Conference on Artificial Intelligence, pages 6350-6357, 2022. (Cited on page 22)
[60] Anpeng Wu, Haoxuan Li, Kun Kuang, Zhang Keli, and Fei Wu. Learning causal relations from subsampled time series with two time-slices, In International Conference on Machine Learning, 2024. (Cited on page 22)
[61] Biwei Huang, Charles Jia Han Low, Feng Xie, Clark Glymour, and Kun Zhang. Latent hierarchical causal structure discovery with rank constraints. Advances in neural information processing systems, pages 5549-5561, 2022. (Cited on page 22)
[62] Xinshuai Dong, Biwei Huang, Ignavier Ng, Xiangchen Song, Yujia Zheng, Songyao Jin, Roberto Legaspi, Peter Spirtes, and Kun Zhang. A versatile causal discovery framework to allow causally-related hidden variables. arXiv preprint, arXiv:2312.11001, 2023. (Cited on page 22)
[63] Yibo Jiang and Bryon Aragam. Learning nonparametric latent causal graphs with unknown interventions. In Advances in Neural Information Processing Systems, volume 36, pages 60468-60513, 2023. (Cited on page 22)
[64] Xiu-Chuan Li, Kun Zhang, and Tongliang Liu. Causal structure recovery with latent vari- ables under milder distributional and graphical assumptions. In International Conference on Learning Representations, 2024. (Cited on page 22)
[65] Aapo Hyvarinen, Hiroaki Sasaki, and Richard Turner. Nonlinear ica using auxiliary variables and generalized contrastive learning. In International Conference on Artificial Intelligence and Statistics, pages 859-868, 2019. (Cited on page 22)
[66] Ilyes Khemakhem, Diederik Kingma, Ricardo Monti, and Aapo Hyvarinen. Variational autoencoders and nonlinear ica: A unifying framework. In International Conference Artificial Intelligence and Statistics, pages 2207-2217, 2020. (Cited on page 22)
[67] Francesco Locatello, Ben Poole, Gunnar Raetsch, Bernhard Schölkopf, Olivier Bachem, and Michael Tschannen. Weakly-supervised disentanglement without compromises. In International Conference Machine Learning, pages 6348 6359, 2020. (Cited on page 22)
[68] David Klindt, Lukas Schott, Yash Sharma, Ivan Ustyuzhaninov, Wieland Brendel, Matthias Bethge, and Dylan Paiton. Towards nonlinear disentanglement in natural data with temporal sparse coding, 2021. (Cited on page 22)
[69] Sebastien Lachapelle, Pau Rodriguez, Yash Sharma, Katie E Everett, Rémi LE PRIOL, Alexandre Lacoste, and Simon Lacoste-Julien. Disentanglement via mechanism sparsity regularization: A new principle for nonlinear ICA. In Conference on Causal Learning and Reasoning, pages 428 484, 2022. (Cited on page 22)
[70] Kartik Ahuja, Jason S Hartford, and Yoshua Bengio. Weakly supervised representation learning with sparse perturbations. In Advances in Neural Information Processing Systems, pages 15516-15528, 2022. (Cited on page 22)
[71] Kartik Ahuja, Divyat Mahajan, Yixin Wang, and Yoshua Bengio. Interventional causal representation learning. In International conference machine learning, pages 372-407,
2023. (Cited on page 22)
[72] Jiaqi Zhang, Kristjan Greenewald, Chandler Squires, Akash Srivastava, Karthikeyan Shan- mugam, and Caroline Uhler. Identifiability guarantees for causal disentanglement from soft interventions. Advances in Neural Information Processing Systems, 2024. (Cited on page 22)
[73] Julius von Kügelgen, Michel Besserve, Liang Wendong, Luigi Gresele, Armin Kekić, Elias Bareinboim, David Blei, and Bernhard Schölkopf. Nonparametric identifiability of causal representations from unknown interventions. Advances in Neural Information Processing Systems, pages 48603-4888 2023. (Cited on page 22)
15

---

[74] Chandler Squires, Anna Seigal, Salil S Bhate, and Caroline Uhler. Linear causal disentangle- ment via interventions. In International Conference on Machine Learning, pages -25400 2023. (Cited on page 22)
[75] Simon Buchholz, Goutham Rajendran, Elan Rosenfeld, Bryon Aragam, Bernhard Schölkopf, and Pradeep Ravikumar. Learning linear causal representations from interventions under general nonlinear mixing. Advances in Neural Information Processing Systems, 2024. (Cited on page 22)
[76] Yonggiang Chen, Yonggang Zhang, Yatao Bian, Han Yang, Kaili Ma, Binghui Xie, Tongliang Liu, Bo Han, and James Cheng. Learning causally invariant representations for out-of- distribution generalization on graphs. In Advances in Neural Information Processing Systems, 2022. (Cited on page 22)
[77] Yongqiang Chen, Yatao Bian, Kaiwen Zhou, Binghui Xie, Bo Han, and James Cheng. Does invariant graph learning via environment augmentation learn invariance? In Advances in Neural Information Processing Systems, 2023. (Cited on page 22)
[78] Yongqiang Chen, Yatao Bian, Bo Han, and James Cheng. Interpretable and generalizable graph learning via subgraph multilinear extension. In ICLR 2024 Workshop on Machine Learning for Genomics Explorations, 2024. (Cited on page 22)
[79] Yongqiang Chen, Kaiwen Zhou, Yatao Bian, Binghui Xie, Kaili Ma. Yonggang Zhang, Han Yang. Bo Han, and James Cheng. Pareto invariant risk minimization. arXiv preprint, arXiv:2206.07766, 2022. (Cited on page 22)
[80] Yonggiang Chen, Wei Huang, Kaiwen Zhou, Yatao Bian, Bo Han, and James Cheng. Under- standing and improving feature learning for out-of-distribution generalization. In Advances in Neural Information Processing Systems, 2023. (Cited on page 22)
[81] Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Carroll L Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to follow instructions with human feedback. arXiv preprint arXiv:2203.02155, 2022. (Cited on page 22)
[82] Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. Training verifiers to solve math word problems. arXiv preprint, arXiv:2110.14166 (Cited on page 22)
2021.
[83] Nouha Dziri, Ximing Lu, Melanie Sclar, Xiang Lorraine Li, Liwei Jiang, Bill Yuchen Lin, Sean Welleck, Peter West, Chandra Bhagavatula, Ronan Le Bras, Jena D. Hwang. Soumya Sanyal, Xiang Ren, Allyson Ettinger, Zaid Harchaoui, and Yejin Choi. Faith and fate: Limits of transformers on compositionality. In Thirty-seventh Conference on Neural Information Processing Systems, 2023. (Cited on page 22)
[84] Bingbin Liu, Jordan T. Ash, Surbhi Goel, Akshay Krishnamurthy, and Cyril Zhang. Trans- formers learn shorteuts to automata. The Eleventh International Conference on Learning Representations, 2023. (Cited on page 22)
[85] Xuan Li, Zhanke Zhou, Jianing Zhu, Jiangchao Yao, Tongliang Liu, and Bo Han. Deepincep- tion: Hypnotize large language model to be jailbreaker, 2024. URL https://arxivv abs/2311. 03191. (Cited on page 22)
[86] Zhanke Zhou, Rong Tao, Jianing Zhu, Yiwen Luo, Zengmao Wang, and Bo Han. Can large language models reason robustly with noisy rationales? In ICLR 2024 Workshop on Reliable and Responsible Foundation Models, 2024. (Cited on page 22)
[87] Brett Drury, Hugo Gonçalo Oliveira, and Alneu de Andrade Lopes. A survey of the extraction and applications of causal relations. Natural Language Engineering, 28:361 - 400, 2022. (Cited page 22)
16

---

[88] Shaobo Cui, Zhijing Jin, Bernhard Schölkopf, and Boi Faltings. The odyssey of common- sense causality: From foundational benchmarks cutting-edge reasoning. arXiv preprint arXiv:2406. 19307, 2024. (Cited on page 22)
[89] Pedram Hosseini, David A Broniatowski, and Mona Diab. Predicting directionality in causal relations in text. arXiv preprint
2021. (Cited on page 22)
[90] Lei Gao, Prafulla Kumar Choubey, and Ruihong Huang. Modeling document-level causal structures for event causal relation identification. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume I (Long and Short Papers), pages I 1808-1817, 2019, (Cited on page 22)
[91] Noah Weber, Rachel Rudinger, and Benjamin Van Durme. Causal inference of script knowl- edge. arXiv preprint arXiv:2004.01 174, 2020. (Cited on page 22)
[92] Xiao Liu, Da Yin. Yansong Feng, Yuting Wu, and Dongyan Zhao. Everything has a cause: Leveraging causal inference in legal text analysis. arXiv preprint arXiv:2 104.09400 2021. (Cited on page 22)
[93] Andrew Kyle Lampinen, Stephanie C.Y. Chan, Ishita Dasgupta, Andrew Joo Hun Nam, and Jane X Wang. Passive learning of active causal strategies in agents and language models. In Advances in Neural Information Processing Systems, 2023. (Cited on page 22)
[94] Kristy Choi, Chris Cundy, Sanjari Srivastava, and Stefano Ermon. Lmpriors: Pre-trained language models as task-specific priors. arXiv preprint, arXiv:2210. 12530, 2022. (Cited on pages 22 and 23)
[95] Stephanie Long, Alexandre Piché, Valentina Zantedeschi, Tibor Schuster, and Alexandre Drouin. Causal discovery with language models as imperfect experts. arXiv preprint, arXiv:2307.02390 2023. (Cited on pages 22 and 23)
[96] Taiyu Ban, Lyuzhou Chen, Xiangyu Wang, and Huanhuan Chen. From query tools to causal architects: Harnessing large language models for advanced causal discovery from data. arXiv preprint, arXiv:2306. 16902, 2023. (Cited on pages 22 and 23)
[97] Moritz Willig, Matej Zečević, Devendra Singh Dhami, and Kristian Kersting, Can foundation models talk causality? In UA/ 2022 Workshop on Causal Representation Learning, 2022. (Cited page 22)
[98] Stephanie Long, Tibor Schuster, and Alexandre Piché. Can large language models build causal graphs? arXiv preprint, arXiv:2303.05279, 2023. (Cited on page 22)
[99] Zhiheng LYU, Zhijing Jin, Rada Mihalcea, Mrinmaya Sachan, and Bernhard Schölkopf. Can large language models distinguish cause from effect? In UAI 2022 Workshop on Causal Representation Learning, 2022. (Cited on page 23)
[100] Yanming Zhang, Brette Fitzgibbon. Dino Garofolo, Akshith Kota, Eric Papenhausen, and Klaus Mueller. An explainable AI approach to large language model assisted causal model auditing and development. arXiv preprint, arXiv:2312.162111 2023. (Cited on page 23)
[101] Lianmin Zheng, Wei-Lin Chiang, Ying Sheng. Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric Xing, et al. Judging llm-as-a-judge with mt-bench and chatbot arena. Advances in Neural Information Processing Systems, 36:46595- 46623, 2023. (Cited on page 40)
[102] Daniel Lyndon, Joseph A Lansley, Jane Evanson, and Anant S Krishnan. Dural masses: meningiomas and their mimics. Insights into imaging, 10(1):11, 2019. (Cited on page 44)
[103] Nisreen Haydar, Khatoun Alyousef, Usama Alanan, Rana Issa, Fawaz Baddour, Zuheir Al- Shehabi, and Moatasem Hussein Al-Janabi. Role of magnetic resonance imaging (mri) in grading gliomas comparable with pathology: A cross-sectional study from syria. Annals of Medicine and Surgery, 82:104679, 2022. (Cited on page 44)
17

---

[104] Birkan Tunç, David A Hormuth II, George Biros, and Thomas E Yankeelov. Modeling of glioma growth with mass effect by longitudinal magnetic resonance imaging. IEEE Transactions on Biomedical Engineering, 68(12)33713-3724 2021. (Cited page 44)
[105] Watts, Box, A Galvin, P Brotchie, N Trost, and T Sutherland. Magnetic resonance imaging meningiomas: pictorial review. Insights into imaging, 5:113-122, 2014. (Cited on page 44)
[106] John Markoff. Competing software goes to web. The New York Times, 2007. URL ht
imes. wttm///01/666///tcccoooogg////// 03-14. (Cited page 47)
html. Accessed: 2024-
[107] Fama Eugene. Efficient capital markets: review of theory and empirical work. Journal of finance, 25:383 417, 1970. (Cited on page 47)
[108] Jessica A Wachter. Can time-varying risk of rare disasters explain aggregate stock market volatility? The Journal of Finance, -87-)3887 2013. (Cited on page 47)
[109] Leland Bybee, Bryan Kelly, Asaf Manela, and Dacheng Xiu. Business news and business cycles. Journal of Finance, Forthcoming, 2023. (Cited page 47)
[110] Michael McPhaden. Genesis and evolution of the 1997-98 el niño. Science, 283(5404): 950-954, 1999. (Cited on pages 49 and 50)
[111] Chester F Ropelewski and Michael S Halpert. Global and regional scale precipitation patterns associated with the niño/southern oscillation. Monthly weather review, 1166))1666-1166 1987. (Cited on page 49)
[112] Wenju Cai, Simon Borlace, Matthieu Lengaigne, Peter Van Rensch, Mat Collins, Gabriel Vecchi, Axel Timmermann, Agus Samtoso, Michael McPhaden, Lixin Wu, et al. Increasing frequency of extreme el niño events due to greenhouse warming. Nature climate change, 4(2):
111-116, 2014. (Cited on page 49)
[113] Chunzai Wang, Clara Deser, Jin-Yi Yu, Pedro DiNezio, and Amy Clement. El niño and southern oscillation (enso): review. Coral reefs of the eastern tropical Pacific: Persistence and loss in a dynamic environment, pages 85-106, 2017. (Cited on page 49)
[114] Joris M Mooij, Sara Magliacane, and Tom Claassen. Joint causal inference from multiple contexts. Journal of Machine Learning Research, 21(99):1-108, 2020. (Cited on page 50)
[115] Yi Ge Zhang, Mark Pagani, Jorijntje Henderiks, and Haojia Ren. A long history equatorial deep-water upwelling in the pacific ocean. Earth and Planetary Science Letters, 467:1-9, 2017. ISSN 0012-82ix. (Cited on page 50)
[116] J. Tarazona and W. Amttz The Peruvian Coastal Upwelling System, pages 229-244 Springer Berlin Heidelberg, 2001. (Cited page 51)
18

---

Appendix of COAT
Contents
A Table of Notations
20
Limitation and Future Opportunities
21
C Related Work
22
Proofs for Theoretical Results
23
D.I Proof for Proposition 2.2
23
D.2 Proof for Proposition 2.4
23
More Details about Experiments
25
E.1
More Details on Constructing AppleGastronome
25
E.2
More Details on Prompts for AppleGastronome
................................
25
E.3 More Details of Results on AppleGastronome
................................
26
E.4 The Full Result on the Apple Gastronome Benchmark
.ineca..........................
29
E.5
Implementation of the FCI algorithm
29
E,6
More Details on Constructing Neuropathic
37
E.7
More Details of Results on Neuropathic
................................
37
E.8 Discussion on the time complexity
37
E.9 Resources
38
COAT with Different Causal Discovery Algorithm
40
Ablation Study
40
Summary of Benchmark Data
43
I Case Study Brain Tumor
43
I.1 The Brain Tumor Dataset
43
1.2 Result and Discussion
44
J
Case Study Stock News
45
J.1 The Stock News Dataset
J.2 Result and Discussion
46
K Case Study on El Niño-Southern Oscillation (ENSO)
49
K.1 Setting and Data processing...
49
K.2 Causal Graph and Discussion
50
Broader Impacts
53
19

---

A
Table of Notations
Notation
Description
V
the LLM for factor proposal.
p'
Prompt used at the f-th round for factor proposal.
,
the LLM for factor parsing.
Pp
Prompt used for factor parsing.
A
Algorithm causal discovery. this paper we mainly use FCI.
F
Abstraction the COAT's Feedback procedure.
D
Full sample set.
D
A subset of sample set
D
The subset of sample used at the t-th round.
C
The predefined value space for L.ro-popeeee factors, c.g., set as (-1,0,
æ
An unstructured sample raw observation drawn from
y
A structured sample of larget variable drawn from y
Y
target variable treated as random variable.
X
raw observation treated as random variable.
w;
The i-th factor proposed by LLMs, like sweetness, size, or scent
w..-)
Factor's corresponding mapping from raw ohservation space the predefined value space C.
W
The set of factors proposed by I.LSs the f-th round.
WS
The set of factors proposed by LLMs the first i rounds.
S
Index set of selected factors in W. e.g., S (1,3) means only select the first and third factors.
LLM-induced representation over selection S, e.g., (-)) (w=1()) wa)))) h(-) = h-n(() hs(-)
Z
Matrix of values of the structured high-level factors proposed by LLMs, i.c., Coxe
z
Matrix of values of structured high-level factors proposed at 1-th COAT round.
z
Matrix of values of the structured high-level factors proposed the first & COAT rounds.
g'
Causal graph at the t-th COAT round over ~ (Y}
k
Number of factors
n
Number of samples
C
Number clusters
C+
Capacity Score of LLM W
Py or p
Perception Score of LLM V
20

---

Application Level
Marketing Analysis
Discase Understanding
Financial Asset Pricing
Task Level
Causal Machine Learning
Treatment Effect Estimation
Counterfactual Reasoning
Causal Foundation Model
Causal Graph
LLMs
COAT: Causal
Causal Discovery
Representation Assistant
Raw Observations from the Real World
Figure 8: COAT combines both strengths of LLMMs that learn the rich knowledge the world, and causal discovery methods to uncover the hidden causal world. With the uncovered causal knowledge. COAT can empower broader applications of causal methods.
B Limitation and Future Opportunities
Future Opportunities Despite the progress in the past decades, existing causal discovery algo- rithms mainly rely on high-quality measured variables given by human experts [3, 4, 5]. However, the causal variables and their measurements applications. For example, Amazon sellers who want to analyze the factors related to user ratings only have user reviews, generated by the underlying user preferences for certain product characteristics. Therefore, the lack of measured high-quality causal variables has been a major impediment to broader real-world applications of causal
usually available in wide range of real-world
methods [6].
This work establishes and demonstrates a preliminary implementation of the system Causal repre- sentatiOn AssistanTT (COAT). We present comprehensive evaluations of COAT and find plentiful evidence that the recent emergence of Large Language Models (LLMs) [7, 8, 9, 10] has a great potential to mitigate the gap. In particular, towards causal foundation models, which consists of two mutually beneficial components: LLMs and causal discovery methods (CDs). On the one hand, LLMs that learn rich world knowledge about the world can assist with discovering high-level hidden variables from low-level observational data [12], certain commonsense causal knowledge [11, 13, 14]. On the other hand, it has also been found that the reliability of LLMs in the reasoning of causality remains a debate [13, 15, 16, 17], due to a series of drawbacks of LLMs [18, 19, 20], well as the ethical considerations [45,46] 46). Nevertheless, CDs can uncover the underlying causal relations between the discovered high-level variables with guarantees. More importantly, CDs can also provide feedback to improve the identification of the variables. The combination of LLMs and CDs iteratively improves the discovery of the hidden causal world, and hence opens up broader adoption of various causal methods.
shown in Fig. 8, envision an entangled system
Limitations
We would like to discuss some technical limitations of the current version of COAT.
Multicollinearity. Sometimes, LLM may output factors with shared similar semantic meanings or overlapped factor-value criteria. The consequence this phenomenon is that multicollinearity could occur in the resulting structured data, the data matrix (each sample per row) would not be full column rank. This would hinder the conditional independent adopt numerical methods like PCA to drop similar factors. More
tests. In experiments, sophisticated methods could be investigated in the future.
Toward complete causal graph. In this work, we mainly focus on one single target variable and identify a set of factors to serve Markov blanket of this target variable. Indeed, one can explore a more complete causal graph by applying COAT procedures on those identified factors to acquire more comprehensive causal graph. One may also introduce multiple target variables let L.LM define suitable target variable (and also parse it out) from pure raw observations. These extensions also important, and we leave it to future work.
21

---

Meanwhile, identifiable causal discovery also requires certain assumptions about the data:
Faithfulness. The empirical distribution of the data reflects the actual data-generating process.
No selection bias. Otherwise, the condition of faithfulness would be violated.
Enough sample size. Our method involves statistical tests; the more, the better.
C
Related Work
Causal discovery aims to discover the unknown causal relations from the observational data [3, 4]. which is critical to both real-world applications and scientific discoveries [23, 47]. We use FCI [27] in this paper just to illustrate the idea. To attain identifiability better than the Markov equivalent class, one can choose more advanced Causal Discovery methods under different assumptions, like constrained functional [48, 49, 50], multiple domain data [51, 52, 53, 54, 55], and non- non-stationary data [51, 56, 57, 58]. There are also some works aiming to address realistic challenges, like structure models with arbitrary distribution [59], subsampled time series [60], and latent hierarchical causal structure [61, 62, 63, 64]. Despite recent theoretical and empirical improvements [5, 28), most existing causal discovery approaches rely well-structured data with human-crafite factors as inputs. They can easily suffer from the low quality of annotated data (e.g., latent confounders).
Causal representation learning aims to establish provable identifiability for latent high-level vari- ables (like location and color of an object) from low-level observations such as images [6]. Such identification can be feasible with certain conditions, like auxiliary information [65, 66, 67]. spar- sity [68, 69, 70], or interventional data [71, 72, 73, 74, 75]. Recent works also generalize the causal representation learning to rraph-structured data [76, 77, 78], and discuss the optimization challenges in realizing causal representation learning [79, 80]. In this work, we show that incorporating LLMs that learn world knowledge from massive training data could effectively relieve the need for artificial causal factor annotation. Meanwhile, COAT also opens up a new line to learn causal representations with rich pre-trained knowledge as well as feedback from causal discovery algorithms.
Reasoning with LLMs has achieved remarkable performance across a variety of tasks with few demonstrations of the samples [7, 8, 10, 81]. The strong capabilities of LLMs show that it is evident that LLMs could acquire and understand commonsense knowledge about the world [12]. The power of LLMs can be further unlocked with suitable context as inputs [29]. Nevertheless, LLMs have also been found to make mistakes in basic algorithmic reasoning [82, 83], easy to hallucinate nonfactual results [18], tend to learn shortcuts or dataset biases [19, 20, 84), vulnerable to adversarial jailbreaks or noisy interruptions [85, 86), and fall short in complex planning and reasoning tasks [12]. The drawbacks of LLMs render it risky to rely on the direct LLM reasoning results to derive any rigorous results. Therefore, we do not directly derive the results from LLMs. Rather, we merely leverage the learned world knowledge in LLMs to find useful causal factors by constructing proper instructions based on causal feedback.
Text mining of causal relations aims to analyze the causal relations implied by the semantics given a text [87, 88, 89]. particular, some works focus on the identification of causal relations among events specified in documents [90, 91, 92]. Different from these tasks, where factors are entities or events described by text, in this work, COAT emphasizes crafting high-level factors that go beyond the unstructured data like text description and also establish identifiability on the Markov blanket for a given target variable. Note that the target variable allowed to be not mentioned the text or other unstructured data. Therefore, this work brings new scope and opportunities for these text-understanding tasks.
Causal learning with LLMs has received muchattention results from the community [11, 13]. Kiciman et al. [11] find that LLMs can recover the pairwise causal relations very well. Lampinen et al. [93] show that transformer based agents can learn causal strategies passively if intervention is allowed during tests. Abdulaal et al. [14], Choi et al. [94], Long et al. [95], Ban et al. [96] propose to incorporate the causal discovery results by LLMs as a prior or constraint to improve the performance of data-driven causal discovery algorithms. However, Zečević al. [15]. Willig et al. [97] find that
LLMS can not understand causality but simply retell the causal knowledge contained in the training data. Zhang et al. [13], Jin et al. [16, 17] find that LLMS can hardly provide satisfactory answers for discovering new knowledge or decision pecision-making tasks. Although Long et al. [98] find that LLMs can
22

---

build causal graphs with 3-4 nodes, Tu et al. [46] find that the performance of LLMs in more complex causal discovery remains limited as LLMs can hardly understand new concepts and knowledge. The aforementioned debate implies the limitations in directly adopting the causal discovery results by LLMMs, which motivates us to incorporate the existing causal discovery algorithms with rigorous guarantees instead of LLMSs to learn the causal relations.
LYU al. [99] find that it is crucial incorporate prompts aligned with the underlying causal story for LLMs to do pairwise causal relation inference. Zhang et al. [100] propose to leverage LLMs to audit causal discovery results in an explainable way.
The closest works to ours are Abdulaal et al. [14], Choi et al. [94], Long et al. [95]. Ban et al. [96] which also incorporates LLMs into the pipeline of causal discovery. Nevertheless, all of the existing combinations of LLMs and causal discovery still focus on artificially curated structured data and rely on the capability of LLMs infer causal relations, therefore, limited in both the reliability and the utility of LLMs in causal learning.
D Proofs for Theoretical Results
D.1 Proof for Proposition 2.2
Proposition D.1 (Restatement of Proposition 2.2). Given the current representation as hig (wi (X), w2 ... ,00 (X)), and a new factor wh+1 satisfying
=
Y y (x+ (X),
(12)
for Markov Blanket S [ks 1] ofY, ie..
Y 1 hik+)))s(() hss())
(13)
we have the following result about conditional mutual information:
1Y;; X hs(()) (Y; X (x)= < 1 (Y; 1ap (X))
(14)
Proof for Prop. 2.2. From Eq. 12, we have
H (y h(n (x) (x)) Hy (x));
(15)
From Eq. 13, we have
Hy hs(l)) h H hs(x)))
(16)
Therefore:
H hs(())) = Hy (ss()),, (1
Hy 11 (z))
(17)
H (y hx)(()) wt (x))
Hy (x))
Also note that
Hy hs(z), H (y x) H (y (x),
(18)
therefore:
Hly hs(=)) - Hyy hs(æ), v) < H (x)()) H(y (zg()),
(19)
which is Eq. 14.
D.2 Proof for Proposition 2.4
Proposition D.2 (Restatement of Proposition 2.4). Given Assumption 2.3, for any small number ê E (0, }), with sufficiently rounds of COAT, i.e..
loge
1++1
(20)
2VD
25
p) log Cy
where 25 is the 5-quantile of the standard normal distribution, we have
I (Y;X) he((X))
Pr
2) 2 1-66. 8.
(21)
((Y;X)
23

---

Proof Prop. 2.4. Let n, be the number of rounds that LLM proposed at least one usable factor satisfying Assumption 2.3. Since large. its Binomial distribution Binom(t, p) can be approximated by Gaussian distribution N (1p,1p(1 p)). To enforce
1 (1/3 x hg)(
<6 2 Pr (1- Co) <の)
Pr
7 (y: x))
loge
Pr
143 >
log (1 Cw)
(22)
n2 tp
loge
- t1))
Pr
ttp(1 p)
/tp(1 - p)
log (1 Cvv)
2 1 - 6.
we have
1
loge
tp
28 -128l
(23)
tt - p) log (1 Cv))
with Gaussian distribution approximation.
Isolating vt 0 from the above inequality, would have the desired result.
24

---

Table 3: Causal relation extraction results in AppleGastronome.
LLM
METHOD
SHD
SID
RECALL
PRECISION
FI
GPT 40
PAIRWISE
5.33::262
0.00:10.00
1.00:t0.00
0.45++007
0.62:t0.7
COAT
1.00
0.00:10.00
1.00:10.00
0.79:1008
0.89=a02
GPT 4
PAIRWISE
8.67
1.89
0.93:10.09
0.35
1:006
COAT
2.33+0447
1.00:
0.93++0.09
0.70++0.09
0.79=0044
3.5
PAIRWISE
5.67+4.03
3
3.00:+2.94
0.67:10.24
0.49:40.36
0.55=0.32
COAT
2.00
2.67::..05
0.75+t0.20
0.63
0.68=0221
MISTRAL-LARGE
PAIRWISE
1.89
0.00:1:.00
1.00:10.00
0.38
0.55=0008
COAT
2.00+10.00
1.33
0.85
0.7411005
0.78=002
MISTRAL-MEDIUM
PAIRWISE
1.00
2.00:10.00
1.51
0.37
0.421007
COAT
0.33+0047
0.00:10.00
1.00:+0.00
0.94+0008
0.97=004
LLAMA-3- ---00
PAIRWISE
4.00#1141
0.33+0447
0.92+0.12
0.51
0.65=0.11
COAT
2.33:2047
3.67
0.75++000
0.70+00077
0.72=004
LLAMA LLAMA-2-70B 2-70B
PAIRWISE COAT
3.33
1.00=+1.41
0.89:0.16
0.48+0.11
0.62=0.13
1.00
0.33
0.89:10.16
0.81
0.84
QWEN- OWEN-1.5-1 10B-
PAIRWISE COAT
6.67
2.33
0.62:10.03
0.38.10.99
0.47:1007
4.00
7.00:14.22
0.47:1.34
0.47:10.4
0.70=n.10
DREPSEEK- V2
PAIRWISE COAT
5.00
1.89
0.93
0.47:0.05
0.621100
1.33+1125
2.33
0.85+0.11
0.87+0.19
0.85=0.14
CLAUDE-3-OPUS
PAIRWISE
7.33
0.33+0.47
0.93++009
0.40+005
0.56+0007
COAT 1.33
0.00
1.00:10.00
0.7911006
0.88=n04
Table 4: Independence tests of the annotation noises with annotated features and other noises AppleGastronome.
LLM
TEST OBJECT
T
P-VALUE
FEATURE
0.2828
0.9997
GPT 4
NOISE
0.0327
0.9962
FEATURE
0.4803 0.0446
0.0325
GPT 3.5
NOISE
0.9962
E More Details about Experiments
E.1
More Details Constructing AppleGastronome
In the AppleGastronome benchmark, we consider the target variable as rating score of the apple by several gastronomes. Each apple has its attributes, including size, smell, and taste (or sweetness). Each gastronome has a unique preference for some attributes of the apple. They will give and rating well write review according the matchness of the apple with respect their preference. We generate the review using GPT 4 by fetching GPT the preferences and the apple attributes.
The prompts for generating the unstructured inputs are given in Fig. 9, The additional results on Relation Extraction are given in Table. 3.
Examples of AppleGastronome are given in Fig. 10.
E.2 More Details Prompts for AppleGastronome
The prompts for factor proposal are given in Fig. 11.
The prompt for factor annotation is given in Fig. 12.
25

---

picky carceoma applas. You are raady to eralaate apples and vzita reviees. your writing should be clear, convinsing rupplieca
Task
Cleaze waite zuviex about the
esslustior reculee for given spple.
Reslustion Reeulte:
high consen:
aveat
cryy and lacking Potecure
night not bring the expected neturre and cocld to laszes
Corbine those avalaation resulta
Fore detailed reasening.
Single pazectaph: Ne quesation racta. Hodern English
Degis pazt by [Reciex Segin].
Figure 9: Illustration of prompts for generating AppleGastronome.
Sooze:
The asseised apple
offering
distinct green drasberes, apple wariety elggeeee
evecahadova
cariety seecing buyara, posing finaccia risk for suplistr.
bespite the distentive atatore, tata tople talls to sast ever. basts qualsty expectations, the presence of austy or rocten acent sigrals underlying cssues that can selther be overlocked by discerning consueers nor excused by vendors seeking profit, Ultinately, stooking this fruit claky venture that's das zore likely incur insses than visis any anticipatad zutamna.
Soore:
variety, rich essencial nuteients Like dietary fiber and vitamin e, ixpreanes with ito large size and delightfel atrong arona. Ita balaneed flavor, leaning rode towards sweet than scut, and abundens zetzeahing gosature quacancen delectabls eating expeziance. Zaia applete signt fioant narket porentiay al to evident, and it certatnly reries brorder aoclain for its highogualsty sterlbuter.
Figure 10: Illustration of examples in AppleGastroonom.
The prompts for constructing feedback are given in Fig. 13.
E.3
More Details of Results on AppleGastronome
The detailed causal graph results are given from Fig. 14 to Fig. 18.
Independent tests about annotation the Apple Gastronome benchmark are shown in Table 4.
26

---

excellantly helpful
assistant for analyzing, abstracting, and processing data. Now Lry your best.
Rating=0
Rating=1
Rating=2
Tasks: Factor Abatraction. what the high-level factors
the text that contribute to the allocation of groups?
Propoze many possible factors based your knowledge,
Adjust your candidate factora by your observation on given data. should diverse and different semantically. Their overlapping should minimized.
proposed
Figure 11: Illustration of the prompt for factor proposal.
lently helptal
andlyx' ngy abstraccing, and processing data,
try best.
Data
apple
randenly picked:
[test)
Tesks
the given is appreeiata factoz based criterion?
[factor_statee _tatte
*our finel purput whould follos this templacate:
value 1st
Figure 12: Illustration of the prompt for factor annotation.
27

---

Dsta
sgroups of semples>
Abstraction.
**ibat ace the highrievel factora behind the Lext that contzibute the allacation groups?**
Propose your candidate factors based so your obaecvation on given data. The progosed factors should be diverse and di Fferent senantion ly. Their overlapping should nininised.
About Output
oucput should
parta described follows.
**part 1et; Considesalion.
part, feel free write down the procees of your considerat Binks You need to abatracty iduntify, and design suitable facuora with corresponeing critersa, and each factor is only lowed to value Fron [-1, 11
*Paatt 2 Factoz filtration. You should decide whethez to use each of the factors rolleving criteris:
Bach new factor should be helptul to distinguich the groups.
concrete and to avold Rach Facuor should focus
overlappine.
Avoid over apping with those exiating factores
sice
Figure 13: Illustration of the prompt for feedback.
28

---

E.4 The Full Result on the Apple Gastronome Benchmark
Full Results on the Apple Gastronome Benchmark are shown in Table
Full Results of Causal Metrics each Round each LLM on Apple Gastronome Benchmark are shown in Table 6.
Table 5: Full Results on the Apple Gastronome Benchmark.
LLMS
METHOD
MB
NMB
OT
RECALL
PRECITION
F1
GPT
META
3.00::0.2
1.00+000
4.00:
0.60++a.6
0.37
0.46=0.13
DATA
3,00:1000
0.67:107
0.00::400
0.60:a00
0.83
0.69=000
DATA+COT
4.67
1.00
0.33+0.58
0.93+a11
0.78
0.85=0.10
COAT
4.00=10.2
0.00
0.00++000
0.80+a.16
1.00=0.00
0.88=0.10
GPT 4
META
2.67:004
0.67:004
2.33
1.
0.46:2008
0.49
DATA
3.00=10.00
0.33:
0.00++000
0.60+4000
0.92
0.72=00.44
DATA+COT
4.33-10.58
0.83+
0.
0.87+aa12
0.8 =0002
0.84
COAT
4.00:10.82
0.33:40.47
0.00:10.00
0.80+.0.66
0.93
0.85
GPT 3.5
META
3.3
0.33:
4.
0.67
0.42
0.51
DATA
2.67=004
0.67=0047
0.00++0000
0.53
0.8 1.014
0.64=0.10
DATA+COT
5.00:10.00
1.00
1.33:10.58
1.00+:0.00
0.68:10.05
0.81
COAT
3.67
+.00
0.00++0.00
0.73++0.9
1.00
0.84::0.7
MISTRAL-LARGE
META
2.00=10.22
0.67
2.
0.40+
0.37=2009
0.38=0.12
DATA
3.00:10.00
0.33
0.00+10.00
0.60++0.00
0.92
0.72=00.0
DATA+COT
4.33:10588
1.00
0.67 +0.58
0.87++a.12
0.73=0007
0.79
COAT
4.3320017
0.00
0.00:10.00
0.87:10.9
1.00::0000
0.93=20.05
MISTRAL-MEDIUM
META
3.00:10.00
0.67=10.7
1.67++..25
0.60+-0.00
0.59
0.59:
DATA
3.00:10.00
0.67
0.00++0.00
0.60++0.00
0.83
0.69
DATA+CoT
4.33:10.88
1.00
0.67:10.58
0.87:1012
0.73=0007
0.79:0005
COAT
4.67
0.00
0.0040000
0.93+40.9
1,00
0.96
LLAMA-3-70B
META
2.67
0.33
4.67
0.53++0.09
0.35
0,42
DATA
2.67:11.25
0.33:10.47
0.00:10.00
0.53
0.93:1009
0.63:20.21
DATA+CoT
2.67
0..78
1.004:.000
0.53
0.62
0.57
COAT
3.67:0047
0.33:1047
0.00:1000
0.73
0.93
0.81
LLAMA-2-70R
META
2.332047
0.67
4.67:+1.25
0.47-40.9
0.32=0007
0.37:0066
DATA
2.33:1094
0.67
0.00+:0.0
0.47-40.1
0.75
0.57
DATA+COT
3.00:11.73
0.67:10.5
0.33
0. 0.600s
0.71
0.65
COAT
3.00:10.00
0.67:10.7
0.00::0.00
0.601:.00
O. 0.83:10.12
0.69:0.01
QWEN-1.5-110B
META
2.00:10.00
1.00
4.000:0.00
0.40-+0.00
0.29:
0.33
DATA
3,00:10.22
1.00
0.00++0.0
0.60++..6
0.74
0.66
DATA+COT
3.67:11.53
0.67710.50
0.001:0.00
0.73
0. 0.83=2017
0.77=0023
COAT
4.00:10.2
0.33
0.00+:0000
0.80+aa 16
0.9 0.939
0.85
DEEPSEEK-VE -22
META
2.33
1.00
3.00+:0.82
0.47
0.37
0.41
DATA
3.33:0017
0.67:10.7
0.00:10.0
0.67
O. 8..11
0.74
DATA+COT
5,00:1000
1. 1.0000000
0.675
1.001..00
0.75
0.86
COAT
3.67:0047
0.67
0.00:tm00
0.73
0.87=1009
0.78=0.02
CLAUDE-3-OPUS
META
2,00.10.00
1.00:1.00
3. 3.00:1:..41
0.40:1:.00
0.35=0.07
0.37=0001
DATA
3.33
0.33:10.17
0.001.0.00
0.67:10.99
0.93
0.77
+ATA+COT
2.67-10.5
0.33-20.58
1. 1000+:7.00
0.53:ta
0.67
0.59
COAT
5,00:10.00
0.33
0.00:10.00
1.00+:.00
0. 0.94:10.08
0.97=0.01
DeepSeek-V V2 is more available when we add this baseline, therefore, we use DeepSeek- V2.5 instead.
E.5 Implementation of the FCI algorithm
We use a third-party open-sourced Python library to perform the FCI algorithm: leam..eadttteooossi////aasss///
https://causal-
set a = 0.05 and dadepennencee.testmeehhoee
hroughout all experiments.
Other parameters are kept as the default.
29

---

Table 6: Full Result of Causal Metrics each Round each LLM on Apple Gastronome Benchmark
LLMs
COAT ROUND
PERCEPTION SCORE
CAPACITY SCORE
GPT 40
ITER
0.82+002
0.24
0.28:0002
ITER 2
0.44
0.26#0002
0.21:0003
ITER 3
0.38-10.12
0.19
0.24:1005
GPT 4
ITER
0.78++0.02
0.18:00.2
0.45:10.04
ITER 2
1.008
0.26
0.28:1005
ITER
0.39+00228
0.22=#0.00
0.29:+0.2
GPT
ITER
0.81
0.21 10.0
0.42:10.05
ITER 2
0.32++0.11
0.23+0002
0.31=00,44
ITER 3
0.38
0.2 10008
0.33
MISTRAL-LARGE
0.78+
0.20=
0.41=:004
ITER 2
0.61
0.20=2003
0.29:1002
ITER 3
0.33
0.32+0000
0.25
MISTRAL-MEDIUM
ITER
0.78
0.20
0.39:1005
ITER 2
0.3
0.19=0.10
0.32+10004
ITER 3
0.36
0.37
0.25
LLAMA-3-70B
ITER
0.75+00.0
0.19=0.02
0.47
ITER
0.36
0.19:0000
0.38
ITER 3
0.27+40.38
0.18+0000
0.36:10.03
LLAMA-2-70B
ITER
0.77+0002
0.21
0,43
2
0.08++0.12
0.35
0.35++003
ITER 3
0.28+0004
0.14
0.33+10.33
QWEN-1.5-110B
ITER
0.77
0. 0.100
0.45:10.09
ITER 2
0.56
0.21
0.32+10.88
ITER 3
0.28+0021
0.18
0.3
DEEPSEEK- V2
ITER
0.89
0.1720.02
0.42:10.05
ITER 2
0.62+t0.10
0.20=0002
0.34++0.02
ITER
0.67+00.24
0.40=0.12
0.29:+0.66
CLAUDE-3-OPUS
ITER
0.77+0002
0.18+0.00
0.43+1006
ITER 2
0.56-10.8
0.39+0.11
0.24
3
0.75
0.15+0000
0.22+10.55
size
aroma
taste
size
smell
taste
nutrition
score
juiciness
nutrition
score
juiciness
market potential
market potential
(a) Ground truth
(b) FCI results
Figure 14: Ground truth and faithful (via FCI algorithm) causal graphs in AppleGastronome.
30

---

Size
Nutrition
Juicy
Taste
Score
Market Potential
(a) GPT 4 reasoning
size
flavor balance
score
moisture and juiciness
nutrient content
market potential
(b) GPT 4 COAT
Figure 15; Causal graphs with GPT in AppleGastronome.
31

---

Nutrition
Size
size
taste profile
Taste
nutritional value
Score
Market Potential
Score
market potential
(a) GPT 3.5 reasoning
(b) GPT 3.5 COAT
Figure 16: Causal graphs with GPT 3.5 in AppleGastronome.
Juicy
Size
size
taste profile
Taste
score
flavor profile
Score
texture
(a) LLaMA2 reasoning
(b) LLaMA2 COAT
Figure 17: Causal graphs with Llama-2 in AppleGastronome.
32

---

Nutrition
Size
Taste
SOUmISS
freshness
size
Score
Aroma
outrient content
Score
flavor profile
Market Potential
marker potential
(a) Mistral reasoning
(b) Mistral COAT
Figure 18: Causal graphs with mistral Medium AppleGastronome.
Nutrition
Taste
sweetness level
Size
Aroma
size
aroma
flavor balance
Score
nutrient content
Market Potential
market potential
(a) Claude-3-One reasoning
(b) Claude-3-Opus COAT
Figure 19: Causal graphs with Claude-3-Opus in AppleGastronome.
33

---

moisture content
Nutrition
Juicy
size
flavor profile
Taste
Size
nutrient content
score
Market Potential
Score
market appeal
(a) DeepSeek- V2 reasoning
(b) DeepSeek- V2 COAT
Figure 20: Causal graphs with DeepSeek- V2 in AppleGastronome.
freshness and quality
Nutrition
sensory appeal
Aroma
Taste
nutrient richness
Score
Score
Market Potential
market potential
(a) Llama-3-70b reasoning
(b) Llama-3-70b COAT
Figure 21: Causal graphs with Llama-3-706 in AppleGastronome.
34

---

Nutrition
Size
Taste
Aroma
size
fresdueess
flavar Ixlanne
Score
profile
score
profile
Market Potential
market potential
(a) mistral-Large reasoning
(b) mistral-Large COAT
Figure 22: Causal graphs with mistral-Large in AppleGastronome.
Nutrition
Aroma
Taste
balance
dominane
perceiveed fresimes
Size
Score
visual physical appeal
Market Potential
markex potential
(a) qwen- qwen-1.5-1100 1.5-110B reasoning
(b) qwen- qwen-1.5-11 1.5-110B COAT
Figure Causal graphs with qwen- 10B in AppleGastronome.
35

---

Size
flavor balance
Taste
taste
size
Score
Score
Market Potential
economic impact
(a) GPT-4o reasoning
(b) 4o COAT
Figure 24: Causal graphs with GPT-4o in AppleGastronome.
36

---

E.6 More Details on Constructing Neuropathic
You are a helpful and professional assistant to Neurologists. Try your best!
Write a clinical notes based on following information:
Patient id: Deuro_(idx:0n Age: (10+np. choice (50)
Gender: @get_gender// ()
Write one summary (one paragraph, do not do inference) about **symptom
diagnosis**: (symptom_non) notes)
L: Left
R: Right
DLI: discoligamentous injury
Radi: radiculopathy
Your clinical notes should surrounded by <note> </note>
Figure 25: Illustration of prompts for generating Neuropathic.
In the Neuropathic benchmark, we convert the dataset into a clinical diagnosis task. In the original dataset, there are three levels of causal variables, including the symptom-level, radiculopathy- In experiments, we mainly consider the target variable of right
and the pathophysiology-leed shoulder impingement. When generating the clinical diagnosis notes as æ using GPT 4, we will avoid any mentioning of variables other than symptoms.
As intend to leverage the Neuropathic benchmark to simulate the real-life diagnosis, after the factor proposal stage, we directly incorporate external experts that measure the values of the candidate factors. The prompts generate the diagnosis records are given in Fig. 25.
Examples of Neuropathic are given in Fig. 26.
E.7
More Details of Results Neuropathic
The detailed causal graph results are given from Fig. 27 to Fig. 31.
E.8 Discussion on the time complexity
Assume there samples with possible factors.
37

---

Group with 1R shoulder impingemant"
Patient
Age: 57
Gender: Male
Sympten Diagnoric Summary:
The patient presents with left shoulder impingenent eymptens include Limited and painful range of notica with positive outcoms when performing a 1imb asseasment also reveals issues; the patient right
intracapsular
"thunba teat the left side. Right
arnband suggesting ongo'ng ng managerent
reports lumbago and psin lecalized to the left thigh. Faysical therapy assessment (L pta) is noted, indicsting involvemen: in therapeutic intervent Joint assessments reveal left hip joinz and right hip arthritis, accompanied by right hanstring disconfort.
Figure 26: Illustration of examples Neuropathic.
ES Raliniiputy
CB Relrriouity
Ralesipuls
Radioakpati
& inpitacmesi
OLICECA
(a) Ground truth
(b) FCI results
Figure 27: Ground truth and faithful (via FCI algorithm) causal graphs Neuropathic.
For Factor Proposal:
META only needs interact once with LLM, so it O(1). DATA runs one single COAT round, so it 0(1). COAT interacts with LLMs multiple rounds. each round, at least one new factor should be proposed; otherwise, the loop will stop. So it is O(n) For Factor Annotation:
META: Not applicable. DATA: At most n factors would proposed single round. And each of them needs m times annotations by LLMS for all samples. So it is O(m)). COAT: At most n factors would be proposed during all rounds. And each of them needs times annotations by LLMs for all samples. So it is O(n)). For Causal Discovery:
Pair-wise reasoning by LLMs: O(n²) COAT: LLM is involved. The computational cost of depends on the numeric methods, The FCI algorithm used by COAT has a time complexity that goes exponentially with n. Learning the causal graph over large number of nodes effectively still an open problem in causal discovery literature.
E.9 Resources
We utilized system comprising two Intel Xeon E5-2630v4 processors with 2.2GHz, two NVIDIA Tesla P40 GPUs, and 256 GB of memory. For conversations with large language models (LLMs), we leveraged the poe.com platform, while annotations facilitated using the OpenAI API and the Mistral API.
38

---

R CS Radiculopathy
R O6 Radiculopathy
Nina
shoulder impingement
GPT reasoning
(b) GPT COAT
Figure 28: Causal graphs with GPT 4 in Neuropathic.
(a) GPT 3.5 reasoning
GPT 3.5 COAT
Figure 29: Causal graphs with GPT 3.5 in Neuropathic.
(a) LLaMA-2-706 reasoning
(b) LLaMA-2-706 COAT
Figure 30: Causal graphs with LLMMA-2-70b Neuropathic.
DULESO
Bulisaedyy
(a) Mistral-Medimm reasoning
(b) Mistral-Modium COAT
Figure 31: Causal graphs with Mistral-Medium in Neuropathic.
39

---

COAT with Different Causal Discovery Algorithm
Table 7: Causal discovery results in Neuropathic. PA, AN, and OT refer to the parents, ancestors, and others, respectively. Accuracy and measure the recovery of the causal ancestors.
FACTOR PROPOSAL
LLM
METHOD
PA
AN
OT
Acc
F1
META
3
0.91
0.59
GPT 4
DATA
2
0.95
0.50
COAT
3
0.96
0.80
DATA (LINGAM)
3
0.96
0.67
COAT (LINGAM)
3
0.95
0.75
META
3
0.91
0.59
GPT 3.5
DATA
3
0.94
0.67
COAT
3
0.96
0.77
DATA (LINGAM)
2
0.91
0.46
COAT (LINGAM)
3
0.94
0.67
META
2
5
0.91
0.53
LLAMA2-70B
DATA
3
0.95
0.60
COAT
3
0.97
0.86
DATA (LINGAM)
3
0.92
0.57
COAT (LINGAM)
3
0.95
0.75
META
0.96
0.80
MISTRAL-MED
DATA
3
3
2
0.94
0.66
COAT
0.97
0.86
DATA (LINGAM)
3
3
0.92
0.50
COAT (LINGAM)
3
0.96
0.80
G Ablation Study
Group Size in Prompt. In the COAT prompt, several samples are given and grouped by the values of the target variable. The samples in each group randomly selected to a fixed number (like samples per group). Empirically, we keep it to be 3 throughout all experiments (sometimes smaller than if samples are not enough). In practice, it is mainly constrained by the LLM'' context length.
The Number of Clusters. When constructing feedback, we first use clustering to separate the dataset and then find the cluster where the target variable is not explained well by current factors (This is heuristic for the problem line 191). Empirically, we set the number of clusters to be one plus the number of current factors.
We conduct ablation studies of COAT with GPT-4 using different hyperparameters:
Table 8: Results about Ablation Study on Hyperparameters.
Method
Cluster Size
Group Size
MB
NMB
OT
Recall
Precision
F1
META
2.67
0.67 =0.47
2.33+044
0.53 19
0.46=aaw
0.49=o.12
DATA
3.0010.00
0.33
0.00
0.60.1..0
0.92.1.12
0.72
COAT
1en(factor)+1
4.0 4.00
0.33
0.00+400
0.80
4.93
0.85=0.1
COAT COAT
1en(factor)+1
4.67
0.00=ax0
0.00#
0.93
1.00
0,96=a.06
3.67
0.00
0.00..000
0.73.1037
1.00
0.82_.222
As shown in Table 8, can observe that COAT is sensitive these hyperparameters and performs robustly well than the baselines under different hyperparameter setups.
Prompt Template. We conduct an ablation study with different prompt template following [101]:
40

---

We put the task description (also the format instructions) in the beginning [System] and we put samples in the last [Data] part of the prompt.
part,
The markdown grammar is replaced by blankets to represent headings, like [Data] and [Groups with Y=1]
[System]
3 COAT iterations performed, which is aligned with the original experimental setup.
TABLE 9: COAT WITH CHANGED PROMPT TEMPLATE
MODEL
MB
NMB
OT
RECALL
PRECISION
GPT-4
0
0
0.80
1.00
0.89
GPT-3.5-TURBO
4
0
0
0.80
1.00
0.89
MISTRAL-MEDIUM
0
0
0.60
1.00
0.75
In Table 9, we observe that COAT is robust to the choice of templates, rejects unexpected factors (zero NMB and OT), and keeps high precision.
41

---

R C5 Radiculopathy
C6 Radiculopathy
C4-C5
DLI C5-C6
R Radiculopathy
R shoulder impingement
DLI C3-C4
(a) Faithful ground truth
R-shniiae impingatnn
THJ C3-A
C7 Reiiskpplly
0.24
0,06
2.84
R.67
Rackelqualy
CS Balkuikpaaay
Ralicukoulll
Q187
A.11
A/S2
0.33
Radiculgaatty
Radicalopatt
(b) GPT-4 COAT with LiNGAM
shouider impingemer
R. Rediculopeity
G28
0.12
0.87
0.13
CS Ralinilopetty
Radirskpphty
L.GZ Radieslopetty
0.87
Releniiqualy
Raberiquelly
(c) GPT-3.5 COAT with LINGAM
Impingemer
Rodicelopatty
0.1
Raticuiopatt
OS Radinnlopatty
0.08
C4 Radicsiogatty
Radicoloyetty
0.87
0.28
DE CICS
4.14
DIJ CS-C
0.22
0.22
R 15 Redieiiiuuuyy
Ratiesiopehy
(d) 1lama-2-70b COAT with LiNGAM
shoulder impingement
C3IC
0.28
0.24
0.06
0.84
L C5 Radicalopathy
0.13
0.08
R C6 Radiculopathy
C4 Radiculopathy
0.89
C5-08
0.92
0.93
42
R C5 Radiculopathy
L CS Radiculopathy
(c) Mistral-med COAT with LiNGAM

---

H
Summary of Benchmark Data
Benchmark
Type
Source
Sample
Sample Size
Ground
Name
Truth
Apple
Synthetic
This paper. will open- open-sourced under CC-BY 4.0.
Textual
200 textual reviews.
Gastronome
Data
Neuropathic
Semi
Tabular samples from Tu et al. [26] on
Textual
100 synthetic textual data for prompt construction; 1000 tabular data for tests.
Yes
Real-world Data
their GitHub repo under CC-BY 4.0.
Textual samples are generated by this paper.
Brain Tumor
Real-world Data
An open Kaggle dataset (kaggle/bruin-tumg brrbrr-ooo------------- classification with an open-sourced project. [38]. MIT License.
Image
60 MRI samples for COAT and CI tests. The sample size is small due to the expensive API cost. One can easily construct much larger dataset by following our instructions.
No
Stock News
Real-world Data
An open Kaggle dataset (kaggle/stock- price-and news- realted-to-it) [39].
Textual with
200 samples for COAT experiments. 400 samples for empirical validation.
No
time index
MIT License.
ENSO
Real-world Data
The NOAA 20th
NetCDF
Monthly observation for 1836/01 to 201 5/12; 1.0 degree latitude 1.0 degree longitude global grid (360x 181); 90+
No
Century Reanalysis
V3 dataset [40] from their website at
http::/psl.noaa.go (with CCO 1.0
Licensc). We
climate variables.
developed set of simple tools for LLMs to utilize the ofticial datahase.
Table 10: Summary of Benchmark Data
I Case Study on Brain Tumor
In this section, we utilize the COAT explore the image dataset.
I.1 The Brain Tumor Dataset
Dataset Magnetic Resonance Imaging (MRI) an important technique for detecting tumors in the human brain. The images from an open Kaggle dataset aaggl//brrannuuuooolaasiiiiiiia
with an open-sourced project. [38]. Each sample is scanning MRI of a human brain. In this case study, the interesting variable is the tumor type. We consider three types of MRI images: glioma, meningioma, and no tumor. We include 20 images for each category and the total sample size is 60.
Data Processing We use vpt-4-vvsion-prennt preview to handle image samples. As the current gpt4 cannot process multiple images simultaneously, we concatenate samples from different categories into one picture, shown in Fig. 33, and give instructions in the prompt explain the format, shown in Fig. 34. For each proposed factor, the L.MSs would go through all sample images individually to evaluate the factor value.
43

---

Figure 33: The initial input to COAT in the Brain Tumor case study. Each row contains 5 samples randomly selected from one category (top-down: glioma, meningioma, and no tumor).
Evaluation
The ground truth not directly available to this dataset. Therefore, each factor in the final result will be evaluated by searching the related literature and will be explained by human interpretation.
1.2 Result and Discussion
Result interpretation As shown Fig. 35, there are two visual factors that appeared in the final causal graph: contrast enhancement and mass effect. The first factor (contrast enhancement) is about whether the bright part is uniform or heterogeneous in the sample image. Another factor (mass effect) is about whether the tumor tissue has caused significant displacement of normal brain structures. One may refer to Fig. 36 to see detailed descriptions of these two factors.
Justification for the final factors We verify the proposed factors by the medical literature [102, 103, 104, 105], shown Fig. 37. Note that these papers are searched according to keywords in factor descriptions and, therefore, can exclude some relevant studies. Therefore, the conclusions this preliminary exploration should be considered reference point for further in-depth validation by domain experts.
The factor description for contrast enhancement directly matches with the related paper (the first two papers displayed in Fig. 37). In addition, this can visually checked in Fig. 33. Therefore, we believe it is good factor.
The second factor, mass effect, may not directly match the papers. is pointed out in the last two papers displayed in Fig. 37 that the two tumors have different axial locations and thus influence the magnitude of the displacement of tissues. Both the keywords "axial location" and "displacement" have occurred in its factor description.
44

---

10plowied Lnngel
You are en excellently helpful AT aseistent for analyring, abstracting, and processing daca. Now try bexs.
Dete
Group samp
pictore.
## Group 21 neningions tumor
See the les in the 2nd in the picture.
# Group 3: no tunor
Bez the camples in the 2rd rov in the pictuze.
Abstraction.
What are the higa-level factord behind the sarple that contribute to the allocation of groups?*4
Figure 34: Illustration the prompt for the Brain Tumor case study. contains instructions about understanding the combined input picture.
Contrast Enhancement
Mass Effect
Tumor Type
Figure 35: Final causal graph by COAT in the Brain Tumor case study.
J Case Study on Stock News
In this section, we utilize COAT to explore time series data with text.
J.1 The Stock News Dataset
Dataset
The dataset consists of the stock value of Microsoft (MSFT) from 2006 to 2009 and its news summary (only include news in the New York Times). This is subset of an open Kaggle dataset sisatocc-rri--andd-nw---eeeeeeee
Fach sample is trading day with the company's close stock price and news. The target value the future return rate, and we are curious about factors in the related news. We fed data during the first 200 trading days COAT, and we used the following 400 trading days for evaluation.
Data processing
The future return rate is calculated by close prices:
close,+4 -
Ft close;
(24)
The target value is a binary variable according to the return rate
Y, 1,,00
(25)
where 1 is the indicator function. News fed to COAT are grouped according to the value of Y, To keep the whole prompt within the context limit, only 3 samples would be randomly included in each group.
Factor processing Given proposed factor, denote its annotated value at the t-th trading day as a, E (-1,0, 0, 1). Each factor is rolling averaged over the past days:
1
S
(26)
M
L-M
45

---

Contrast Enhancement
Strong, unáfore entancerent tysical of miringions.
ahntreal rnhancement
Interogerenus
typical
Nass Effect
displacenent
structures (con with both wordnngorn gloow pronounced in gliow
dut letro addal location).
Na
speritic critecta for -1 not
cousing sininal
brain structures
axial location).
Figure 36: Detailed descriptions of these two final factors the Brain Tumor case study. These descriptions were also directly fed to LLMs for factor annotation on images.
"They [meningiomas) almo always netrnstrate uniform strong enhancement on post-contrast imaging.
Lyndon, Dontel, et al. "Dural masse:: meningiomas and their mimics." Insights into imaging (2019):
"High-grade (gliomas) ) tumors were heterogeneour contrast enhancement with other medical imaging aspects of MRI such as bleeding, necrosis, and edema"
Haydar, Nisreen, al. "Role of Magnetic Resonance Imaging (MRI)
grading gliomas comparuble with pathology: A cross sectional study from Syria. Annals of Medicine and Surgery 82 (2022): 104679.
"Glioblastoma multiforme (GBM) is the most common primary brain tumor and is associated with a poor prognosis. One of its defining characteristics is the significant deformation of surouuddnng tissue. the so-cal so-called "mass effect".
Tung, Birkan, et al. "Modeling of glioma growth with mass effect by longitudinal magnetic resonance imaging." IEEE Transactions on Hiomedical Engineering
(2021): 131373724.
"Meningiomas typically appear as lobular extra axial masses with well-circumscriber margins. They typically have a broad-based dural attachment and, if sufficiently large, inward displacement the cortical grey matter.
Watts, J. al. "Mognetic resononce imoging of meninglomas:: pictorial review." Insights into imaging (2014): 113-122.
Figure 37: Medical literature about the brain tumor. Searched according to keywords in factor descriptions.
Each variable, including return rate, is normalized by the rolling standard deviation over the past M days:
St. F. Sd (()))))2111)
(27)
An example of a processed factor can be seen in Fig. 38.
J.2 Result and Discussion
Result Causal Graph As shown in Fig. 39, four factors are identified in the final causal graph to form Markov blanket of return rate: Product Focus, Legal and Regulatory Issues, Market Strategy, and Innovation and Technology Focus. One can refer to Fig. 40 for more detailed descriptions of these factors. One factor (Innovation and technology focus) is identified as possible cause of the return rate; this matches the nature of the company' type and reflects people's 's expectation of the company to keep creating innovative computer software. It is also interesting to see that COAT captures structure between factors and market stralegy, where product focus and legal and regulatory issues are identified as potential causes. also implies the existence of latent confounder between market strategy and return rate that may not be significantly reflected in news text.
46

---

innovation and technology focus
innovation and technology
0.6
-0.2
2006-12
2007-01
2007-02
2007-03
2007-04
2007-05
2007-06
2007-07
2007-00
2007-00 -0-
2007-10
Date
Figure 38: Example one processed factors Innovation and Technology Focus during the first 200 trading days. The red star marks the highest value. In June 2007, there discussion about future competition in desktop operating systems and the trend towards web-based services [106].
Innovation Technology Focus
Product Focus
Logal Regulatory
Retum
Macket Scrategy
Figure 39: The final causal graph by COAT in the Stock News case study
Evaluation by Trading Strategies For each factor {Fi} processed in Eq. 27, we establish trading strategy on it to see the performance in the out-of-sample trading days. At each trading day t, we fit the following model
F1 a FFt
(28)
using samples {r., Fiht-20giic: Note that r; is the future return rate defined in Eq. 24 and is not available this day. After the estimation, make decision based f; â ,F, If f. O, go long with I unit capital, i.e., purchase that amount of stock, the gain would be gain would be x 1 go short with unit capital, i.e., borrow and sell that amount of stocks immediately and buy them back next time, the gain would be - x units capital.
units capital. If <
To align with the definition in Eq. 24, make trading decisions every 4 trading day. We introduce an additional Buy and Hold baseline to always go long one unit capital. The trading evaluation is after the 200-th day and thus has no overlapping with data fed to COAT.
The cumulative return plot shown Fig. 41, and the metrics commonly used in economic literature [107, 108, 109] are shown in Table 11. One important metric to see trading strategy's 's effectiveness the sharp ratio: a measure of risk-adjusted return, showing the excess return (over the risk-free rate, we set it to be 2%) per unit of standard deviation. A higher Sharpe ratio indicates better performance per unit risk. We see that the Innovation and Technology Focus factor yields the highest sharp ratio and outperforms other non-causal factors.
47

---

Product Focos
Text primarily discusses the launch, parformance, feotures of coppary's protuct.
products
aspects company strategy, practices, procesdings, regelatory actions, oversight company's
and Regulatory Issues
Othervise; not mentiones.
-I: Text does not discuss lagal ragilatory issses but rocuses on otter topica lice product inforaation or naclet parformance.
Strategy
Text siscusses compary strategles related sarket expanston or conpetitive actions.
8: Othervise; or not centiones.
Text focuses internal comany
details, ingal/raguiatory netters without direce lisk warket
Innovation and technology Focus
Articles highlighting now tecinologies, Lnnovations, on sigrificant advercements existing tectnology.
etscuesing technclgges, updates,
Figure 40: The detailed factor descriptions proposed by COAT in the Stock News case study
Out-of-Sammle Strategy Performance Comparison
1.25
Buy and Hold
product
legal/regulatory issues
1.00
market strategy
innovation and technology focus
0.75
Cumulative Retums
0.50
0.25
2007.09 09
2007-11
2008-01
2008-03
2008.0 OS
2008-07
2006-08 09
2008-11 11
2009-01
2009.03
2009 05
Date
Figure 41: The out-of-sample cumulative return trading strategies based on different factors in the Stock News case study
Table 11: Performance about trading strategy according to each factors
BUY AND HOLD
PRODUCT POCUS
LEGAL/REGULATON ISSUES
MARKET STRATEGY
INNOVATION AND TECHNOLOGY FOCUS
EXPSCTED RETURN
-3.50
0.29
0.38
0.48
0.72
SHARP RATIO
-2.58
0.70
0.94
1.20
1.86
T STAA
3,00
0.89
1.19
1.51
2.35
INTORMATION RATIO
2.45
2.50
2.56
2,73
3.79
3,88
3.98
4,22
T-STAT
3.09
3.15
3.23
3.44
MAX LOSS
0.50
0.15
.0.43
-0.38
48

---

K
Case Study on El Niño-Southern Oscillation (ENSO)
In this section, we use COAT explore scientific problem using open datasets in climate science. The case study shows its potential to handle more complex data formats when combined with tools.
The ENSO Background EI Niñio-Southen Oscillation (ENSO) an important phenomenon in the Pacific Ocean and is characterized by irregular fluctuations in sea surface temperatures[ I 10]. ENSO consists of the warm phase (known El Niño) and the cold phase (known La Niña). It profoundly influences global weather patterns, including precipitation, storm development, and temperature anomalies [111]. The prediction of ENSO events involves the complex interplay between oceanic and atmospheric systems and, therefore, is still an open problem [11 113].
NOAA Dataset The NOAA 20th Century Reanalysis V3 dataset contains high-dimensional infor- mation about Earth's 's atmosphere with fine-grained time and space coverage from the I 19th century to the early 21st century [40]. This dataset amalgamates many observational data and uses the ensemble filter data assimilation method to reconstruct the historical state of the global atmosphere. The dataset includes fine-grained spatial coverage with 360 181 grids. We only use monthly data this case study.
K.1 Setting and Data processing
The Target Variable Construction The focus of this study is the future change in monthly SST in the Nino3 region, which could be an important indicator ENSO events. The target variable is manually crafted from the dataset using API and visualized in Fig. 42, where the oscillation pattern can be seen.
nino3 sst y Observation
measurement='Ske Skin Temperature',
level=' Surface', region=( Latiildde_miiin -5, 'latitude_max': 5, Longitudd_mii min 'longitude_max 270}
210, )
next month shift y y. shift y
Listing Code to define the target variable in ENSO case study
1.5
380
2000
Figure 42: The visualization of the target variable defined in the ENSO study. The oscillation pattern can be seen.
Prepare Tools for L.MMs The dataset is in the NetCDF format (network Common Data Format), so it is not convenient to be directly fed to LLMs. We prepare function Dbservationneeasurrenn
49

---

level, region) to help define factors. For example, in Lst. 1, we define the target variable with the help of this function. The measurement specifies the climate variables like precipitation rate or temperature; the level specifies the vertical location, like surface or specific pressure level; the region a rectangle about the area concerned. This function would output a single time series about the measurement on the specified level averaged within the specified region. Note that the target variable is the only factor defined by humans, all other factors are defined by code from LLMs.
def
():
Air Temperature at Cloud Layer Top (Nino3 Region)
Declare observation
observation_name = Observation(
measurement= Temperature",
level="Isentroph Levels",
region=("latitude _iiieem__nn -5, 'latitude_max's 'longi Longitude__ee tude_ min 'Longitude_maxxs 270},
210, detailed_level=300
10
)
13
Return the observation values directly return observation_nam:
14
15
factor_dict[' Temperature at Cloud Layer Top (Nino3 Region) '] @eL_arr_temprruur__.tccuuuul_ - ()
Listing 2: LLM can be prompted to propose factors using complex tools
Feedback Construction The dataset is in a special format; we can only draw a tabular subset of data from it in this setting. Therefore, is difficult make groups on samples like what we did in the previous cases. Instead, we construct feedback with some summary information. As shown in Fig. 43, two types of information are included in the feedback: intermediate causal graph and OLS regression result of factors in the current estimation of a Markov blanket on the target variable. These two types information can drawn naturally from the COAT's intermediate results and are not complex, so LLMs can always access them and easily understand them. Intuitively. these factors would help LLMs better understand existing factors and propose further possible factors based on the provided dataset and its knowledge.
K.2 Causal Graph and Discussion
Result Interpretation In this case study, we are treating non-stationary climate data, while assum- ing the causal structure invariant. Under this assumption, the non-stationarity actually helpful for causal structure learning [51, 114]. To this end, we utilize the CD-NOD algorithm [51] to fully utilize the changing causal modules for better identifiability. CD-NOD would first identify the non- con-stationary nodes, whose conditional distribution given the causal parents are changing with time, and then use them for better causal structure recovery. Four factors are identified to non-stationary, as shown in Fig. 44.
We have adjusted the nodes' names, shapes, and colors for better visualization. There are 14 nodes in total, with 13 factors identified by COAT. Each factor is a time series about a certain climate measurement above specific level averaged over a specific region. For simplicity, we only considered instantaneous causal relations among those time series.
There three regions identified to be relevant to the ENSO phenomenon:
Equatorial Pacific Region (Orange Nodes). This region (5N-5S, 120W-280W) is one of the most active places about ENSO. It becomes significantly warm during the El Niño phase and becomes significantly cold during the La Niña phase [115].
Nino3 Region (Blue Nodes). This region (5N-5S, 150W-90W) is one of the most classical regions monitor the El Niño events by scientists. also used by humans construct the target variables [1]
50

---

marthly chengy undowe remprrttrr (nirat)
(rnmotttiear nelentte)
temperauue mgjon
pressare (equasonal porfic reglon)
(equountal poctfic region)
(sima3 rglco)
complese
Heazt
At Casall Gragh Reults
or the pizzon
Reahss
Dap. Vanable:
roeeth skilt for niae3 (ies sariice tomposatur)
R R-aqzared (anzoniened:
0,005
Modet
OI.S
Adj it B-seponal (mncreeedd)
094
Melad
Loss Squaus
F- F-sutisiic: Frub (B-statencc
851.9
2024
28:22443
Lags Recisbedddd
DiMaddl.
Coveunnee
cadf
4d
P-H
[0:025
sot Eace dir torpenturt gnad err of peciffi)
0.0003
0.001
0.548
.0001
0.001
occan atrmephere
casten oquakeri paofic)
1.813
fog poem siroephrre pacrft)
equaooia
duil
Vlak
What new faveesteve night influmnee tounthly change in om or faoe seeperatur prioty"
Rogascon rap exp.
you
Figure 43: The prompt with feedback based on previous loop in the ENSO case study
South American Coastal Region (Green Nodes). This region (ON-20S, 80W-60W) includes the Peruvian coastal up-welling system, and is noticed to be relevant the ENSO cycle [116].
Some insights are delivered by paths the output causal graph:
Sea level Pressure. This factor is about the sea-level pressure on the equatorial Pacific region. The pressure gradient would influence the movement of warm water, and thus influence the sea surface temperature (SST) change [41]. In addition, pressure can influence the water evaporation and thus regulate through the water circulation. This also matches another indirect path Sea level Pressure + Sensible Heat Net Flux + Volumetric Soil Moisture Cloud Cover > SST Change.
Momentum Flux, V-compoent. This factor is about the vertical movement of air. It is crucial in driving atmospheric convection [41], and it is related to the Walker Circulation, which is an important component the ENSO system[42] Also, it could influence the change in sea level pressure and indirectly influence the SST change.
Cloud Cover. The factor is the fraction of the sky covered by clouds the NINO3 region. It could influence the SST Change through solar radiation well as water circulation. It is confirmed to have a significant correlation [43] with ENSO events and plays an important role in the atmospheric circulation and hydrological eycle [44].
Soil Temperature. This might be novel hypothesis proposed by COAT, since we found no sufficient research confirm this point the best of our knowledge. This causal graph has also suggested two possible indirect mechanisms: (1) through Volumetric Soil Moisture
51

---

and Cloud Cover, and (2) through Convective Precipitation Rate and Sea Level Pressure. Therefore, this finding encourages more serious investigations of these hypotheses.
Dewmmand
Air
Longwove Redurion Flux
Tempentune
Equaterial Pactfic
Roughness
Ninod Region
South American Coastal Region
Monesturaa Flxx, U- Composeett
Claud Cover
Zonal Wind
Target
Proposed Factor (stalonary)
Vollmmenee Scal Mossure
Sensible Hove Not Flux
Latent Hear Net Flux
Proposed Factor (oon statioary)
Convectve
Prripiistio Rane
Temperstrry
Momentum Ples V-- Componers
SST Change
Sea Level Pressure
Figure 44: The final causal graph found by COAT in the ENSO case study
Discussion on Additional assumption The constructed feedback contains the causal structure among existing factors and their OLS regression analysis on the target variable. By doing so, some additional assumptions are implicitly made. For example, it assumes linear relation between factors and the target variables. More detailed theoretical analysis is out of the scope of this case study, and we left it for future work.
Discussion on Hallucination Although we have clarified how to use tools prompts, LLM can sometimes propose unsupported data requests. For example, only certain pressure levels are supported in the dataset. To this end, we include Python code in the prompt, as listed in Lst. 3. This function can check whether the LLM's request supported. We ask LLM to run this function in its code interpreter to make sure the proposed factors are valid.
52

---

import pandas
def choike (measurement, level, detaille__eveeINonee)
meta = pd. read_ csv Measurement__Level. weteeeeellleeaa.eeeeee csv
measurement_sse set set (meta. Measurement. values) measurement__eeel.nnng
meta. meta. groupby (I Measurement }} agg (("
Level": lambda set(
detailed_level_rane
10
'Pressure Levels': [1000, 975, 950, 925, 900, 850, 800 700, 650. 600, 550, 500, 450, 480, 350, 300, 258, 200, 150, 100,
750,
70, 50, 30, 20, 10, 5, 1J.
MMltiple Subsurface Levels' :000 [o, 10, 40, 108],
12
FFxed Levels Above the Surface': [12, 30, 50, 80,
100, 150, 200, 250, 300, 500],
13
"Isentropic Levels': [300, 330, 350]
16
if measurement not in measurement_set, return False, f"measurement error"
17
18
if level not in measurement_
1oc [measurement, 'Level'
]:
return False, f"level error, only allowed: { measurenent_.evel_map 1oc[measuremmett 'Level ']
20
if level in detailed_Level_rnngeeee k)ys
if @rtailed_lever not in detailed wew.__eeeeeeeeeeeeeeeeeeeeeeeeee return False, f"need detailed level in {
detailed leaerled_eeel__rnneeeeeeeelll
24
return True, None
Listing LLM required to use this function overcome hallucination
L Broader Impacts
This work focuses on fully leveraging the rich knowledge learned by LLMs during pre- pre-trainnn training to facilitate causal discovery from unstructured data, with the hope of empowering broader applications and social benefits. Besides, this paper does not raise any ethical concerns. This study does not involve any human subjects, practices to data set releases, potentially harmful insights, methodologies and applications, potential conflicts of interest and sponsorship. discriimnation/iisfaiinne concerns, privacy and security issues, legal compliance, and research integrity issues.
53

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
See the end of the introduction part.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results
be expected to generalize other settings.
is fine to include aspirational goals as motivation as long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: See Sec. 2.5 and Appendix B.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification. asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low images are taken in low lighting. Or speech-to-text system might be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach address problems of privacy and faimess.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
Question:
For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
54

---

Justification: see Appendix D.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
Justification: See Appendix E.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how to reproduce that algorithm.
(b) the contribution is primarily a new model architecture, the paper should describe the architccture clearly and fully.
(c) If the contribution is new model (e.g., a large language model), then there should either way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reprodueibility. the case of closed-source models, may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
55

---

Answer: [Yes]
Justification:
https://causalconn github. io/
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.cm cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https: / //nips. httpps.c//puut cc/publ gegggddee/oeeeeiieeeeeeeee
weisssssooooooooo for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification:
See Sec. 3.1, Sec. 4.1 and Appendix E.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment Statistical Significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification: Standard Deviations are included.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, at least for the experiments that support main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
should be clear whether the error bar is the standard deviation the standard error of the mean.
56

---

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of enrors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: See Appendix E.9.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g.. preliminary or failed experiments that didn't make into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https: htt////rrippp..pppppp/////ccccce
Answer: [Yes]
Justification: Authors follow the NeurIPS Code of Ethies.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., if there is a special consid- eration due to laws or regulations their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: See the Appendix I..
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that improvement in the quality of generative models could be used to
57

---

generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
there are negative socictal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
No sensitive risky content is included in this paper.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: See Appendix H.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paper swi thcode com com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
this information is not available online, the authors are encouraged to reach out to the asset's is creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and the documentation provided alongside the assets?
58

---

Answer: [Yes]
Justification: See Appendix H.
Guidelines:
The answer NA means that the paper does release new assets.
Researchers should communicate the details the dataset/codeemoodll as part their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification: not involve crowdsourcing research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then much detail as possible should be included the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: This paper does not involve crowdsourcing research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors to adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
59


Reviews:
[{"review_id": "25NpGC5ozw", "reviewer": "Reviewer_AG9s", "summary": "This paper combines the power of LLMs with that of causal discovery by proposing a Causal representatiOn AssistanT (COAT) approach. Specifically, it considers datasets with textual descriptions, and tries to identify the Markov blanket with respect to a target variable (such as customer ratings and medical diagnosis). The key contribution is discovery of the causal factors through a pipeline that uses both LLMs and a causal discovery algorithm.", "questions": "Could the authors share more about the scope of the work? Are there some other restrictions on the problem setting, besides needing a discrete label y? My assessment is that there is a gap between the scope mentioned in the problem setting and what is described in the experiments, which seems more restricted. Perhaps the authors can clarify.\n\nIdentifiability is mentioned loosely on page 2, with some technical references, but seems to have been used in an imprecise way here. The connection here seems tenuous at best.\n\nThere is a comment on pg. 3: “Note that the target variable Y serves as a guider, and no specific relation between x and Y is assumed.” What is a guider? And what do you mean no relation between x and Y is assumed? I thought the entire point is to do causal discovery: x is a function of z, and Y is a function of z. This line seems incorrect.\n\nSection 3.2 would be much easier to follow with some illustrative examples. The content in Fig. 2 is too abstract to be really useful. I think the authors missed a trick here.\n\nThe meaning of C and p are unclear to me from what is described on pg. 6. How does one assess the significance of Proposition 3.3?\n\nThe details about benchmarks are incredibly important, and it should be easy for anyone to understand at least a high-level sense of a benchmark – basic things like the number of data points, for instance. Please fix Section 4 accordingly.\n\nWhat is OT in Section 4.2? Is it the same as OT in the next section? Define MB, NMB, OT somewhere. I don’t see them mentioned anywhere clearly, although I understand from context that MB means Markov blanket.\n\nAre Table 3 and Fig. 5 in the Appendix? If so, then mention that.\n\nI’m not convinced that the “realistic” benchmarks are realistic. It’s too bad I can’t gauge this from the main text.\n\nPlease add a detailed limitations section. Mention all the limitations around evaluation in particular, as well as the significant risks of relying on LLMs for causal discovery.\n\nMinor comments: line 181: “an potential” should be “a potential”; line 208: “Ability” should be “ability”; lines 211 and 212: seems there is a grammatical error here; line 216: what are “shared notations”?", "limitations": "The authors have not suitably described limitations. I consider this a major weakness; please see my previous comments.", "rating": 5, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 3, "strengths": "I find this an interesting and practical paper that combines the advantages of LLMs – such as the vast amount of knowledge that they encode – and that of causal discovery approaches. The ideas around combination are generally simple but novel, and I believe the approach could potentially be valuable in a suite of applications, although the extent of the value is unclear from the paper.", "weaknesses": "A major limitation of the work is the empirical evaluation, even though it comes across on the surface as being extensive. I sympathize with the authors about benchmarks for causal discovery, but it seems they have used GPT-4 to generate the textual description of the data, and then used LLMs in their COAT procedure. This is clearly a synthetic dataset that can be problematic. Even the “realistic” benchmarks do not come across as sufficiently realistic, based on my understanding and the lack of details in the main paper.\n\nI don’t understand why key aspects of the evaluation were moved to the appendix. I find it impossible to fully evaluate the work based solely on the contents of the main paper. I understand the need to make space and to move things to the appendix, but it’s never suitable to move key aspects such as the description of the benchmarks and the key results that show value of the work. This has impacted my assessment of this work and I have had to decrease my score because of the authors’ choices around appendix content.\n\nA related weakness is the lack of any attempt at describing limitations, of which there are clearly many.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for the detailed and insightful comments on our work. We hope our response can sufficiently address your concerns.\n\n> W1. Reality of benchmarks\n\n**The choice of synthetic and realistic benchmarks is because of the evaluation purpose**. Since we usually do not have access to the ground truth causal graph of the realistic benchmarks, we need to synthesize ones to effectively evaluate the performances of COAT.\n\nMeanwhile, **we do evaluate COAT in realistic data**, for which we have revised our manuscript to make it clearer:\n- Brain Tumor: This is an open-sourced dataset containing MRI images for brain tumor classification. \n- Stock News:  This dataset contains the close price of a company from 2006 to 2009 and its 804 news summary (from the New York Times).\n- ENSO: This dataset contains high-dimensional information about Earth’s atmosphere with fine-grained time and space coverage from the 19th century to the early 21st century. This is a popular real-world dataset in climate science.\n\n> W2 Evaluation details.\n\nThank you for the suggestion! We have revised the paper to include more details of the evaluation in Sec 4.1, and please kindly let us know if you feel any additional revision could further improve the clarity:\n- Purpose: We evaluate whether COAT can propose and identify a set of high-level factors belonging to the Markov Blanket of the target variable Y.\n- Construction: We prepare different high-level factors: 3  parents of Y, 1 child of Y, and 1 spouse of Y. These factors form a Markov blanket of Y. In addition, we also prepare one \"disturbing\" factor related to Y but not a part of this blanket. \n- Expectation: A good Method is expected to propose the 5 high-level factors (up to semantical meanings) and exclude the \"disturbing\" factor.\n\nThe key findings from the experiments are summarized below: \n- COAT is more resistant to the \"disturbing\" factor, which is supported by the lower *NMB* column (number of factors out of the Markov blanket) in both table 1 (page 8) and table 5 (page 24). \n- COAT filters out irrelevant factors from LLMs' prior knowledge that are not reflected by the data, which is supported by the lower *OT* column (number of other irrelevant factors).\n- COAT robustly encourages LLM to find more expected factors through the feedback, which is supported by the lower *MB* column (number of factors in the Markov blanket).\n\n> W3 Lack of limitation discussion.\n\nWe indeed provided a discussion on limitations in the future work section on line 629, page 17. We revised the section title to **Limitations and Future Directions** to make it clearer. \n\n> Q1 Scope of the work.\n\nThe problem setting in Sec 3.1 establishes the objective of this work that reliably leverages an LLM to identify the underlying high-level factors in the Markov Blanket of a given target variable.\n\nThe inputs are merely the target variable and the unstructured data, which can be either text or images. After identifying a set of candidate high-level factors, the values of those factors can be either obtained by LLM annotations or by some external tools.\n\nMeanwhile, identifiable causal discovery also requires certain assumptions about the data:\n- **Faithfuness**. The empirical distribution of the data reflects the actual data-generating process.\n- **No selection bias**. Otherwise, the faithfulness condition would be violated. \n- **Enough sample size**. Our method involves statistical tests, the higher the better.\n\n\n> Q2 Use of identifiability mentioned on page 2.\n\nThe references here are classic books and survey papers about causal discovery methods with rigorous theoretical guarantees of identifiability. **We use them to show the important role of identifiability**, which is lacking in the current literature on using LLMs for causality-related tasks. The discussion in line 33 to 38 gives rise to our main research question: How can LLMs **reliably** assist in revealing the causal mechanisms behind the real world? \n\n> Q3 Description of problem setting.\n\nThe meanings of the sentence are:\n- Why Y serves as a `guider`: Only a subset of hidden high-level factors belongs to a Markov blanket of Y, that correlates with Y. Therefore, one may use Y to guide the identification of the desired high-level factors.\n- The `relation` actually means causal relations. In a Markov Blanket, there are three types of causal relations: Parents, Children, and Spouses. In the second type, some elements in z could be functions of Y.\n\nWe have revised the sentence to \"Note that the target variable Y serves as a guider, and no prior assumption between x and Y is assumed\" to avoid any potential misunderstandings.\n\n> Q5 The meaning of C and p, and the significance of Proposition 3.3?\n\nWe define the two concepts to formalize the LLMs' ability to propose useful factors:\n- **Perception Score** $p$: the probability that the LLM proposes such a new factor. This can be seen as a measure of the LLM's responsiveness to feedback.\n- **Capacity Score** $C_\\Psi$: the decreasing ratio of the conditional mutual information, as described in Eq 8. This can be seen as a **measure of the quality of the proposed factors**.\n\nThe significance of Proposition 3.3:\n- It shows that COAT can provably find a set of high-level factors in the Markov blanket of Y with sufficient rounds if $p>0$ and $C_\\Psi >0$.\n- It also characterizes the influence of LLM's ability ($p$ and $C_\\Psi$) on the efficiency of COAT by Eq 9.\n\n> Q6 The details about benchmarks.\n\nThanks for the suggestion. These details (initially in appendix G) are added to the main paper now. A related discussion can be found in our response to W1.\n\n> Q10 & L1 Limitations.\n\nNow the limitation part (initially a part of appendix B) is a single section. We also add discussions about *faithfulness*, *selection bias* , and *sample size*, as we responded. Note that COAT is relying on LLM for factor proposal, whose risk can be controlled given the faithfulness condition, as implied by our theoretical results.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "xr73jSl76W", "reviewer": "Reviewer_szEm", "summary": "This work proposes COAT (Causal representation AssistanT), a novel framework to leverage LLMs to assist with causal discovery from unstructured data. COAT aims to combine the advantages of LLMs and causal discovery algorithms. To do so, COAT employs LLMs to identify high-level variables and parse unstructured data into structured data. On the other hand, causal discovery algorithms read the parsed data to identify causal relations. To improve the reliability of the results, COAT also constructs feedback from the causal discovery results to iteratively improve the high-level variable identification. The authors conduct extensive case studies ranging from synthetic data to realistic data, and find COAT effectively helps with discovering meaningful causal structures that well explain the target variable.", "questions": "Please refer to \"Weaknesses\".", "limitations": "No concerns regarding limitations.", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 4, "strengths": "1. This work identifies a crucial and timely problem for how to advance the causal tasks including causal learning and reasoning with foundation models likes LLMs;\n2. COAT is novel, interesting and well-motivated. The authors also provide theoretical discussion to justify its soundness;\n3. COAT is model-agnostic and robust to the choice of LLMs, and input data modalities;\n4. The authors construct several benchmarks, present comprehensive case studies, and conduct extensive experiments to verify their claims. The improvements over direct prompting LLMs are significant.", "weaknesses": "1. The authors should provide more comparisons with advanced prompting techniques such as CoT.\n2. More discussions should be provided on the hyperparameters used in COAT, such as the group size in feedback.\n3. Model names are inconsistent. The name in Fig 4(c) is not the same with other names.\n4. GPT-4 reasoning in Fig 7(c) is unclear in meaning.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your support and constructive comments on our work. We hope our response can sufficiently address your concerns.\n\n> W1 More comparisons with advanced prompting techniques such as CoT.\n\n**We construct a CoT baseline** based on *DATA*, where the LLM is prompted to \"Think step by step to consider factors\", and to output the desired factors in the same format as other methods.\n\n| LLM           | method           |         MB        |       NMB       |        OT       |      Recall     |    Precision    |        F1       |\n|----------------|------------------|:-----------------:|:---------------:|:---------------:|:---------------:|:---------------:|:---------------:|\n| GPT-4          | CoT baseline | 4.33   $\\pm$ 0.58 | 0.83 $\\pm$ 0.29 | 0.17 $\\pm$ 0.29 | 0.87 $\\pm$ 0.12 | 0.81 $\\pm$ 0.02 | 0.84 $\\pm$ 0.06 |\n|                |       COAT       |   4.00 $\\pm$   0.82   |   0.33 $\\pm$ 0.47   |   0.00 $\\pm$ 0.00   |   0.80 $\\pm$ 0.16   |   0.93 $\\pm$ 0.09   |   0.85 $\\pm$ 0.11   |\n| GPT-3.5        | CoT baseline | 5.00 $\\pm$   0.00    | 1.00 $\\pm$ 0.00       | 1.33 $\\pm$ 0.58 | 1.00 $\\pm$ 0.00       | 0.68 $\\pm$ 0.05 | 0.81 $\\pm$ 0.04 |\n|                |       COAT       |   3.67 $\\pm$   0.47   |   0.00 $\\pm$ 0.00   |   0.00 $\\pm$ 0.00   |   0.73 $\\pm$ 0.09   |   1.00 $\\pm$ 0.00   |   0.84 $\\pm$ 0.07   |\n| Mistral-Medium | CoT baseline | 4.33   $\\pm$ 0.58 | 1.00 $\\pm$ 0.00   | 0.67 $\\pm$ 0.58 | 0.87 $\\pm$ 0.12 | 0.73 $\\pm$ 0.07 | 0.79 $\\pm$ 0.05 |\n|                |       COAT       |   4.67 $\\pm$   0.47   |   0.00 $\\pm$ 0.00   |   0.00 $\\pm$ 0.00   |   0.93 $\\pm$ 0.09   |   1.00 $\\pm$ 0.00   |   0.96 $\\pm$ 0.05   |\n\n\nIt shows that LLMs with CoT can:\n- be aware of high-level factors behind data (lower OT than *META*);\n- still struggle to distinguish the desired factors in Markov Blanket (higher *NMB* than *COAT*).\n\n\n> W2 More discussions should be provided on the hyperparameters used in COAT, such as the group size in feedback.\n\nThanks for the suggestion. We have revised the manuscript to include a discussion on hyperparameters:\n- **group size in prompt**: In the COAT prompt, several samples are given and grouped by the values of the target variable. The samples in each group are randomly selected and are kept in a fixed number (like 3 samples per group). Empirically, we keep it to be 3 throughout all experiments (sometimes smaller than 3 if samples are not enough). In practice, it is mainly constrained by the LLM's context length.\n- **the number of clusters**: When constructing feedback, we first use clustering to separate the dataset and then find the cluster where the target variable is not explained well by current factors (This is a heuristic for the problem in line 191). Empirically, we set the number of clusters to be one plus the number of current factors.\n\nFurthermore, we conduct ablation studies of COAT with GPT-4 using different hyperparameters:\n\n| Method |   cluster size  | group size |        MB       |     NMB     |      OT     |      Recall     |  Precision  |        F1       |\n|--------|:---------------:|:----------:|:---------------:|:-----------:|:-----------:|:---------------:|:-----------:|:---------------:|\n| META   |        -        |      -     |  2.67 $\\pm$   0.94  | 0.67 $\\pm$ 0.47 | 2.33 $\\pm$ 0.47 |   0.53 $\\pm$ 0.19   | 0.46 $\\pm$ 0.08 |   0.49 $\\pm$ 0.13   |\n| DATA   |        -        |      3     |   3.00 $\\pm$ 0.00   | 0.33 $\\pm$ 0.47 | 0.00 $\\pm$ 0.00 |   0.60 $\\pm$ 0.00   | 0.92 $\\pm$ 0.12 |   0.72 $\\pm$ 0.04   |\n| COAT   | len(factor) + 1 |      3     |  4.00 $\\pm$   0.82  | 0.33 $\\pm$ 0.47 | 0.00 $\\pm$ 0.00 |   0.80 $\\pm$ 0.16   | 0.93 $\\pm$ 0.09 |   0.85 $\\pm$ 0.11   |\n| COAT   | len(factor) + 1 |      1     | 4.67 $\\pm$ 0.58 | 0.00 $\\pm$ 0.00   | 0.00 $\\pm$ 0.00   | 0.93 $\\pm$ 0.12 | 1.00 $\\pm$ 0.00   | 0.96 $\\pm$ 0.06 |\n| COAT   |        2        |      3     | 3.67 $\\pm$ 1.53 | 0.00 $\\pm$ 0.00   | 0.00 $\\pm$ 0.00   | 0.73 $\\pm$ 0.31 | 1.00 $\\pm$ 0.00   | 0.82 $\\pm$ 0.22 |\n\nOne can observe that COAT is not sensitive to these hyperparameters, and performs robustly well than the baselines under different hyperparameter setups.\n\n\n\n> W3 Model names are inconsistent. The name in Fig 4(c) is not the same as other names.\n\nWe have fixed them in the revised manuscript.\n\n> W4 GPT-4 reasoning in Fig 7(c) is unclear in meaning.\n\nFig 7(c) shows the result based on directly prompting LLM to reason for the causal relations among given factors. We add explanations in the figure caption now.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "V5lJaRrPaW", "reviewer": "Reviewer_1cJQ", "summary": "The paper tackles the problem of discovering relevant features for recovering the underlying causal graph in the absence and/or in lieu of a human domain expert. The proposed method, COAT, first queries an LLM through a prompt elucidating the task (for eg., discovering relevant features that affect a product review using a few text reviews), then the proposed variables are fed into another LLM that assigns a value to each of these variables, thus outputting structured/tabular data that can be used for causal discovery. Finally, the tabular data is used in conjunction with a traditional causal discovery algorithm (FCI and LiNGAM in this case) to retrieve a causal graph with respect to a target variable (for eg., review score) using the proposed variables. The process repeats until the proposed variables form a markov blanket for the target variable w.r.t the raw unstructured input data (for eg., the text reviews), progressively expanding the markov blanket in each iteration. Additionally, the LLM can receive feedback at the end of each iteration in the form of samples that the proposed variables cannot sufficiently explain. In particular, the authors propose clustering the samples w.r.t the latent variables induced by the LLM and picking the samples in the cluster with the largest conditional entropy.\n\nInitial theoretical analysis of the proposed method implies that the proposed method is able to identify the markov blanket for a target variable using the proposed variables given that enough iterations of COAT are performed.\n\nThe authors evaluate COAT empirically over two synthetic datasets and three real-world datasets. They compare COAT against two simple baselines 1) factors being directly proposed by the LLM based on the prompt without further iterations 2) factors being proposed by LLM when queried using both the prompt and some samples of raw observations. The second baseline is essentially COAT without the LLM receiving any feedback after each iteration. The experiments are conducted using 10 different LLMs and primarily one causal discovery algorithm (FCI), with additional experiments on one dataset using LiNGAM. Additionally, the paper proposes two novel metrics for quantitatively assessing the performance of LLMs for feature proposals to be used for causal discovery.\n\n\nUpdate: I moved my rating up in the hope that teh authors will add the experiments as they promised to the final version. We have no way of enforcing it but hopefully, the authors will follow up on their promise.", "questions": "The paper addresses an important and timely problem and proposes a simple and intuitive solution, leveraging the strengths of LLMs and traditional causal discovery methods. While the experiments demonstrate the effectiveness of the proposed method over two simple baselines, stronger baselines and more ablations on prompts and factor annotations would strengthen this claim. Theoretical analysis is limited to the well-studied causal discovery aspect of the pipeline while making strong assumptions on the powerfulness of the LLMs, diversity and faithfulness of the raw observational data, and the number of iterations being sufficiently large, seems rather unsurprising.", "limitations": "See the weakness and questions above.", "rating": 5, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "The paper addresses the important problem of causal discovery and employs an effective two pronged approach involving LLMs and traditional causal discovery algorithms. This approach leverages the strengths of both the LLMs and the causal discovery algorithms i.e, ability to respond to complex prompts and unstructured data with high-level and possibly noisy information, and robust causal discovery with strong theoretical guarantees although requiring strong assumptions on the faithfulness of the data and causal mechanisms, respectively. Overall, I believe this is a promising direction wherein the two components complement each other effectively.\n\nThe empirical evaluation is sufficient in terms of the large number of LLMs considered and the moderate amount of datasets evaluated. The results, based on the chosen metrics, sufficiently demonstrate the effectiveness of the proposed method over the simple baselines.\n\nFinally, the paper is well-written and clearly explains the steps involved in each iteration. The further explanations provided in the appendix also aid in this.", "weaknesses": "The theoretical aspects of the proposed algorithm are exaggerated in the introduction. Given the strong assumptions of “sufficiently powerful” LLMs, “sufficiently diverse” examples and further assumptions pertaining to the chosen causal discovery method, the propositions, while appreciated, are rather straightforward. In particular, it would be far more interesting to theoretically analyse the impact of modules involving the LLMs themselves, such as the chosen prompt template, quality of factor annotations and responsiveness of LLMs to feedback regarding causal discovery, even though some of these are evaluated empirically. Also, an analysis on the rate of convergence of COAT would be beneficial.\n\nSecondly, while the modularity of the proposed approach facilitates utilising a cross product of LLMs, causal discovery methods and feedback mechanisms, it also necessitates extensive ablation studies. In particular, the paper would be strengthened by a thorough ablation of the initial prompts and feedback. In particular, a discussion and ablation on the chosen prompt template and its effect on the proposed factors, or lack thereof, is needed. A robust template would allow more seamless adoption of the proposed method. Finally, the chosen baselines are far too simple to make any strong claims on the effectiveness of COAT. Comparing against some of the methods covered in the related work section would help bolster this claim.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for the insightful and constructive comments on our work. We hope our response can sufficiently address your concerns.\n\n> W1.1 \"strong assumptions\" of “sufficiently powerful” LLMs...\n\n\n**Thank you for pointing out these potentially confusing words. We revised the paper to clarify that \"sufficiently powerful\" and \"sufficiently diverse\" are not part of assumptions.** These words appear in Sec 3.3, which are used to give an intuitive and concrete description of the COAT algorithm. \n\n**Our theoretical results in Sec 3.4 do not rely on those assumptions.** \n\n- Proposition 3.1 is to show why we expect new factors to satisfy Eq 6. Test on the Eq 6 is a conditional independent test. **The assumption behind this is the faithfulness condition**, i.e., the conditional independence reflects the d-separation of the causal graph. \n- Proposition 3.3 concretely characterizes the impact of LLM's ability on factor identification, and it requires $p>0$ and $C_\\Psi>0$ so that Eq 9 is about a finite number. \n\n\n> W1.2 Impacts of modules.\n\n**The impacts of each module in COAT can be characterized by our theoretical framework.** Let $Y$ be the target variable. Before entering one certain COAT iteration, $k$ factors *(denoted by $h_{[k]}(X)$ as defined on page 6 line 202)* have been proposed and verified. In this iteration, the LLM receives a feedback constructed with those factors, and then *we expect the LLM to propose a new factor $w_{k+1}$ such that $Y \\perp w_{k+1}(X)\\mid h_{[k]}(X)$ doesn't hold*. **In definition 3.2 (page 6 line 209), we define two measures to quantify the capabilities of LLMs in the COAT framework**: \n\n- **Perception Score** $p$: the probability that the LLM proposes a new factor. This can be seen as a measure of the LLM's responsiveness to the given prompts and the feedback.\n- **Capacity Score** $C_\\Psi$: the decreasing ratio of the conditional mutual information, as described in Eq 8 on page 6.  This can be seen as a **measure of the quality of the factors proposed by the LLM**.\n\nAt each iteration, we prompt LLM to propose factors multiple times so the two measures can be estimated, as shown in Table 6 (page 25, appendix). With the help of these two metrics, **we then theoretically analyze the impact of the LLM-involved modules on the number of COAT rounds**, as shown by Eq 9 in proposition 3.3 (page 6). \n\n\n> W1.3 Convergence rate of COAT.\n\n**As suggested by the reviewer, we are happy to further improve proposition 3.3 with the result about rate of convergence** (the proof is similar): *with probability at least $1-\\delta$, the following inequality holds*:\n$$\n\\frac{I(Y;X\\mid h_{\\le t}(X))}{I(Y;X)} \\le \\left(\\frac{1}{1-C_\\Psi}\\right)^{-tp-z_{\\delta}\\sqrt{tp(1-p)}}\n$$\nThat is, under the setting in proposition 3.3, with both $C_\\Psi$ and $p$ being positive, COAT would converge exponentially with its number of rounds.\n\n> W2.1 Ablation of the prompt templates.\n\nWe conduct an ablation study with a different prompt template following [1]:\n- We put the task description (also the format instructions) in the beginning *[System]* part; and we put samples in the last *[Data]* part of the prompt.\n- The *markdown grammar* is replaced by blankets to represent headings, like `[System]`, `[Data]`, and `[Groups with Y=1]` ...\n- 3 COAT iterations are performed, which is aligned with the original experimental setup.\n\nCOAT with changed prompt template:\n\n|                | MB | NMB | OT | Recall | Precision |   F1  |\n|----------------|:--:|:---:|:--:|:------:|:---------:|:-----:|\n| GPT-4          |  4 |  0  |  0 |  0.80  |   1.00    | 0.89  |\n| GPT-3.5-Turbo  | 4  |  0  |  0 |  0.80  |   1.00    | 0.89  |\n| Mistral-Medium | 3  |  0  |  0 |  0.60  |   1.00    | 0.75  |\n\nWe observe that COAT is robust to the choice of templates, rejects unexpected factors (zero *NMB* and *OT*), and keeps a high precision.\n\n\n[1] Judging llm-as-a-judge with mt-bench and chatbot arena, NeurIPS'23.\n\n\n\n> W2.2 Baselines are too simple.\n\nWe need to clarify that, to the best of our knowledge, the *META* baseline is already a strong baseline, as the extensive empirical evidence about LLM-based methods in causality-related tasks [2]. If you happen to know a stronger baseline, please let us know. \n\nMeanwhile, **we additionally construct a stronger baseline with CoT** based on *DATA*, where the LLM is prompted to \"Think step by step to consider factors\", and to output these factors in the same format as other methods.\n\n| LLM           | method           |         MB        |       NMB       |        OT       |      Recall     |    Precision    |        F1       |\n|----------------|------------------|:-----------------:|:---------------:|:---------------:|:---------------:|:---------------:|:---------------:|\n| GPT-4          | CoT baseline | 4.33   ± 0.58 | 0.83 ± 0.29 | 0.17 ± 0.29 | 0.87 ± 0.12 | 0.81 ± 0.02 | 0.84 ± 0.06 |\n|                |       COAT       |   4.00 ±   0.82   |   0.33 ± 0.47   |   0.00 ± 0.00   |   0.80 ± 0.16   |   0.93 ± 0.09   |   0.85 ± 0.11   |\n| GPT-3.5        | CoT baseline | 5.00 ±   0.00       | 1.00 ± 0.00       | 1.33 ± 0.58 | 1.00 ± 0.00       | 0.68 ± 0.05 | 0.81 ± 0.04 |\n|                |       COAT       |   3.67 ±   0.47   |   0.00 ± 0.00   |   0.00 ± 0.00   |   0.73 ± 0.09   |   1.00 ± 0.00   |   0.84 ± 0.07   |\n| Mistral-Medium | CoT baseline | 4.33   ± 0.58 | 1.00 ± 0.00       | 0.67 ± 0.58 | 0.87 ± 0.12 | 0.73 ± 0.07 | 0.79 ± 0.05 |\n|                |       COAT       |   4.67 ±   0.47   |   0.00 ± 0.00   |   0.00 ± 0.00   |   0.93 ± 0.09   |   1.00 ± 0.00   |   0.96 ± 0.05   |\n\n\nIt shows that LLMs with CoT can:\n- be aware of high-level factors behind data (lower OT than *META*);\n- still struggle to distinguish the desired factors in Markov Blanket (higher *NMB* than *COAT*).\n\n\n[2] Causal reasoning and large language models: Opening a new frontier for causality, arXiv'23.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "QtxXeDpucn", "reviewer": "Reviewer_PF9K", "summary": "This paper presents Causal representatiOn AssistanT (COAT), which introduces large language models (LLMs) to bridge this gap. LLMs are trained on massive observations of the world and have shown great capability in extracting key information from unstructured data. Thus, employing LLMs to propose useful high-level factors and craft their measurements is natural. COAT also uses CDs to find causal relations among the identified variables and provides feedback to LLMs to iteratively refine the proposed factors. This mutual benefit enhances both LLMs and CDs.", "questions": "1. How can we ensure that LLM does not introduce erroneous prior knowledge for reasoning?", "limitations": "N/A", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. Interesting topic. Employing LLMs to propose useful high-level representations for causal discovery.\n2. Develop two benchmarks for the unstructured causal discovery. AppleGastronome and Neuropathic.\n3. Derives the first metrics that measure the causal representation learning capabilities of various LLMs.", "weaknesses": "1. ‘We will release an anonymous link during the discussion period.’ I will consider raising my score if the code is reasonable.\n2. The contribution of LLM in COAT is a little small. I assume LLM is used as a representation tool to learn the conceptual level attributes, including iterative refining. The causal structural learning can still be considered as the downstream task.\n3. COAT will inherit the shortcomings of downstream causal structure learning algorithms.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "> W1 ‘We will release an anonymous link during the discussion period.’ I will consider raising my score if the code is reasonable.\n\nWe send an anonymized link of code to the AC in a separate comment, as we are not allowed to include any links to external pages in the responses this year.\n\n> W2 The contribution of LLM in COAT.\n\nLLM is extensively involved in the factor proposal phase as a representation assistant:\n- In the factor proposal phase, LLMs are extensively leveraged to identify useful high-level factors from the given samples. The pre-trained knowledge and the reasoning ability of LLMs are heavily involved during this phase. Although COTA reduces the reliance on LLMs, **the final results still depend on the capabilities of LLMs**, as shown in our theories and experiments.\n- Moreover, when an external interface to annotate the factors is not available, LLMs will be leveraged to parse the data. Without LLMs, we can still not obtain tabular data readily to be used for further causal structural learning.\n- For the causal feedback module, it is still crucial that LLMs need to properly distinguish the already verified factors and other potential factors that may affect the partitioning of the provided groups.\n\nAlthough in the current design of COAT, the causal structural learning module can be separated, the causal structural learning module plays an important role in providing various feedback, such as the existence of a hidden confounder, when extending COAT to more challenging scenarios.\n\n> W3 COAT will inherit the shortcomings of downstream causal structure learning algorithms.\n\nIt is common for all rigorous causal discovery methods to have certain assumptions and requirements on the data, while COAT may relieve the requirements:\n- **Well-specified variables**: Classic causal discovery methods work on tabular data whose columns are meaningful variables. Here COAT extends their scope on unstructure data utilizing the power of LLMs. \n- **Assumptions on the data and population distribution**: Assumptions are inevitable for any causal identifiability guarantee. COAT's factor proposal phase is disentangled with causal discovery, so its identifiability can hold even when one of the chosen causal discovery methods doesn't. For instance, the Apple Gastronome benchmark doesn't satisfy the linear non-Gaussian assumption, yet it still works on the factor proposal task when cooperating with LiNGAM (Appendix F).\n\n> Q1 How can we ensure that LLM does not introduce erroneous prior knowledge for reasoning?\n\n\nThanks for the good question. This question shares exactly the same spirit as the motivation of the paper. All factors proposed by LLM will be checked on data in the factor proposal phase. LLM can propose irrelevant factors (as witnessed on the *META* baseline), but they will be filtered out in each COAT iteration (as witnessed on the *DATA* baseline, the single COAT iteration, and *COAT* itself).", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
Dear Reviewers,

Thank you for your time and constructive comments on our work. To summarize, all reviewers agree **the paper's proposal to reliably advance rigorous causal discovery methods with the advantages of foundation models like LLMs is novel and valuable** (AG9s, szEm, 1cJQ, PF9K). The method is sufficiently evaluated in constructed benchmarks (szEm, 1cJQ, PF9K). The soundness is justified by the provided theoretical results (szEM, 1cJQ). The paper also develops novel metrics to measure the LLMs' ability to propose desired factors (1cJQ, PF9K). Comprehensive case studies are presented on three realistic data (szEM, 1cJQ).


We believe all of the reviewers' concerns can be addressed. In the following, we brief our responses to the main concerns and suggestions raised in the review:

- **The guarantee on factor proposal** (AG9s, 1cJQ, PF9K)
    - In this paper, we show COAT can provably identify a Markov Blanket of a given target variable. In section 3.4, we propose two new metrics ($p$ and $C_\Psi$) to characterize the capability of LLMs in identifying useful high-level factors and also establish the corresponding guarantee.
    - We further discuss the intuition behind $p$ and $C_\Psi$, and also give the convergence rate of COAT in the response to Reviewer [1cJQ](https://openreview.net/forum?id=w50ICQC6QJ&noteId=jw35g23fxl).
- **Baselines and ablation study** (szEm, 1cJQ)
    - We construct an additional stronger baseline with CoT prompting to enhance the evaluation. We present the key results and interpretation in the response to Reviewer [szEm](https://openreview.net/forum?id=w50ICQC6QJ&noteId=6vJZbvuIRV) and [1cJQ](https://openreview.net/forum?id=w50ICQC6QJ&noteId=jw35g23fxl).
    - We provide empirical evidence to show COAT is not sensitive to hyperparameters like group size or cluster size in the response to Reviewer [szEm](https://openreview.net/forum?id=w50ICQC6QJ&noteId=6vJZbvuIRV).
    - We also provide empirical evidence to show COAT is not sensitive to the choice of prompt templates in the response to Reviewer [1cJQ](https://openreview.net/forum?id=w50ICQC6QJ&noteId=jw35g23fxl).

We also provided an anonymous link to our sample code for reproducing the results in our paper to the Area Chair, according to the NeurIPS requirements.

Please let us know if there are any other concern and we are happy to discuss them. We would appreciate it if you could take our responses into consideration when making the final evaluation of our work.

Sincerely, 

Authors.

Meta-review: The discussion on the manuscript was intense and of high quality between authors and reviewers. I have to recognize that the high engagement of reviewers (more tha 30 posts were published) and authors significantly contributed to understand each other on the presented topic, and this made me more confident to propose acceptance for this paper.

Indeed, all reviewers agreed that the outcome of the rebuttal and subsequent discussions were satisfying. The authors managed to address and solve all the main issues raised by reviewers, they all agree on this and some of them consequently raised their score.

As also some reviewers noticed, I suggest the authors to develop all fixing discussed and promised during the rebuttal and discussion period (revise their exposition in various sections of the manuscript and move some content to the main file). In particular, all the revisions that have been promised by the authors, see the many posts, must be honored.
Decision: Accept (poster)