Paper: Causal Modelling Agents: Causal Graph Discovery through Synergising Metadata- and Data-driven Reasoning (ICLR-2024)

Reviewer comment (Experiments):
Can the authors provide further evidence from experimental or observational data for the discovered causal relationships with the CMA?

Author response:
>The statistical correlations presented in Appendix A.3.4 do suggest the causal relationship between LW and HFLX. An experiment that intervenes LW may help reveal such causality.

  

Thank you for your response. We had previously conducted an experiment which intervenes on the LW variable in **Appendix A.3.4**. To maximise clarity, we have amended the manuscript to illustrate these results in two separate figures (instead of the single **Figure 13**). For the experimental/counterfactual intervention, please also see the [anonymised figure](https://i.imgur.com/3T1eMcb.png).
Indeed, for each of our discovered novel relationships, we have included:

1.  A regression (observational) analysis, as requested, to assess for possible statistical associations between the variables.
    
2.  An experiment which intervenes on the parent variable (counterfactual intervention) to assess a given relationship from a causal perspective.
    
3.  Experimental/observational evidence from the literature to contextualise and/or corroborate our findings.
    

Please find references for these analyses below:

-   LW → HFLX:
    1.  Observational analysis: **Figure 13, Appendix A.3.4**. 
    2.  Counterfactual intervention: **Figure 14, Appendix A.3.4**. 
    3.  Literature: **Section 4.2**, main manuscript.  
	    

-   Sex → P-tau:
     1.  Observational analysis: **Figure 34, Appendix A.7**
    2.  Counterfactual intervention: **Figure 3B**, main manuscript.   
    3.  Literature: **Section 4.3**, main manuscript.
    4.  Additional literature: **Appendix A.7.1**
    

-   sTREM2 → P-tau:
    1.  Observational analysis: **Figures 32 and 33, Appendix A.7**
    2.  Counterfactual intervention: **Figure 31, Appendix A.7**
    3.  Literature: **Section 4.3**, main manuscript.
    4.  Additional Literature: **Appendix A.7.1**
    
___

Thank you once more for your time. Please let us know if there are any additional points you’d like to discuss, and if not, we hope that the reviewer might consider an update.

Reviewer comment (Novelty):
The use of LLM to tweak intermediate results seems clever but it is hard to assess its technical contribution.

Author response:
Thank you for noting these important related works. In contrast to previous work, the CMA is the first framework to use LLMs as priors, critics during an iterative learning procedure, and as post-processors. We demonstrate the utility of this approach as per our results in **Section 4.2**, and in **Section 4.3**, we identify novel causal structures in the challenging real-world task of jointly modelling the radiological and clinical phenotype of Alzheimer’s disease. An additional novel aspect of our work is that our framework is the first causal discovery approach to output a multi-modal structural causal model (that is, we generalise and extend the DSCM framework), which enables reasoning over multi-modal data (e.g., for computing counterfactuals in the imaging space, as can be seen in Figure 3, Panel C of the main manuscript).  
  
The method developed by Long et al.[1] assumes knowledge of the Markov Equivalence Class (MEC) containing the true Directed Acyclic Graph (DAG). This is a strong assumption for real-world applications as it presumes optimal outputs from causal discovery algorithms. Their work shows promising results in substituting an LLM in lieu of a human expert in this setting, but their approach serves as a post-processing step, reliant on a priori knowledge of a MEC which contains the true causal graph. In contrast, the CMA does not require any prior assumptions about the graph's structure.  
  
Ban et al.[2] use an LLM's output as a prior for causal discovery algorithms, demonstrating its effectiveness with synthetic datasets, an approach also explored in the ‘LMPriors’ paper by Choi et al.[3]. However, this method lacks a feedback mechanism for hypothesis refinement, potentially limiting novel causal link detection by ignoring signals from the data. An example of this is the 'biological sex' -> 'phosphorylated tau protein' link identified by the CMA in **Section 4.3**, but overlooked by domain experts. We additionally note that in contrast to Ban et al.[2], we consider a real-world multi-modal dataset as one of our experiments, demonstrating our framework’s empirical utility for more complex and varied data scenarios beyond synthetic cases.  
  
We have revised the text in the Related works section of the paper to better highlight these differences.

Reviewer comment (Theory):
It is unknown how LLM is doing with respect to its memory. Does LLM always try to update edges in order to maximize data fitting? If the data fitting is based on the currently predicted causal graphs, how can it improve its causal graph? It does not work like an EM algorithm. Does LLM ‘regret’ its previous decision if fitting becomes worse? Considering developing a causal discovery algorithm that is based on local search (incrementally updating causal graph based on its likelihood), how would you compare their learning trajectories?

Author response:
As you correctly point out, the data fitting step at a given iteration is based on the currently predicted causal graph. However, the CMA uses both the current data fit, as well as the data fit from the previous iteration to produce a memory. As we show in the ‘post-processing’ paragraph of **Section 3**: $\mu_t=\textsf{LLM}{\mu}(\mathcal{G}^s_t,\mathcal{G}^s_{t-1},F_t,F_{t-1})$, where for iteration $t$, $\mathcal{G}_t$ is the causal graph and $F_t$ is the metric of fit.


 
 The prompting strategy and procedure for memory generation is further outlined in ‘Listing 3: System prompt for memory generation’, and ‘Algorithm 5: Post-processing and memory generation’ in **Appendix A.1.5**. Exactly as you intuited, the LLM is therefore prompt-engineered to ‘regret’ its previous decisions if fitting degrades, and to select an action or set of actions to improve fit. The LLM constrains the super-exponential search space of potential causal graphs, which a naive local-search-based algorithm would need to explore.

Reviewer comment (Experiments):
LLM’s stochastic nature is ignored. LLM may answer differently for the same question.

Author response:
Thank you for this very important insight. We very much agree that LLMs do not always answer the same question deterministically, however we take account of this by querying the LLMs multiple times to mitigate for potential stochasticity in the outputs. We also attempt to account for failure modes in LLM reasoning. For example, Berglund et al. [4] identified a 'reversal curse' in LLMs, where they fail in logical deduction based on question order, not always generalising from 'A is B' to 'B is A'. To additionally address this layer of variability, queries are also made in both causal directions for local phase amendments. This process is described in more detail in **Appendix A.1.3**.

Reviewer comment (Evaluation):
It is essential to thoroughly examine the behavior of LLM. How does it adjust the result based on its belief (GPT-4 etc) and intermediate results passed. There are more questions remained than answered.

Author response:
Thank you once more for this important point. We strongly agree that investigating the behavioural patterns of LLMs is important to better understand their outputs, and indeed had performed a behavioural analysis in **Appendix A.2.4**: ‘Additional Results II - LLM Behavioural Patterns’. Here, we investigate behavioural patterns for three types of relationships: 1) Relationships which might plausibly exist in either causal direction (i.e., A -> B or B -> A), 2) Relationships which should only exist in a single direction (i.e., A -> B, but not B -> A), and, 3) Variables which should have no functional causal relationship between them. Furthermore, for each of these types of relationship, we assess LLM behaviour by first assuming the non-existence of the relationship, and then its existence in the A -> B direction, and then in the B -> A direction (**Appendix A.2.4; Figure 5**). 
We additionally investigated how these behavioural patterns change with the use of a Retrieval Augmented Generation (RAG) pattern [5], and find that in-context learning [6] strongly enforces causal relationships given the retrieved context (**Appendix A.2.4; Figure 6**). 

We have amended the manuscript to more explicitly reference this analysis.

Reviewer comment (Writing):
The word “metadata” is somewhat used in a mixed manner between domain knowledge already encoded in LLM and memory passed through JSON format. It should be more formally defined.

Author response:
We appreciate this suggestion. We’ve now made a number of amendments to keep the distinction between both concepts clear.

Reviewer comment (Novelty):
Given the abundance of similar papers (Long, S., Piché, A., Zantedeschi, V., Schuster, T., & Drouin, A. (2023). Causal discovery with language models as imperfect experts. arXiv preprint arXiv:2307.02390., Ban, T., Chen, L., Wang, X., & Chen, H. (2023). From query tools to causal architects: Harnessing large language models for advanced causal discovery from data. arXiv preprint arXiv:2306.16902.) in the field, the contribution is not clear.

Author response:
Thank you for noting these important related works. In contrast to previous work, the CMA is the first framework to use LLMs as priors, critics during an iterative learning procedure, and as post-processors. We demonstrate the utility of this approach as per our results in **Section 4.2**, and in **Section 4.3**, we identify novel causal structures in the challenging real-world task of jointly modelling the radiological and clinical phenotype of Alzheimer’s disease. An additional novel aspect of our work is that our framework is the first causal discovery approach to output a multi-modal structural causal model (that is, we generalise and extend the DSCM framework), which enables reasoning over multi-modal data (e.g., for computing counterfactuals in the imaging space, as can be seen in Figure 3, Panel C of the main manuscript).  
  
The method developed by Long et al.[1] assumes knowledge of the Markov Equivalence Class (MEC) containing the true Directed Acyclic Graph (DAG). This is a strong assumption for real-world applications as it presumes optimal outputs from causal discovery algorithms. Their work shows promising results in substituting an LLM in lieu of a human expert in this setting, but their approach serves as a post-processing step, reliant on a priori knowledge of a MEC which contains the true causal graph. In contrast, the CMA does not require any prior assumptions about the graph's structure.  
  
Ban et al.[2] use an LLM's output as a prior for causal discovery algorithms, demonstrating its effectiveness with synthetic datasets, an approach also explored in the ‘LMPriors’ paper by Choi et al.[3]. However, this method lacks a feedback mechanism for hypothesis refinement, potentially limiting novel causal link detection by ignoring signals from the data. An example of this is the 'biological sex' -> 'phosphorylated tau protein' link identified by the CMA in **Section 4.3**, but overlooked by domain experts. We additionally note that in contrast to Ban et al.[2], we consider a real-world multi-modal dataset as one of our experiments, demonstrating our framework’s empirical utility for more complex and varied data scenarios beyond synthetic cases.  
  
We have revised the text in the Related works section of the paper to better highlight these differences.

Reviewer comment (Evaluation):
I noticed discrepancies between what was mentioned and the results such as the performance of gpt-4 in the table 7 in Kıcıman et al. (2023). For example, NHD of GPT 4 in Kıcıman et al. (2023) was reported as 0.22 but you reported 0.35 for GPT 4 in the table 2 in your paper.

Author response:
Thank you for this point, and well noticed. We re-ran all experiments in this work, but simply had not moved this information from our Appendix to the main manuscript for the data-driven approaches for the arctic sea ice benchmark. This has now been rectified. To replicate the metadata-based benchmarks for gpt-3.5-turbo and gpt-4, we used the same ‘single prompt’ approach as in the kiciman paper (this is described in kiciman et al. [8] in their Appendix A.1, Table 14). They augment this prompt so that it can be used for ‘full graph discovery’ (this modification is described in kiciman et al. [8]; Section 3.2.1, page 13). For convenience, we provide the prompt template in full in **Appendix A.3.2, Table 5** of our own work. We believe the slightly different results could be down to a couple of factors. First, it appears that gpt-4’s performance on a multitude of tasks can change over time, as demonstrated by Chen et al. [11], which may partly explain these differences. Second, due to a lack of description as to how they handled errors, we implemented our own error-handling strategy. Namely, the model is meant to produce a single letter in {A,B,C} which reflects one of three options: 




- A: Changing $A$ causes a change in $B$

- B: Changing $B$ causes a change in $A$

- C: No causal relationship exists 





Occasionally the model might output ‘C/A’. This would be handled as an error in our work, and no edge would be proposed in these cases. A more detailed description of the experimental setup can be found in our **Appendix A.3.2**.


We have updated **Appendix A.3.3** to include the points above in our interpretation of the results for the arctic sea ice dataset.

Reviewer comment (Novelty):
While the authors do a good job regarding the related work, we consider this could be further enhanced by citing surveys that provide an overview of the relevant topics and domains. E.g., the authors may be interested on the following works: (a) for causal deep modelling: Li, Zongyu, and Zhenfeng Zhu. "A survey of deep causal model." arXiv preprint arXiv:2209.08860 (2022); (b) for Alzheimer disease neuroimaging: Varghese, Tinu, et al. "A review of neuroimaging biomarkers of Alzheimer’s disease." Neurology Asia 18.3 (2013): 239. and Márquez, Freddie, and Michael A. Yassa. "Neuroimaging biomarkers for Alzheimer’s disease." Molecular neurodegeneration 14 (2019): 1-14; and (c) Huang, Yiyi, et al. "Benchmarking of data-driven causality discovery approaches in the interactions of arctic sea ice and atmosphere." Frontiers in big Data 4 (2021): 642182, and Kretschmer, Marlene, et al. "Using causal effect networks to analyze different Arctic drivers of midlatitude winter circulation." Journal of climate 29.11 (2016): 4069-4081.

Author response:
Thank you for your kind words. We have included a reference to the causal deep modelling survey at the beginning of our discussion of LLMs and causality in the related works section.  
The Alzheimer’s disease (AD) neuroimaging references provide a nice exposition of the 18F-AV-45 tracer in particular, which is a variable we consider in our AD models. We therefore additionally include these references in the ‘Experiments’ section.  
Finally, Huang et al.’s work on data-driven causal discovery for the arctic sea ice dataset is referenced in **Section 4.2: ‘Benchmarking Experiments’.**

Reviewer comment (Presentation):
In the related work section, the authors may consider weighting the views and findings regarding LLMs and causality expressed in the following paper: Zečević, Matej, et al. "Causal parrots: Large language models may talk causality but are not causal." arXiv preprint arXiv:2308.13067 (2023).

Author response:
Thank you for pointing us to this contemporary work. We have added a discussion of this paper in a new section on the use of LLMs for causal reasoning. This can be found in the ‘Related work’ section of the main manuscript.

Reviewer comment (Evaluation):
When reporting results in Section 4.1, the authors measure average data likelihood and the deviation. It would be helpful to have some reference value to understand whether the reported values are good or not and why.

Author response:
In **Section 4.1**, the most meaningful comparison lies in the relative difference in data likelihoods under each model. We have amended the manuscript to highlight that the model that matches the data-generating process acts as the reference likelihood.  
We then demonstrate that making incorrect modelling assumptions about the data-generating process leads to a lower data likelihood. Finally, the difference in data likelihoods is compared with a pairwise Tukey HSD test. Assessing relative model performance in this way has been employed by other related works[1].

Reviewer comment (Experiments):
How is the threshold for DAG-GNN selected?

Author response:
We assessed two thresholds for the DAG-GNN algorithm and chose the threshold with the best performance for inclusion in the main manuscript. We began with a threshold of 0.3 as per the DAG-GNN paper by Yu et al.[2], who themselves selected this parameter based on prior work by Zheng et al.[3]. We also assessed a threshold of 0.1, which had worse performance on our benchmarks. Results of the DAG-GNN algorithm with different thresholds can be found in **Tables 6, 9, and 12** of **Appendix Sections A.3.3, A.4.3, and A.5.3**, respectively. We had also conducted a similar hyperparameter tuning experiment for the NOTEARS algorithm and reported those results in the same tables.

Reviewer comment (Presentation):
Table 1: align results to the right so that differences in magnitude are quickly visualized.

Author response:
We very much appreciate the formatting/readability suggestions. We’ve amended the tables and re-ordered some of the columns in **Table 1** to improve clarity.