Paper: Causal Inference Using LLM-Guided Discovery (ICLR-2024)

Reviewer comment (Evaluation):
The experimental result could have been highly affected by the popularity of the datasets and domain knowledge on the internet and using LLMs to guide causal discovery can be very limited to those commonly available data.

Author response:
We agree that there is a possibility that LLMs have seen the BNLearn datasets in its pretraining setup, and highlight this in the limitations section of our paper as well. However to tackle this claim we include the Neuropathic Pain Diagnosis dataset in our updated paper draft and show the effectiveness of our methods on the same. The dataset is less popular and requires a very nuanced medical understanding in order to correctly assess the causal relationship, therefore depicting how LLMs can be employed in real world settings which are not straightforward. That being said, we still agree that memorization of causal discovery datasets is an issue in current evaluation of LLMs which is scope for future research work.

Reviewer comment (Experiments):
It is not clear what the advantages of using LLMs as a source of domain knowledge are as it may have issues with hallucinations unless there are large-scale experiments that show some domain knowledge are impractical to obtain via domain experts and need LLMs to guide such effort.

Author response:
One of the practical implications of using LLMs can be to aid human experts as they can look at the LLM generated graphs and edit it further thus saving time as well as efforts. While LLMs do suffer from issues like hallucinations, they still carry an understanding of the world which can be used to assist the human experts to improve and fasten the causal graph creation process. It will be on human experts to accept the suggestions and reasoning provided by LLMs while constructing causal graphs thus removing any hallucinated or false outcome.

Reviewer comment (Experiments):
It is also not clear to me why the estimation is not compared against with those estimation methods that use causal graphs or simply a Markov equivalence class of DAGs [4] even if there is only the information of causal orders available to show the merits of using only the causal order for estimation.

Author response:
Thank you for the review, we have added a comparative analysis (Refer Table A16 in Appendix) on Asia dataset to show estimation using backdoor set from the given graph vs given causal/topological order of the graphs.

Reviewer comment (Theory):
Is it possible that the causal order output by LLMs orient a new unshielded collider in the output of other causal discovery algorithms?

Author response:
Since we do not have a check for this in our current algorithm, there is a slight possibility that LLMs might orient new unshielded colliders when used as priors with PC Algorithm. We will incorporate this check in our pipeline.

Reviewer comment (Experiments):
Have the authors tried to provide background knowledge to PC and compare that with PC+LLM? For example, randomly sample from the ground truth and provide such background knowledge to PC or other algorithms.

Author response:
We have added Table A15 in Appendix to show comparison between LLM and Ground truth prior for causal discovery.

Reviewer comment (Novelty):
The first contribution regarding the sufficiency about knowing the causal order is not novel and a rather straightforward insight seeing that conditioning on any 'upstream' node of a treatment variable in a DAG results in a valid adjustment set. Therefore, it is certainly good to point this out again, but this is not a new contribution by this work.

Author response:
The technical result may not be a new result for the backdoor criterion. But its application leads to a valid order-based method for causal discovery using LLMs, whereas we argue that existing pairwise-based prompts that elicit edges may not be valid. The pre-existing work has heavily leveraged LLMs for causal edge detection between any given nodes by optimizing on metrics like SHD for full graph discovery. But the identification of direct causal edge between any given pair of nodes heavily depends on the presence of other nodes, therefore we believe probing LLMs for getting the order between any given pair is a better question framing since the pairwise order only depends on the variables in the question. 

For example, consider the data-generating process, lung cancer ->doctor visit -> Positive Xray. If asked, an expert would affirm a causal edge from ‘Lung cancer’ to ‘Positive Xray’ (indeed, such an edge exists in the BNLearn Cancer [1]). However, if they are told that the set of observed variables additionally includes doctor visit, then the correct answer would be to not create a direct edge between ‘Lung Cancer’ and ‘Positive Xray’, but rather create edges mediated through ‘doctor visit’. Note that the causal order, ‘Lung Cancer’ < ‘Positive Xray’ remains invariant in both settings. To fill this conceptual gap in the existing work, our work focuses on causal order and probes LLMs specifically from this perspective since the causal order is useful for most downstream tasks and in practice has huge applications as it is locally consistent in terms of estimation (refer table 1).

Reviewer comment (Novelty):
The paper overall seems rather incremental, seeing that the paper by Kiciman et al. is already providing some significant prior work in this regard for causal discovery. However, I acknowledge the incorporation of LLM generated knowledge with statistical approaches such as PC.

Author response:
While our work draws inspiration from prior research to enhance LLM-based pipelines for aiding in causal discovery, our approach stands out by prioritizing the exploration of how LLMs can effectively identify causal order. This emphasis on identifying causal order is more valuable compared to solely focusing on direct and indirect edge connections, which often rely heavily on other variables. But since order between any given pair of variables remains invariant to other present variables and they directly correlate with downstream tasks like Causal effect estimation, we emphasize and build our frameworks towards that. While previous papers have focused on pairwise causal relationship discovery, our work takes inspiration from pre-existing algorithms to propose novel LLM based pipelines (such as the Triplet prompt). Also, we present how LLMs can be easily adapted with different classes of discovery algorithms whether it is score based or constraint based, for enhancing discovery performance. Most importantly, we emphasize on how current LLMs can be used for downstream applications like causal inference, effect estimation, prediction, etc by leveraging them as knowledge base for finding correct causal order whereas previous work (like Kiciman et al.) have focused more on pairwise causal relationship identification for graph discovery. We show how this approach is not suitable and does not leverage LLMs potential to help in causal tasks by helping get the correct order of the graph by aiding discovery algorithms like PC, CamML, etc.

Reviewer comment (Evaluation):
You are arguing that looking at SHD is often the wrong metric. However, these works using SHD typically address the problem of inferring the whole DAG structure without any particular causal task in mind, while you are only concerned with the causal order for effect estimation problems. In that sense, the SHD makes sense as a metric to see how good the inferred DAG structure is.

Author response:
While we do agree that SHD is a good metric for evaluating similarity of two graphs, applying this metric specifically for evaluating LLMs’ performance for causal discovery is something we disagree with since direct and indirect edge connections cannot be determined due to the subjectivity of the task and it’s dependence on other variables present (as demonstrated in the ‘Lung Cancer’, ’Positive X-Ray’ and ‘Doctor Visit’ example above from Cancer dataset). While consideration of other variables can have a direct impact on edge formation between any given pair of nodes, the causal order remains invariant. Therefore probing LLMs to find order is more valid, and also transfers to downstream causal tasks like effect estimation, prediction etc. which further helps position LLMs for utility in real world applications.

Reviewer comment (Experiments):
While you reference the work by Kiciman et al., a more direct comparison is missing. In particular, the related open source package https://github.com/py-why/pywhy-llm has several prompting techniques for inferring structural information. That being said, they do not combine it with methods like PC, which is the novel part in your work.

Author response:
Thank you for pointing this out, the base prompt which we have used in our work has been taken directly from Kiciman et al. We further compare this with various prompting strategies including our Triplet prompt for a more direct comparative analysis on DTop and SHD. We have updated the manuscript and cited Kiciman et al. for the Base prompt.

Reviewer comment (Theory):
Assumption 3.3 states there is no latent confounding between treatment and target, but you actually need the stronger assumption that there is no latent confounding between any observed variables. Otherwise for instance proposition 4.2 will fail: Suppose we want to find a valid backdoor set for $X \to Y$, and there is a third observed variable $Z$ that is not a cause or effect of $X$ or $Y$, but there is a latent variable causing $X$ and $Z$, and another causing $Y$ and $Z$. Then a valid topological ordering of the observed variables is $Z < X < Y$, but adjusting for $Z$ actually opens the backdoor path.

Author response:
**Question**: Assumption 3.3 states there is no latent confounding....observed variables is Z<X<Y, but adjusting for Z actually opens the backdoor path.

**Answer**: Thank you for pointing it out and we agree with you. We revised the manuscript to include the line “We assume that the underlying causal graph has no unobserved confounders''.

Reviewer comment (Theory):
Proposition 4.2 requires the further assumption that $i < j$.

Author response:
**Question**: Proposition 4.2 requires the further assumption that i<j.

**Answer**: Thank you for pointing it out. We’ve updated the proposition statement in the revised manuscript.

Reviewer comment (Theory):
Paragraph below proposition 4.2, "causal effect practitioners tend to include all confounders ...": Can you provide a reference for this claim? Either way, what you propose goes further than including all *confounders*: you also include variables that cause either the target or the treatment but not both.

Author response:
**Question**: Paragraph below proposition 4.2, "causal effect practitioners tend to include all confounders ...":....the treatment but not both.

**Answer**: The statement is based on causal inference practice in statistics and econometrics where all observed covariates (assumed to be pre-treatment) are included in the estimation model. As noted by Cinelli and Pearl (2020), variables that cause only the target are “good” variables to condition on, so they do not lead to any trouble. Including causes of treatment only can lead to estimation issues (e.g., including an instrumental variable) that increase with the strength of the instrument’s causal effect on treatment. In practice, however, strong instruments are rare. https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3689437

Reviewer comment (Theory):
The definitions of $E_m, E_f, $E_d$ for SHD are incorrect: a wrongly oriented edge will add one to each of these three variables. Further, I think you mean to add the cardinalities rather than the sets themselves.

Author response:
**Question**: The definitions of Em, Ef, E_d$ for SHD are incorrect: a wrongly oriented edge will add one to each of these three variables. Further, I think you mean to add the cardinalities rather than the sets themselves.

**Answer**: Apologies. We’ve removed the mathematical formula for SHD as it is not required to understand the rest of the paper and the detailed formula will affect the readability.

Reviewer comment (Presentation):
Algorithm in section 5.2: Steps 2 and 3 and the difference between them are unclear from the text. For algorithms, it may be better to use pseudocode, or at least some mathematical notation.

Author response:
**Question**: Algorithm in section 5.2: Steps 2 and 3 and the difference between them are unclear from the text. For algorithms, it may be better to use pseudocode, or at least some mathematical notation.

**Answer**: We’ve updated the Algorithm 2 in the revised manuscript to clearly explain the steps.