Title: When Additive Noise Meets Unobserved Mediators: Bivariate Denoising Diffusion for Causal Discovery
Venue: NeurIPS 2025
Accepted: yes
Keywords: causal discovery

additive noise model

diffusion
Abstract: Distinguishing cause and effect from bivariate observational data is a foundational problem in many disciplines, but challenging without additional assumptions. Additive noise models (ANMs) are widely used to enable sample-efficient bivariate causal discovery.
However, conventional ANM-based methods fail when unobserved mediators corrupt the causal relationship between variables. This paper makes three key contributions: first, we rigorously characterize why standard ANM approaches
break down in the presence of unmeasured mediators.  Second, we demonstrate that prior solutions for hidden mediation are brittle in finite sample settings,
limiting their practical utility. To address these gaps, we propose Bivariate Denoising Diffusion (BiDD) for causal discovery, a method designed to handle latent noise introduced by unmeasured mediators. Unlike prior methods that infer directionality through mean squared error loss comparisons, our approach introduces a novel independence test statistic: during the noising and denoising processes for each variable, we condition on the other variable as input and evaluate the independence of the predicted noise relative to this input. We prove asymptotic consistency of BiDD under the ANM, and conjecture that it performs well under hidden mediation. Experiments on synthetic and real-world data demonstrate consistent performance, outperforming existing methods in mediator-corrupted settings while maintaining strong performance in mediator-free settings.


Peer reviews:
comments: SUMMARY:
This paper discusses the problem of causal direction identification under the addictive noise model with unmeasured mediator (ANM-UM). The authors first analyze the equivalent condition for ANM-UM to reduce to common ANM. Then, they analyze how existing methods based on ANM can fail under unmeasured mediator. Next, under the linear setup, they develop an identification rule of the causal direction via denoising. Experiments on synthetic and real-world datasets are conducted for verification.

STRENGTHS AND WEAKNESSES:
**Strength**

**1**. This paper establishes the necessary and sufficienct condition for the ANM-UM to reduce to the ANM.

**2**. The authors conducts exhausive analyses on why existing methods fail under unmeasured mediation.

**3**. They conduct both synthetic and real-world experiments for validation.

**Weakness**

**1**. The consistency of the decision rule is shown only for the case where ANM-UM reduces to ANM. For the irreducible case, e.g., nonlinear causal relationship, there is no theoretical intuition why this method can work. 

Indeed, the authors consider empirical verifications in Sec. 5. But the improvement of BiDD over baseline methods seems to be insignificant. I not sure whether this result can support the conjecture that BiDD also works under the irreducible case.

QUESTIONS:
**1.** Can the authors explain the major challenge when extending Thm. 4.2 to the irreducible case? Is there any potential idea for the extension?

**2.** If BiDD works under the irreducible case (as conjectured in Sec. 4.2) and the baselines do not work under ANM-UM (as shown in Sec. 3), why their performance seems to be comparable?

LIMITATIONS:
Yes
TITLE: Official Review
CLARITY: 3
IMPACT: 3
ORIGINALITY: 3
DATE: 2025-07-03

comments: SUMMARY:
This paper proposes Bivariate Denoising Diffusion (BiDD), a method for identifying causal direction in the presence of unobserved mediators. BiDD leverages diffusion models, hypothesizing that noise predictions are less dependent on the cause than the effect. Empirical evaluations demonstrate BiDD’s superiority over baselines on synthetic and real-world data.

STRENGTHS AND WEAKNESSES:
Strengths:  
- BiDD adapts diffusion models to bivariate causal discovery.  
- Extensive experiments validate BiDD’s performance.

Weaknesses:
- First, the paper does not adequatly explain why diffusion models are theoretically better suited for this task compared to alternatives like VAEs.   
- Decision Rule 1 appears incorrect, as it ignores the edge case where \( MI(\epsilon_Y, X) = MI(\epsilon_X, Y) \).
- Theorem 4.2 only addresses identifiability when no nonlinear mediator exists. The paper should further provide guarantees for the more general case where nonlinear mediators are present.

QUESTIONS:
- What properties of diffusion (e.g., noise-perturbation process) make them more effective in handling unobserved mediators? Can you offer a theoretical analysis to it?
- What is the identifiability when nonlinear mediator exists.

LIMITATIONS:
See Weaknesses.
TITLE: Official Review
CLARITY: 3
IMPACT: 2
ORIGINALITY: 3
DATE: 2025-07-05

comments: SUMMARY:
This paper aims at inferring causal direction between two variables when unobserved mediators are present with nonlinear relations with the output , where additive noise model (ANM) assumptions may not hold. The author provided a through discussion why existing methods failed, and then proposed a Bivariate Denoising diffusion that is provable to learn the causal relation under ANM. Experiments on synthetic and real-world data are conducted.

STRENGTHS AND WEAKNESSES:
Strengths. 

1. The paper considered an important scenario in bivariate causal discovery, and was overlooked by the literature. 
2. The authors rigorously analyze when and why existing methods fail under hidden mediators, and provide consistency guarantees for BiDD under standard ANM.
3. BiDD outperforms existing methods on synthetic datasets across a range of mechanisms (linear, nonlinear, invertible/non-invertible).

Weakness. 

1. The theoretical consistency of BiDD is only proved in the standard ANM setting, and only conjectured in more realistic settings with nonlinear mediators.
2. Training two conditional denoising diffusion models and estimating mutual information is computationally intensive, which may limit scalability to large datasets.
3. Assumption 2.2 is difficult to decipher. It is difficult to comprehend which constraints on f1,...,fN and noise to make the assumption holds.

QUESTIONS:
See the weakness above.

LIMITATIONS:
Yes.
TITLE: Official Review
CLARITY: 3
IMPACT: 3
ORIGINALITY: 2
DATE: 2025-07-08

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe proposed method in this paper offers a causal direction identification method while addressing the hidden mediator case. The authors show precisely where existing work does not perform well and why. The proposed method uses denoising diffusion to obtain a new conditional independence test.  Finally, they demonstrated empirical performance on both synthetic and real-world datasets.\n\nSTRENGTHS AND WEAKNESSES:\nBelow I share my comments.\n\n### Strength:\n* The paper is written in a clear and well-organized manner. The proposed method appears to be theoretically sound.\n* I appreciate the authors' discussion on the failure modes of prior works in different scenarios. The analysis of failure cases is helpful for understanding the motivation and contributions of the proposed method.\n\n### Weakness:\n* Line 198: Typo. I think it should be “noising/forward process”.\n* Line 200: In $(\\epsilon_Y - f_{\\epsilon_Y} (\\tilde{Y}, X))^2$, why is it $X$ and not $\\tilde{X}$, given that noises were added to both $\\tilde{X}$ and $\\tilde{Y}$? Are the noising processes in Eq $4.1$ and $4.2$ applied to $X$ and $Y$ at the same time, or are they applied individually while keeping the other variable fixed? The authors should clarify this point.\n* The authors mentioned that they choose the direction with lower dependence (mutual information). Shouldn’t there be no dependence in the anti-causal direction? What does it mean when both directions have high dependence (but unequal), or both have low dependence (but unequal)? What does the gap between the two dependence values imply? These questions should be addressed in the paper.\n\nQUESTIONS:\nBelow I share my questions.\n\n* Why can we not consider all $Z_i$ as a single variable $Z$ and then apply existing methods?\n* The authors mentioned (line 202) that “inclusion may also introduce dependence between the predicted noise and the unnoised variable”. Why does this happen?\n* Line 271: Is it possible that the inferred direction keeps flipping at different timestamps $t$? What does this behavior indicate? Does the method eventually converge to a specific direction?\n* (Line 297) For which setup was the causal direction the hardest to infer?\n* How many mediators can the proposed method accommodate before performance degrades?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 2, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-10"}