Efficient Ensemble Conditional Independence Test Framework for Causal Discovery
Authors: Zhengkang Guan
Kun Kuang
Venue: ICLR 2026 Poster (2026)
Keywords: Conditional Independence Testing
Causal Discovery
P-value Combination
Primary area: causal reasoning

Abstract: Constraint-based causal discovery relies on numerous conditional independence tests (CITs), but its practical applicability is severely constrained by the prohibitive computational cost, especially as CITs themselves have high time complexity with respect to the sample size. To address this key bottleneck, we introduce the Ensemble Conditional Independence Test (E-CIT), a general-purpose and plug-and-play framework. E-CIT operates on an intuitive divide-and-aggregate strategy: it partitions the data into subsets, applies a given base CIT independently to each subset, and aggregates the resulting p-values using a novel method grounded in the properties of stable distributions. This framework reduces the computational complexity of a base CIT to linear in the sample size when the subset size is fixed. Moreover, our tailored p-value combination method offers theoretical consistency guarantees under mild conditions on the subtests. Experimental results demonstrate that E-CIT not only significantly reduces the computational burden of CITs and causal discovery but also achieves competitive performance. Notably, it exhibits an improvement in complex testing scenarios, particularly on real-world datasets.

Published as a conference paper at ICLR 2026
EFFICIENT ENSEMBLE CONDITIONAL INDEPENDENCE TEST FRAMEWORK FOR CAUSAL DISCOVERY
Zhengkang Guan, Kun Kuang"
College of Computer Science and Technology, Zhejiang University zhengkang guan @ ju. edu. cn, kunkuangezju. eew.e/.jeueuccoom
ABSTRACT
Constraint-based causal discovery relies on numerous conditional independence tests (CITs), but its practical upplicability is severely constrained by the prohibitive computational cost, especially as CITs themselves have high time complexity with respect to the sample size. To address this key bottleneck, we introduce the Ensemble Conditional Independence Test (E-CTT), a general-purpose and plug- intuitive dwvide-and-aggeeuun strategy:
and-play framework. E-CIT operates it partitions the data into subsets, applies a given base CIT independently to each subset, and aggregates the resulting p- p-values using novel method grounded in the properties of stable distributions. This framework reduces the computational complexity of base CIT linear in the sample size when the subset size is fixed. Moreover, our tailored p-value combination method offers theoretical consistency guarantees under mild conditions on the subtests. Experimental results demonstrate that E-CIT not only significantly reduces the computational burden of CITs and causal discovery but also achieves competitive performance. Notably, exhibits an improvement in complex testing scenarios, particularly on real-world datasets.
INTRODUCTION
Conditional independence testing (CIT) serves as a foundational tool in statistics and machine learning, particularly central to causal discovery algorithms (Spirtes et al. 20000; 1995] Glymour
ctt al. 2019. Vowels et al.) 2022). which fundamentally rely on CIT to examine whether variables X and are independent given a conditioning set Z. Formally, it evaluates the following hypotheses:
Ho X 1L Y Z
versus H1 : X
However, the heavy reliance constraint-based causal discovery on numerous CIT's creates a severe computational bottleneck, significantly limiting its practical use. While many studies (Akbari et al.) 2021; Mokhtarian et al.] [021 2023) 2025 Shiragur et al., 2024: Rohekar et al., 2021) have focused on reducing the number of CIT's to streamline the discovery process, more fundamental challenge lies in the high time complexity of CITss themselves (Zhang et al. [011]] Scethon et al. (022). Despite
some research (Strobl et al.] 2019 Schacht & Huang (025) on mitigating the cubic time complexity of the popular kernel-based conditional independence test (KCIT) (Zhang et al. 201 Shah &
Peters (2018) demonstrate that single CIT is uniformly effective across all conditional dependence structures. Thus, critical open question is how to generally reduce the computational cost of CITs
while preserving their testing power.
To address this challenge, we propose the Ensemble Conditional Independence Test (E-CIT), general- purpose and-paay framework that can be seamlessly applied to existing CTT methods to mitigate the computational burden while maintaining competitive performance. E-CIT adopts an intuitive wwride-and-aggregnn strategy: given a CIT method, it partitions the dataset into multiple subsets, conducts independent tests on each subset, and aggregates the resulting p-values. For this combination, we introduce novel method based on the properties of stable distributions, which is theoretically consistent under mild conditions overall procedure. When the subset size is fixed, this strategy controls the computational complexity of the base CIT linear in the sample size.
the subtests, and ensures the reliability of the
"ooresponniig author.

---

Published as a conference paper at ICLR 2026
The main contributions of this paper are summarized follows:
We introduce E-CIT, a general- divide-ann-aggregaat framework that systemati- cally mitigates the computational complexity of CITs, thereby addressing fundamental computational bottleneck in causal discovery with respect to sample size.
We develop novel p-value combination method grounded in the closure property of stable distributions, which offers validity and consistency under mild conditions the subtests, while remaining flexible across different settings.
Through extensive experiments on both synthetic and real-world datasets, we show that CIT yields substantial efficiency gains while achieving competitive performance, especially in challenging heavy-tailed or real-world scenarios.
2 RELATED WORK
2.1 CONDITIONAL INDEPENDENCE TESTING
We briefly review several representative and recent approaches CIT, while referring Li & Fan (2020) for a comprehensive overview. A typical approach in CIT is to define criteria for conditional independence. One of the most widely used measures is Conditional Mutual Information (CMI) (Runge, 2018: Mukherjee et al. 2020) Jamshidi et al., 2024), along with several other metrics (Yu al. 2020; Wang al. 2015 Cai al., [2022). Sen al. (2017) reformulate CIT as a binary classification problem and apply modern classifiers for hypothesis testing. Recently, the Conditional Randomization Test (CRT) (Candes et al. 2018 has inspired several new methods. For example, Bellot & van der Schaar (2019); Shi et al. (222)) utilize GANs for conditional sampling, whereas Li et al. (2023a b) employ nearest-nes neeistt sampling techniques. These methods are particularly effective for handling large conditioning sets.
KCIT (Zhang et al. 201 a widely used CIT method leveraging reproducing kernel Hilbert spaces (RKHS), and has inspired many kernel-based extensions (Doran et al. 2014, (Scetbon et al.) 2022 Zhang et al. 2022, Pogodin et al. 2024). Efforts to accelerate CITs have primarily focused on approximations of KCIT, notably RCIT (Strobl et al., 2019) and FastKCIT (Schacht & Huang 2025). RCIT employs random Fourier features for efficient approximation, while FastKCIT partitions the dataset with Gaussian mixture models of the conditioning variable Z. Although similar in spirit, CaT is specifically tailored to KCIT rather than serving as a general framework like E-CIT.
2.2 COMBINATION TEST
The problem of combining individual p- values into overall test has long been cen- tral in statistics, with important applications in fields such genomics. Consider the sce- nario where the same hypothesis tested m times, yielding corresponding p-values Pr, Pi,...-Pm. Pm- Classical methods for combin- ing these p-values are summarized in Table where (--) denotes the cumulative distribu- tion function of the standard normal distribu- tion. Heard & Subin-Delance tigate the conditions under which each clas- sical method for combining p-values is most appropriate.
Table I: Classical p-value combination methods
Method
Formula
Tippett 1931
min (p:)
Edgington 1972
Pa
Fisher 1934
2 i=1 lup;
Pearson [933
2 1- In(1 -p:)
Mudholkar & George 1979
In [p./ p.)]
(2018) inves-
i=1
Stouffer ct al
1949
Σ m
② (p:)
1=1
Lipták 1958
1 pc)
Recently, the problem of combining multiple p-values has received renewed attention (Vovk &
Wang 2020 Geistkemper 2024), particularly in high-dimensional settings with dependent tests, as is common in biostatistics. Liu & Xie (2020) propose a Cauchy-based method using inverse probability weighting, which performs well under such conditions (Long et al., (023). Building on this technique, Liu et al. (2024) develop ensemble testing method, and Ling & Rho (2022) further generalize this approach using stable distributions. Notably, these specific methods are explicitly designed for traditional parametric settings, primarily for whole-genome sequencing (WGS) association studies.
2

---

Published as a conference paper at ICLR 2026
p-value
p-valuez
Ensemble p-value
p-valuex
Overview of the E-CIT framework. Each scatter plot displays samples of variables X and Y. with color indicating the value of Z. Despite smaller subset sizes, the marginal dependence (black contours) and conditional independence given Z (blue contours) remain clearly distinguishable.
Figure 1:
3 METHOD
3.1 ENSEMBLE CONDITIONAL INDEPENDENCE TEST FRAMEWORK
Given n samples from joint distribution over X, Y, and Z, our goal is to test whether X TT Y Z using an arbitrary base CIT method. However, the sample size grows, the computational cost of many CIT methods can become prohibitive due to their high complexity. Inspired by ensemble learning, we propose the Ensemble Conditional Independence Test (E-CIT) framework to address this issue, as shown in Figure ! We partition the entire dataset into K subsets of size ng. where n = K Kng- The base CIT is applied independently to each subset, yielding p p-values {Pi..... Pk}. which are then combined into final p-value. When ni is fixed, this ensures that the overall computational cost scales linearly with n, regardless of the original complexity of the CIT method.
........Pk,
As illustrated in Figure [ subsets of sufficient size can effectively capture conditional dependence. However, unlike classical parametric hypothesis tests and their associated p-value combination methods (Liu & Xie| (2020) Ling & Rho 2022), CITs have a more complex alternative hypothesis, leading to p-value distributions that vary significantly across data-generating mechanisms and base CIT methods. This variability poses challenge for p-value combination, since the statistical properties of a combination method depend heavily on the alternative distribution of p-values (Heard) & Rubin-Delanchy 2018). Therefore, to ensure broad applicability across scenarios and methods, flexible aggregation strategy that maintains statistical properties under diverse conditions essential. In the next section, we propose such method based on the properties of stable distributions for the E-CIT framework.
3.2 COMBINING P-VALUES VIA STABLE DISTRIBUTIONS
We utilize the properties of stable distributions to combine p-values to construct the aggregation strategy. We begin with a brief introduction to stable distributions, as detailed in Nolan 2001 2020). Definition (Stable Distribution). A random variable X follows a Stable Distribution with parame- ters a (0, B E [-1, 7 > O, and 8 E IR denoted as S(a, 8.7, if its characteristic function is given by:
E (ixp( exp (-70 /u/ i3 (tan ga (sign(u))) + iōu), exp [-/|u| [1 + 2 (sign (u)) log [u]] + idu). a = 1.
The formulations above represent one of the parameterizations of the characteristic function used to define stable distributions. The parameter a, known the stability parameter, controls the tail
3

---

Published as a conference paper at ICLR 2026
heaviness, with smaller values of corresponding to heavier tails. The parameter / is the skewness parameter, where 3 = 0 corresponds to a symmetric distribution. In this case, the normal and Cauchy distributions arise when
2 and corresponds to 3 = 1 and a = 0.5. The scale parameter 7 determines the spread of the distribution, and the location parameter shifts the distribution along the real axis.
1, respectively, whereas the skewed Lévy distribution
The most important property stable distributions is their generalization of closure under summation, as the sum of independent stable-distributed random variables remains stable (detailed in Appendix (A) In our method, we utilize the closure property in a specific case, as shown below.
Proposition 1. Let X1, X2...., X2, be independent and identically distributed (i.id.) random variables following a stable distribution:
X, (,,3,,,,,, 11,,2.....
Then, the normalized sum
S X1 x
also follows stable distribution: S, (a, 3, 8),, where y J -1g.
This elegant property gives rise to the name of the stable distribution. Building upon this property, we define the core of our proposed method:
Definition (Ensemble Test). Given a set of i..dd. p-values Pt, P1.P2...... PK derived from independent and identical subtests H. the ensemble test He (H, K; B,7,ö) 5) is defined by the statistic Te:
1 K K
T = Fs
where Fg is the inverse cumulative distribution function (CDF) of the stable distribution S(a, 3,7, is evident that we obtain the lower-tail p-value, referred to as the ensemble p- value Pes given by:
Pe = FTe
where Fsr the CDF of the stable distributioo S(a, 3.7/0, 8) with o' -ā-lyy
The ensemble test combines individual p-values into a single test statistic by leveraging the properties shown in Proposition [ This approach offers greater flexibility by allowing adaptive selection of the stable distribution parameters a, 3, 7, to accommodate different types of CIT and underlying conditional dependence structures in the data. Among these parameters, controls the tail heaviness of the stable distribution and has the greatest influence on its CDF Fs. Therefore, in practice, we recommend fixing B, 5 and varying only a, which provides a simple yet effective way to adjust the flexibility of E-CIT.
is important to distinguish our approach from related work such as Ling & Rho (2022). Structurally, our method is generalized form of Stouffer et al. (1949), whereas Ling & Rho (2022) generalizes
Liu & Xie (2020) and Lipták (1958). More importantly, our subsequent theoretical analysis is tailored for the challenges of CIT's and, consequently, makes no parametric assumptions the form of the
subtests (such as the normality of subtests' statistics (Liu & Xie 2020: Ling & Rho 2022 ).
To formally establish the reliability of our method, present its key theoretical properties below (see Appendix B for detailed proofs).
Theorem 1. The ensemble test He (for exact subtest p-values) satisfies the following properties:
Validity: Under the null hypothesis, the ensemble p-value is uniformly distributed on [o, 1). ensuring Type I error control.
2. Admissibility: The ensemble test is admissible, indicating that no other test uniformly outperforms in terms of error rates and decision-making optimality.
3. Unbiasednes 11 The ensemble test is unbiased if its subtests are unbiased, meaning the ensemble does not compromise the unbiasedness of the individual subtests.
Unnbissedness in hypothesis testing is defined as the rejection probability under the alternative hypothesis heing at least the pre-specified significance level (1 (Tehmann & Romano, 2005), distinct from unbiasedness in estimation.
4

---

Published as a conference paper at ICLR 2026
Theorem ensures that our ensemble test is valid for exact base p-values. However, we acknowledge that due to the challenges inherent to CITs, these p-values are often approximate. This implies that the guarantees of Theorem may not hold exactly in practice, point discuss further in Appendix F
We further examine the ensemble test's power. Let ae. Be, and Te = 1 - 3. denote the Type error, Type II error, and power of the ensemble test, respectively. The following theorem establishes key result that describes the power of our ensemble test.
Lemma 1. Assume that Fsg (p is integrable. The power of the ensemble test He (H, K; a. 7, approaches I as K- + 00, i.e. limx-sso Fe
1, if the following condition holds:
E
s (F
< F5
(aa),
where 11 for k = 1,2,..... .,,...,.K. K are i.id. p-values from the subtest H. under the alternative hypothesis.
Lemma I establishes sufficient condition for the power of our ensemble test converge to 1. However, the conditions stated are not directly interpretable and may be difficult to verify. Thus, we present the following theorem, which provides more relaxed and practically verifiable conditions for convergence.
Theorem 2. Consider the ensemble test He (H, a,,,,,8,,,,, 9,7,5), and assume that Tp'(p is inte- grable. If the following conditions hold:
1. ElpE" #1l S Ccer
2. (p) 2 fi(1 - for p [0, 111 where f1 is the probability density function of Pk H1
3. a 2 3
Then, we have limx -0c Te 1.
Remark 1. Theorem highlights the reliability of the E-CIT framework. It shows that E-CIT not only preserves the consistency of the base CIT's but also offers potential way improve the power of methods lacking theoretical consistency guarantees. More importantly, while many existing CITs have consistency guarantees (Scetbon al. 2022 Jamshidi et al. 2024), their underlying assumptions can be difficult to satisfy in complex scenarios (Appendix C In contrast, the consistency of E-CIT established by Theorem 2 does not directly impose assumptions on the testing scenario itself. Instead, it only requires the individual subtests to be reasonably effective. This property enhances the general applicability of E-CIT in challenging situations (see experimental results in Section 4
The three conditions in Theorem @aneeemmmaann be easily satisfied in practice. The first condition requires the performance of individual subtests, specifically that the expected p-value of the subtest H under the alternative hypothesis H₁ is below the significance level. This condition is mild and can be directly translated to requiring that the power of H. exceeds a threshold determined by fi, which can be as low as 0.5 in edge cases. See further illustrations in Appendix D
Remark 2.
The second condition concerns the shape of the p-value distribution under Hi, requiring the density on the left side of fi to exceed the symmetric value on the right. This is natural since p-values under H1 tend to concentrate near O, and it is automatically satisfied when the first condition holds and p-values are approximated by a Beta distribution (Heard & Rubin- Rubin-Delanchy 2018) (Appendix DD).
The third condition restricts the stable distribution used in He. The requirement a 2 ensures the tail no heavier than Cauchy distribution, which aligns with statistical intuition about tail behavior (Liu) & Xie, 2020). Although 8 8 = 0 can be relaxed in theory (see Eq 3), Appendix B.3). we fix them to simplify both the proofs and implementation, while using a to control tail heaviness in practice.
Remark It is important note that this desirable convergence property occurs with respect the number of subtests K, rather than the sample size n. Moreover, the condition ensuring convergence primarily imposes requirements on the effectiveness of the individual subtests. Therefore, for fixed total sample size, the ensemble approach benefits from increasing K only if the effectiveness of each subtest can be maintained. simple increase K alone may not improve performance.
Lastly, we discuss the rationale for maintaining flexibility in our E-CIT framework. Since valid p-values follow a uniform distribution under the null hypothesis, the Neyman- Pearsson lemma dic- tates that the uniformly most powerful test statistic for combining p-values should correspond to
5

---

Published as a conference paper at ICLR 2026
monotonic transformation of Ik=1 log fi (px) Casella & Berger 2024 2018 Therefore, the ensemble test is optimal when
Heard & Rubin- Delanchy
K Σ
(px)
Σ log fa (pk)
where is an arbitrary monotonic function. However, for CIT. unlike traditional parametric tests, the conditional dependence structure the data under the alternative hypothesis H1 can lead different distributions of Pk Hi even with the same CIT method. Therefore, our E-CIT allows the flexibility of adjusting to concisely control Fs, enabling the test statistic to satisfy the above condition closely as possible. However, we acknowledge that the theoretically optimal choice of a is context-dependent and requires further analysis for specific CIT methods (see Appendix F for a discussion).
4 EXPERIMENTS
In this section, we comprehensively evaluate the effectiveness of E-CIT. We first demonstrate its ability to reduce computational costs while maintaining performance (Section 4.1], followed by its broad applicability across different CTT methods (Section 4.2) and strong performance on real-world datasets (Section 4.3). Additionally, we apply E-CIT causal discovery (Section[4.4) Other results, including the impact of subset size and the advantages of our p-value combination method for CIT,
can be found in Appendices E.6 and B.7
We conduct our synthetic experiments under the post-nonlinear model, following the setup of prior works (Zhang et al. 22011; Doran et al. 2014 Bellot & van der Schaar 2019. Scetbon et al. [2022 Li ct al. 2023a b). Specifically, we consider the null hypothesis Ho : :X
Z and the alternative
hypothesis H1 X y Y Z, with data generated as follows:
Ho: =x fx ( Z + = (W7 +r))
H1: X= fx Z = (W7 Z + 3x +
Here, Z is drawn from either standard normal or standard Laplace distribution. The weight matrices Wx and Wy are initialized with entries from U(0, 1) and column-normalizze so each column sums one, with 8x set The nonlinear functions fx and fy are randomly selected from {z, x², _², tanh(x), cos(x)) The noise terms Ex and Ey are i.i.d. samples from a standard Student's s t, Laplace, or Cauchy distribution.
All CITs are evaluated at significance level of 0.05. We compare each CIT with the original method and its ensemble version. For all ensemble tests, we fix the subtest sample size at nı 400, while the number of subtests K varies with the total sample size, which ensures linear computational complexity. This value of nı is chosen based on empirical experience (Zhang et al. 2011 Scetbon et al. 2022 Runge, 2018), 8), which indicates that CIT methods exhibit sufficient empirical behavior at this sample size, as further discussed in Appendix [F While n; 400 already yields good performance, our ablation study in Appendix E. indicates that further optimization is possible. Following Theorem 2 we set B = 5 = 0 and 7 = 1. We use two values of a (1.75 and 2) to illustrate how the tail heaviness of the stable distribution affects performance, based experiments reported in Appendix F.1
4.1 EFFICIENCY THROUGH ENSEMBLE FRAMEWORK
In this experiment, we show that the ensemble framework reduces computational cost while main- taining competitive performance. We compare our ensemble-enhanced KCIT (E-KCIT) with RCIT (Strobl et al. 2019). FastKCIT (Schacht & Huang 2025) (the only other methods known to accelerate CITss, and the original KCTT over 1000 independent trials. For all these methods, the kemel bandwidth is determined using the median heuristic. While advanced bandwidth optimization strategies represent an active area of research (Wang et al. 2025). we strictly adhere to the standard median heuristic as ensure controlled comparison and minimize potential confounding factors.
Figure 2 shows results under standard Student's (with two degrees of freedom). Cauchy, and Laplace noise, with Z normally distributed, evaluating Type I error (left), test power (middle), and runtime (right). Е-КCIT significantly reduces computational costs while maintaining competitive test power. Notably, under the more challenging heavy-tailed noise distributions (Figures 2a and 2b E-KCIT

---

Published as a conference paper at ICLR 2026
Type Error
Power
Execution Time
1.0
KGII
Rale
FaiKKIT
0.08
Power
0.8
Time (s)
MCIT 1755
E-KCIT (2-2)
0.04
0.7
15
0.00
0.5
190
2400
5000
4000
4b00
1600
1200
4000
Sample Size (n)
(a) f-distributed noise
Type Error
Power
Execution Time
1.0
0.08
0.8
45
RCIT
FreKKII
0.08
Power
30
1.75)
Error
0.6
Time
E-NGIT (a.m) 2)
15
0.02
0.4
0.00
1839
2630
400
1600
4000
850
1600
240
3500
400
Sample (n)
Cauchy-distributer noise
Type Error
Power
Execution Time
0.10
KDIT
0.08
RGIT
Rale
Power
EIITIT ta 1.75)
Time
0.02
0.7
1600
2400
800
2e10
3000
4000
805
1000
2600
4000
Sample Size (n)
(c) Laplace-ppsseet distributed noise
Figure 2: Comparison of Type error (left; 0.05 significance level marked by solid black line), test power (middle), and runtime (right) for KCIT, RCIT, FastKCIT, and Е-КСIT under different noise distributions.
demonstrates more consistent performance. Although methods generally maintain the Type I error near the nominal significance level, the ensemble framework may slightly affect this control in some scenarios. As discussed Appendix F this effect independent of the test's power. some scenarios (as observed in subsequent experiments), it can lead to more conservative Type error.
4.2 ENSEMBLE EFFECTIVENESS ACROSS CONDITIONS
To evaluate the ensemble framework across diverse settings, we further compare five CIT methods: RCIT (Strobl al. 2019). LPCIT (Scethon et al. 2022), CMIknn (Runge) 2018). CCIT (Sen et al.. 2017). and Fisher Z-test (FisherZ) (Fisher 1921). both their original (Orig.) and ensemble versions (a = 1.75 and a = 2). Simulations are conducted with sample sizes of 800, 1200, and 1600, with Z sampled from standard normal or Laplace distribution, and standard t-distributed noise with three different degrees of freedom (df=2, 3, 4).
Each setting is repeated 1000 times for RCIT, LPCIT, and Fisher Z-test, and 500 times for the more computationally intensive CMIknn and CCIT. Table [ and additional results in Tables 6 and 8 (Appendix E.2 report Type error rates and test powers. Bold values indicate statistically
7

---

Published as a conference paper at ICLR 2026
Table 2: Results for n 1200, Standard Normal Z. Bold (Bold italics) indicates the ensemble (original) version is statistically significantly better.
df of t-distributed Noise
df 4
df = 3
df = 2
CIT Methods
Type
Power
Type
Power
Type
Power
Orig.
0.037
0.845
0.037
0.765
0,037
0.548
RCIT
Ensemble (a=1.75)
0.046
0.891
0.042
0.823
0.042
0.623
Ensemble (a=2200)
0.054
0.906
0.066
0.838
0.044
0.609
Orig. Ensemble (a=1.75)
0.054
0.742
0.054
0.659
0.021
0.422
LPCIT
0.042
0.755
0.031
0.675
0.013
0.447
Ensemble (a=2.00)
0.054
0.767
0.026
0.669
0.017
0.418
Orig. Ensemble (a=1.75)
0.122
0.990
0,116
0.986
0.124
0.982
CMlken
0.164
0.994
0.138
0,988
0.136
0.988
Ensemble (a=2.00)
0.124
0.990
0.136
0.980
0.104
0.982
Orig. Ensemble (a=1.75)
0.450
0.896
0,430
0.928
0.454
0.904
CCIT
0.336
0.856
0.334
0,828
0.286
0.816
Ensemble (ax22.0)
0.322
0.848
0.350
0.830
0.308
0.812
Orig-
0.217
0.695
0.144
0.613
0.093
0.510
FisherZ
Ensemble (x=1.75)
0.197
0.766
0.138
0.659
0,094
0.561
Ensemble (a=2.000
0.213
0.719
0.124
0.656
0.078
0.508
Table Performance comparison on the flow-Cytometry dataset (results for both a merged). Bold (Bold italics) indicates the ensemble (original) version is statistically significantly better.
Method
Precision
Recall
F1-score
Orig.
0.580
0.674
0.624
KCIT
Ensemble
0.730
0.664
0.695
Orig. 0.684
0.647
0.665
RCIT
Ensemble
0.715
0.662
0.687
Orig.
0.740
0.649
0.691
LPCIT
Ensemble
0.838
0.664
0.741
Orig. 0.880
0.698
0.779
CMiknn
Ensemble
0.872
0.668
0.756
Orig. 0.520
0.722
0.605
CCIT
Ensemble
0.618
0.680
0.646
Orig-
0.840
0.656
0.737
FisherZ
Ensemble
0.852
0.699
0.767
significant improvement of the ensemble method over the original (based on a one-sided test at the 0.1 level), whereas bold italics denote cases where the original method performs better.
Across various simulation settings, the ensemble test consistently enhances the test power of RCIT, LPCIT, and the Fisher Z-test, while maintaining appropriate Type error control. In contrast, the benefit for CMlknn is less pronounced. CCIT represents special case: in our experiments, it fails to properly control the Type error. Interestingly, applying the ensemble test significantly reduces the Type error in this case, albeit with a minor reduction in power. We also observe that the choice of the E-CIT parameter a affects performance across different CIT methods and data configurations. For example, in Table 2 with ensemble framework with a = 2 performs better for RCIT, whereas & 1.75 performs better for the Fisher Z-test. These observations align with our analysis in Section 3 highlighting the importance of flexibility in the E-CIT framework. We also evaluate the impact of the dimensionality of the conditioning set Z in Appendix E.3
1200 and Z drawn from standard normal distribution, the
8

---

Published as a conference paper at ICLR 2026
F1 Score
SHD
Execution Time
KGTT
Score
E-KCIT te 2.75)
SHD
E-CCIT
0.6
Time
500
2
0.4
5d0
1688
2606
3205
4000
100
1600
2600
3000
Sample Size (n)
(a) t-distributed noise
F1 Score
SHD
Execution Time
0.8
KCIT
750
Score
KDIT (e = 1.75)
0.6
SHD
E-KOIT
Time
250
pleg
4000
800
2000
Sample Size (n)
(b) Cauchy-distributed noise
F1 Score
SHD
Execution Time
1.0
1500
KGIT
Score
E-KGI (a 2.75)
0.8
Time
1000
KCIT 2)
500
0.6
800
188
2600
3300
4900
1e00
2400
4080
800
1885
280
Sample Size (n)
(c) capllac-distributae noise
Figure 3: Comparison of causal discovery performance (F1-score, SHD, and runtime) of KCIT, RCIT,
and E- KCIT under different noise distributions. Shaded areas indicate #1 standard deviation.
4.3 REAL DATA EXPERIMENT: FLOW-CYTOMETRY DATASET
We evaluate E-CIT on the Flow-Cytometn dataset, with the experimental details provided in Ap- pendix E.4 As shown in Table 3] the ensemble framework enhances the performance of most CTT methods complex real-world datasets. While there slight performance drop for CMIknn, table gains are observed for KCIT, RCIT. LPCIT, and FisherZ. For CCIT, consistent with Section 4.2 the ensemble framework improves its Type eπтor control, as reflected in higher precision. Overall, these results further confirm the broad applicability of E-CIT.
4.4 APPLICATION CAUSAL DISCOVERY
Similar to the experiments in Section 4. we evaluate the performance of E-KCIT against RCTT and KCIT on synthetic causal graphs generated with nonlinear functional mechanisms and additive noise. Detailed settings are provided in Appendix E.5 We consider Student's & (df 2), Cauchy, and Laplace noise distributions, with results shown in Figure In most settings, E- E-KCIT outperforms RCIT and KCIT in both F1-score and Structural Hamming Distance (SHD), while its runtime remains comparable to RCIT. These results demonstrate that E-CIT is both practical and effective for causal discovery.

---

Published as a conference paper at ICLR 2026
5 DISCUSSION
In this paper, we have introduced the Ensemble Conditional Independence Test (E-CTT), a general- purpose, plug-and-play framework that addresses the critical computational bottlenck in constraint- based causal discovery. By employing a andded----agggggatee strategy, E-CIT can linearize the complexity of base CIT. Moreover, based on stable distributions, our novel p-value combination method ensures statistical properties under mild conditions. Our theoretical and empirical findings suggest that the framework is especially effective complex real-world scenarios. The significance of our work lies in its modularity: instead of proposing another specific CIT, we present a framework that enhances the scalability and can provide consistency for a broad class of CIT methods.
Applicability and Scope. While E-CIT provides a broadly applicable plug-and- aan--nddplay framework for reducing the computational burden of CITs, is crucial to delineate its present theoretical and practical scope. First, our current analysis primarily assumes independent and identical subtests, yielding i.i.d. p-values. This condition may not be satisfied in certain scenarios (see the discussions on correlated p-values and distribution drifts in Appendix (F). Second, although our p-value combination method ensures power consistency under mild conditions, its ultimate performance remains intrinsically tied to the base CIT. Consequently, while E-CIT can linearize complexity with respect sample size, does not resolve fundamental statistical challenges, such as the curse of dimensionality in high- dimensional conditioning sets Z, inherent in specific methods. Finally, the theoretical guarantees of E-CIT are conditional on the subtests being reasonably effective. As a Swvide-and-aggeegan framework, E-CIT cannot substitute for the fundamental statistical validity of the underlying subtests.
Practical Implementation. Grounded in our theoretical analysis and empirical findings, we provide the following current guidelines for hyperparameter selection (see Appendix [ for detailed discussion). First, regarding the stability parameter a, we recommend a general default setting of a E (1.75, 2}. While our framework offers the flexibility to tune a for optimality relative to specific base CITs, these values have demonstrated robustness across diverse settings in experiments. Second, regarding the subset size Tgg, it should chosen such that the base CIT exhibits reasonable asymptotic behavior and power. A practical rule of thumb is to adopt the sample size recommended in the base CIT's original literature (e.g., ng = 400 for KCIT (Zhang et al. 2011)).
Limitations and Future Directions. We further provide a detailed discussion the limitations and future directions of our method in Appendix This includes an analysis of the potential impact on Type error control, the handling of super-uniform p-values generated by permutation tests, and promising avenues for future research such as handling correlated p-values and developing method- specific enhancements. We believe E-CIT offers a powerful solution that balances computational efficiency with statistical power, thereby paving the way for causal discovery in large-scale and complex scientific problems.
ACKNOWLEDGMENTE
This work was supported in part by the National Natural Science Foundation of China (62376243), the National Key Research and Development Program of China (2024YFF0203300), and "Pioneer" and "Leading Goose" R&D Program of Zhejiang (2025C02037)..
A preliminary version of this work was presented as the bachelor's thesis of the first author, Zhengkang Guan. He would like thank the faculty members of the Department of Statistics and Data Science at Xiamen University for their support and mentorship during his undergraduate studies. He is particularly grateful Prof. Jingyuan Liu for her insightful discussions on general statistical methodologies, as well as her invaluable guidance regarding overall academic and career development.
REFERENCES
Sina Akbari, Ehsan Mokhtarian, AmirEmad Ghassami, and Negar Kiyavash. Recursive causal structure learning the presence of latent variables and selection bias. Advances in Neural Information Processing Systems, 34: 11111111111000 2021.
Fadoua Balabdaoui, Harald Besdziek. and Yong Wang. Parametric convergence rate of a non- parametric estimator in multivariate mixtures of power series distributions under conditional independence. arXiv preprint arXiv:2509.05452, 2025.
10

---

Published as a conference paper at ICLR 2026
Alexis Bellot and Mihaela der Schaar. Conditional independence testing using generative adversarial networks. Advances in neural information processing systems, 32, 2019.
Zhanrui Cai, Runze Li, and Yaowu Zhang. A distribution free conditional independence test with
applications to causal discovery. Journal of Machine Learning Reseurch, 23(85):1-41, 2022.
Emmanuel Candes, Yingying Fan, Lucas Janson, and Jinchi Lv. Panning for gold:'model-x' model-x koookkoff for high dimensional controlled variable selection. Journal of the Royal Statistical Society Series B: Statistical Methodology, 80(3):551-577, 2018.
George Casella and Roger Berger. Statistical inference. CRC press, 2024.
Gary Doran, Krikamol Muandet, Kun Zhang, and Bernhard Schölkopf. A permutation-based kernel conditional independence test, In Proceedings of the Thirtieth Conference on Uncertainty in Artificial Intelligence, UAI 2014, Quebec City, Quebec, Canada, July 23-27, 2014, 2014.
Eugene Edgington. An additive method for combining probability values from independent experiments. The Journal of Psychology, 80(2):5513363, 1972.
Ronald Fisher. On the" probable error" of coefficient of correlation deduced from small sample. Metron, 1:3-32, 1921.
Ronald Aylmer Fisher. Statistical methods for research workers. 1934.
Tiffany Geistkemper. Review of Optimal Procedures for Combining P-Values with Proposal of Bayesian Approach to Identify Auxiliary Covariates. The University of Memphis, 2024.
Clark Glymour, Kun Zhang, and Peter Spirtes. Review causal discovery methods based on graphical models. Frontiers in genetics, 10:524, 2019.
Nicholas A Heard and Patrick Rubin-Delanchy. Choosing between methods of combining-values. Biometrika, 105(1):239-244 2018.
Fateme Jamshidi, Luca Ganassali, and Negar Kiyavash. On the sample complexity of conditional independence testing with von mises estimator with application to causal discovery. Forty-first International Conference on Machine Learning, ICML 2024, Vienna, Austria, July 21-27, 2024, 2024.
Immu Kim, Matey Neykov, Sivaraman Balakrishnan, and Larry A. Wasserman. Local permutation tests for conditional independence. The Annals of Statistics, 2021.
Erich Leo Lehmann and Joseph P Romano. Testing statistical hypotheses. Springer, 2005.
Chun Li and Xiaodan Fan. On nonparametric conditional independence tests for continuous variables.
Wiley Interdisciplinary Reviews: Computational Statistics, 12(3):e1489, 2020.
Shuai Li, Ziqi Chen, Hongtu Zhu, Christina Dan Wang, and Wang Wen. Nearest-neighbor sampling based conditional independence testing. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pp. 8631-8639, 2023a.
Shuai Li, Yingjie Zhang. Hongtu Zhu, Christina Wang, Hai Shu, Ziqi Chen, Zhuoran Sun, and Yanfeng Yang. K -eeearest-neighbor local sampling based conditional independence testing. Advances in Neural Information Processing Systems, 36, 2023b.
Xing Ling and Yeonwoo Rho. Stable combination tests. Statistica Sinica, 32:641-644 2022.
Tamás Lipták. On the combination independent tests= független mozgó szintes próbák összevont értékelésérö1. A Magyar Tudományos Akadémia Matematikai Kutató Intézetének Közleményei, 1714): 1958.
Yaowu Liu and Jun Xie. Cauchy combination test: powerful test with analytic p-value calculation under arbitrary dependency structures. Journal of the American Statistical Association, 115(529): 393-402, 2020.
11

---

Published as a conference paper at ICLR 2026
Yaowu Liu, Zhonghua Liu, and Xihong Lin. Ensemble methods for testing a global null. Journal of the Royal Statistical Society Series B: Statistical Methodology, 86(2):461-486, 2024.
Mingya Long, Zhengbang Li, Wei Zhang. and Qizhai Li. The cauchy combination test under arbitrary dependence structures. The American Statistician, 77(2): 34 142, 2023.
Ehsan Mokhtarian, Sina Akbari, AmirEmad Ghassami, and Negar Kiyavash. A recursive markov boundary-based approach to causal structure learning. In The KDD'21 Workshop on Causal Discovery, pp. 26-54. PMLR, 2021.
Ehsan Mokhtarian, Mohmmadsadegh Khorasani, Jalal Etesami, and Negar Kiyavash. Novel ordering- based approaches for causal structure learning in the presence of unobserved variables. In Pro- ceedings of the AAAI Conference on Artificial Intelligence, volume 37, pp. 12260-12268, 2023.
Ehsan Mokhtarian, Sepehr Elahi, Sina Akbari, and Negar Kiyavash. Recursive causal discovery. Journal of Machine Learning Research, 26(61): (:061))1-655 2025.
Joris M. Mooij and Tom Heskes. Cyclic causal discovery from continuous equilibrium data. In Ann Nicholson and Padhraic Smyth (eds.), Proceedings of the Twenty-Ninth Conference on Uncertainty in Artificial Intelligence, UAI 2013, Bellevue, WA, USA, August 11-15, 2013. AUAI Press, 2013.
Govind S Mudholkar and EO George. The logit statistic for combining probabilities-an overview. Optimizing methods statistics, 345:365, 1979.
Sudipto Mukherjee, Himanshu Asnani, and Sreeram Kannan. Cemi:
Classifier based conditional mutual information estimation. In Uncertainty in artificial intelligence, pp. 1083-1093, PMLR, 2020.
Ignavier Ng, AmirEmad Ghassami, and Kun Zhang. On the role of sparsity and dag constraints for learning linear dags. Advances in Neural Information Processing Systems, 33:17943-177 2020.
John P Nolan. Stable distributions. 2012.
John P Nolan. Univariate stable distributions. Springer Series in Operations Research and Financial Engineering, 10:978-3, 2020.
Karl Pearson. On method of determining whether sample of size supposed to have been drawn from a parent population having a known probability integral has probably been drawn at random. Biometrika, pp. 379-410, 1933.
Roman Pogodin, Antonin Schrab, Yazhe Li, Danica J. Sutherland, and Arthur Gretton. Practical kernel tests of conditional independence. ArXiv, abs/2402.13196. 2024.
Raanan Y. Rohekar, Shami Nisimov, Yaniv Gurwicz, and Gal Novik. Iterative causal discovery in the possible presence of latent confounders and selection bias. Advances in Neural Information Processing Systems 34: Annual Conference on Neural Information Processing Systems 2021, NeurlPS 2021, December 6-14, 2021, virtual, pp. 2454 2465, 2021.
Jakob Runge. Conditional independence testing based on nearest-neighhor estimator of conditional mutual information. In International Conference on Artificial Intelligence and Statistics, pp. 938-947. Pmir, 2018.
Karen Sachs, Omar Perez, Dana Pe'er, Douglas Lauffenburger, and Garry Nolan. Causal protein- signaling networks derived from multiparameter single-cell data. Science, 308(5721):533-59, 2005.
Meyer Scetbon. Laurent Meunier, and Yaniv Romano. An asymptotic test for conditional indepen- dence using analytic kernel embeddings. In International Conference on Machine Learning, pp. 19328-19346. PMLR, 2022.
Oliver Schacht and Biwei Huang. A fast kernel-bused conditional independence test with application to causal discovery. arXiv preprint arXiv:2505. //055 2025.
12

---

Published as a conference paper at ICLR 2026
Rajat Sen, Ananda Theertha Suresh, Karthikeyan Shanmugam, Alexandros G Dimakis, and Sanjay Shakkottai. power-powered conditional independence test. Advances in neural information processing systems, 30, 2017.
Rajen Dinesh Shah and J. Peters. The hardness of conditional independence testing and the generalised covariance measure. The Annals of Statistics, 2018.
Chengehun Shi, Tianlin Xu, Wicher Bergsma, and Lexin Li. Double generative adversarial networks for conditional independence testing. Journal of Machine Learning Research, 22(285):1-32, 2021.
Kirankumar Shiragur, Jiaqi Zhang, and Caroline Uhler. Causal discovery with fewer conditional independence tests. In Forty-first International Conference on Machine Learning, ICML 2024, Vienna, Austria, July 21-27, 2024, 2024.
Peter Spirtes, Christopher Meek, and Thomas S. Richardson. Causal inference in the presence of latent variables and selection bias. In UAI '95: Proceedings of the Eleventh Annual Conference on Uncertainty in Artificial Intelligence, Montreal, Quebec, Canada, August 18-20, 1995, pp.
499-506, 1995.
Peter Spirtes, Clark N Glymour, and Richard Scheines. Causation, prediction, and search. MIT press, 2000.
Samuel A. Stouffer, Edward A. Suchman, Leland C. De Shirley A. Star, and Robin M. Williams. The American Soldier: Adjustment During Army Life. Princeton University Press, Princeton, 1949.
Eric V Strobl, Kun Zhang, and Shyam Visweswaran. Approximate kemne-bbased conditional indepen- dence tests for fast non-parametric causal discovery. Journal of Causal Inference, (11):20180011 2019.
Leonard Henry Caleb Tippett. The methods of statistics. 1931.
Vladimir Vovk and Ruodu Wang. Combining p-values via averaging, Biometrika, 107(4):791-808, 2020.
Matthew J Vowels, Necati Cihan Camgoz, and Richard Bowden. D'ya like dags? survey on structure learning and causal discovery. ACM Computing Surveys, ---4) 2022.
Wenjie Wang, Mingming Gong. Biwei Huang, James Bailey, Bo Han, Kun Zhang, and Feng Liu. Practical kernel selection for kernel-based conditional independence test. Advances in neural information processing systems, 2025.
Xueqin Wang, Wenliang Pan, Wenhao Hu, Yuan Tian, and Heping Zhang. Conditional distance correlation. Journal of the American Statistical Association, 1106512):1266-1777 2015.
Guodu Xiang, Hao Wang, Kui Yu, Xianjie Guo, Fuyuan Cao, and Yukun Song. Bootstrap- layerwise refining for causal structure learning. IEEE Transactions on Artificial Intelligence, 2708-2722, 2024.
Shujian Yu, Ammar Shaker, Francesco Alesiani, and José C. Príncipe. Measuring the discrepancy between conditional distributions: Methods, properties and applications. In Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence, IJCAI 2020, pp. 2777-2784, 2020. doi: 10.24963/ICAAL2200033.
Hao Zhang, Shuigeng Zhou, Kun Zhang, and Jihong Guan. Residual similarity based conditional independence test and its application in causal discovery. In AAAI Conference on Artificial Intelligence, 2022.
Kun Zhang, Jonas Peters, Dominik Janzing, and Bernhard Schölkopf. Kernel-based conditional independence test and application in causal discovery. In UAI 2011, Proceedings of the Twenty- Seventh Conference Uncertainty in Artificial Intelligence, Barcelona, Spain, July 14-17, 2011, pp. 804-813. AUAI Press, 2011.
Shengyu Zhu, Ignavier Ng. and Zhitang Chen. Causal discovery with reinforcement learning. arXiv preprint arXiv: 190..944000077.. 04477, 2019.
13

---

Published as a conference paper at ICLR 2026
A CLOSURE PROPERTY OF STABLE DISTRIBUTIONS
Proposition A.1 NNolan 2012)). ) A stable distribution S(a,, 3, 7, 6) has the following properties:
1. X S(,, 1,,,,,,,8 7,,6, then for any + 0,0 TR,
(S (sign 3, |0 aδ b)
a #
aX +bb
_ 3 log lal) a a= (a, B1, 01) and X2 S (a: B2, 72, 82) are independent, then X1 X2
(1, (sign |a aδ
2. If 11 (a, S(r, 3, 7, 6), where
8 817 + 8272 = = ri + 7 8 = 61 62
1 72
This is the general version of Proposition T which is one the core properties of stable distributions. For detailed properties of stable distributions, refer to Nolan (2012 2020).
B OMITTED PROOFS
B. PROOF OF THEOREM I
Theorem The ensemble test He (for exact subtest p-values) satisfies the following properties:
1. Validity: Under the null hypothesis, the ensemble p-value is uniformly distributed on [0, 1), ensuring Type error control.
2. Admissibility: The ensemble test is admissible, indicating that no other test uniformly outperforms in terms of error rates and decision-making optimality.
3. Unbiasedness: The ensemble test is unbiased its subtests are unbiased, meaning the ensemble does not compromise the unbiasedness of the individual subtests.
Proof. We first establish validity, and then jointly prove admissibility and unbiasedness. Validity:
According the definition of the ensemble test and p-value (Definition 2. the validity property follows directly. First, by the definition of the p-value, under the null hypothesis, we have:
Pk UO0, 1), k l,....,, K.
Consider a stable distribution S ~ S(a, B, y, 5) with cumulative distribution function (CDF) Fst which is invertible, with its inverse denoted by Fz1. Therefore, we have:
P ( (pk) S 3) = P (pk s Fs (s)) = Fs (8).
Thus, we conclude
F5 (pk) $ s S(a, 8,,,
Furthermore, since Pl, P2.
PK are derived from independent tests (and are thus i.i.d), it follows
that:
FS (p1), F5' (p.,)....... Fg' (px) sid
S(a, 8,,8,
By Proposition
obtain:
K Tg K
p (pa) ~ S (a,,8.'.,))
k=1
where 7/ -2
Thus, by the Probability Integral Transform, it is evident that:
Pe (T.) UO, 1),
14

---

Published as a conference paper at ICLR 2026
where Fsr is the CDF of the stable distribution 3(a,8.1y'0
Admissibility and Unbiasedness:
We build our proof on an earlier result presented by Lipták (9988), from which we formulate the following lemma.
Lemma B.1 Lipták 1958 Let Tg be an aggregated statistic defined as
K
Tg x
Σ wsz(bx))
where ar-) any strictly increasing and continuous function, and wi represent weights satisfying Σ w; = 1 and w;
E [0, 11, 1..... K. Then, the test based on T, is admissible. Furthermore, the p-values pa; 1...., K are from unbiased tests, then the lest based Tg is also unbiased.
In Lemma B.1 the presence the outer function x +-) of Tg yields an equivalent test, since sr.)) is strictly increasing. By further setting equal weights and discarding constant terms, we derive the simplified statistic
K
T
3 (p:).
The test based on T also preserves both admissibility and unbiasedness.
Clearly, the ensemble test statistic
K T.
F (p&)
K
k=1
differs from this structure by only constant factor. Therefore, the ensemble test also preserves admissibility and unbiasedness when pk: k l..... K are from unbiased subtests.
B.2 PROOF OF LEMMA 1
Lemma 1 Assume that F5 (pk is integrable. The power of the ensemble test He (H, a; 3, approaches I as K + co, i.e, lim +00 1, if the following condition holds:
E F (
Fs. (ace),
where Pk Hi for k: = 1,2, K are i...d. p-values from the subtest H under the alternative hypothesis.
Proof. By the definition of Type II error:
B. P (I's (T:) ae (11)
(To > F5' (ax)) H1)
K
P
Σ
F5 (ot
H:
Fz (ac)
Since F5 ( (p
) b = 1,2,...
K are i.i.d. and integrable, by the Strong Law of Large Numbers
(SLLN), we have:
K To
1
Σ Fs
(p
(p
M1)
a.s.
Fs (p!
(P (p as - co.
K
k=1
Then for any > there exists (almost surely) Ko such that for 2 Ko:
To
<
Fs (p #1) te.
15

---

Published as a conference paper at ICLR 2026
Take = FF F's (pk ) surely for large enough K:
> 0 since we have E
F5 (pk }]
< Fs (ae), then almost
Te < (ac) (ac).
Therefore:
P (To > Fs (a+)) + 0 as K - 00,
which implies:
lim Be = 0.
K-000
This is equivalent to:
lim Te 1.
K-+00
B.3 PROOF OF THEOREM! 2
Theorem 2 Consider the ensemble test He (H, K; 3,,,, and assume that F5 (px:) is inte-
grable. If the the following conditions hold:
1E ElpE"
#1 S ase
2. (p) 2 fi(1 for E [0, } where f1 is the probability density function of Ps.
3. a 2 3 = 8 0. Then, we have limg- Te
Proof. is sufficient to show that under these conditions, derive
Fs (p ]
< Fs. (ax)
which directly yields the conclusion via Lemmal 1
The first step is to show that Fs 1 (p
< F5
E
Pk
We begin by reformulating E
Fg
to facilitate bounding:
E
Fg
(pk 11
F5 (p) f1( dp
0
F (p) f1()) dp +
F3 (p) dp
F5' (p) (.((p)) dp // F5' fi dp
F3 (p) (v())) fi - -p) dp
Consider the tangent line I(-) of F51 at E Pk Since B = 5 = 0, F's is monotonically increasing
and convex on [o. ! z and therefore F=1 is monotonically increasing and concave on [0, 11 Ad-
ditionally, since E pk fh ae }, it follows that ((p) > F3s (p) for [0, f]. and I(.) is also
monotonically increasing.
Meanwhile, since (p) 2 (1 p) for p E [0, 1. have (i()) fi(1 p) for p E []
16

---

Published as a conference paper at ICLR 2026
Therefore, obtain:
1 F' (p) ( fi ((1-p)) p)] dp f-" -f((p) - fi(1 -p]] dp
((p) f((() dp S.² 1(p) 100ffil1 - p) dp
/ (()) (1( dp [ -1(1 - -1))p)))))))
/
Since 3 = 8 = 0, it follows from the definition of the stable distribution that F'gg (2) 0 holds.
Furthermore, z(-) is the tangent line of F'5 at
IE
Hi [P
and F5s' is concave [0, 2/ have
1
00 -1 -(22
1
Fgg
2
Moreover, it follows that l(p) and -1(1 - p) are parallel, which implies that ((p) > -1(1 - p) for any p. Consequently, we obtain
{.² (()) fi(p) dp ' -1(1 -1(1-p) (1()) dp / ((p) fi(p) dp + [ 1(p) 1()) fp))
((p) fz(p) dp
Because 1(-) is linear and tangent to F5 at Elp (1 ).
E [(PP
) ]
E Pk
11 H1 1)
TE
Thus, we obtain
E Fs'
(pk Hi }]
< Fgg
E Pk 1).
(I)
Since F is monotonically increasing and IF
|
11
<
aes we further deduce that
Fg
E
Pk
1)
S F5 (ae).
(2)
Moreover, given that a: 2 1, we have 7 = K -ly S 7, indicating that S(a, 8,)) has smaller scale parameter compared to S(a, B, 7, 6). Furthermore, when 3 = 0, both distributions are symmetric, and it is evident that:
Fg1 (ac) < Fzg (ac)
(3)
Combining inequalities [ and (3, we conclude that
E F (p
| }]
< Fz. (ace.
By applying Lemma 1 obtain limx -oc ne 1.
n
17

---

Published as a conference paper at ICLR 2026
C PRACTICAL CONSIDERATIONS OF CERTAIN CONSISTENCY GUARANTEES
While some CIT methods offer theoretical consistency guarantees, it is important to note that their practical performance can be compromised by the challenging nature of real-world data. We acknowledge the asymptotic guarantees, but highlight how complex data environments can limit their practical applicability. has been shown that no single CIT can be effective in all scenarios (Shah Peters 2018: Kim et al. (02T), which further implies that universal consistency across all scenarios is unattainable for any single CIT.
To illustrate these practical limitations, consider LPCIT (Scetbon al., 2022 example, focusing on how its assumptions and estimation procedures can affect its convergence speed in practice:
Assumption Violations: LPCIT's consistency derivation relies on Assumption 3.5 (Scetbon et al. (022), which requires the variables under test, after kernel mapping, to possess higher-order moments with controlled growth rates. However, in many practical situations, the variables might be heavy-tailed, leading to heavy-tailed properties even after kernel mapping. This can violate the assumption, potentially causing a failure consistency in practice. Experiments in Section 4.2 using t-distributions with varying tail thicknesses also show that LPCIT performs better in relatively thin-tailed scenarios.
Estimation Difficulties: LPCIT's estimation of conditional means relies on Regularized Least Squares (RLS), which minimizes squared error and is highly sensitive extreme values. This implies that as the sample size increases, extreme values can disproportionately affect the squared error term, making it difficult for the estimator's variance effectively decrease, thus limiting the improvement in test performance.
Hyperparameter Optimization Challenges: LPCIT employs Gaussian process regression for selecting kernel bandwidth and RLS regularization parameters. In our experiments. we found that this optimization process in LPCIT is highly non-convex. The complexity of CTT scenarios makes it challenging to perfectly solve for the aforementioned hyperparameters, which may further limit the power improvement as the sample size increases.
D CERTAIN ILLUSTRATIONS OF THE CONDITIONS IN THEOREM 2
PDF of p*
a. = 0.05
f ~ Beta(5, 95)
Power 0.55
0.000
0.025
0.050
0.075
0.100
0.125
0.150
0.175
Figure 4: An Example Satisfying the First Two Conditions of Theorem 2
Pk (5)
Hh
Consider the first two conditions of Theorem for a subtest H:
1. E
Pk H <
are
2. (p) 2 f1 p) for E [0. }]. where fi is the probability density function of Hi
Here we consider modeling the distribution of ps using the Beta distribution (Heard Rubin Delanchy 2018).
an idealized case where the significance level 0.05 Hi Beta(5, 95), so that its expectation equals 0.05, which satisfies the first condition of and pk
18

---

Published as a conference paper at ICLR 2026
Hi Theorem 2 In this setting, the power of the test corresponds the probability that Pk 0.05, meaning the probability of correctly rejecting the null hypothesis under the alternative. As shown by the shaded area in Figure this probability is approximately 0.55. In more extreme cases, the area can approach 0.5. This indicates that the first condition of Theorem 2 can be seen as a requirement on the power of the subtest under f1, and this requirement is relatively mild.
Next, demonstrate that when Pk Mi follows Beta distribution, the second condition naturally follows from the first.
Assuming Pk Hi follows Beta distribution, its probability density function is:
1 R (1 p) 0 <<p<1
Li B(a BB)
From the first condition of Theorem
we have:
Elpl QB
S Che
OB + 3B
Thus, have
3B
1
121 1.
a
For p E [0, }). taking the ratio gives:
fi aB, 3B)
p
an-Bp
[1(1 CR Bn)
ag-88 3g
Since BB < 0 and i-p S for E [0, 21. we have
2 1. Therefore:
1-p
1
(p) 2 f1(1p), p),
Vp
0,
2
which is the second condition of Theorem 2
E ADDITIONAL EXPERIMENTS RESULTS
E. EMPIRICAL STUDY THE SELECTION OF
We investigate how the parameter affects the performance of E-CIT, under post-nonlinear model similar to Section 4.1 We use standard Laplace-distributed noise and normally distributed Z, with 1200. Ensemble KCIT (E-KCIT) with n& 400 is used representative, with E {0.25, 0.5, 0.5,..., As a baseline, we also include a mean-p method that directly averages p-values.
Figure shows that the power of E-CI increases with larger a, consistent with Theorem 2 while Type error follows non-monotonic trend. Overall, = 1.75 and yield the best performance under this setting and are used subsequent experiments under different data scenarios.
E.2 ADDITIONAL RESULTS FOR SECTION 4.2 EXPERIMENTS
In the main text, we present the representative Table 2 which reports results for the setting with 1200 and Z following the standard normal distribution. The results under other data settings shown in the following tables.
In addition, we note that the experiments in this section conducted using (-distributions with different degrees freedom. The rationale is that some methods behave uncontrollably under extreme distributions such the Cauchy, while the Gaussian distribution and the Laplace distribution pose little challenge for many methods. Therefore, we choose f-distributions with varying degrees of freedom to ensure both reasonable and diverse comparisons.
19

---

Published as a conference paper at ICLR 2026
Type Error
Power
0.07
Ensemble
Averaging
0.08
Error Rate
0,950
Power
0.05
0.925
0.875
0.25
0.50
0.76
1.00
1.26
1.50
1,75
2,00
0.25
0.50
0.76
1.00
1.25
1.50
1,75
2.00
Figure 5: Empirical evaluation of a using E-CT Type error (left) and power (right). The red line indicates the power of mean-p.
Table 4: Results for = 800, Standard Normal Z. Bold (Bold italics) indicates the ensemble (original) version statistically significantly better.
df of t-distributed Noise
df 4
df 3
df
CTT Methods
Type
Power
Type
Power
Type
Power
Orig.
0.037
0,801
0.041
0.724
0.042
0.512
RCIT
Ensemble (a=1.75)
0.041
0.833
0.039
0.768
0.033
0.545
Ensemble (a=2.00)
0.060
0.852
0.048
0.786
0.045
0.562
Orig. Ensemble (a=1.75)
0.065
0.740
0.052
0.627
0,016
0.423
LPCIT
0.051
0.745
0.030
0.681
0.021
0.436
Ensemble (am2.00)
0.045
0.720
0.033
0.662
0.015
0.423
Orig- Ensemble (a=1.75)
0.086
0.978
0.124
0.984
0.106
0.954
CMlknn
0.110
0.974
0.104
0.978
0.128
0,958
Ensemble (a=2.00)
0.088
0.976
0.116
0.972
0.108
0.964
Orig-
0.414
0.904
0.426
0.886
0.414
0.882
CCIT
Ensemble (a=1.75)
0.372
0.838
0.358
0.844
0.352
0.816
Ensemble (a=2.00)
0.400
0.844
0.424
0.848
0.392
0.814
Orig.
0.170
0.702
0.129
0.585
0.090
0.502
FisherZ
Ensemble (omm177)
0.172
0.702
0.125
0.625
0.081
0.520
Ensemble (a=22.00)
0.185
0.683
0.109
0.636
0.066
0.532
20

---

Published as a conference paper at ICLR 2026
Table 5: Results for 800, Standard Laplace Bold (Bold italies) indicates the ensemble
(original) version is statistically significantly better.
df of t-distributed Noise
df 4
df = 3
df = 2
CIT Methods
Type
Power
Type
Power
Type
Power
Orig.
0.043
0,801
0.056
0.688
0.019
0.521
RCIT
Ensemble (a=1.75)
0.046
0.803
0.048
0.720
0.045
0.536
Ensemble (a=2200)
0.058
0.796
0.051
0.745
0.047
0.552
Orig. Ensemble (a=1.75)
0.055
0.714
0.042
0.636
0.017
0.422
LPCIT
0.050
0.719
0.037
0.637
0.019
0.434
Ensemble (a=2.00)
0.046
0.730
0.039
0.647
0.021
0.449
Orig. Ensemble (a=1.75)
0.122
0.960
0.114
0.956
0.090
0.938
CMlken
0.116
0,962
0.130
0,962
0.118
0.956
Ensemble (a=2.00)
0.120
0.970
0.126
0.952
0.122
0.954
Orig. Ensemble (a=1.75)
0.424
0.874
0,434
0.882
0.430
0.850
CCIT
0.332
0.862
0.356
0,806
0.378
0.818
Ensemble (ax22.00
0.386
0.828
0.386
0.832
0.418
0.824
Orig-
0.476
0.688
0,274
0.645
0.118
0.525
FisherZ
Ensemble (x=1.75)
0.484
0.682
0.320
0.648
0.127
0.550
Ensemble (a=2.00)
0.459
0.667
0.299
0.630
0.119
0.594
Table 6: Results for n 1200, Standard Laplace Z. Bold (Bold italics) indicates the ensemble (original) version statistically significantly better.
df of t-distributed Noise
df 4
df 3
df
CTT Methods
Type
Power
Type
Power
Type
Power
Orig.
0.049
0,824
0.046
0.730
0.041
0.532
RCIT
Ensemble (a=1.75)
0.067
0.860
0.053
0.792
0.040
0.578
Ensemble (a=2.00)
0.060
0.869
0.068
0.822
0.040
0.622
Orig. Ensemble (a=1.75)
0.065
0.762
0.034
0.631
0,028
0.430
LPCIT
0.039
0.755
0.034
0.690
0.013
0.464
Ensemble (am2.00)
0.050
0.747
0.041
0.678
0.015
0.436
Orig- Ensemble (a=1.75)
0.100
0.992
0.108
0.968
0.084
0,960
CMlknn
0.150
0.980
0.136
0.982
0.130
0,982
Ensemble (a=2.00)
0.082
0.982
0.128
0.972
0.098
0.984
Orig-
0.442
0.898
0.428
0.900
0.458
0.892
CCIT
Ensemble (a=1.75)
0.318
0.806
0,296
0.816
0.326
0.788
Ensemble (a=2.00)
0.348
0.862
0.318
0.810
0.312
0.812
Orig.
0.484
0.691
0.287
0.682
0.138
0.520
FisherZ
Ensemble (omm1.7)
0.545
0.726
0.371
0.685
0.134
0,591
Ensemble (a=22.00)
0.489
0.731
0.332
0.692
0.136
0.577
21

---

Published as a conference paper at ICLR 2026
Table 7: Results for n 1600, Standard Normal Z. Bold (Bold italics) indicates the ensemble
(original) version is statistically significantly better.
df of t-distributed Noise
df 4
df = 3
df = 2
CIT Methods
Type
Power
Type
Power
Type
Power
Orig.
0.039
0.866
0,042
0.799
0,037
0.551
RCIT
Ensemble (a=1.75)
0.048
0.932
0.047
0.870
0.043
0.642
Ensemble (a=2200)
0.069
0.931
0.069
0.883
0.065
0.681
Orig. Ensemble (a=1.75)
0.072
0.744
0.055
0.674
0.024
0.415
LPCIT
0.035
0.755
0.023
0.707
0,012
0.465
Ensemble (a=2.00)
0.053
0.754
0.023
0.696
0.015
0.423
Orig. Ensemble (a=1.75)
0.102
0.994
0.104
0.996
0.114
0.994
CMlken
0.146
0.998
0.128
0.998
0.164
0.986
Ensemble (a=2.00)
0.142
1.000
0.104
0.996
0.130
1.000
Orig. Ensemble (a=1.75)
0.428
0.932
0,434
0.928
0.428
0.926
CCIT
0.232
0.834
0.256
0,806
0.230
0.752
Ensemble (ax22.00
0.270
0,828
0.260
0.826
0.260
0.802
Orig-
0.232
0.756
0.130
0.634
0.105
0.526
FisherZ
Ensemble (a=1.75)
0.248
0.759
0.142
0.672
0.112
0.576
Ensemble (a=2.00)
0.216
0.744
0.119
0.665
0.101
0.535
Table 8: Results for n 1600, Standard Laplace Z. Bold (Bold italics) indicates the ensemble
(original) version is statistically significantly better.
df of t-distributed Noise
df = 4
df 3
df 2
CIT Methods
Type
Power
Type
Power
Type
Power
Orig.
0.046
0.862
0.043
0.757
0.038
0.553
RCIT
Ensemble (a=1.75)
0.062
0.913
0.053
0.849
0.039
0.639
Ensemble (a=2.00)
0.071
0.916
0.060
0.882
0.060
0.673
Orig. Ensemble (a=1.75)
0.075
0.755
0.062
0.663
0.018
0.421
LPCIT
0.043
0.750
0.037
0.716
0.009
0.482
Ensemble (a=2.00)
0.045
0.772
0.040
0.714
0.014
0.453
Orig. Ensemble (a=1.75)
0.098
0.990
0.112
0.980
0.114
0.982
CMIkon
0. 146
0.992
0.160
0.994
0.144
0.994
Ensemble (a=2.00)
0.116
0.994
0.134
0.990
0.106
0.994
Orig. Ensemble (a=1.75)
0.452
0.936
0,462
0.910
0.436
0.908
CCIT
0.266
0,810
0.230
0.788
0.254
0.768
Ensemble (a=2.00)
0.282
0.788
0.248
0.818
0.280
0.758
Orig-
0.509
0.707
0.337
0.682
0.121
0.525
FisherZ
Ensemble (c75)
0.554
0.747
0.375
0.703
0.164
0.558
Ensemble (a=2.00)
0.556
0.720
0.361
0.704
0.157
0.612
E.3 IMPACT OF DIMENSIONALITY OF CONDITIONING SET Z
Figure 6 illustrates the effect of conditioning set Z dimensionality on the performance of KCIT, RCIT, LPCIT, and Fisher Z-test, using fixed sample size of 1200, with Z sampled from a standard Gaussian and noise from a t-distribution with df=3. For each method, we plot the Type I error rate (left subfigure) and the test power (right subfigure) function of the dimensionality of Z.
22

---

Published as a conference paper at ICLR 2026
Type/ Error
Power
Type Error
Power
0.6
0.8
Bnor Ral
0.6
Power
0.8
Power
KDIT
Emor
ER 175)
0.0
0.0
2
2
3
Dimension of
Dimension
Dimension
Dimension
(a)
KCIT
(b) RCIT
Type Error
Power
Type Error
Power
0.6
0.8
Eioor Ra0
0.6
Power
0.6
Power
Emor
0.2
0.0
0.0
2
3
Dimensiion of
Dimenston
Dimensioo
Dimension of
(c) LPCIT
(d) Fisher Z-test
Figure 6: Effect of the conditioning set dimensionality on the ensemble performance of KCIT, RCIT, LPCIT, and Fisher Z-test.
Although E-КCIT does not perform well at this relatively small sample size, the ensemble framework generally provides performance improvements across methods and conditioning set dimensionalities, further demonstrating its effectiveness for CIT.
E.4 DETAILS OF EXPERIMENT ON REAL-WORLD FLOW-CYTOMETE DATA
We conduct experiments on the Flow-Cytometry dataset, a widely used benchmark for evaluating CIT methods and causal discovery algorithms (Li et al. 2023b Sen et al., 2017; Mukherjee al., 2020 Ng et al. 2020 Zhu et al. 2019 Mooij & Heskes 2013). This dataset originates from the seminal study by Sachs et al. (2005), which employed multiparameter flow cytometry to measure single-cell signaling in primary human CD4+ T cells. The dataset is available in the supplementary material of Sachs et al. (2005). (It is important to clarify that this dataset consists of real-world biological data, not computer simulations. The term "stimulations" in the original study refers to biological reagents used to perturb the cells, not synthetic data generation.)
The dataset comprises measurements of 11 phosphorylated proteins and phospholipids. The consensus causal graph, derived by domain experts and validated in the original study, serves as the ground truth. follow the standard protocol in CIT literature (Li et al. 2023b] Sen et al. [017]] Mukherjee et al.) 20200 Ng et al.) 2020: Zhu et al. 2019; Mooij & Heskes 2013) by treating specific subsets as observational data. This is justified
Although the data were collected under biological stimulations,
because the stimulations act on unobserved exogenous root nodes, preserving the downstream causal
structure among the observed proteins.
To provide comprehensive evaluation and address different sample size regimes, we utilize two configurations of this dataset:
Dataset A (n = 1755): Following the setup of Li et al. 2023b), we combine the samples from the cd3cd25 and cd3cd28 cam2 experimental conditions. This results in a total of 1755 samples. This setting tests the methods' performance on moderate sample size.
Dataset B (n = 853): To align with benchmarks such as CCMI (Mukherjee et al. 2020) and evaluate performance on smaller sample sizes, we explicitly evaluate on the ed cd2 alone, which contains 853 samples.
From the consensus graph, we extract comprehensive set of 50 conditionally independent and 50 conditionally dependent pairs for evaluation. Similar to Section we compare six CIT methods and their ensemble variants. We evaluate performance using precision, recall, and F1-score.
23

---

Published as a conference paper at ICLR 2026
Table 9: Performance comparison on the Flow-Cytometry dataset (results for both a merged). Bold
(Bold italics) indicates the ensemble (original) version is statistically significantly better.
Method
Precision
Recall
Fl-score
Orig.
0.880
0.667
0.759
KCTT
Ensemble
0.862
0.660
0.747
Orig. 0.888
0.654
0.753
RCIT
Ensemble
0.872
0.665
0.754
Orig.
0.912
0.660
0.766
LPCIT
Ensemble
0.924
0.673
0.778
Orig.
0.920
0.667
0.773
CMiknn
Ensemble
0.904
0.667
0.767
Orig.
0.560
0.636
0.596
CCTT
Ensemble
0.594
0.658
0.623
Orig. 0.940
0.671
0.783
FisherZ
Ensemble
0.942
0.672
0.784
Precision and recall are defined as TP/(TP- + FP) and TP/(TP + FN), respectively. The F1 F1-score the harmonic mean of the two. Here, TP and TN denote correctly identified conditionally dependent and independent instances, respectively, while FP and FN denote incorrect predictions.
Consistent with our hyperparameter guideline to maintain subset size лą R 400, we set K 5 for Dataset (n& 351) and K for Dataset B (ng s 426). We evaluate E-CIT with both 1.75 and 2. As the results were stable across these values, we report them jointly. Due to the randomness in data partitioning. E-CIT results are averaged over 10 runs. RCIT (Strobl et al.) 2019) and its ensemble version averaged over 100 runs account for their inherent randomness. The results for the two datasets are presented in Tables 3 and
E.5 DETAILS OF EXPERIMENT ON THE APPLICATION IN CAUSAL DISCOVERY
In the causal discovery experiments, generate synthetic causal graphs as follows. Each graph contains a backbone path generated according fixed topological order, while all other possible edges are added independently with probability 0.3. Data are then simulated according to the graph structure: each variable is computed as the sum of its parent variables after transformation by nonlinear function (randomly chosen from {x, 2 x², tanh(a (z). cos(x))), with additive noise drawn from a standard Student's (df = 2). Laplace, or Cauchy distribution. We apply the PC algorithm (Spirtes et al. 2000) 50 times using each CIT method.
We exclude FastKCIT (Schacht Huang 2025) from this comparison because its assumption that the conditioning set can be well approximated by Gaussian mixture with V components results in highly unstable performance in strongly non-Gausser nn--ausssian scenarios. Figure 3 reports results in terms of F1-score (left), SHD (middle), and runtime (right). SHD measures the number of edge operations required to transform the estimated graph into the ground-truth graph. To isolate the effect of conditional independence testing from that of edge orientation rules in the PC algorithm, both Fl-score and SHD are computed on skeleton graphs only.
E.6 ABLATION STUDY OF SUBSET SIZF
We investigate how the choice of the subset size n& affects the performance of E-CIT under the same
post-nonlinear model setup as in Section 4.1 Specifically, we fix the total sample size to n = 2000 and consider пı {200, 285, 333, 400, 500, 666, 1000). For each configuration, conduct 1000 independent experiments and compare the performance of E- KCTT (with a 1.75 and a = 2) against the original KCIT. Results are summarized in Figure 7
Overall, although the subset size ng has some impact on E- KCIT's performance, the ensemble method shows competitive performance relative the original KCTT across most values, demonstrating the
24

---

Published as a conference paper at ICLR 2026
Type| Error
Power
Orgi.
0.85
Enor Rate
0.15
a=2
Power
Significance level
0.50
0.10
0.85
0.08
0.80
0.00
230
686
1000
reap
ng.
ng
(a) t-distributed noise
Type Error
Power
Orgi.
Ongi
Error Rate
0
2
Power
0.7
Sign lfcaanc level
0.06
0.00
1020
1000
nk.
nk
(b) Gauchy-distribun noise
Type Error
Power
Orgi.
Ongi
0.98
Enor Rate
Power
0.96
Signnficacce level
0.82
0.00
1099
0.80
280
2365
66
nk
(c) daplace-distriueted noise
Figure 7: Comparison of Type error (left), test power (right) for E-КCIT (with different ny.) and KCTT under different noise distributions.
robustness of E-CIT with respect to this parameter. Importantly, the choice of nı = 400 adopted in other experiments (an empirical choice following previous CIT studies) is not necessarily optimal. This observation highlights the potential for further gains within our framework. Nevertheless, we note that ny influence the control of Type I error. This arises because the asymptotic distribution of subtest p-values may deviate from Uniform(0, 1) depending on 74. As shown Figure 7 this effect is mild and can be mitigated by simply avoiding excessively small values of ng to ensure the asymptotic distribution remains close to Uniform 1). For more detailed discussion, please refer to Appendix F
E.7 P-VALUE COMBINATION METHOOSS FOR E-CIT
We further compare proposed p-value combination method with several classical alternatives. We fix the total sample size at 2000 and subset size at n4 = 400, while keeping all other settings identical those in Section 4.1 We perform 1000 repetitions of E-KCIT under three different noise distributions, and report results Table 10
Across all scenarios, method achieves the highest power while maintaining valid Type error control, consistently outperforming the classical alternatives. worth noting that when a = 2, our method reduces to Stouffer al. (1949). In this case, omit it from the table.
25

---

Published as a conference paper at ICLR 2026
Table 10: Comparison of different combination methods
Combination Methods
Noise
Metric
Ours
Tippett
Edgington
Fisher
Pearson
Mudholkar
Type I
0.033
0.035
0.289
0.035
0.096
0.018
Power
0.919
0.735
0.956
0.876
0.933
0.876
Type
0.032
0.031
0.463
0.028
0.196
0.015
Cauchy
Power
0.649
0.467
0.925
0.546
0.821
0.547
Type 1
0.065
0.076
0.254
0,067
0.110
0.046
Laplace
Power
0.950
0.784
0.971
0.935
0.956
0.928
KCIT
under H₁
E-KCIT under
E-KCIT H1
20
15
15
10
10
0.0
0.4
0.8
p-value
p-value
p-value
p-value
(a) KCIT and E-КCIT
LPCIT under Ho
LPCIT under H1
E-LPCIT under Ho
E-LPCIT under H1
20
20
Denely
15
5
15
2
10
0.0
0.2
0.4
0.6
0.8
1.0
0.0
0.2
0.4
0.6
088
1.0
0.0
0.
0.4
0.6
0.8
1,0
0.0
0.2
0.4
0.6
0.8
1.0
p-value
p-valie
(b) LPCIT and E-LPCIT
Figure Empirical distribution of KCIT, E-KCIT, LPCIT, and E-LPCIT under the null and alternative hypotheses. Results are obtained under the post-nonlinear setup of Section 4.1 with 2000, ny 400, and standard t-distributed noise (df = 3), with a total of 1000 replications.
F LIMITATIONS AND FUTURE DIRECTIONS
The primary goal of E-CIT is to provide a general-purpor framework for reducing the computational cost of CIT. Many in-depth discussions that require access to the alternative hypothesis distribution of the base CIT methods fall outside the main scope of this work, but they represent important directions for future research. We diseuss several key points below:
Type Error Control. Controlling Type error is a central problem in hypothesis testing and remains one of the major challenges for CIT. In Theorem [ provide theoretical guarantees for Type I error control of E-CIT, in the ideal case where the subtest p-values are perfect. Although the asymptotic validity of specific CIT methods is the focus of this work, in practice, deviations of subtest p-values from the Uniform 1) may induce shifts in Type I error, which can be amplified by E-CIT. As illustrated in Figure the p-value distributions of the original and ensemble versions of KCIT and LPCIT under the null and alternative hypotheses show that under the null, KCIT p-values are slightly deflated both ends, while L.CIT p-values are slightly inflated. These deviations lead to right-skew and left-skew in their ensemble versions, which may result in slightly higher or lower Type I errors.
Such deviations are not necessarily detrimental. Indeed, we observe many ensemble versions exhibiting slightly lower Type I error. Nonetheless, Type I error should ideally be maintained at the nominal level in hypothesis testing. Although this is challenging in the context of CIT, it is possible to minimize such deviations within the E-CIT framework in practice. We recommend that the subtest sample size n, be sufficiently large to ensure that the performance requirements under the alternative hypothesis (Theorem 2 satisfied, while also guaranteeing sufficiently good asymptotic
26

---

Published as a conference paper at ICLR 2026
Uniform(0, 1) behavior under the null. Our ablation studies subset size in Appendix E.6 support
this recommendation.
Super-Unifore p-values from Permutation Tests. Many CIT methods, particularly those based on the Conditional Randomization Test (CRT), obtain p- values via permutation. These p-values are super- uniform under the null, meaning that for any [0, 1]. P(p I a) S The discrete, stepwise nature of their CDF can lead to conservative behavior. However, as the number of permutations increases, the super-uniform distribution approaches a strict uniform distribution, and this theoretical convergence justifies the approximation. The validity guarantee in Theorem relies on transforming strictly uniform p-values to stable distribution via the inverse probability integral transform. Applying the same transform to super-unifomn p-values does not yield an exact stable distribution, but with a sufficiently large number of permutations, the resulting distribution closely approximates the ideal transformation. Hence, in practice, super-uniform p-values from permutation tests do not significantly impair the performance of E-CIT.
Additionally, permutation-basee p-values may take values exactly equal to 0 or 1, corresponding to extreme rejection or acceptance of the null. Although these values are within the framework's definition, they can create practical computational issues. We suggest adding small uniform random perturbation to permutation p-values to avoid such issues and to produce a more continuous distribution, closer to strict uniformity.
should be noted that for permutation- based CIT, the computational savings of E-CIT are limited, as the primary cost arises not from sample complexity but from repeated permutations. Developing meth- ods that specifically reduce the computational burden of the permutation procedure itself represents an important direction for improving the practical scalability of causal discovery algorithms.
Our experiments indicate that E-CIT achieves competitive even superior test power. The key factor is the comparison of two convergence rates: whether adding more samples directly to single test or performing multiple subtests and aggregating leads faster growth in test power. For traditional parametric tests, power grows rapidly with sample size, making separate subtests inefficient. In contrast, for CIT, both the inherent convergence limitations of the base test and the difficulty of satisfying its consistency assumptions in practice suggest that separate subtests may be more efficient. While prior work has investigated the convergence rates of specific CIT methods (Jamshidi et al.) 2024] Balabdaoui et al., 2025), establishing theoretical guarantees comparing the two strategies remains a fundamental open problem.
Compatibility of the Ensemble Framework with CTT and Convergence Rates.
Hyperparameter Selection and Theoretical Optimality. As general framework, E-CIT does not require access to the alternative hypothesis distribution and therefore cannot provide a theoretically optimal combination strategy, particularly with respect to the stability parameter Currently, our framework primarily offers flexibility that allows adaptation to different CIT methods and data distributions. Empirical choices for are discussed in Appendix [I. Future work could explore the framework's ' full potential by investigating theoretically optimal aggregation strategies for specific CIT methods based on their properties under the alternative hypothesis. While this is complex problem, we currently recommend using empirical values (a 1.75 or 2), which, although not theoretically optimal, have proven be effective in our experiments.
The second core hyperparameter is the number K of subtests (or equivalently, the subset size ng)). In practice, it is sufficient to ensure that each subset is large enough to maintain good asymptotic Uniform(0 00, 1) behavior under the null and adequate power under the alternative of the subtests, so that Theorems [ and 2 hold approximately. The choice of ny can be guided by empirical results from studies of the original CIT methods, balancing statistical performance and computational cost.
The Curse of Dimensionality. The curse of dimensionality While our experiments real-world datasets encompass varying dimensions for the conditioning set Z, with a dedicated empirical analysis in Appendix EE33 addressing this issue directly is not the primary objective of the E-CIT framework. Instead, E-CIT is designed to be orthogonal to the internal mechanisms of base tests. Indeed, numerous specialized CIT methods (Sen et al., 2017: Bellot & van der Schaar 2019 Shi et al.) [02T Li et al., 2023a b) have already been developed to handle the high-dimensional conditioning sets Z, and H-CIT' can serve as a scalable wrapper for these methods.
fundamental challenge in CIT.
27

---

Published as a conference paper at ICLR 2026
Correlated p-values. One may consider using resampling methods such bootstrap to generate additional p-values and improve small-sample test performance, In fact, a similar idea has already been validated in small-sample causal discovery scenarios (Xiang al. (024). However, this approach inevitably introduces correlations among p-values. In CIT, the theoretical form of the p-value distribution under the alternative is inherently challenging, making it difficult model correlations between tests, unlike in the parametric setting studied by (Liu & Xie, 2020) [ing & Rho 2022). While E-CIT focuses on reducing computational in large-sample settings, exploring strategies to exploit overlapping data splits or correlated subtests is an interesting direction for improving CIT performance.
Distribution Drifts. The current theoretical guarantees of E-CIT rely on the assumption that the K subtests are independent and identical, resulting in i.i.d. p-values. However, in many real- world scenarios, such as data collected across heterogeneous environments or time-series data, the underlying data-generating mechanisms may experience distribution shifts. such instances, the theoretical guarantees of our current framework may not strictly hold. Understanding how these drifts impact the aggregated p-value, and developing adaptive subset partitioning or robust aggregation strategies enhance resilience, represent highly practical direction for future research.
-ethod-specific Enhancements. E-CIT is designed as a general framework and does not incorporate method-speciffe optimizations. While we demonstrate its effecttiveness across multiple CIT methods, further improvements may be possible by integrating E-CIT more deeply with specific methods. For example, DGCIT (L-folds) (Shi al., 202T) and NNSCIT (3-folds) (Li al. 2023a) internally data splitting. Combining E-CIT principles with these internal schemes may further enhance sample efficiency.
28


Reviews:
[{"review_id": "lbombbugpj", "reviewer": "Reviewer_CASq", "summary": "This paper proposes an efficient ensemble independence testing approach that reduces computational cost by first partitioning the data into subsets, then applying a base conditional independence test (CIT) on each subset, and finally aggregating the resulting p-values. This strategy reduces the complexity of a base CIT to be linear in the sample size. Empirical results show that the proposed method improves efficiency while maintaining comparable accuracy.", "questions": "1.\tCould the authors provide references to support the claim that ensemble learning can address high computational complexity (lines 131–134)?\n2.\tIs this the first work to propose an ensemble-based conditional independence testing approach? \n3.\tWhat happens if the assumptions in Definition 1 are violated? Does the derived ensemble test still hold under such conditions?\n4.\tWhat are the implications if the unbiasedness assumption in Theorem 1 is violated? For instance, subtests may be biased in practice if the selected data subsets are not representative. While I understand there is a theoretical–practical gap, it would be helpful to analyze how sensitive the proposed algorithm is to such violations, given that the paper emphasizes its asymptotic optimal properties.\n5.\tWhy do the authors assume nonlinear causal functions? It would be informative to also report results for linear causal functions. Additionally, can the proposed algorithm be applied to data with Gaussian noise? If so, does it still demonstrate advantages in accuracy and efficiency compared to state-of-the-art methods?\n6.\tThe experiments on real data are not entirely convincing. The paper uses synthetically generated interventional data rather than directly collected real-world data. A more rigorous and widely accepted evaluation strategy would rely solely on the 783 observational samples. Otherwise, it is difficult to substantiate the claim that the proposed method performs well on real-world data.\n7.\tThe sample size may not be the dominant factor affecting causal discovery efficiency. The primary challenge for constraint-based approaches lies in the accuracy and scalability of conditional independence tests when the conditioning set is high-dimensional. How does the proposed approach perform under such settings?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "-\tThe paper presents a strong motivation and a practical approach for improving the efficiency of conditional independence tests, and consequently, constraint-based causal discovery.\n-\tThe paper is well-written and clearly presented.", "weaknesses": "-\tThis paper provides limited evaluation of generality; see Questions 5, 6, and 7 for details.\n-\tThe paper makes strong claims of optimality but lacks sensitivity analysis for violations of the theorem’s assumptions, some of which appear overly strong.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "DhrP0VJXfN", "reviewer": "Reviewer_nRmx", "summary": "This paper introduces E-CIT, a divide-and-aggregate framework to solve the high computational cost of conditional independence (ci) tests. E-CIT partitions the data into subsets, applies base CI tests to each subset, and combines the resulting p-values. Its core contribution is a p-value combination method based on stable distributions, which provides theoretical consistency. This framework reduces the complexity of base CITs to linear in sample size (when subset size is fixed) and demonstrates competitive or superior power, especially in heavy-tailed scenarios.", "questions": "1. Is $n_k=400$ still a good choice with a larger sample size?\n2. Please elaborate more details on the differences and the relation with Stouffer et al. (1949).", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. The paper is well-structured, and the proposed framework is also intuitive.\n2. The framework can be combined with different CI methods. \n3. Table 1 is clear and useful for unfamiliar readers.", "weaknesses": "1. It is intuitive to divide the sample set into several subsets to reduce computational burden. However, I find it difficult to understand why this simultaneously enhances statistical power in experiments. The theoretical justification throughout the paper does not focus on or support this point. Consequently, we are still unclear under which conditions the E-CIT framework outperforms single-test execution.\n2. The paper's core theoretical guarantees depend on the p-values from the subtests being well-defined and effective. To ensure the subtest p-values are accurate, $n_k$ should be sufficiently large and tend to infinity. On the other hand, the paper claims that the framework reduces time complexity to $O(n)$ when $n_k$ is treated as a fixed constant, which raises a contradiction. $n_k$ cannot be simultaneously treated as a small, fixed constant to justify the $O(n)$ complexity claim, and as an asymptotically large number to justify the theoretical validity and consistency of the theorems. \n3. The paper states that the median heuristic is used for all methods to ensure a fair comparison. However, bandwidth is sample-size dependent. To isolate the effect of the ensemble strategy, I suggest the authors attempt to find the optimal bandwidth for each test in its respective setting (e.g., via bandwidth optimization [1]). \n4. The framework does not provide a principled or automated method to select $n_k$. \n5. The authors claim in the contribution that the p-value combination method proposed is novel. It may be slightly overstated. The method shares a related technical foundation with existing works on p-value combination (e.g., Ling & Rho (2022), Liu & Xie (2020)), which are already mentioned in the paper's related work. \n\nReferences:   \n[1]  W. Wang et al, Practical Kernel Selection for Kernel-based Conditional Independence Test. NeurIPS 2025.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "jPoW9maO4Q", "reviewer": "Reviewer_cUG3", "summary": "This paper proposes E-CIT, a framework for scalable conditional independence testing (CIT) that divides data into subsets, applies any base CIT independently, and aggregates p-values through a stable-distribution–based combination. The goal is to reduce computational cost while preserving Type I error control and statistical stability. The paper provides theoretical proofs of validity and convergence, and empirical results on synthetic and real datasets showing comparable accuracy to full-sample CITs but much better efficiency.", "questions": "Since E-CIT combines subset-level p-values, how sensitive is it to the number of partitions. does too few or too many subsets affect power consistency?\n\nThe theoretical analysis assumes independence between subsets; in real data where partitions can still share dependence structure, how much does this violate the validity guarantees?\n\nCould the authors compare their stable-distribution aggregation directly against the more recent Cauchy Combination Test (Liu & Xie 2020) to clarify relative advantages?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. The paper addresses a practical bottleneck in causal discovery with a general framework that can wrap around existing tests such as without retraining.\n\n2. The use of stable-distribution aggregation is mathematically sound and provides more stable inference than classical methods, with clear theoretical foundation.\n\n3. Experiments show that E-CIT achieves similar Type I and power performance as full CIT while reducing computation time by up to an order of magnitude on large datasets.", "weaknesses": "1. The proposed framework is mainly an adaptation of existing ensemble ideas; it does not introduce a fundamentally new conditional independence test or inference principle.\n\n2. Although the theoretical proofs are solid, they mostly re-establish known results from p-value combination theory in the CIT context, making the methodological novelty limited.\n\n3. The evaluation focuses mostly on synthetic data and standard causal discovery tasks; there is little evidence on whether E-CIT maintains robustness on high-dimensional or real-world noisy domains.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "JQZ1qQPBMD", "reviewer": "Reviewer_7jSe", "summary": "The paper is related to causal discovery and proposes a generic Ensemble Conditional Independence Test (E-CIT), which uses a divide-and-aggregate strategy to reduce the typical superlinear numerical complexity to a linear one. The approach is compatible with various existing CIT methods. Several numerical experiments are performed to illustrate the performance of the ensemble approach. Results for different baseline setups look promising. Type I error and statistical power remain competitive while saving computational time.", "questions": "Does it make a difference how the data is partitioned into the K subsets?\n\nHow to automatically choose the hyperparameter \\alpha in general applications? How to choose the number of samples K?\n\nAre the results worse in Table 2 and 3 if alpha is chosen smaller or larger?\n\nThe left windows in Figure 2 for the Type I Error do not have a good y-axis scaling. The key differences cannot be seen. Please use a e.g. range between 0.0 and 0.08.\n\nUnder which conditions the approach performs not good (e.g. regarding the data partitioning)?\n\nMinor comments:\n\n- line 391,408,410: Why are some numbers italic? What does it mean?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "S1 The considered problem is interesting and relevant.\nS2 The paper is technically sound and overall well-written.\nS3 To obtain linear complexity is advantageous.\nS4 The evaluation and comparison against baselines is convincing.", "weaknesses": "W1 It remains unclear whether hyper parameters have to be tuned.\nW2 Limitations of the paper’s methods could be better discussed.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)