Title: Practical Kernel Selection for Kernel-based Conditional Independence Test
Venue: NeurIPS 2025
Accepted: yes
Keywords: conditional independence testing

kernel method

causal discovery
Abstract: Conditional independence (CI) testing is a fundamental yet challenging task in modern statistics and machine learning. 
One pivotal class of methods for assessing conditional independence encompasses kernel-based approaches, known for assessing CI by detecting general conditional dependence without imposing strict assumptions on relationships or data distributions.
As with any method utilizing kernels, selecting appropriate kernels is crucial for precise identification. 
However, it remains underexplored in kernel-based CI methods, where the kernels are often determined manually or heuristically.
In this paper, we analyze and propose a kernel parameter selection approach for the kernel-based conditional independence test (KCI).
The kernel parameters are selected based on the ratio of the statistic to the asymptotic variance, which approximates the test power for the given parameters at large sample sizes. 
The search procedure is grid-based, allowing for parallelization with manageable additional computation time.
We theoretically demonstrate the consistency of the proposed criterion and conduct extensive experiments on both synthetic and real data to show the effectiveness of our method.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a kernel parameter selection method for kernel-based conditional independence (KCI) testing, serving as an alternative to the commonly used median heuristic. The method is theoretically proven to be consistent and explicitly accounts for model estimation bias. Extensive empirical studies on both synthetic and real-world datasets are conducted to validate the effectiveness of the proposed approach.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths: 1. This paper investigates a valuable topic and provides a well-motivated problem statement. \n\n2. The KCI test is an important tool in machine learning and data analysis, and kernel parameter selection has long been a challenging research problem. This paper proposes a practical parameter selection method that can encourage the broader application of KCI methods in scientific research. \n\n3. The paper is well-structured and easy to follow, the mathematical descriptions are mostly clear and well-written. \n\n4. The paper provide both theoretical proofs and empirical evidence to validate the superiority of the proposed method.\n\nWeaknesses: 1. The comparison of baseline CI testing methods in the simulation study is unfair. To the best of my knowledge, the conditional mean in GCM is typically estimated using kernel ridge regression or XGBoost regression. However, in lines 863 to 864 of the paper, Lasso, a linear regression method, is used instead. This can lead to poor performance of the GCM method under nonlinear simulation settings, which is indeed reflected in its empirical results.\n\n2.The distinction between the methods in the figures is not very clear and requires very careful examination — for example, \"Power\" and \"Median\" in Figure 1, and \"Median\" and \"Org\" in Figure 2.\n\n3.There is no evidence provided for the parameter selection, such as the choice of the candidate weight list.\n\nQUESTIONS:\nQuestions. 1. I suggest presenting the results of the ablation study for \"On the selectable kernels\" and \"On the decomposition\" separately, so that it is easier to see the impact of each component on the empirical results. \n\n2.The results on the CHILD dataset in Table 2 show that KCIT with the median heuristic for kernel parameter selection performs better. Why isn't it highlighted in bold?\n\n3.I suggest that the authors include the running times of other methods in the main experiments in Section 5.1.3. In addition, I am curious about the running times when these two KCI testing methods are used within the PC algorithm for causal discovery.\n\n4.I'm not sure how the candidate weight list in line 233 is chosen. Is it selected based on the theoretical results provided, or is it chosen empirically?\n\n5.Could you explain in more detail how the p-value of the test is computed in line 237? Is this based on the theoretical results provided in the paper?\n\n6.Why does SelZ not improve the Type II error compared to Median? Please provide an intuitive explanation.\n\nIf the authors can address my concerns, I will consider increasing my score.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-06-10"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper presents a new parameter selection method for Kernel Conditional Independencet (KCI) testing. Specifically, the kernel selection criterion is defined as the ratio of the estimates of $C^2_{KCI}$ and $2\\sigma_1$, which both depend on the proposed estimator \\hat{C}^2_{KCIu}. The convergence rate is provided in Theorem 3 and several experiments are conducted to demonstrate the empirical performance.\n\nSTRENGTHS AND WEAKNESSES:\nAlthough I cannot understand all the math involved, I like the research question a lot. Such (hyper)-paremeter-choice research questions resolve important practical challenges and potentially have a high impact both for practitioners and for researchers who focus on more applied research, yet are often neglected. \n\nStrength: \n- The paper is quite well-written and easy to follow. \n- I presume the convergence rate is not bad, i.e., C_2 n^{-1/2} + C_1 n^{-(beta -1) / (2(\\beta + p))}, considering the range of \\beta and $p$. \n- The experiment results are strong: within those methods that can effectively control type-I error, the proposed method \"Power\" achieves the lowest type-II error in general. \n\nWeakness:\n- I would apprecaite a more thorough related work review about kernel (paremeter) selection in the main body. \n- It's a bit unclear to me if the U-statistic (eq.5) has been considered by other previous work. I feel the writing a bit unclear here.\n\nQUESTIONS:\nFirst, I have some minor suggestions on writing/typos: \n- Eq.12: I believe it should X/Y \\sim f( . ) + E?\n- Line 100 - 102: the sentence starts with g(.) but ends with \"mappings of (X, Z) and Y onto Z\". I believe, (X, Z) --> Y is about h(.) not g(.)?\n\nSecond, two questions: \n- Line 199, the Boundedness assumption: Could you explain how we can obtain \"Then h_{ij} and \\hat{h}_{ij} are bounded within [−ν_z ν^2,ν_z ν^2]?\n- Of course It's already a lot of experiments and a lot of work for a conference paper, but have you considered checking the empirical performance on other causal structures, i.e., chain or collider?\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 2, "CLARITY": 4, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-22"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a simple yet effective kernel selection method for conditional independence testing (CIT). The method enhances the power of CIT by maximizing the ratio of the KCI statistic to its variance. The authors also provide rigorous theoretical guarantees, establishing the convergence rate of their estimator. Empirical results demonstrate that selecting kernel parameters using the proposed approach improves performance by approximately 5%–10% over original methods without kernel selection. Overall, this paper is technically solid and constitutes a valuable contribution to the kernel-based CIT family.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths: \n1. The method is very simple and intuitive in practice, yet proves to be effective. And the authors provided corresponding theoretical analysis.\n2. Cleverly leverage parallelization to carry out the parameter selection, without substantially increasing the overall computational time.\n3. Theorem 3 appears to make a significant contribution to the development of the KCI family of methods.\n\nWeakness: \n1. My primary concern is that the proposed method can only be applied to improve algorithms within the KCI family, and the improvement may not be substantial.\n2. My second concern pertains to the runtime of the algorithm when both the sample size and the dimensionality of Z are high (e.g., dz=30-50, n=1000-2000). The authors have only provided runtime results for cases where dz<10. Given the computational complexity of kernel methods (O(n^3)), I believe this issue could significantly impact the applicability of the proposed method to large-scale datasets.\n\nQUESTIONS:\n1. Is it possible to extend your method to scenarios where both X and Y are high-dimensional, despite the well-known fact that kernel methods generally suffer from the curse of dimensionality? Plus, will high-dimensional X, Y obstacle to the proof of theory\n2. Given that the topic of this paper is CI **testing**, it would be very beneficial to extend Theorem 3 to bound type I and type II errors of the tests with the tuned kernel.\n3. I am not quite clear on how the weight list in line 233 is chosen. Could you provide further explanation or clarification?\n4. From Figure 1,3, it can be observed that compared to the ‘Median’ approach, the ‘Power’ method achieves not only a lower type II error (which is the main improvement highlighted in the paper) but also a lower type I error (or at least does not worsen it). This seems somewhat counter-intuitive. This question may be correlated to Q2. Could the authors provide futher explanations for this phenomenon?\n5. To better show the improvement of 'Power' beyond 'Median' and other tests, it is recommended to show the densities/histograms of p-values.\n6. When developing your Th 3, you used yourTh 7 to handle the estimation error of CME. While Liu et al. developed their Th 6 in [1]. The first term in their error bound exhibits a convergence rate of $n^{-1/3}$. Could you elaborate on what advantages your result offers compared to theirs, in terms of the first term in your (11)?\n\n[1] Liu, Learning Deep Kernels for Non-Parametric Two-Sample Tests, ICML2020.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 4, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-06-30"}

comments: SUMMARY:
This paper addresses the problem of kernel selection in conditional independence testing. It introduces a test power–based criterion along with its corresponding finite-sample estimate. Based on this criterion, a grid search is performed to identify the optimal bandwidth. Experiments on both synthetic and real-world datasets demonstrate that the proposed kernel selection method improves test power compared to the original approach.

STRENGTHS AND WEAKNESSES:
Strengths：
1. The paper provides solid theoretical support for the proposed methodology, and I did not identify any major issues regarding its correctness.
2. The approach is straightforward and enhances efficiency by parallelizing the grid search process, making it practical for real-world applications.

Weakness:
1. While the focus on CI testing is appreciated, the underlying idea of optimizing bandwidth through power maximization has been previously considered in related contexts [1], which somewhat limits the novelty of the contribution.

[1] Feng Liu, Wenkai Xu, Jie Lu, Guangquan Zhang, Arthur Gretton, and Danica J Sutherland. Learning deep kernels for non-parametric two-sample tests. In International conference on machine learning, pages 6316–6326. PMLR, 2020.

2. In the “Overall Procedure”, ω_z  is the median heuristic value (no selection); Fixing ω_z without tuning may limit the flexibility of the kernel ridge regression step and reduce the adaptability of the method in different settings.

3. The experiments are conducted solely using Gaussian kernels. While the proposed method works well under this setting, its applicability to more complex or adaptive kernel designs (e.g., deep kernels) appears limited.

QUESTIONS:
1.what is "K" in line 216?

LIMITATIONS:
yes
TITLE: Official Review
CLARITY: 3
IMPACT: 3
ORIGINALITY: 2
DATE: 2025-07-02