SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Tao Zhang
Shanghai 200433, China
dalizhangt@163.com
Yaowu Zhang*
MoE Key Laboratory of Interdisciplinary Research of Computation and Economics
zhang.yaowu@mail.shufe.edu.cn
Tingyou Zhou
School of Data Sciences
Zhejiang University of Finance and Economics
Hangzhou 310018, China
zhoutingyou@zufe.edu.cn
Measuring the nonlinear dependence between random vectors and testing for their statistical independence is a fundamental problem in statistics. One of the most popular dependence measures is the Hilbert-Schmidt independence criterion (HSIC), which has attracted increasing attention in recent years. However, most existing works have focused on either fixed or very high-dimensional covariates. In this work, we bridge the gap between these two scenarios and provide statistical insights into the performance of HSIC when the dimensions grow at different rates. We first show that, under the null hypothesis, the rescaled HSIC converges in distribution to a standard normal distribution. Then we provide a general condition for the HSIC based tests to have nontrivial power in high dimensions. By decomposing this condition, we illustrate how the ability of HSIC to measure nonlinear dependence changes with increasing dimensions. Moreover, we demonstrate that, depending on the sample size, the covariate dimensions and the dependence structures within covariates, the HSIC can capture different types of associations between random vectors. We also conduct extensive numerical studies to validate our theoretical results.
Let  $\mathbf{x} = (X_1, \ldots, X_p)^{\mathrm{T}} \in \mathbb{R}^p$  and  $\mathbf{y} = (Y_1, \ldots, Y_q)^{\mathrm{T}} \in \mathbb{R}^q$  be two random vectors. The problems of measuring nonlinear dependence between  $\mathbf{x}$  and  $\mathbf{y}$  and testing for their independence are fundamental and have a wide range of applications in statistics and machine learning. For example, dependence measures can be applied in feature screening (Fan et al., 2020b), model checking (Clarke et al., 2018), graphical models (Maathuis et al., 2018), and causal inference (Imbens & Rubin, 2015).
Many statistical tools have been developed to measure the dependence between random variables. Classical measures such as Pearson correlation (Pearson, 1920), Spearman's  $\rho$  (Spearman, 1904) and Kendall's  $\tau$  (Kendall, 1938) can only capture linear or monotonic dependence and may fail to detect complex associations. For univariate covariates, several rank-based measures have been proposed to overcome this limitation, such as those by Hoeffding (1948), Blum et al. (1961), Bergsma & Dassios (2014) and Chatterjee (2021). For multivariate covariates, various nonparametric tests have been introduced to quantify arbitrary associations, including the distance correlation (Székely et al., 2007), the projection correlation (Zhu et al., 2017), the mutual information (Berrett & Samworth, 2019), the kernel canonical correlation (Bach & Jordan, 2002), the constrained covariance (Gretton et al., 2005), and the Hilbert-Schmidt independence criterion (HSIC, Gretton et al., 2007). Among these measures, the distance correlation and the HSIC are perhaps the most widely used and studied in both statistics and machine learning. For instance, Li et al. (2012) applied the distance correlation to perform feature screening. Fan et al. (2020a) generalized the distance correlation to a conditional dependence measure under a factor model setting. Albert et al. (2022) developed an adaptive test of independence by aggregating HSIC measures. Pfister et al. (2018) defined a  $d$ -variable HSIC to characterize the mutual dependence among  $d$  random variables. It is worth noting that the distance correlation is equivalent to HSIC with a specific kernel choice (Sejdinovic et al., 2013).
High-dimensional settings pose new challenges and opportunities for dependence measures. Székely & Rizzo (2013) suggested to estimate the distance correlation with the  $U$ -statistic theory to avoid bias accumulations when the dimension is high. Ramdas et al. (2015) found that the power of kernel and distance based independence tests decreases polynomially with increasing dimension against some fair alternatives. Zhu et al. (2020) proved that, when both  $p$  and  $q$  grow much faster than the sample size  $n$ , the sample distance correlation and HSIC can only detect componentwise linear dependences. Gao et al. (2021) established a general condition for the distance correlation based test to have power approaching 1. They also showed that the distance correlation can capture certain pure nonlinear relationship when  $p = q = o(n^{1/2})$  under a specific alternative hypothesis.
In this paper, we aim to provide statistical insights into the HSIC in high dimensions, motivated by the increasing popularity of HSIC and the prevalence of high-dimensional data. We focus on the asymptotic distributions of HSIC under both null and alternative hypotheses when the dimensionality grows with the sample size. Under the null hypothesis, we prove that the rescaled HSIC converges in distribution to a standard normal distribution. We also derive a general condition for the HSIC based tests to have power asymptotically approaching one. These results extend those of Gao et al. (2021) on the distance correlation. Moreover, our analysis is more comprehensive than theirs. By decomposing the general condition, we reveal that HSIC can detect different types of dependences, depending on the dimensionality and sample orders. This sheds light on the performance of HSIC in high dimensions, which has been overlooked in the literature. For illustrative purposes, we let  $\mathbf{x}$  and  $\mathbf{y}$  be two random vectors with zero mean and identity covariance matrix. When HSIC is used to test for statistical independence between them, we obtain the following conditions for nontrivial power:
- If  $p \to \infty$  and  $q$  is fixed as  $n \to \infty$ , and there only exists the conditional mean of  $\mathbf{x}$ 's  $s$ -th polynomial given  $\mathbf{y}$ , then  $p^{(s - 1)}$  must grow slower than  $n$ .
- If both  $p$  and  $q \to \infty$  as  $n \to \infty$ , and there only exists the covariance between  $\mathbf{x}$ 's  $s_1$ -th polynomial and  $\mathbf{y}$ 's  $s_2$ -th polynomial, then  $p^{(s_1 - 1)}q^{(s_2 - 1)}$  must grow slower than  $n$ .
These two conditions reveal significant statistical insights and have important implications in practice. For example, when the data dimension  $q$  is small and  $p$  is larger than the sample size  $n$ , according to the first condition, to ensure  $p^{(s - 1)}$  smaller than  $n$ ,  $s$  can only be 1. In other words, the HSIC can only measure the conditional mean of  $\mathbf{x}$  given  $\mathbf{y}$ . When both  $p$  and  $q$  are larger than  $n$ , according to the second condition, to ensure  $p^{(s_1 - 1)}q^{(s_2 - 1)}$  smaller than  $n$ , both  $s_1$  and  $s_2$  must be 1. That is, the HSIC can only measure the covariance between  $\mathbf{x}$  and  $\mathbf{y}$ . In summary, our main contributions are:
1. We generalize the results of Gao et al. (2021) for distance correlation to HSIC, which is a more flexible and widely used measure of dependence.
2. We show that HSIC can capture different types of dependences, depending on the dimensionality and sample orders, which has rarely been realized in the literature before.
3. In contrast to Zhu et al. (2020), who only focused on the case when both  $p$  and  $q$  grow much faster than  $n$ , our results characterize a full picture of the performance of HSIC based test in high dimensions.
The rest of the paper is organized as follows. We give some preliminaries about the HSIC in Section 2. Then we present the asymptotic null distribution as well as the power analysis for high-dimensional HSIC in Section 3. In Section 4, we provide statistical insights into the HSIC through connecting the association type with the sample size, the high dimensionality, and the dependence structures within covariates. We conduct comprehensive numerical studies in Sections 5 and conclude this paper with a brief discussion in Section 6. All technical details are relegated to the Supplement.
In this section, we give a brief review on HSIC measures proposed by Gretton et al. (2007), which is derived from the notion of cross-covariance operator Baker (1973). We assume that  $X \in \mathcal{X} \subseteq \mathbb{R}^p$  and  $Y \in \mathcal{V} \subseteq \mathbb{R}^q$  are random vectors taking values in Euclidean spaces respectively. Let  $\mathcal{F}$  and  $\mathcal{G}$  be the reproducing kernel Hilbert spaces (RKHSs) on  $\mathcal{X}$  and  $\mathcal{V}$ , with associated kernels  $K(\cdot, \cdot)$  and  $L(\cdot, \cdot)$ , respectively. Then the cross-covariance operator  $C_{\mathbf{xy}}$  associated to  $\mathcal{F}$  and  $\mathcal{G}$  is defined as the operator mapping from  $\mathcal{G}$  to  $\mathcal{F}$ , such that for all  $f \in \mathcal{F}$  and  $g \in \mathcal{G}$ ,  $\langle f, C_{\mathbf{xy}} g \rangle_{\mathcal{F}} = \operatorname{cov}\{f(\mathbf{x}), g(\mathbf{y})\}$ . Instead of using the largest singular value of this operator, Gretton et al. (2007) suggested to adopt the squared Hilbert-Schmidt norm, which admits a closed form expression
$$
\begin{array}{l} \operatorname {H S I C} (\mathbf {x}, \mathbf {y}) = E \left\{K \left(\mathbf {x} _ {1}, \mathbf {x} _ {2}\right) L \left(\mathbf {y} _ {1}, \mathbf {y} _ {2}\right) \right\} + E \left\{K \left(\mathbf {x} _ {1}, \mathbf {x} _ {2}\right) \right\} E \left\{L \left(\mathbf {y} _ {1}, \mathbf {y} _ {2}\right) \right\} \\ - 2 E \left[ E \left\{K \left(\mathbf {x} _ {1}, \mathbf {x} _ {2}\right) \mid \mathbf {x} _ {1} \right\} E \left\{L \left(\mathbf {y} _ {1}, \mathbf {y} _ {2}\right) \mid \mathbf {y} _ {1} \right\} \right], \\ \end{array}
$$
where  $(\mathbf{x}_1,\mathbf{y}_1)$  and  $(\mathbf{x}_2,\mathbf{y}_2)$  are independent copies of  $(\mathbf{x},\mathbf{y})$ . When the RKHSs,  $\mathcal{F}$  and  $\mathcal{G}$ , are well chosen, (e.g., characteristic, Sriperumbudur et al., 2010), the HSIC can be served as an independence criterion in the sense that it is nonnegative and equals zero if and only if  $\mathbf{x}$  and  $\mathbf{y}$  are independent. To normalize the HSIC to range from zero to one, we follow Zhu et al. (2020) to define the squared Hilbert-Schmidt correlation as
$$
\operatorname {h C o r r} ^ {2} (\mathbf {x}, \mathbf {y}) \stackrel {\text {d e f}} {=} \operatorname {H S I C} (\mathbf {x}, \mathbf {y}) \operatorname {H S I C} ^ {- 1 / 2} (\mathbf {x}, \mathbf {x}) \operatorname {H S I C} ^ {- 1 / 2} (\mathbf {y}, \mathbf {y}). \tag {1}
$$
In addition, similar to Zhu et al. (2020) and Albert et al. (2022), we focus on the kernels that can be written in the form of  $K(\mathbf{x}_1,\mathbf{x}_2) = k(\| \mathbf{x}_1 - \mathbf{x}_2\| /\gamma_{\mathbf{x}})$  and  $L(\mathbf{y}_1,\mathbf{y}_2) = l(\| \mathbf{y}_1 - \mathbf{y}_2\| /\gamma_{\mathbf{y}})$ , where  $k(\cdot)$  and  $l(\cdot)$  are some real-valued functions defined in  $[0,\infty)$ ,  $\gamma_{\mathbf{x}}$  and  $\gamma_{\mathbf{y}}$  are the bandwidth parameters. This kind of kernel is very commonly used in the literature and includes many positive-definite kernel such as the Gaussian kernel, the Laplacian kernel, the rational quadratic kernel, and the Kernel generating Sobolev spaces, etc. For example, when  $k(x) = \exp (-x^2)$ , it corresponds to the Gaussian kernel, and when  $k(x) = \exp (-x)$ , it corresponds to the Laplacian kernel. One may refer to Genton (2001) for more examples and discussions about isotropic kernels.
3 Asymptotic Properties in High Dimensions
At the sample level, with the random sample  $\{(\mathbf{x}_i,\mathbf{y}_i),i = 1,\dots ,n\}$ , we estimate the HSIC with  $U$ -statistics,
$$
\begin{array}{l} \mathrm {H S I C} _ {n} (\mathbf {x}, \mathbf {y}) \stackrel {\text {d e f}} {=} \frac {\sum_ {(i _ {1} , i _ {2})} K \left(\mathbf {x} _ {i _ {1}} , \mathbf {x} _ {i _ {2}}\right) L \left(\mathbf {y} _ {i _ {1}} , \mathbf {y} _ {i _ {2}}\right)}{n (n - 1)} - \frac {2 \sum_ {(i _ {1} , i _ {2} , i _ {3})} K \left(\mathbf {x} _ {i _ {1}} , \mathbf {x} _ {i _ {2}}\right) L \left(\mathbf {y} _ {i _ {1}} , \mathbf {y} _ {i _ {3}}\right)}{n (n - 1) (n - 2)} \\ + \frac {\sum_ {\left(i _ {1} , i _ {2} , i _ {3} , i _ {4}\right)} K \left(\mathbf {x} _ {i _ {1}} , \mathbf {x} _ {i _ {2}}\right) L \left(\mathbf {y} _ {i _ {3}} , \mathbf {y} _ {i _ {4}}\right)}{n (n - 1) (n - 2) (n - 3)}, \tag {2} \\ \end{array}
$$
where the indexes in the summands, i.e.,  $(i_1,i_2),(i_1,i_2,i_3)$  and  $(i_{1},i_{2},i_{3},i_{4})$ , are all distinctive from each other. Then the squared Hilbert-Schmidt correlation  $\mathrm{hCorr}^2 (\mathbf{x},\mathbf{y})$  can be estimated as
$$
\mathrm {h C o r r} _ {n} ^ {2} (\mathbf {x}, \mathbf {y}) \stackrel {\mathrm {d e f}} {=} \mathrm {H S I C} _ {n} (\mathbf {x}, \mathbf {y}) \mathrm {H S I C} _ {n} ^ {- 1 / 2} (\mathbf {x}, \mathbf {x}) \mathrm {H S I C} _ {n} ^ {- 1 / 2} (\mathbf {y}, \mathbf {y}).
$$
We now discuss how to use the sample squared Hilbert-Schmidt correlation to distinguish between the null hypothesis  $H_0: \mathbf{x}$  is independent of  $\mathbf{y}$ , and the alternative hypothesis  $H_1: \mathbf{x}$  is not independent of  $\mathbf{y}$ . To implement the HSIC based test, it is required to study the asymptotic distributions for  $\mathrm{hCorr}_n^2(\mathbf{x}, \mathbf{y})$  under the null hypothesis. When the dimensions are fixed, Gretton et al. (2007) showed that  $n \mathrm{hCorr}_n^2(\mathbf{x}, \mathbf{y})$  converges in distribution to a weighted sum of independent chi-squared random variables as long as  $\mathbf{x}$  is independent of  $\mathbf{y}$ . We now study its asymptotic null distribution in high
dimensions. Before that, we define the two quantities  $H_{\mathbf{x}}(\mathbf{x}_1,\mathbf{x}_2)$  and  $H_{\mathbf{y}}(\mathbf{y}_1,\mathbf{y}_2)$  as
$$
H _ {\mathbf {x}} \left(\mathbf {x} _ {1}, \mathbf {x} _ {2}\right) \stackrel {{\mathrm {d e f}}} {{=}} K \left(\mathbf {x} _ {1}, \mathbf {x} _ {2}\right) - E \left\{K \left(\mathbf {x} _ {1}, \mathbf {x} _ {2}\right) \mid \mathbf {x} _ {1} \right\} - E \left\{K \left(\mathbf {x} _ {1}, \mathbf {x} _ {2}\right) \mid \mathbf {x} _ {2} \right\} + E \left\{K \left(\mathbf {x} _ {1}, \mathbf {x} _ {2}\right) \right\},
$$
$$
H _ {\mathbf {y}} \left(\mathbf {y} _ {1}, \mathbf {y} _ {2}\right) \stackrel {{\text {d e f}}} {{=}} L \left(\mathbf {y} _ {1}, \mathbf {y} _ {2}\right) - E \left\{L \left(\mathbf {y} _ {1}, \mathbf {y} _ {2}\right) \mid \mathbf {y} _ {1} \right\} - E \left\{L \left(\mathbf {y} _ {1}, \mathbf {y} _ {2}\right) \mid \mathbf {y} _ {2} \right\} + E \left\{L \left(\mathbf {y} _ {1}, \mathbf {y} _ {2}\right) \right\}. \tag {3}
$$
We further define another two quantities  $G_{\mathbf{x}}(\mathbf{x}_1,\mathbf{x}_2)$  and  $G_{\mathbf{y}}(\mathbf{y}_1,\mathbf{y}_2)$  as
$$
G _ {\mathbf {x}} (\mathbf {x} _ {1}, \mathbf {x} _ {2}) \stackrel {{\text {d e f}}} {{=}} E \{H _ {\mathbf {x}} (\mathbf {x} _ {1}, \mathbf {x} _ {3}) H _ {\mathbf {x}} (\mathbf {x} _ {2}, \mathbf {x} _ {3}) | \mathbf {x} _ {1}, \mathbf {x} _ {2} \},
$$
$$
G _ {\mathbf {y}} (\mathbf {y} _ {1}, \mathbf {y} _ {2}) \stackrel {{\text {d e f}}} {{=}} E \{H _ {\mathbf {y}} (\mathbf {y} _ {1}, \mathbf {y} _ {3}) H _ {\mathbf {y}} (\mathbf {y} _ {2}, \mathbf {y} _ {3}) | \mathbf {y} _ {1}, \mathbf {y} _ {2} \}.
$$
Then we summarize the asymptotic null distribution for  $\mathrm{hCorr}_n^2 (\mathbf{x},\mathbf{y})$  in the following Theorem.
Theorem 1. Assume the kernels are symmetric with finite fourth moment, i.e.,  $K(\mathbf{x}_1,\mathbf{x}_2) = K(\mathbf{x}_2,\mathbf{x}_1)$ ,  $L(\mathbf{y}_1,\mathbf{y}_2) = L(\mathbf{y}_2,\mathbf{y}_1)$ ,  $E\{K^4 (\mathbf{x}_1,\mathbf{x}_2)\} < \infty$  and  $E\{L^4 (\mathbf{y}_1,\mathbf{y}_2)\} < \infty$ . Further assume that  $p + q\to \infty$
$$
\frac {E \left\{H _ {\mathbf {x}} ^ {4} (\mathbf {x} _ {1} , \mathbf {x} _ {2}) \right\} E \left\{H _ {\mathbf {y}} ^ {4} (\mathbf {y} _ {1} , \mathbf {y} _ {2}) \right\}}{n \left\{\mathrm {H S I C} (\mathbf {x} , \mathbf {x}) \mathrm {H S I C} (\mathbf {y} , \mathbf {y}) \right\} ^ {2}} \rightarrow 0, \quad a n d \quad \frac {E \left\{G _ {\mathbf {x}} ^ {2} (\mathbf {x} _ {1} , \mathbf {x} _ {2}) \right\} E \left\{G _ {\mathbf {y}} ^ {2} (\mathbf {y} _ {1} , \mathbf {y} _ {2}) \right\}}{\left\{\mathrm {H S I C} (\mathbf {x} , \mathbf {x}) \mathrm {H S I C} (\mathbf {y} , \mathbf {y}) \right\} ^ {2}} \rightarrow 0, \tag {4}
$$
as  $n\to \infty$  . Then under the null hypothesis, we have  $2^{-1 / 2}n\mathrm{hCorr}_n^2 (\mathbf{x},\mathbf{y})\stackrel {d}{\to}N(0,1)$
Different from that when the dimensions are fixed, Theorem 1 reveals that, under certain conditions,  $n\mathrm{hCorr}_n^2 (\mathbf{x},\mathbf{y})$  is asymptotically normal in high dimensions. The asymptotic normality makes the limiting null distribution tractable in practice. It greatly expedites the implementation of HSIC based tests because no additional permutations are required to decide critical values. We remark here that the assumptions imposed in Theorem 1 to ensure the asymptotic normality are generally very mild. For the restrictions on the kernels, many commonly used kernels, including Gaussian and Laplacian kernels, are symmetric with finite fourth moment. For the assumption in (4), it is used to restrict the weak dependence within covariates in  $\mathbf{x}$  and  $\mathbf{y}$ . It holds true when  $\mathbf{x}$  and  $\mathbf{y}$  follow the multivariate normal distributions with bounded eigenvalues. It also holds true when  $\mathbf{x}$  and  $\mathbf{y}$  are  $m$ -dependent random sequences (Definition 9.1 of DasGupta, 2008). One may refer to Gao et al. (2021) for more discussions about this assumption.
Next, we explore the power performance of the HSIC based test in high dimensions. As discussed in Section 2, we restrict our attentions to the isotropic kernels. Specifically, in the remaining of this paper, we consider  $K(\mathbf{x}_1,\mathbf{x}_2)$  and  $L(\mathbf{y}_1,\mathbf{y}_2)$  to be in the form of  $k(\| \mathbf{x}_1 - \mathbf{x}_2\| /\gamma_{\mathbf{x}})$  and  $l(\| \mathbf{y}_1 - \mathbf{y}_2\| /\gamma_{\mathbf{y}})$ , respectively. Because of the kernel form, we assume without loss of generality that  $E\mathbf{x} = \mathbf{0}$  and  $E\mathbf{y} = \mathbf{0}$ . We denote by  $\mathbf{z}^{*}\stackrel {\mathrm{def}}{=}\mathbf{z} / \gamma_{\mathbf{z}}$ , where  $\mathbf{z}\in \mathbb{R}^d$  can be either  $\mathbf{x}$  or  $\mathbf{y}$ . We say that  $f(n)\asymp g(n)$  if  $f(n) = O\{g(n)\}$  and  $g(n) = O\{f(n)\}$ . Before summarizing the asymptotic power performance, we make the following assumptions:
(A1) There exists some  $\kappa_{\mathbf{z}} > 0$  such that  $E\{\| \mathbf{z}^{*}\|^{2} - E(\| \mathbf{z}^{*}\|^{2})\}^{2k}\asymp E(\mathbf{z}_{1}^{\mathrm{T}}\mathbf{z}_{2}^{*})^{2k}\asymp d^{-k}\kappa_{\mathbf{z}}$  for all  $k\in \mathbb{N}^+$ .
(A2) Let  $k_{0}(x) = k(x^{1 / 2})$  and  $l_{0}(y) = l(y^{1 / 2})$ . The first and second derivatives of  $k_{0}(\cdot)$  and  $l_{0}(\cdot)$  are uniformly bounded away from zero to infinity around  $E\| \mathbf{x}_1^* -\mathbf{x}_2^*\| ^2$  and  $E\| \mathbf{y}_1^* -\mathbf{y}_2^*\| ^2$ , respectively.
We remark here that Assumption (A1) is used to restrict the dependence structures within the coordinates of  $\mathbf{z}$ . Because  $\mathbf{z}^{*} = \mathbf{z} / \gamma_{\mathbf{z}}$  and  $E\mathbf{z} = \mathbf{0}$ , we have  $E\mathbf{z}^{*} = \mathbf{0}$  as well. Hence, both  $\| \mathbf{z}^{*}\|^{2} - E(\| \mathbf{z}^{*}\|^{2})$  and  $\mathbf{z}_1^{\mathrm{T}}\mathbf{z}_2^*$  are sums of  $d$  random variables with mean zero. Their standardized versions may converge in distribution to standard normal distributions under mild conditions (e.g., mixing conditions) by applying the central limit theorem (CLT). For instance, Volny (1989) established a CLT for non-stationary mixing processes. Alternatively, there exists some  $\kappa_z > 0$  such that  $d^{\kappa_z / 2}\{\| \mathbf{z}^*\|^2 -E(\| \mathbf{z}^*\|^2)\}$  and  $d^{\kappa_z / 2}\mathbf{z}_1^{\mathrm{T}}\mathbf{z}_2^*$  both converge in distribution to normal random variables. Then Condition (A1) is satisfied. Assumption (A2) imposes some conditions on the kernels, which covers the Gaussian and Laplacian kernels. Moreover, it also makes several requirements on the bandwidth parameters as well. For example, suppose the Gaussian kernel is used, i.e.,  $k_0(z) = \exp (-z)$ , when  $\gamma_{\mathbf{z}}$  is small enough such that  $E\| \mathbf{z}_1^* -\mathbf{z}_2^*\| ^2\to \infty$ , then we have  $k_0'(E\| \mathbf{z}_1^* -\mathbf{z}_2^*\| ^2)\to 0$ , which violates Assumption (A2). Theoretically, the bandwidth parameter  $\gamma_{\mathbf{z}}$  can be chosen from a wide range of values, as long as it satisfies condition (A2). In practice, we use the median of
$\| \mathbf{z}_1 - \mathbf{z}_2\|$  as a default value for  $\gamma_{\mathbf{z}}$ . When it has the same magnitude with  $E\| \mathbf{z}_1 - \mathbf{z}_2\|$ , Assumption (A2) can be easily satisfied for many commonly used kernels. Moreover, as demonstrated in the simulated examples in the Supplementary Material, our method exhibits robustness across various selections of  $\gamma_{\mathbf{z}}$ .
With these two assumptions, we are ready to provide the asymptotic power performance of the HSIC based test in high dimensions.
Theorem 2. Assume (A1) and (A2) hold true. Then under the alternative hypothesis, if  $n^{1/2} \mathrm{hCorr}^2(\mathbf{x}, \mathbf{y}) \to \infty$  as  $n \to \infty$ , we have  $n \mathrm{hCorr}_n^2(\mathbf{x}, \mathbf{y}) \to \infty$  in probability.
Theorem 2 reveals that, as long as the dependence measured by  $\mathrm{hCorr}^2 (\mathbf{x},\mathbf{y})$  is not too small,  $n\mathrm{hCorr}_n^2 (\mathbf{x},\mathbf{y})$  converges to infinity in probability. This, together with Theorem 1, guarantee that the HSIC based test can have nontrivial power in high dimensions. However, as the dimension increases, the signal measured by  $\mathrm{hCorr}^2 (\mathbf{x},\mathbf{y})$  may decay to zero, which makes the test lose power in high dimensions. To gain more insights about how the power is influenced by the dimensions, it is desired to connect  $\mathrm{hCorr}^2 (\mathbf{x},\mathbf{y})$  with the dimensionality, as well as the association types between  $\mathbf{x}$  and  $\mathbf{y}$ .
4 Statistical Insights in High Dimensions
We have discussed the asymptotic properties of HSIC based tests in high dimensions, and demonstrated that it can detect the alternative hypothesis as long as the signal strength measured by  $\mathrm{hCorr}^2 (\mathbf{x},\mathbf{y})$  does not decay to zero too fast. However, it is still unclear that how  $\mathrm{hCorr}^2 (\mathbf{x},\mathbf{y})$  is influenced by the dimensions. To gain more insights into the performance in high dimensions, in this section, we study what kind of association types the HSIC can detect under different relationships between the sample size and the covariate dimensions. To this end, we expand  $\mathrm{hCorr}^2 (\mathbf{x},\mathbf{y})$  defined in (1) to gain some insights. Because HSIC(x,y) characterizes the dependence between x and y, while HSIC(x,x) and HSIC(y,y) are the normalizing factors, we expand HSIC(x,y) and calculate the orders for HSIC(x,x) and HSIC(y,y).
We first calculate the orders for  $\mathrm{HSIC}(\mathbf{x},\mathbf{x})$  and  $\mathrm{HSIC}(\mathbf{y},\mathbf{y})$  at the population level as the dimensions diverge to infinity. The results are summarized in the following Proposition.
Proposition 1. Under Assumptions (A1) and (A2),  $\mathrm{HSIC}(\mathbf{z},\mathbf{z})\asymp d^{-\kappa_{\mathbf{z}}}ifd\to \infty$
According to Proposition 1, the condition  $n^{1/2}\mathrm{hCorr}^2(\mathbf{x},\mathbf{y}) \to \infty$ , which guarantees the HSIC based test would have power approaching 1 in high dimensions, will boil down to  $n^{1/2}p^{\kappa_{\mathbf{x}}/2}q^{\kappa_{\mathbf{y}}/2}\mathrm{HSIC}(\mathbf{x},\mathbf{y}) \to \infty$ .
Next, we expand  $\mathrm{HSIC}(\mathbf{x},\mathbf{y})$  at the population level. Before that, we introduce some additional notations. Similar to Shao & Zhang (2014), we define  $\mathrm{MD}^2 (\mathbf{x}\mid \mathbf{y})$  as