A Kernel-based Test of Independence for Cluster-correlated Data

1 Introduction . We are often interested in studying the dependence between two multivariate variables . For example , in genetic studies , we may want to assess the association between multiple genetic variants within a gene and a group of traits that likely share a common genetic mechanism [ 1 , 2 ] . In microbiome studies , we may wish to investigate the association between the overall composition of human microbiota , including hundreds of microbial taxa , and multiple host metabolites from a particular metabolic pathway [ 3 , 4 ] . Such multivariate analyses aggregate information across variables and are often more powerful than univariate analyses . Meanwhile , correlated observations arise in many practical situations . Family-based designs are common in genetic studies [ 5 ] , where multiple family members are recruited together into the study . Longitudinal data are common in epidemiological studies [ 6 ] , where variables of interest are measured on the subjects repeatedly over time . Such study designs introduce clustered dependence among the observations and require proper accommodation . In this work , we aim to develop an approach for assessing the dependence between two multivariate variables , based on cluster-correlated data . A variety of parametric and semi-parametric methods has been proposed to study the association between one or multiple exposure variables and a multivariate longitudinal ( or other cluster-correlated ) outcome . These methods often extend upon existing tools for univariate longitudinal data . For example , many studies stack the multivariate outcome into a single response vector and then apply the 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . usual methods that account for clustered data , such as generalized estimating equations ( GEE ) [ 7 , 8 ] and random effects models [ 9 , 10 ] . Such approaches generally apply to a low-dimensional setting [ 11 ] and are subject to limitations typical of parametric and semi-parametric methods . Randomeffect-based methods require assumptions on the distribution of the multivariate outcome . GEE-based methods rely on good estimation of the correlation structure within clusters as well as across different outcome variables to achieve a high efficiency . Finally , both approaches assume parametric ( often linear ) relationships between the exposure and the outcome . Therefore , they can only evaluate a limited number of dependence patterns , and are not sufficient as independence tests . Here we base our approach on the Hilbert-Schmidt Independence Criterion ( HSIC ) , a non-parametric kernel-based measure for assessing the generalized dependence between two multivariate , and potentially high-dimensional , variables [ 12 ] . By mapping the two variables into reproducing kernel Hilbert spaces ( RKHS ’ s ) , the population HSIC can be viewed as a measure of maximized covariance between functions in the two RKHS ’ s . When the RKHS ’ s being used are characteristic [ 13 ] , the population HSIC is zero if and only if the two variables are independent . This measure makes no assumption on the distributions of the variables or the nature of the dependence . The original HSIC-based independence test [ 14 ] applies to independent and identically distributed ( i.i.d . ) observations . Several extensions have been made to accommodate non-i.i.d . data , but none of the tests , to our knowledge , directly applies to clustered data at an observation level . Zhang et al . ( 2008 ) [ 15 ] extended the HSIC to certain sequence data , such as XOR sequence and Gaussian process , by specifying the correlation structure of the data as a graphical model and deriving the test statistic based on the maximal cliques . Chwialkowski et al . [ 16 ] and Wang et al . [ 17 ] developed HSIC-based tests to evaluate the dependence between two time series or random processes in general . Flaxman et al . [ 18 ] considered spatial and temporal data ; they proposed to first use Gaussian process regression to remove dependence on space and time from the raw variables , and then perform the HSIC test on the resulting de-correlated residuals . However , their approach generally applies to independence testing between two univariate variables . A study with an aim closest to ours is by Rudra et al . [ 19 ] : They analyzed the association between multiple genetic variants and a multivariate longitudinal outcome . Rudra et al . concatenated the outcome measurements from different time points at the subject level , and then applied the HSIC test to the subject-level data . Although this is a straightforward approach to deal with clustered correlation , there could be a loss of statistical power by analyzing data at the subject/cluster level rather than observation level . In this work , we present the first HSIC-based independence test for cluster-correlated data . Using the empirical HSIC [ 14 ] as our test statistic , we derive its asymptotic distribution under the null hypothesis of independence between the two variables but in the presence of clustered correlation among observations . We also examine the behavior of the test statistic under the alternative hypothesis and establish the consistency of our test . Furthermore , we provide a way to approximate the asymptotic null distribution of the test statistic and allow for statistical testing in practice . In simulation studies , our proposed test controls type I error rates well and has a much higher statistical power than competing methods across a range of scenarios . In an application to a longitudinal microbiomemetabolite data set , compared to other approaches , our proposed test identifies a larger number of metabolic pathways significantly associated with the overall microbiome composition , highlighting the value of our test in scientific studies . The remaining sections are organized as following . In Section 2 , we provide our background assumption on clustered data and an overview of the HSIC statistic . In Section 3 , we study the asymptotic behavior of the HSIC statistic under null and alternative hypotheses , and construct a statistical test of independence for cluster-correlated data . In Section 4 , we demonstrate the performance of our proposed test on both simulated and real data . In Section 5 , we summarize our work , discuss the limitations of our proposed test and provide a conclusion . 2 Background . In this section , we introduce our assumption on cluster-correlated data and give an overview of the HSIC statistic . 2.1 General setting . Let PXY be a probability measure defined on a sample space X × Y , where both X and Y can be multi-dimensional . Let PX and PY be the marginal distributions on X and Y , respectively . The variables X and Y are statistically independent if PXY = PXPY ( equivalently , we can write X ⊥⊥ Y ) . We consider a sample of clustered data { ( Xj , Yj ) } nj=1 drawn from PXY , where the pattern of clustered correlation is balanced and complete : Assumption 2.1 . The observations ( X1 , Y1 ) , · · · , ( Xn , Yn ) are identically distributed according to PXY , and can be divided into m clusters of fixed size d ( i.e. , n = md ) . In particular , the m clusters { [ ( Xdi−d+1 , Ydi−d+1 ) , · · · , ( Xdi , Ydi ) ] } m i=1 are independent from one another while having identical within-cluster correlation structure . The specific correlation structure among the observations in each cluster can be arbitrary . We are interested in studying the dependence between X and Y based on the sample { ( Xj , Yj ) } nj=1 . 2.2 Hilbert-Schmidt Independence Criterion . We briefly review the Hilbert-Schmidt Independence Criterion ( HSIC ) proposed by Gretton et al . ( 2005a ) [ 12 ] . The HSIC measures the generalized dependence between two variables X and Y , by embeddingX and Y into reproducing kernel Hilbert spaces ( RKHS ’ s ) and maximizing the covariance between functions of X and Y in the RKHS ’ s . Let HX be an RKHS on X with associated kernel function ( i.e. , inner product in the RKHS ) kX : X × X → R , and letHY be an RKHS on Y with associated kernel function kY : Y × Y → R. Following Gretton et al . ( 2007 ) [ 14 ] , the cross-covariance operator CXY : HY → HX can be defined such that , for any f ∈ HX and g ∈ HY , 〈f , CXY g〉HX = EXY [ ( f ( X ) − EX [ f ( X ) ] ) ( g ( Y ) − EY [ g ( Y ) ] ) ] = Cov ( f ( X ) , g ( Y ) ) . As shown by Gretton et al . ( 2005b ) [ 20 ] , the operator norm ( i.e. , the largest singular value ) of CXY , defined by ‖CXY ‖ : = supf∈HX , g∈HY , ‖f‖∞≤1 , ‖g‖∞≤1 Cov ( f ( X ) , g ( Y ) ) , is zero if and only if X ⊥⊥ Y , given that the kernels kX and kY are universal ( see Definition 5 of [ 20 ] ) . In this sense , ‖CXY ‖ is a measure of independence between X and Y . The largest singular value of CXY becomes zero when the sum of all squared singular values , denoted as the squared Hilbert-Schmidt norm [ 12 ] , is zero . Therefore , the squared Hilbert-Schmidt norm of CXY , ‖CXY ‖2HS , is also an independence criterion . This measure is defined as the population HSIC , which can be expressed conveniently in terms of kernel functions : HSIC ( PXY ) : = ‖CXY ‖2HS = EXX′Y Y ′ [ kX ( X , X ′ ) kY ( Y , Y ′ ) ] + EXX′ [ kX ( X , X ′ ) ] EY Y ′ [ kY ( Y , Y ′ ) ] − 2EXY [ EX′ [ kX ( X , X ′ ) ] EY ′ [ kY ( Y , Y ′ ) ] ] , where X ′ is an independent copy of X . It is obvious that , if X is independent from Y , then we have HSIC ( PXY ) = 0 . Furthermore , for certain characteristic kX and kY [ 13 ] , HSIC ( PXY ) = 0 if and only if X ⊥⊥ Y . Example characteristic kernels include Gaussian kernels and Laplacian kernels [ 21 ] . To estimate the population HSIC from a sample { ( Xj , Yj ) } nj=1 , the empirical HSIC can be used : HSIC ( Pn ) : = 1 n2 n∑ i , j kX ( Xi , Xj ) kY ( Yi , Yj ) + 1 n4 n∑ i , j , q , r kX ( Xi , Xj ) kY ( Yq , Yr ) − 2 n3 n∑ i , j , q kX ( Xi , Xj ) kY ( Yi , Yq ) . Define the kernel matrices KX and KY such that the ( i , j ) -th element of KX is kX ( Xi , Xj ) and the ( i , j ) -th element ofKY is kY ( Yi , Yj ) . Then the empirical HSIC can also be written in terms of KX andKY : HSIC ( Pn ) = 1 n2 tr ( HKXHKY ) , whereH = I − 1n11 T is a centering matrix . Both Gretton et al . ( 2007 ) [ 14 ] and Zhang et al . ( 2012 ) [ 22 ] have derived the asymptotic distribution of HSIC ( Pn ) under the null hypothesis of independence between X and Y as a weighted sum of chi-square variables , when the observations are i.i.d . In Section 3 , we examine the asymptotic behavior of HSIC ( Pn ) based on cluster-correlated observations . It turns out that the null distribution in this case is still a weighted sum of chi-square variables , where the weights are now modified .

Summary: This submission introduces a calibrated testing procedure for the independence between two variables under non-iid sampling. The sampling is assumed to follow a clustered structure, where samples within clusters are correlated but the clusters are independent from each others. The procedure relies on the Hilbert-Schmidt Independence Criterion (HSIC), and a first contribution is an asymptotic null distribution under the clustered sampling assumption. The test is also shown to be consistent. Experiments on simulated and real data show that the proposed test guarantees a control on the type I error, and has higher power than existing alternatives in a variety of scenarios.