A fast score test for generalized mixture models

In biomedical studies, parametric mixture models are widely used for modeling data collected from a non-homogenous population. For instance, in genetic linkage analysis, parametric admixture models have been considered to account for the fact that linkage may exist only in a proportion of families (Liang and Rathouz, 1999; Fu et al., 2006; Di and Liang, 2011). In microarray data analysis, mixture models have been applied to test for partially differentially expressed genes (Ghosh and Chinnaiyan, 2009; Van Wieringen and Van de Viel, 2009). Generally, testing for homogeneity in mixture models is a nonregular problem because the mixture proportion parameter lies on the boundary of its parameter space and some nuisance parameters are absent under the null hypothesis of homogeneity. Therefore, theoretical development for the asymptotic distributions of tests is more difficult than that in regular statistical models. Much work has been done for testing homogeneity in parametric mixture models. For example, a score test is proposed by Liang and Rathouz (1999), and (modified) likelihood ratio tests are derived by Lemdani and Pons (1999), Chen et al. (2001) and Di and Liang (2011), among many others. For more comprehensive overview of parametric mixture models, see McLachlan and Basford (1988) and Lindsay (1995). However, in many modern applications such as the DNA methylation data analysis, the distributions of methylation values are often highly heterogeneous across different CpG sites (Wang, 2011), such that one single parametric mixture model cannot provide sufficient goodness of fit for the methylation data measured at different CpG sites. To relax the parametric assumption, Qin and Liang (2011) proposed a two-sample semiparametric exponential tilt mixture model (ETMM), where the distribution of the reference group is completely unspecified and the distribution of the other group is a mixture of two unknown distributions linked by an exponential tilt with a single parameter. Such an ETMM contains the mixture of natural exponential family (NEF) models (Morris, 2006). However, this ETMM still does not provide sufficient degrees of freedom in modeling, because it only allows one single free parameter which reduces to the non-homogeneity of means in normal models. In applications such as DNA methylation data, the two experimental groups can be different in variations but not in means, or different in both means and variations, and such difference in variation may suggest important associations between the CpG sites and diseases (Teschendorff et al., 2014; Sun et al., 2017). In such circumstances, existing methods are not sufficient to capture these key features of data and can lead to a substantial loss of power. Hong et al. (2017) proposed a pseudolikelihood EM based test (PLEMT) using a multi-parameter generalized exponential tilt mixture model (GETMM). The GETMM generalizes the ETMM by incorporating pre-specified multidimensional basis functions, which offers more flexible characterization of the non-homogeneity between two groups. Unlike the empirical likelihood method in Qin and Liang (2011), a pairwise conditioning procedure was used to eliminate the unspecified distribution function, and a penalized pseudolikelihood was constructed to handle the identifiability problem. Despite the superior power of the PLEMT over existing tests in capturing the signals in mean and variations, there are several limitations. First, it involves choosing multiple tuning parameters including the penalty parameter and number of steps for the EM algorithm. There is no universally accepted values for the tuning parameters. Second, this method is computationally expensive since EM algorithm is used to obtain the test statistic. In this paper, we propose a novel score test for homogeneity based on GETMM. We adopt the same procedure as in Hong et al. (2017) in elimination of the unspecified distribution function of the reference group. Instead of using penalization, we work on the pseudolikelihood directly and show that the parameters of interest are still identifiable. To address the challenge that some nuisance parameters disappear under the null hypothesis, we modify the score function by a novel procedure and derive the asymptotic null distribution of the proposed score test. Unlike the standard asymptotic results in Davies (1977, 1987), we unveil new Wilks phenomenon of the proposed score test, i.e., asymptotic null distributions are independent of nuisance parameters. The simple asymptotic distribution controls the Type I error well, and avoids the need of permutation based procedures. In addition, we show that, with fixed nuisance parameters, the proposed score test is locally most powerful among a class of tests. This justifies the optimality of our method in finite samples. To obtain further insight about the power of the proposed test, we derive the asymptotic power functions under local alternative hypotheses. Two types of local alternatives are studied, with respect to two different geometric paths for approximating the null hypothesis. This result is particularly useful in performing sample size calculation and experimental design. The rest of this article is organized as follows. In Section 2, we introduce the GETMM and construct the pseudolikelihood. In Section 3, we derive the score test based on the constructed pseudolikelihood and derive the asymptotic null distribution. In Section 4, we derive the local asymptotic power of the proposed tests and provide insights on experimental designs. In Section 5, we conduct simulation studies to compare the proposed tests with the existing ones. In Section 6, we apply these tests to the methylation data from an ovarian cancer study. Concluding remarks are given in Section 7.