A Shrinkage Likelihood Ratio Test for
High-Dimensional Subgroup Analysis with a
Logistic-Normal Mixture Model
Shota Takeishi∗
Risk Analysis Research Center
The Institute of Statistical Mathematics
shotakeishi2@gmail.com
January 9, 2024

Abstract
In subgroup analysis, testing the existence of a subgroup with a diﬀerential treatment eﬀect
serves as protection against spurious subgroup discovery. Despite its importance, this hypothesis
testing possesses a complicated nature: parameter characterizing subgroup classiﬁcation is not
identiﬁed under the null hypothesis of no subgroup. Due to this irregularity, the existing methods
have the following two limitations. First, the asymptotic null distribution of test statistics often
takes an intractable form, which necessitates computationally demanding resampling methods tocalculate the critical value. Second, the dimension of personal attributes characterizing subgroupmembership is not allowed to be of high dimension. To solve these two problems simultaneously,
this study develops a novel shrinkage likelihood ratio test for the existence of a subgroup using
a logistic-normal mixture model. The proposed test statistics are built on a modiﬁed likelihood
function that shrinks possibly high-dimensional unidentiﬁed parameters toward zero under thenull hypothesis while retaining power under the alternative. This shrinkage helps handle theirregularity and restore the simple chi-square-type asymptotics even under the high-dimensional
regime.

1

Introduction

Subgroup analysis is routinely conducted in clinical trials that aim to account for patients’ heterogeneous responses to treatment (Wang et al., 2007). By exploring the interaction between
treatment eﬀect and patients’ characteristics, subgroup analysis searches for a subgroup with certain attributes who have a more beneﬁcial or adverse treatment eﬀect compared to the rest of∗

I would like to thank Katsumi Shimotsu for helpful suggestions and sharing his computational resources with me,
as well as Xuming He for pointing me to the relevant literature. I also appreciate comments from participants of theweekly Bayesian seminar at the Department of Statistics, the Graduate School of Economics, the University of Tokyo.
This research was supported by JSPS KAKENHI Grant Number JP22J12024 and in part through computational
resources and services provided by Advanced Research Computing at the University of Michigan, Ann Arbor.

1the population. Despite its widespread usage, one possible concern is that the treatment eﬀect is
actually homogeneous so that the detected subgroup is spurious.
To prevent such false subgroup discovery, this study proposes a novel testing method for the
existence of a subgroup that is computationally eﬃcient and scales with high-dimensional patients’characteristics. Following Shen and He (2015) and Shen et al. (2017), our hypothesis testing is basedon a logistic-normal mixture model. This is a type of normal mixture regression model in which
the means for diﬀerent Gaussian components express distinctive treatment-outcome relationships
and the mixing proportion varies as a logistic function of covariates. These covariates and theirassociated parameters pertain to subgroup classiﬁcation and are thus called classiﬁcation covariates
and classiﬁcation parameters, respectively. With this model, hypothesis testing for the existence ofa subgroup reduces to testing the number of components. Speciﬁcally, the null hypothesis of one
component suggests that no subgroup characterized by the classiﬁcation covariates exists, and thealternative hypothesis of two components indicates otherwise.
Numerous model-based methods have been developed for testing the existence of a subgroupwith a diﬀerential treatment eﬀect. Within the framework of the logistic-normal mixture model, apioneering work by Shen and He (2015) considers an EM-test for subgroup existence while Shenet al. (2017) extend their approach to unequal variance cases. For survival data, Wu et al. (2016)
consider the logistic-Cox mixture model and develop an EM-test for the existence of a subgroup.
In contrast to those mixture-based modelings, Fan et al. (2017) deal with a structurally similarchange-plane model where two regression functions switch according to single-index thresholding
characterized by covariates and parameters. They then propose a score-type test for the existenceof a subgroup. This change-plane-based approach has been adapted to many other contexts: Kang
et al. (2017) for survival data and Huang et al. (2021) for binary response data.
Hypothesis testing for the existence of a subgroup possesses an irregular structure: the classiﬁcation parameter is not identiﬁed under the null hypothesis of no subgroup. Due to the presenceof unidentiﬁed parameters, the aforementioned works have the following two limitations. First,
the asymptotic null distribution of test statistics takes complicated forms. In fact, the asymptotic
null distribution derived in Fan et al. (2017) is a functional of stochastic processes indexed byunidentiﬁed parameters. The intractability of the limiting distribution necessitates computationally demanding resampling methods to calculate the critical value. The dependence of the null
distribution on the unidentiﬁed parameters further gives rise to the second limitation. Namely,
the asymptotic null distribution of test statistics is even not well deﬁned when the dimension ofclassiﬁcation covariates, and accordingly, the classiﬁcation parameter increases with sample size.
Considering the growing availability of high-dimensional personal characteristics such as biomarkers and genetic information, this restriction can be a hurdle to the practical use of testing for the
existence of a subgroup.
To address these challenges, this study develops a novel testing procedure. Our test statistics
are based on the likelihood ratio as in Shen and He (2015) and Shen et al. (2017). However, instead of naively estimating the model parameter, we propose to estimate the parameter under the2alternative model with a modiﬁed likelihood function that penalizes L1 -norm of the classiﬁcation
parameter. When the null hypothesis is true, this penalization strongly shrinks the unidentiﬁed
classiﬁcation parameters toward zero. Owing to this shrinkage, the asymptotic null distribution of
the resulting shrinkage likelihood ratio test statistics (SLRT ) follows the half chi-square distribution, whose quantile is easy to calculate. This asymptotic result holds even when the dimension of
the classiﬁcation covaraites and parameters increases with sample size under some rate conditions.
Meanwhile, the shrinkage eﬀect is not as strong when the alternative hypothesis is true. Hence,the proposed test is more powerful than the test that ﬁxes the classiﬁcation parameter to zero inadvance, as conﬁrmed in our simulation study.
Besides tackling the computational burden and the high dimensionality associated with testing
the existence of a subgroup, this study further connects to the following strands of the literature.
First, in testing the number of components in ﬁnite mixture models, several works employ penalized
likelihood to realize simpliﬁed asymptotic null distributions. See, for example, Chen et al. (2001)
and Li et al. (2009) for non-normal mixture models; Chen and Li (2009) for normal mixture models;and Kasahara et al. (2014) for Markov regime-switching models. Although those predecessors
and the present study share the same spirit of using penalization, the penalized parameters aresubstantially diﬀerent; the former penalize a one-dimensional mixing proportion while the latter
penalizes possibly high-dimensional parameters that can be associated with covariates. Especially,
dealing with the high dimensionality requires a distinct proof strategy for deriving the asymptotic
distribution.
Beyond the context of ﬁnite mixture models, hypothesis testing with unidentiﬁed parameters
under the null hypothesis for more general settings has been a long-standing interest in statistics
and econometrics. A partial list of the examples includes Davies (1977), Davies (1987), Andrews
and Ploberger (1994), Hansen (1996), Song et al. (2009) and Andrews and Cheng (2012). This
literature, however, does not cover the case where the unidentiﬁed parameters under the null
hypothesis are high-dimensional. Furthermore, our shrinkage approach is a novel way to treat sucha non-identiﬁcation problem in hypothesis testing. A recent preprint by Yoshida and Yoshida (2023)
considers penalized quasi-maximum likelihood estimation with part of parameter unidentiﬁed andproposes to use the Lq -norm penalty (0 < q ≤ 1) to stabilize the asymptotic behavior. However,hypothesis testing and the high-dimensional unidentiﬁed parameter are beyond the scope of theirresearch.
We note that Wang (2016) develops a information-criterion-based method for selecting thenumber of components in the logistic-mixture model with high-dimensional covariates. As Chenet al. (2012) point out, however, such a model selection procedure and hypothesis testing often
serve diﬀerent purposes. The former is expected to ﬁnd the simplest model consistent with theobserved data, while the latter is useful to check the validity of scientiﬁc propositions, for example,
the (non)existence of a subgroup in our context.
The rest of the paper is organized as follows. Section 2 introduces the formal model setup,
the proposed testing methodology, and the notation. Section 3 then investigates the theoretical3

properties of the proposed test statistics after providing the required assumptions. In particular, weestablish the asymptotic distribution of the test statistics under the null hypothesis of no subgroup.
Section 4 illustrates the ﬁnite sample performance of the proposed method through Monte Carlosimulations. We also discuss the choice of a tuning parameter. Subsequently, in Section 5, weanalyze real-world data with the proposed test. Section 6 concludes the article. All the proofsof the propositions in the main text are relegated to Appendix A, while Appendix B collects theauxiliary results and their proofs.

2

Methodology

Let {(Yi , Xi , Di , Zi , εi , δi )}ni=1 be i.i.d. copies of sample size n deﬁned on some underlying probability
space (Ω, F, P) with the following logistic-normal mixture model:
Yi = Xi′ α + Di (β + δi λ) + εi ,P(δi = 1|Xi , Zi ) = exp(Zi′ γ)/(1 + exp(Zi′ γ)),P(δi = 0|Xi , Zi ) = 1 − P(δi = 1|Xi , Zi ),(1)

where Yi ∈ R is the outcome of interest, Di ∈ R is a treatment variable, Xi ∈ Rq is other confounding
variables, and εi is an independent error term that follows the normal distribution with mean
zero and variance σ 2 . Furthermore, δi ∈ {0, 1} is a latent subgroup membership indicator, andZi ∈ Rdn is possibly high-dimensional classiﬁcation covariates that may be predictive of subgroupmembership. Among the unknown parameters (α, β, λ, γ, σ 2 ), β expresses treatment eﬀect common
to the entire population, λ is an additional treatment eﬀect speciﬁc to a subgroup and γ is theclassiﬁcation parameter that governs how Z inﬂuences the subgroup classiﬁcation. Based on thismodel, we perform the following hypothesis test based on the observable {(Yi , Xi , Di , Zi )}ni=1 :H0 : λ = 0 against Ha : λ > 0,
where the positivity of λ under the alternative hypothesis reﬂects an identiﬁability issue in thelogistic-normal mixture model (Jiang and Tanner, 1999). In this formulation, the null hypothesis
indicates that there exists no subgroup characterized by Z.
As in Shen and He (2015), our likelihood ratio-based test starts with the following conditional
density function of Yi given Wi := (Xi′ , Di , Zi′ )′ :f (Yi |Wi ; θ, γ) := π(Zi′ γ)ϕσ (Yi − Xi′ α − Di (β + λ)) + (1 − π(Zi′ γ))ϕσ (Yi − Xi′ α − Di β),where θ collects the parameter (α, β, λ, σ 2 ) except for γ, π(x) := exp(x)/(1 + exp(x)) and ϕσ isa density function of the normal distribution with mean 0 and variance σ 2 . Letting ln (θ, γ) :=Pn
i=1 log f (Yi |Wi ; θ, γ), the likelihood ratio test statistics take the form 2(ln (θ̂, γ̂) − ln (θ̂0 )), where4(θ̂, γ̂) is the MLE under the full logistic-normal mixture model while θ̂0 denote the MLE under thenull model with the restriction λ = 0 (so that γ is dropped for brevity).
The standard chi-square-type limit theory is expected to break down for the likelihood ratio testin the logistic-normal mixture model. To illustrate this point intuitively, the non-identiﬁability ofγ keeps γ̂ from having any clear probability limit under the null hypothesis, λ = 0. This non-limit
property implies that γ̂ freely moves across the whole parameter space even asymptotically, whichshould translate into the complex asymptotic null distribution characterized by the parameter spaceof γ as in Fan et al. (2017). Although this argument does not directly apply to the EM-test ofShen and He (2015), their test is also not free from the intractable asymptotic null distribution.
Furthermore, the adaptability of their method to high-dimensional Z is not known.
For a simple chi-square-type asymptotic distribution and high-dimensional adaptability, ourproposal aims to ﬁx the aforementioned non-limit issue of γ̂. To introduce our idea, we deﬁne apenalized log-likelihood function:ln∗ (θ, γ) :=nXlog f (Yi |Wi ; θ, γ) − pn ∥γ∥1 ,

i=1

Pd

j=1 |γj | is a L1 -norm and pn is a tuning parameter such that pn /n goes to zero
as n → ∞. We then use a penalized estimator (θ̂∗ , γ̂ ∗ ) := argmaxθ∈Θ,γ∈Γ ln∗ (θ, γ) in place of (θ̂, γ̂)where ∥γ∥1 :=for likelihood ratio test statistics where Θ and Γ are parameter spaces for θ and γ, respectively.
Hence, our proposed shrinkage likelihood ratio test statistics (SLRT ) are deﬁned as
SLRT := 2(ln (θ̂∗ , γ̂ ∗ ) − ln (θ̂0 )).
The idea of penalizing γ is inspired by the following insight. Under the null hypothesis, λ = 0,an estimate of λ is expected to be close to zero asymptotically. In this case, variation of γ has little
eﬀect on ln (θ, γ). Then, the eﬀect of γ on ln∗ (θ, γ) is more through −pn ∥γ∥1 than through ln (θ, γ),
which strongly shrinks γ to zero. This shrinkage of γ toward zero solves the non-limit problem ofγ. In contrast, under the alternative λ > 0, an estimate of λ should be bounded away from zero
when the sample size is large. The eﬀect of the variation of γ on ln (θ, γ) is nontrivial this time.
Combining this with the fact that pn /n is set to be asymptotically negligible, the eﬀect of γ onl∗ (θ, γ) is more through ln (θ, γ) so that the shrinkage eﬀect of γ to zero is not as strong. Thisavoids substantial power loss of the test.
In the remainder of the paper, we use the following notation. Collect the covariate as Ui =(Xi′ , Di )′ .

We suppress the dependence of dn and pn on n and just write d and p. Let := denote“equals by deﬁnition.” For a ∈ Rk , ∥a∥r denotes the Lr -norm of a in Euclidean space. In particular,
we suppress r and just write ∥a∥ when referring to the L2 -norm. For a := (a1 , . . . , ak )′ ∈ Rk anda real-valued function g(a), let ∇a g(a∗ ) := (∂g(a∗ )/∂a1 , . . . , ∂g(a∗ )/∂a2 )′ be a vector of derivativeevaluated at a = a∗ . The subscript 0 as in θ0 signiﬁes the true parameter value. For two real
numbers a and b, a ∧ b and a ∨ b denote min(a, b) and max(a, b), respectively. Let C be a universal
5ﬁnite positive constant whose value may change from one expression to another. For two real
sequences {an }n∈N and {bn }n∈N , the notation an ≲ bn means that there exists a ﬁnite constant D
independent of n such that an ≤ Dbn for all n ∈ N. All the limits below are taken as n → ∞ unlessstated otherwise. Throughout the article, we assume n ∧ d ≥ 2.

3

TheoryThe goal of this section is to establish the asymptotic null distribution of SLRT , which is crucial
for the implementation of the test. After setting the required assumptions, we ﬁrst show theconsistency of θ̂∗ and the convergence rate of ∥γ̂ ∗ ∥1 to zero (Proposition 1). We then introduce
the reparameterization of θ to deal with the non-regularity inherent in the logistic-normal mixture
model. Built on the reparameterization, we establish the quadratic approximation for the penalized
log-likelihood function and derive the convergence rate of the reparameterized estimator and the2asymptotic null distribution of SLRT . In the following, let Θ := Θα × Θβ × Θλ × Θσ be theparameter space for θ = (α′ , β, λ, σ 2 )′ , and Z := (Z(1) , · · · , Z(d) )′ . Throughout the section, we
assume that the null hypothesis holds: λ0 = 0.

3.1

Assumptions

In addition to the basic model setup, the following set of assumptions is required for the subsequent
theoretical results.
2

Assumption 1. (a) Θα and Θβ are compact, convex sets, (b) Θσ = (0, ∞), (c) Θλ = [0, uλ ] forsome 0 < uλ < ∞, (d) Γ = Rd and (e) (α0′ , β0 )′ lies in an interior of Θα × Θβ .
Assumption 2. (a) E[∥U ∥10 ] < ∞, (b) E[U U ′ ] is positive deﬁnite, (c) Z is uniformly subGaussian: there exist ﬁnite K, C > 0, independent of n and j, such that P(|Z(j) | > t) ≤ Ke−Ct

2

for all 0 < t < ∞ and 1 ≤ j ≤ d, and (d) D is bounded and nondegenerate, i.e., V ar(D) > 0.
√
Assumption 3. n, d and p satisfy the following rate condition: (a) n7/4 log n log d/p2 = o(1) and(b) log d = o(n1/4 ).
Compactness in Assumption 1(a) and (c) is required for the proof of consistency (Proposition1), which extends that of Lemma A1 of Andrews (1993) to our context. As noted by Assumption
1(b) and (d), the parameter spaces for σ 2 and γ are unrestricted. Still, Lemma 1 suggests thatthose parameter spaces can essentially be regarded as compact with probability approaching one.
Assumption 2(a) is set because we expand the log-likelihood function ﬁve times and the tenth-orderterms of U appear when establishing the quadratic approximation for the penalized log-likelihood
function (Proposition 2). Similar higher-order moment conditions are employed when higher-order
expansion of the log-likelihood function is necessary, as in Assumption 2(a) of Kasahara and Shimotsu (2015). Positive deﬁniteness of E[U U ′ ] in Assumption 2(b) is standard in hypothesis testingfor ﬁnite mixture models and can be seen in Theorem 2 of Shen and He (2015) and Assumption
6

2(b) of Kasahara and Shimotsu (2015). We, however, do not require positive deﬁniteness of E[ZZ ′ ],which is a major departure from Shen and He (2015) (see Theorem 2 therein). Sub-Gaussianity
in Assumption 2(c) is for the sake of repeated use of Lemma 2.2.1 and 2.2.2 of van der Vaart andWellner (1996) to deal with the high-dimensionality of Z. For Assumption 2(d), the boundedness is
a technical requirement for the proof of Lemma 6 while the nondegeneracy avoids the complication
associated with the quadratic approximation, as discussed in the paragraph following Proposition
2. This assumption should be satisﬁed in most of our intended applications where D denotes atreatment variable in clinical trials. Note that Fan et al. (2017) also consider bounded and nondegenerate D in their setting for subgroup analysis. Assumption 3(b) indicates that d can grow much
faster than n. However, given Assumption 3(a), larger d requires larger p, which leads to stronger
shrinkage and thus power loss of the test.

3.2

Asymptotic Null Distribution of SLRTBased on the assumptions, we start by showing the consistency of θ̂∗ and the convergence rate of∥γ̂ ∗ ∥1 to zero.
Proposition 1. Assume Assumptions 1-3 hold. Then θ̂∗ →p θ0 and ∥γ̂ ∗ ∥1 = op (n−1/4 (log d log n)−1/2 ).
Note that the choice of the convergence rate of ∥γ̂ ∗ ∥1 , n−1/4 (log d log n)−1/2 , is for the sake of theproof of Proposition 2 and not essential in itself. The proof is built on that of Lemma A1 of Andrews(1993), a consistency result when some parameters are not identiﬁed. We make modiﬁcations so thatthe unidentiﬁed parameter can be of high-dimension and the L1 norm of the unidentiﬁed parameter
converges to zero in probability due to the L1 penalty. The key instrument for handling the highdimensionality is a judicious use of the multivariate contraction principle (Lemma 3) that leverages
the contraction property of the multivariate function (x1 , x2 , x3 ) → log(π(x1 )e−x2 +(1−π(x1 ))e−x3 )appearing in the log-density.
We proceed to analyze the asymptotic properties of SLRT . As λ0 = 0 is on the boundary of Θλunder the null hypothesis, we employ the method of Andrews (1999) for quadratic approximationof the penalized log-likelihood function with the score ∇θ log f (Y |W ; θ0 , γ̂ ∗ ). We, however, notethat the standard analysis is hampered by the irregular structure of the score:∂D(Y − X ′ α0 − Dβ0 ),log f (Y |W ; θ0 , γ̂ ∗ ) =∂βσ02∂
D(Y − X ′ α0 − Dβ0 ).
log f (Y |W ; θ0 , γ̂ ∗ ) = π(Z ′ γ̂ ∗ )
∂λ
σ02

(2)

There are two aspects to this irregularity. First, (∂/∂λ) log f (Y |W ; θ0 , γ̂ ∗ ) depends on γ̂ ∗ , thedimension of which possibly increases with the sample size n. However, this problem can be solved
by using the convergence ∥γ̂ ∗ ∥1 = op (n−1/4 (log d log n)−1/2 ) in Proposition 1. Speciﬁcally, Lemma
6 clariﬁes that π(Z ′ γ̂ ∗ ) can be approximated by π(0) asymptotically, which enables us to treat(∂/∂λ) log f (Y |W ; θ0 , γ̂ ∗ ) essentially as D(Y −X ′ α0 −Dβ0 )/2σ02 . Unfortunately, this approximation7

leads to the second aspect of the irregularity: (∂/∂β) log f (Y |W ; θ0 , γ̂ ∗ ) and D(Y −X ′ α0 −Dβ0 )/2σ02
are linearly dependent. This linear dependence degenerates the Fisher information matrix so thatthe standard second-order quadratic approximation is no longer valid.
We overcome this challenge by considering reparameterization inspired by Kasahara and Shimotsu (2015), which are built on the result of Rotnitzky et al. (2000). Let us introduce thefollowing one-to-one mapping between the original parameter (α′ , β, σ 2 , λ)′ and the reparameterized one (α′ , ν, σ 2 , λ)′ with β = ν − λ/2. Collect the reparameterized parameter as ψ := (η ′ , σ 2 , λ)′where η := (α′ , ν)′ . Accordingly, let Θψ := {(α′ , β + λ/2, σ 2 , λ)′ : (α′ , β, λ, σ 2 ) ∈ Θ} denote thereparameterized parameter space. With the abuse of notation, the reparameterized density, thelog-likelihood function, and the penalized log-likelihood function are given byf (Y |W ; ψ, γ) := π(Z ′ γ)ϕσ (Y − X ′ α − D(ν + λ/2)) + (1 − π(Z ′ γ))ϕσ (Y − X ′ α − D(ν − λ/2)),ln (ψ, γ) :=Pn∗
i=1 log f (Yi |Wi ; ψ, γ) and ln (ψ, γ) := ln (ψ, γ) − p∥γ∥1 . With this reparameterization,

the original score structure (2) can be transformed into∂D(Y − X ′ α0 − Dν0 )log f (Y |W ; ψ0 , γ̂ ∗ ) =∂νσ02
D(Y − X ′ α0 − Dν0 )∂log f (Y |W ; ψ0 , γ̂ ∗ ) = (2π(Z ′ γ̂ ∗ ) − 1)
∂λ
2σ02
2

 
∂2
D2 (Y − X ′ α0 − Dν0 )2∂∗∗
log f (Y |W ; ψ0 , γ̂ ) = 2
log f (Y |W ; ψ0 , γ̂ ) .
−1 −
∂λ2
∂λ
4σ0
σ02
As suggested by Lemma 6, the following approximation holds for the second and the third line:∂
log f (Y |W ; ψ0 , γ̂ ∗ ) ≈ 0,
∂λ


∂2
D2 (Y − X ′ α0 − Dν0 )2∗
log f (Y |W ; ψ0 , γ̂ ) ≈ 2
−1 .
∂λ2
4σ0
σ02

(3)

As (∂/∂ν) log f (Y |W ; ψ0 , γ̂ ∗ ) and the approximation for (∂/∂λ2 ) log f (Y |W ; ψ0 , γ̂ ∗ ) in (3) are linearly independent, the derivative of the penalized log-likelihood function with respect to α, ν, σ 2and λ2 can play the role of the scores for the quadratic approximation under this parameterization.
Namely, let

Sn := n−1/2



Pn







Ui 1
σ0 Hi




1
H 2
tn (ψ) := n1/2 (σ 2 − σ02 ) , si := 
 2σ02 i  ,Di
n1/2 λ2H2
8σ02 ii=1 si , In := n−1n1/2 (η − η0 )Pn′ki=1 si si and Hi:= H k (εi /σ0 ) for k ∈ N where H k (z) is the

Hermite polynomial of order k given by, for example, H 1 (z) = z and H 2 (z) = z 2 − 1. Then,the following proposition formally establishes the quadratic approximation for the penalized loglikelihood function.
8

Proposition 2. Assume Assumptions 1-3 hold. Then (a) ln∗ (ψ, γ̂ ∗ ) − ln∗ (ψ0 , γ̂ ∗ ) = Sn′ tn (ψ) −
1
′
∗
∗
2
2 tn (ψ) In tn (ψ) + Rn (ψ, γ̂ ), where supψ∈{ψ∈Θψ :∥ψ−ψ0 ∥≤κ} |Rn (ψ, γ̂ )|/(1 + ∥tn (ψ)∥) →p 0 for any

sequence κ converging to zero, (b) Sn →d N (0, I), and (c) In →p I, where I is nonsingular.
Proposition 2 of Kasahara and Shimotsu (2015) obtains a similar quadratic approximation result
under reparameterization for their analysis of normal mixture regression models. Our proposition
2, however, departs from that work in two respects. First, the setting of Kasahara and Shimotsu(2015) accommodates heterogeneous intercept terms across diﬀerent mixture components, whichcorresponds to the case where D equals unity in our context. Such heterogeneity is yet anothersource of singularity of the Fisher information matrix: the ﬁrst and second derivatives of the logdensity with respect to variance and an intercept term, respectively, are linearly dependent. Due tothis complication, Kasahara and Shimotsu (2015) employ a more involved reparameterization thanours. Second, our proof faces a new challenge of the presence of possibly high-dimensional γ̂ ∗ andhandle the problem in a novel approach. Namely, we show that the eﬀect of γ̂ ∗ on the quadratic
approximation vanishes asymptotically (Lemma 6) with the help of Proposition 1. Consequently,
the resulting quadratic approximation in Proposition 2(a) is the same as if γ̂ ∗ is ﬁxed to zero up
to the remainder term.
Deﬁne the reparameterized version of the penalized MLE as (ψ̂ ∗ , γ̂ ∗ ) := argmaxψ∈Θψ ,γ∈Γ ln∗ (ψ, γ).
Then, based on Proposition 2, the following proposition derives the asymptotic null distributionof SLRT . Note that χ21 /2 + χ20 /2 denotes the half chi-square distribution, which is a mixture of
the chi-square distribution with one degree of freedom and a point mass at zero with equal mixing
weights.
Proposition 3. Assume Assumptions 1-3 hold. Then (a) tn (ψ̂ ∗ ) = Op (1) and (b) SLRT →dχ21 /2 + χ20 /2.
According to the proof, the limiting null distribution, χ21 /2 + χ20 /2, is the same as that for the
likelihood ratio test statistics with γ ﬁxed to zero. Moreover, the diﬀerence between the lattertest statistics and SLRT converges to zero in probability. The half chi-square distribution often
appears in likelihood-ratio-based tests for the number of components in ﬁnite mixture models inwhich penalization is imposed on one-dimensional mixing proportion independent of covariate (e.g.,
Chen et al., 2001; Li et al., 2009). In this respect, our result witnesses the generalizability of thepenalization approach in the literature to covariate-dependent and high-dimensional settings.

4

Monte Carlo Simulation

In this section, we ﬁrst discuss the criteria for the choice of tuning parameter p. We then conduct
a simulation study to evaluate the ﬁnite sample performance of the test under several settings.
We compute (θ̂∗ , γ̂ ∗ ) via a version of EM algorithm (Jordan and Jacobs, 1994) where the standard weighted logistic regression in the M step is replaced by its L1 -penalized counterpart. The

9monotonicity of the algorithm can be easily shown. All the simulations are conducted in R language (R Core Team, 2022). The simulation codes are available at the author’s GitHub repository(https://github.com/stakeish/shrinkage_subgroup).

4.1

Empirical Formula for p

While Assumption 3 indicates the order of p relative to n and d, this assumption per se does not provide the exact value of p the practitioners should choose. Inspired by Chen et al. (2012), we derive
the empirical formula determining the speciﬁc value of p given n and d through numerical experiments. The construction of the formula proceeds as follows. For each n ∈ {100, 250, 500, 750, 1000}
and d ∈ {10, 25, 50, 75, 100}, we generate 2000 datasets of i.i.d. random variables {Yi , Xi , Di , Zi }ni=1 ,with a random seed set to 10, from the following null distribution: X ∼ N (0, 1), D ∼ Bernoulli(0.5),
Z(1) = 1, (Z(2) , . . . , Z(d) )′ ∼ N (0, I), Y ∼ N (1 + 2X + D, 1), where I is an identity matrix and
X, D and Z are independent of each other. Let P be a candidate set for p. For each n, we ﬁrst
calculate the rejection frequency of the likelihood ratio test with γ ﬁxed to zero (the benchmark
rejection frequency); then, for each d and for all p in P, we compute rejection frequencies of theshrinkage likelihood ratio tests. We set the level of the tests to 5% and use χ21 /2 + χ20 /2 to calculate
the critical value. Now, for each n and d, we deﬁne pn,d as the smallest value in P with which
the rejection frequency of the shrinkage likelihood ratio test falls within 0.3% from the benchmark
rejection frequency. With 25 observations of (pn,d , n, d) at hand, consider the regression model:√
pn,d = a + bn7/8 log d, motivated by Assumption 3(a). We obtain an estimate of (a, b)′ simply by
the method of least squares, which results in the following empirical formula for p.
p = 6.3383 + 0.0086n7/8

p
log d

(4)

We use the likelihood ratio test statistics with γ ﬁxed to zero as the benchmark because SLRT
approach these statistics asymptotically under the null as the argument following Proposition 3
discusses. Even though we set Z to the standard normal variable, the empirical formula is robust
to deviation from this speciﬁc distribution, as clariﬁed later in this section.

4.2

Size and PowerWe move on to assess the ﬁnite sample properties of the proposed methods. A random seed is set
to 20 for each n, d, and parameter value. We use the empirical formula (4) to determine p. The
level is set to 5% throughout. We ﬁrst examine the size property of the test. Consider the four nullsettings: X ∼ N (0, 1), D ∼ Bernoulli(0.5), Z(1) = 1, Y ∼ N (1 + 2X + D, 1) andSetting I: (Z(2) , . . . , Z(d) )′ ∼ N (0, I),

Setting II: (Z(2) , . . . , Z(d) )′ ∼ N (0, Σ)

Setting III: (Z(2) , . . . , Z(d) )′ ∼ i.i.d. Rademacher, Setting IV: (Z(2) , . . . , Z(d) )′ ∼ i.i.d. Skew normal,

10where Σ in Setting II is a randomly generated positive deﬁnite matrix based on Cholesky decomposition. Furthermore, “Rademacher” in Setting III indicates a Rademacher variable, which takes its
value at −1 or 1 with equal probability and “Skew normal” in Setting IV means the skew normal
distribution with the shape parameter set to 4 and the other parameters speciﬁed to ensure thatthe distribution has mean zero and variance one. We benchmark the shrinkage likelihood ratio
tests in these four settings against the likelihood ratio tests with γ ﬁxed to zero. Table 1 showsthat the proposed method works reasonably well in preserving the nominal level, especially when
the sample size is no smaller than 500. The proposed method also generally attains type I errors
close to those of the benchmark. Notably, the deviation of Z from the standard normality has
little eﬀect on the performance, which can be supporting evidence for the generalizability of theempirical formula (4).
Table 1: Type I Errorn = 100n = 250n = 500n = 750n = 1000B(I)0.06400.05280.05200.04880.0548S(I)0.0646
0.0552
0.05620.05180.0572

B(II)0.06400.05280.05200.04880.0548n = 100n = 250n = 500n = 750n = 1000B(I)0.06640.05820.0506
0.0546
0.0480S(I)0.06880.06480.0566
0.05780.0492B(II)0.06640.05380.05000.05040.0480n = 100n = 250n = 500n = 750n = 1000B(I)0.05600.0576
0.05340.05200.0468S(I)0.05920.06580.0636
0.05700.0490B(II)0.05840.05580.04740.0496
0.0546d = 10
S(II)
B(III)
0.0646 0.0558
0.0554 0.0560
0.0540 0.0500
0.0514 0.0526
0.0560 0.0470
d = 50
S(II)
B(III)
0.0690 0.0592
0.0592 0.0618
0.0564 0.0586
0.0544 0.0522
0.0498 0.0506
d = 100
S(II)
B(III)
0.0624 0.0576
0.0656 0.0576
0.0540 0.0514
0.0530 0.0526
0.0570 0.0542

S(III)
0.05700.0602
0.05340.05520.0486B(IV)0.06320.04640.04980.0492
0.0560S(IV)0.06400.04880.05320.05220.0582

S(III)
0.06040.06540.06400.05520.0522B(IV)0.06620.05120.05240.05020.0490S(IV)0.06820.05700.06000.05380.0504

S(III)
0.06040.06600.05820.05900.0552B(IV)0.06080.05700.05380.04980.0478S(IV)0.06380.06780.06320.05440.0514

Notes: The columns “B(I)”, “B(II)”, “B(III)” and “B(IV)” report the rejection frequencies of thelikelihood ratio test with γ ﬁxed to zero (benchmark), for Setting I, II, III and IV, respectively. The
columns “S(I)”, “S(II)”, “S(III)” and “S(IV)” report the rejection frequencies of the shrinkage likelihood
ratio tests for Setting I, II, III and IV, respectively. The number of replications is 5000.

We then investigate the power properties of the proposed tests. Consider the same setting as
the null case except for Y ∼ N (1 + 2X + (1 + δ)D, 1) and P(δ = 1|X, Z) = π(Z ′ γ), where γ is avector with all the elements equal 1. Similarly to the null case, we benchmark the proposed method11against the likelihood ratio test with γ ﬁxed to zero. For fair comparison, we use the size-adjusted
critical values, obtained from the simulation under the null. As table 2 indicates, the proposed
methods generally improve upon the power over the benchmark. The rate of improvement is ashigh as 20% in some cases. We note that the degree of improvement decreases with d, whichcoincides with the observation regarding Assumption 3. Interestingly, the correlation of Z seems
to boost the power improvement. This phenomenon suggests the possibility that the power of thetest depends on the correlation structure for Z.
Table 2: Size-adjusted Powern = 100n = 250n = 500n = 750n = 1000B(I)0.17120.32420.51400.67300.7750S(I)0.17680.36920.65400.8246
0.9238B(II)0.17400.31940.4992
0.66240.7574n = 100n = 250n = 500n = 750n = 1000B(I)0.17420.32780.49900.67040.8128S(I)0.1736
0.34460.57440.71840.8450B(II)0.17780.33380.55640.70380.8080n = 100n = 250n = 500n = 750n = 1000

B(I)0.2006
0.34320.53580.70900.8204S(I)0.20440.35540.5806
0.73100.8344B(II)0.19580.32140.5706
0.69700.8048d = 10
S(II)
B(III)
0.1850 0.1922
0.4066 0.3242
0.7394 0.5290
0.9050 0.6614
0.9634 0.7864
d = 50
S(II)
B(III)
0.1810 0.1808
0.4062 0.3154
0.6960 0.5422
0.8442 0.6958
0.9158 0.8024
d = 100
S(II)
B(III)
0.1976 0.1930
0.3744 0.3400
0.6608 0.5508
0.7950 0.6898
0.8846 0.8068

S(III)
0.19600.36980.65480.83540.9234B(IV)0.17220.35520.53240.68680.7556S(IV)0.1764
0.39600.64900.83320.9168

S(III)
0.1846
0.34700.60000.74600.8428B(IV)0.17680.34400.5432
0.69700.8042S(IV)0.17880.36080.59240.74540.8416

S(III)
0.19440.3466
0.58380.72020.8274B(IV)0.1766
0.34140.54500.71360.8130S(IV)0.18100.3506
0.57200.72860.8230Notes: The columns “B(I)”, “B(II)”, “B(III)” and “B(IV)” report the rejection frequencies of thelikelihood ratio test with γ ﬁxed to zero (benchmark), for Setting I, II, III and IV, respectively. The
columns “S(I)”, “S(II)”, “S(III)” and “S(IV)” report the rejection frequencies of the shrinkage likelihood
ratio tests for Setting I, II, III and IV, respectively. The number of replications is 5000.

5

Real-World Data Analysis

We apply the proposed method to data from AIDS Clinical Trials Group Protocol 175 (ACTG175),
which is available in R package speﬀ2trial. This study randomizes 2139 HIV-infected patients into
four diﬀerent treatment arms: zidovudine (ZDV) only, ZDV plus didanosine (ddI), ZDV plus zalcitabine (zal), and ddI only. As common in previous works (e.g., Lu et al., 2013; Fan et al., 2017),12

we consider the CD4 count at 20 ± 5 weeks after the randomization as the outcome variable ofinterest. Following Lu et al. (2013), we include the following 12 covariates plus the intercept termfor X and Z in (1): age, weight, Karnofsky score, CD4 count at baseline, CD8 count at baseline, hemophilia, homosexual activity, history of intravenous drug use, race, gender, antiretroviral
history, and symptomatic status. We standardize those 12 covariates when used for Z. For thetreatment variable D, we conduct the following four analyses as in Lu et al. (2013).
• Analysis 1: D = 0 for ZDV only and D = 1 for the other three treatment combined together.
• Analysis 2: Consider only those with ZDV plus ddI or ZDV plus zal. D = 0 for ZDV plus
zal and D = 1 for ZDV plus ddI.
• Analysis 3: Consider only those with ZDV plus ddI or ddI only. D = 0 for ddI only andD = 1 for ZDV plus ddI.
• Analysis 4: Consider only those with ZDV plus zal or ddI only. D = 0 for ddI only andD = 1 for ZDV plus zal.
Fan et al. (2017) also test the existence of a subgroup; however, they include only two covariates(age and homosexual activity) for X and Z and are limited to Analysis 2.
The results are summarized in Table 3. The proposed test rejects the null hypothesis of no
subgroup at the 5% level for Analysis 1-3, while it fails to reject the null hypothesis for Analysis
4. In particular, the p-value for Analysis 2 is less than 0.001, suggesting strong evidence for the
existence of a subgroup, which is in accordance with the ﬁnding of Fan et al. (2017). Furthermore,even though Lu et al. (2013) do not perform the hypothesis test, they suggest the absence oftreatment heterogeneity explained by covariate for Analysis 4 based on their estimation result.
This conclusion is consistent with our failure to reject the null hypothesis for Analysis 4.
Table 3: The p-values for the real-world data analysis
p-value

Analysis 10.0098Analysis 20.0005

Analysis 30.0067

Analysis 4
0.5

Notes: Each entry shows the p-value of SLRT for the corresponding analysis.

6Conclusion

This study develops a novel shrinkage testing method for the existence of a subgroup with adiﬀerential treatment eﬀect in logistic-normal mixture models. Compared to the existing works, theproposed test is computationally easy to implement due to the tractable asymptotic null distributionof the test statistics and accommodates high-dimensional covariates characterizing the classiﬁcationof the subgroup. Furthermore, we conﬁrm the good ﬁnite sample performance of the proposed
method through numerical simulation.
13

There are several interesting directions for future research. First, the theoretical properties of
the test statistics under the alternative hypothesis remain unknown. Even though we expect thepresence of L1 -penalization to complicate the situation, the theoretical analysis of the power merits
investigation in order to fully characterize the performance of the proposed method. Second, theproposed test hinges on the correctness of the parametric speciﬁcation of the model, which canbe too restrictive in real data. Hence, the development of a speciﬁcation test or extension of the
proposed shrinkage method to more general nonlinear models as in Andrews and Cheng (2012)is an important research topic. Lastly, in clinical trials, it is often the case that the outcome ofinterest is survival time and thus possibly right-censored. Tailoring the proposed method for such
survival data is of practical importance.

14

A

ProofsWe employ the notations in the empirical process theory: for any random variable R and measurable
P
function h in a class H, we write P h(R) = E[h(R)] and Pn h(R) = n1 ni=1 h(Ri ), and let ∥·∥H denotethe supremum of the absolute value over H. For example, ∥Pn h(R)−P h(R)∥H = suph∈H |Pn h(R)−P h(R)|.
In all the proofs in Appendix A and the proofs of Lemma 2, 4 and 6 in Appendix B, we use thefollowing Assumption 1′ in place of Assumption 1. Assumptions 1′ (a), (c) and (e) are identical toAssumptions 1(a), (c) and (e), while Assumptions 1′ (b) and (d) replace Θσ and Γ in Assumptions
2

2

1(b) and (d) with their compactiﬁed versions Θ̃σ and ΓM . Lemma 1 in Appendix B shows thatthe replacement by Assumption 1′ is valid with probability approaching one under Assumptions 1
2

and 2. Let Θ̃ := Θα × Θβ × Θλ × Θ̃σ denote the compactiﬁed parameter space.
Assumption 1′ . (a) Θα and Θβ are compact, convex sets, (b) Θ̃σ = [lσ02 , uσ02 ] for some 0 < lσ02 <2

σ02 < uσ02 < ∞, (c) Θλ = [0, uλ ] for some 0 < uλ < ∞, (d) ΓM = {γ ∈ Rd : ∥γ∥1 ≤ M n/p} forsome M , and (e) (α0′ , β0 )′ lies in an interior of Θα × Θβ .

Proof of Proposition 1. We apply Lemma 2 with cn = n−1/4 (log d log n)−1/2 . To this end, we
divide the proof into three steps: (i) characterization of an , (ii) characterization of bε,n , and (iii)veriﬁcation of an = o(bε,n ).
(i) characterization of an . Let f˜(Y |W ; θ, γ) := (2πσ 2 )1/2 f (Y |W ; θ, γ) = π(Z ′ γ)e−
(1 − π(Z ′ γ))e(Y −X ′ α−Dβ)2−2σ 2(Y −X ′ α−D(β+λ))22σ 2. Then a straightforward calculation yields
(Pn − P ) log f˜(Y |W ; θ, γ)

an = E

Θ̃×ΓM

.

(5)

n (see Lemma 2.2.7
Let ξ1 , . . . , ξn be i.i.d. Rademacher random variables independent of {(Yi , Wi )}i=1

of van der Vaart and Wellner (1996) for the deﬁnition). Lemma 2.3.1 of van der Vaart and Wellner(1996) (the symmetrization inequality) gives thatE(Pn − P ) log f˜(Y |W ; θ, γ)

Θ̃×ΓM


≲E


Pn ξ log f˜(Y |W ; θ, γ)

Θ̃×ΓM

.

(6)

Letting θ∗ := (α∗′ , β∗ , λ∗ , lσ02 )′ with α∗ = 0, β∗ = 0, λ∗ = 0 and lσ02 deﬁned in Assumption 1′ (b), by
the triangle inequality,EPn ξ log f˜(Y |W ; θ, γ)

Θ̃×ΓM





≤ E Pn ξ log f˜(Y |W ; θ, γ) − log f˜(Y |W ; θ∗ , 0)hi
+ E Pn ξ log f˜(Y |W ; θ∗ , 0)) .




Θ̃×ΓM

(7)

We bound each of the two terms on the right side. For the ﬁrst term, we start by considering
the function g(w1 , w2 , w3 ) := log(π(w1 )e−w2 + (1 − π(w1 ))e−w3 ). By a straightforward calculation
15+

using diﬀerentiation and the mean value theorem, |g(w1 , w2 , w3 ) − g(u1 , u2 , u3 )| ≤ |w1 − u1 | + |w2 −
u2 | + |w3 − u3 | for any (w1 , w2 , w3 ) and (u1 , u2 , u3 ) ∈ R3 . It now follows from Lemma 3 that




˜
˜
E Pn ξ log f (Y |W ; θ, γ) − log f (Y |W ; θ∗ , 0)
Θ̃×ΓM



h
i
′
(Y − X α − D(β + λ))2
(Y − X ′ α − Dβ)2
≲ E Pn ωZ ′ γ Γ + E Pn ω
+
E
Pn ω
2
M
2σ
2σ 2
Θ̃



(, 8)
Θ̃

where ω is a standard normal random variable as deﬁned in the statement of Lemma 3. The right√side is further bounded by C( n log d/p + n−1/2 ) from Lemma 4, by which we obtainEPn ξ log f˜(Y |W ; θ, γ) − log f˜(Y |W ; θ∗ , 0)

√




Θ̃×ΓM

≲

1
n log d
+√ .
p
n

(9)

For the second term on the right hside of (7), Lemma 8 of iChernozhukov et al. (2015) combinedwith Assumption 2(a) gives that E Pn ξ log f˜(Y |W ; θ∗ , 0)) ≲ n−1/2 . In light of this inequality
and (9), an ≲ (n log d)1/2 /p + n−1/2 follows from (5), (6) and (7).
(ii) characterization of bε,n . Let rn be an arbitrary sequence of positive real numbers converging−1
rn
to zero. Deﬁne Ψrεn := {(θ′ , γ)′ ∈ Θ̃ × ΓM : ∥θ − θ0 ∥ + c−1n ∥γ∥1 ≥ ε, cn ∥γ∥1 ≤ rn } and Ψε,c :=rn
rn−1{(θ′ , γ)′ ∈ Θ̃ × ΓM : ∥θ − θ0 ∥ + c−1n ∥γ∥1 ≥ ε, cn ∥γ∥1 > rn }. Then, because Ξε,n = Ψε ∪ Ψε,c (Ξε,nis deﬁned in the statement of Lemma 2), bε,n is no smaller thanE[log f (Y |W ; θ0 , 0)] − (An ∨ Bn ) ≥ (E[log f (Y |W ; θ0 , 0)] − An ) ∧ (E[log f (Y |W ; θ0 , 0)] − Bn ), (10)
where
An :=

sup
(θ ′ ,γ ′ )′ ∈Ψrεn(E[log f (Y |W ; θ, γ)] − p/n∥γ∥1 ) , Bn :=supn(θ ′ ,γ ′ )′ ∈Ψrε,c(E[log f (Y |W ; θ, γ)] − p/n∥γ∥1 ) .

We bound each of E[log f (Y |W ; θ0 , 0)] − An and E[log f (Y |W ; θ0 , 0)] − Bn from below. For theﬁrst quantity, a straightforward calculation gives that E[log f (Y |W ; θ0 , 0)] − An is no smaller thanE[log f (Y |W ; θ0 , 0)]−sup(θ ′ ,γ ′ )′ ∈ΨrεnE[log f (Y |W ; θ, 0)]−∥E[log f (Y |W ; θ, γ)] − E[log f (Y |W ; θ, 0)]∥Ψrεn .
(11)Note that for suﬃciently large n with rn ≤ ε/2, (θ′ , γ ′ )′ ∈ Ψrεn implies that ∥θ − θ0 ∥ ≥ ε/2. Hence,for such n, it holds thatE[log f (Y |W ; θ0 , 0)] −≥ E[log f (Y |W ; θ0 , 0)] −sup(θ ′ ,γ ′ )′ ∈ΨrεnE[log f (Y |W ; θ, 0)]supθ∈{θ∈Θ:∥θ−θ0 ∥≥ε/2}

E[log f (Y |W ; θ, 0)],(12)

where the right side is positive from the information inequality combined with the model identiﬁabil-

16

ity and Assumption 1′ (a)-(c). Meanwhile, similarly to the argument leading to (8), | log f (Y |W ; θ, γ)−log f (Y |W ; θ, 0)| ≤ |Z ′ γ|, which is bounded by ∥γ∥1 max1≤j≤d |Z(j) |. Consequently,∥E[log f (Y |W ; θ, γ)] − E[log f (Y |W ; θ, 0)]∥Ψrεn ≤ cn rn E max |Z(j) | ≲ cn rn1≤j≤dplog d = o(1), (13)where the second inequality follows from Lemma 2.2.1 and 2.2.2 of van der Vaart and Wellner(1996) and Assumption 2(c) and the last equality follows from the choice of cn . Combining (11),(12) and (13), we obtain
E[log f (Y |W ; θ0 , 0)] − An ≥ cε ,(14)

for some positive constant cε for suﬃciently large n.
For E[log f (Y |W ; θ0 , 0)] − Bn , note thatE[log f (Y |W ; θ0 , 0)]−Bn ≥ E[log f (Y |W ; θ0 , 0)]−supn(θ ′ ,γ ′ )′ ∈Ψrε,cE[log f (Y |W ; θ, γ)]+infn(θ ′ ,γ ′ )′ ∈Ψrε,cp/n∥γ∥1 ,n E[log f (Y |W ; θ, γ)] ≥ 0 from the information inequality
where E[log f (Y |W ; θ0 , 0)] − sup(θ′ ,γ ′ )′ ∈Ψrε,c√5/4 log d log n)n p/n∥γ∥1 ≥ prn cn /n = (pn rn )/(nand the model identiﬁability, and inf (θ′ ,γ ′ )′ ∈Ψrε,cn and the choice of c . As a result,from the construction of Ψrε,cnE[log f (Y |W ; θ0 , 0)] − Bn ≥ (pn rn )/(n5/4plog d log n)(15)holds.

√
Combining (10), (14) and (15), we have bε,n ≳ pn rn /(n5/4 log d log n).
(iii) veriﬁcation of an = o(bε,n ). Combine the results from (i) and (ii) to obtain the inequality

an
1
≲
bε,n
rn

n7/4

√

log n log d
+
p2

n3/4

√

log n log d
p



!
≲

1 
rn

n7/4

√

s
log n log d
+
p2

n7/4

√


log n log d .
p2

Taking the convergence rate of rn to zero suﬃciently slow, the right side converges to zero by
Assumption 3(a). Lemma 2 now completes the proof.
Proof of Proposition 2. First, we prove part (a). Let ψ = (ζ1 , . . . , ζr )′ with r = q + 3. Collectησ := (η ′ , σ 2 )′ , tn (ησ ) := n1/2 (ησ − ησ 0 ) and sηi σ := (Ui′ Hi1 /σ0 , Hi2 /(2σ02 )′ )′ . Accordingly, deﬁnePP′Snησ := n−1/2 ni=1 sηi σ and Inησ := n−1 ni=1 sηi σ sηi σ . We abbreviate f (Y |W ; ψ0 , γ̂ ∗ ) to f0 . Let {ησ }be a set consisting of the elements of ησ . Furthermore, let V and ρ(ε) denote a random variablewith the ﬁnite second moments that is independent of ε and a polynomial of ε whose value andform may vary from one expression to another, respectively. Expanding ln∗ (ψ, γ̂ ∗ ) around ψ0 ﬁve

17

times using Taylor’s theorem yields ln∗ (ψ, γ̂ ∗ ) − ln∗ (ψ0 , γ̂ ∗ ) =P5

k=1 Dk where1D1 := ∇ψ ln (ψ0 , γ̂ ∗ )′ (ψ − ψ0 ), D2 := (ψ − ψ0 )′ ∇ψψ′ ln (ψ0 , γ̂ ∗ )(ψ − ψ0 )
2
r
r
r
1 XXX
D3 :=
∇ζi ζj ζk ln (ψ0 , γ̂ ∗ )(ζi − ζi,0 )(ζj − ζj,0 )(ζk − ζk,0 ),
3!
D4 :=
D5 :=

1
4!
1
5!

i=1 j=1 k=1
r
r X
r X
r X
X

∇ζi ζj ζk ζl ln (ψ0 , γ̂ ∗ )(ζi − ζi,0 )(ζj − ζj,0 )(ζk − ζk,0 )(ζl − ζl,0 ),i=1 j=1 k=1 l=1
r X
r X
r X
r
r X
X

∇ζi ζj ζk ζl ζm ln (ψ̄, γ̂ ∗ )(ζi − ζi,0 )(ζj − ζj,0 )(ζk − ζk,0 )(ζl − ζl,0 )(ζm − ζm,0 ),i=1 j=1 k=1 l=1 m=1

where ψ̄ lies on the path connecting ψ0 and ψ. We now investigate each of D1 -D5 .
(i) D1 . By a straightforward calculation using Lemma 5,
D1 = (Snησ )′ tn (ησ ) +n1 Xn1/4 i=1

Di Hi1(2π(Zi′ γ̂ ∗ ) − 1)2σ0!
n1/4 λ = (Snησ )′ tn (ησ ) + op (1)n1/4 λ,where the second equality follows from Lemma 6(a).
(ii) D2 . Let µ1 , µ2 be any elements of ψ. Then ∇µ1 µ2 log f0 = ∇µ1 µ2 f0 /f0 −(∇µ1 f0 /f0 )(∇µ2 f0 /f0 ).
When µ1 , µ2 ∈ {ησ }, a straightforward calculation with Lemma 5 gives that E[∇µ1 µ2 f0 /f0 ] = 0. In
particular, this implies(ησ − ησ 0 )′ ∇ησ ησ′ ln (ψ0 , γ̂ ∗ )(ησ − ησ 0 ) = tn (ησ )′ op (1)tn (ησ ) − tn (ησ )′ Inησ tn (ησ ),(16)

where op (1) follows from the law of large numbers. At the same time, a straightforward calculation with Lemma 5 gives that (ησ − ησ 0 )′ ∇ησ λ ln (ψ0 , γ̂ ∗ )λ = tn (ησ )′ Snησ λ n1/4 λ − tn (ησ )′ Inησ λ n1/4 λ,′Pwhere we deﬁne Snησ λ := n−3/4 ni=1 (2π(Zi′ γ̂ ∗ ) − 1)Ui′ Di Hi2 /(2σ02 ), (2π(Zi′ γ̂ ∗ ) − 1)Di Hi3 /(4σ03 )P
and Inησ λ := n−3/4 ni=1 sηi σ (2π(Z ′ γ̂ ∗ ) − 1)Di Hi1 /(2σ0 ). By Lemma 6(b), Snησ λ = op (1) and Inησ λ =op (1). Hence, we have(ησ − ησ 0 )′ ∇ησ λ ln (ψ0 , γ̂ ∗ )λ2 = tn (ησ )′ op (1)n1/4 λ.

(17)Lastly, by a straightforward calculation with Lemma 5, ∇λ2 ln (ψ0 , γ̂ ∗ )λ2 equals
n−1/2

n
X

!
Di Hi2 /(4σ02 ) n1/2 λ2 −

n−1/2

i=1

n
X

!
(2π(Zi′ γ̂ ∗ ) − 1)2 (Di Hi1 )2 /(2σ0 )2

n1/2 λ2 .

i=1However, observe that n−1/2

Pn

′ ∗
2
1 2
2
i=1 (2π(Zi γ̂ ) − 1) (Di Hi ) /(2σ0 ) = op (1) by Lemma 6(c). Hence,

we have∗∇λ2 ln (ψ0 , γ̂ )λ =
2

n

−1/2

n
X

!
Di Hi2 /(4σ02 )

i=1

18

n1/2 λ2 + op (1)n1/2 λ2 .

(18)

Combining (16), (17) and (18), we obtain
1
D2 = − tn (ησ )′ Inησ tn (ησ ) +
2

n

−1/2

n
X

!
Di Hi2 /(8σ02 )n1/2 λ2i=1′′+ tn (ησ ) op (1)tn (ησ ) + tn (ησ ) op (1)n1/4 λ + op (1)n1/2 λ2 .
(iii) D3 . Note that
1 X
D3 =
2

r
X X

∇ζi ζj ζk ln (ψ0 ; γ̂ ∗ )(ζi − ζi,0 )(ζj − ζj,0 )(ζk − ζk,0 )ζi ∈{ησ } ζj ∈{ησ } k=1

+

1 X
1
∇ζλ2 ln (ψ0 ; γ̂ ∗ )(ζ − ζ0 )λ2 + ∇λ3 ln (ψ0 ; γ̂ ∗ )λ3 .
2
3!

(19)

ζ∈{ησ }

We take a close look at each term on the right side. First, by a straightforward calculation with
Lemma 5 and Assumption 2(a), for any elements µ1 , µ2 , µ3 of ψ, |∇µ1 µ2 µ3 log f0 | is bounded by aintegrable function g(W, ε) that does not depend on γ̂ ∗ . Hence, by the law of large numbers,
X

r
X X

∇ζi ζj ζk ln (ψ0 ; γ̂ ∗ )(ζi − ζi,0 )(ζj − ζj,0 )(ζk − ζk,0 )ζi ∈{ησ } ζj ∈{ησ } k=1

=

X

X

Op (1)n1/2 (ζi − ζi,0 )n1/2 (ζj − ζj,0 )O(∥ψ − ψ0 ∥).

(20)

ζi ∈{ησ } ζj ∈{ησ }Subsequently, for ζ ∈ {ησ }, a straightforward derivative calculation yields∇ζλ2 f0∇ζλ f0 ∇λ f0 ∇λ2 f0 ∇ζ f0∇ζ f 0∇ζλ2 log f0 =
−2
−
+2
f0
f0
f0
f0
f0
f0



∇λ f 0f0

2.

By Lemma 5, it is easy to verify that E[∇ζλ2 f0 /f0 ] = 0. Furthermore, Lemma 5 also suggests that(∇ζλ f0 /f0 )(∇λ f0 /f0 ) and (∇ζ f0 /f0 )(∇λ f0 /f0 )2 can be written as the form V (π(Z ′ γ̂ ∗ ) − 1/2)2 ρ(ε).
Hence, by the law of large numbers, Lemma 6(b) and applying Lemma 5 to (∇λ2 f0 /f0 )(∇ζ f0 /f0 )
yield
X
1 X
∇ζλ2 ln (ψ0 ; γ̂ ∗ )(ζ − ζ0 )λ̂2 =op (1)n1/2 (ζ − ζ0 )n1/2 λ22ζ∈{ησ }

ζ∈{ησ }

− tn (ησ )

′

n

−1

!
n
X
ησ
2
2
si Di Hi /(8σ0 ) n1/2 λ2 .

(21)

i=1

Lastly, by a straightforward derivative calculation with Lemma 5, ∇λ3 log f0 can be written as theform D3 (π(Z ′ γ̂ ∗ ) − 1/2)k ρ(ε) with k ∈ N. It follows from Lemma 6(b) that ∇λ3 ln (ψ0 ; γ̂ ∗ )λ3 =19op (1)n1/4 λn1/2 λ2 . Combining this with (19), (20) and (21) yields that
D3 = − tn (ησ )
+

X

′

n

−1

!
n
X
ησ
2
2
si Di Hi /(8σ0 ) n1/2 λ2
i=1X

Op (1)n1/2 (ζi − ζi,0 )n1/2 (ζj − ζj,0 )O(∥ψ − ψ0 ∥)ζi ∈{ησ } ζj ∈{ησ }+X

op (1)n1/2 (ζ − ζ0 )n1/2 λ2 + op (1)n1/4 λn1/2 λ2 .

ζ∈{ησ }

(iv) D4 . Observe that
D4 =

r
r
r
1 X XXX∇ζζi ζj ζk ln (ψ0 ; γ̂ ∗ )(ζ − ζ0 )(ζi − ζi,0 )(ζj − ζj,0 )(ζk − ζk,0 )3ζ∈{ησ } i=1 j=1 k=1+

1∇ 4 ln (ψ0 ; γ̂ ∗ )λ4 .
4! λ

(22)

For the summation on the right side, a straightforward calculation with Lemma 5 and Assumption2(a) gives that, for any elements µ1 , µ2 , µ3 , µ4 of ψ, |∇µ1 µ2 µ3 µ4 log f0 | is bounded by a integrable
function g(W, ε) that does not depend on γ̂ ∗ . Hence, by the law of large numbers,
r X
r X
r
X X

∇ζζi ζj ζk ln (ψ0 ; γ̂ ∗ )(ζ − ζ0 )(ζi − ζi,0 )(ζj − ζj,0 )(ζk − ζk,0 )ζ∈{ησ } i=1 j=1 k=1

=

X

X

Op (1)n1/2 (ζ − ζ0 )n1/2 (µ − µ0 )O(∥ψ − ψ0 ∥),(23)ζ∈{ησ } µ∈{ησ ,λ2 }where {ησ , λ2 } is a set consisting of the elements of ησ and λ2 . Furthermore, a straightforward
derivative calculation yields∇λ4 log f0 =∇λ 4 f 0∇ 3 f 0 ∇λ f 0
−4 λ
−3
f0
f0
f0



∇λ 2 f 0
f0

2
+ 12

∇λ 2 f 0
f0

∇λ f 0
f0

2


−6

∇λ f 0f0

4.

By Lemma 5, E[∇λ4 f0 /f0 ] = 0. Furthermore, a straightforward calculation with Lemma 5 givesthat (∇λ3 f0 /f0 )(∇λ f0 /f0 ), (∇λ2 f0 /f0 )(∇λ f0 /f0 )2 and (∇λ f0 /f0 )4 all have the form V (π(Z ′ γ̂ ∗ ) −1/2)k ρ(ε) where k ∈ N and V has the ﬁnite second moment. Combining these facts with applying
Lemma 5 to (∇λ2 f0 /f0 )2 in conjunction with the law of large numbers and Lemma 6(b),1∇ 4 ln (ψ0 , γ̂ ∗ )λ4 = −
4! λ

1 X
(Di Hi2 /(8σ02 ))2
2n
n

i=1

20

!
(n1/2 λ2 )2 + op (1)(n1/2 λ2 )2 .

(24)

In view of (22), (23) and (24), we obtain
1 X
2n
n

D4 = −i=1

Di Hi28σ02

2 !
(n1/2 λ2 )2 +X

X

Op (1)n1/2 (ζ − ζ0 )n1/2 (µ − µ0 )O(∥ψ − ψ0 ∥)ζ∈{ησ } µ∈{ησ ,λ2 }+ op (1)(n1/2 λ2 )2 .
(v) D5 . For any elements µ1 , . . . , µ5 of ψ, a straightforward calculation with Lemma 5 inconjunction with Assumptions 2(a) and 1′ (a)-(c) implies that |∇µ1 ...µ5 log f (Y |W ; ψ̄, γ̂ ∗ )| is boundedby an integrable function independent of the values of ψ̄ and γ̂ ∗ . Hence, the law of large numbers
implies thatX

D5 =Op,ψ̄ (1)n1/2 (ζ − ζ0 )n1/2 (µ − µ0 )O(∥ψ − ψ0 ∥),ζ∈{ησ ,λ2 },µ∈{ησ ,λ2 }where |Op,ψ̄ (1)| ≤ Op (1) for some Op (1) independent of ψ̄ and γ̂ ∗ as discussed above.
Collecting the terms from D1 -D5 , we obtain
1ln∗ (ψ, γ̂ ∗ ) − ln∗ (ψ0 , γ̂ ∗ ) = Sn′ tn (ψ) − tn (ψ)′ In tn (ψ) + Rn (ψ, γ̂ ∗ ),2
where, by a straightforward calculation, supψ∈{ψ∈Θψ :∥ψ−ψ0 ∥≤κ} |Rn (ψ, γ̂ ∗ )|/(1 + ∥tn (ψ)∥)2 = op (1)for any sequence κ converging to zero. This completes the proof of part (a). Part (b) follows fromthe central limit theorem. For part (c), the law of large numbers implies that In →p I := E[si s′i ].
′′σλThe nonsingularity of I follows from the nonsingularity of E[sηi sηi ] and E[sσλi si ] by Assumption′

1 2
2(b) and Assumption 2(d), respectively, and the fact that E[sηi sσλi ] = 0 because E[Hi Hi ] = 0,
2
2
2
2 ′
where sηi := (Ui Hi1 /σ0 ) and sσλi := (Hi /(2σ0 ), Di Hi /(8σ0 )) .

Proof of Proposition 3. Part (a) follows from a simple adaptation of the proof of Proposition 3(a)of Kasahara and Shimotsu (2015) to our quadratic approximation in Proposition 2(a). For part(b), our proof is based on that of Proposition 3(b) and (c) of Kasahara and Shimotsu (2015).
We suppress ψ from tn (ψ), and let t̂n := tn (ψ̂ ∗ ).

Deﬁne Iλ := E[Di2 (Hi2 )2 /(8σ02 )2 ], Iησ λ :=(01,q+1 , E[Di (Hi2 )2 /(16σ04 )])′ ,"Iησ :=

E[Ui Ui′ (Hi1 )2 /σ02 ]0q+1,101,q+1

E[(Hi2 )2 /(2σ02 )2 ]#", I :=I ησ

I ησ λIη′ σ λIλwhere 0s,t is a s × t zero matrix. For ϑ := (α′ , β, σ 2 )′ , let l0,n (ϑ) :=log-likelihood function under the null model, where g(Y |W ; ϑ) := ϕσ

#
,

Pn

i=1 log g(Yi |Wi ; ϑ) be a
(Y − X ′ α − Dβ) for a density

function ϕσ of the normal distribution with mean zero and variance σ 2 . Letting ϑ̂ denote the MLE
under the null model, observe that SLRT = 2(ln (ψ̂ ∗ , γ̂ ∗ ) − ln (ψ0 , γ̂ ∗ )) − 2(l0,n (ϑ̂) − l0,n (ϑ0 )) becauseln (ψ0 , γ̂) = l0,n (ϑ0 ). We investigate each of (i) 2(ln (ψ̂ ∗ , γ̂ ∗ )−ln (ψ0 , γ̂ ∗ )) and (ii) 2(l0,n (ϑ̂)−l0,n (ϑ0 ))below.
21

(i) 2(ln (ψ̂ ∗ , γ̂ ∗ ) − ln (ψ0 , γ̂ ∗ )). Because ln (ψ̂ ∗ , γ̂ ∗ ) − ln (ψ0 , γ̂ ∗ ) = ln∗ (ψ̂ ∗ , γ̂ ∗ ) − ln∗ (ψ0 , γ̂ ∗ ), Proposition 2(a) and part (a) of this proposition yields ln (ψ̂ ∗ , γ̂ ∗ ) − ln (ψ0 , γ̂ ∗ ) = Sn′ t̂n − 21 t̂′n In t̂n + op (1).

Let Wψ = (Wη′ σ , Wλ )′ := I −1 Sn , where Wησ is the ﬁrst q + 2 elements of Wψ . It now follows from
2Sn′ t̂n − t̂′n I t̂n = Wψ′ IWψ − (t̂n − Wψ )′ I(t̂n − Wψ ), Proposition 2(c) and part (a) of this proposition
that2(ln (ψ̂, γ̂) − ln (ψ0 , γ̂)) = Wψ′ IWψ − (t̂n − Wψ )′ I(t̂n − Wψ ) + op (1).

(25)

Partition Sn = (Sη′ σ , Sλ )′ with Sησ being the ﬁrst q + 2 elements of Sn . Furthermore, deﬁneW̄ησ := Iη−1Sησ and Iησ ·λ := Iλ − Iη′ σ λ Iη−1
Iησ λ . By a tedious but straightforward calculation usingσσ

the formula of inverse of a partitioned matrix for I −1 (e.g., Exercise 5.16(a) of Abadir and Magnus(2005)), we haveWψ′ IWψ = W̄η′ σ Iησ W̄ησ + Wλ′ Iησ ·λ Wλ .

(26)

For the second term on the right side of (25), the proof of Theorem 2 of Andrews (1999) givesthat (t̂′n − Wψ )′ I(t̂n − Wψ ) = inf t∈Λn (t − Wψ )′ I(t − Wψ ) + op (1), where Λn := {tn (ψ) : ψ ∈ Θψ }.
By Assumption 1′ (a)-(c) and (e), the set {tn (ψ)/bn : ψ ∈ Θψ } is locally approximated by a cone
Λ := Rq+2 × [0, ∞) for any sequence bn such that bn → ∞ and bn = o(n1/2 ) (see page 1358
of Andrews (1999) for the deﬁnition of “locally approximated by a cone”). Hence, Lemma 2 ofAndrews (1999) yields inf t∈Λn (t − Wψ )′ I(t − Wψ ) = inf t∈Λ (t − Wψ )′ I(t − Wψ ) + op (1), by which
we have(t̂′n − Wψ )′ I(t̂n − Wψ ) = inf (t − Wψ )′ I(t − Wψ ) + op (1).
t∈Λ

(27)

Furthermore, a straightforward calculation with the formula of inverse of a partitioned matrix forI −1 , (t−Wψ )′ I(t−Wψ ) = (t1 +Iη−1Iησ λ t2 −W̄ησ )′ Iησ (t1 +Iη−1Iησ λ t2 −W̄ησ )+(t2 −Wλ )′ Iησ ·λ (t2 −Wλ ),σσwhere we partition t = (t′1 , t2 )′ with t1 being the ﬁrst q + 2 elements of t. Because t1 ranges overRq+2 independently of the value of t2 , we observeinf (t − Wψ )′ I(t − Wψ ) =t∈Λ=

inf (t1 − W̄ησ )′ Iησ (t1 − W̄ησ ) +t1 ∈Rq+2

inf

t2 ∈[0,∞)(t2 − Wλ )′ Iησ ·λ (t2 − Wλ )inf (t1 − W̄ησ )′ Iησ (t1 − W̄ησ ) + 1{Wλ < 0}Wλ′ Iησ ·λ Wλ .

t1 ∈Rq+2(28)

Combining (25), (26), (27) and (28) gives2(ln (ψ̂, γ̂) − ln (ψ0 , γ̂))= W̄η′ σ Iησ W̄ησ + 1{Wλ ≥ 0}Wλ′ Iησ ·λ Wλ −inf (t1 − W̄ησ )′ Iησ (t1 − W̄ησ ) + op (1)t1 ∈Rq+2(ii) 2(l0,n (ϑ̂) − l0,n (ϑ0 )). Deﬁne un (ϑ) := (n1/2 (α − α0 )′ , n1/2 (β − β0 ), n1/2 (σ 2 − σ02 ))′ and sηi σ :=(Ui′ Hi1 /σ0 , Hi2 /(2σ02 ))′ . By a similar argument to the proof of Proposition 2, we obtain the following
quadratic approximation.
1
l0,n (ϑ) − l0,n (ϑ0 ) = Snησ ′ un (ϑ) − un (ϑ)′ Inησ un (ϑ) + Rn (ϑ),
222

where Snησ := n−1/2Pn

P n ησ ησ ′
ησ
ησ
−1
2
i=1 si and In := ni=1 si si , and supϑ:∥ϑ−ϑ0 ∥≤κ |Rn (ϑ)|/(1+∥un (ϑ)∥) =op (1) for any κ converging to zero. Then, similarly to part (a) of this proposition, we can showun (ϑ̂) = Op (1). In view of the above quadratic approximation and un (ϑ̂) = Op (1), repeating theargument for part (i) gives that2(l0,n (ϑ̂) − l0,n (ϑ0 )) = W̄η′ σ Iησ W̄ησ −inf (t1 − W̄ησ )′ Iησ (t1 − W̄ησ ) + op (1).

t1 ∈Rq+2

Consequently, combining the results from (i) and (ii) yields SLRT = 1{Wλ ≥ 0}Wλ′ Iησ ·λ Wλ +
1/2

1/2

op (1) = 1{Iησ ·λ Wλ ≥ 0}(Iησ ·λ Wλ )2 + op (1) from Iησ ·λ > 0. By the central limit theorem andSlutsky’s theorem, Wψ →d N (0, I −1 ). In particular, because Wλ is the last coordinate of Wψ andIη−1is the bottom right element of I −1 , Wλ →d N (0, Iη−1). Hence, Iησ ·λ Wλ →d N (0, 1). Because
σ ·λ
σ ·λ
1/2

the map x → 1{x ≥ 0}x2 is continuous almost everywhere with respect to Lebesgue measure, thecontinuous mapping theorem completes the proof.

B

Auxiliary results

Lemma 1 compactiﬁes the parameter space Θ × Γ in Assumption 1, which is helpful for the proofs
in Appendix A.
Lemma 1. Assume Assumptions 1 and 2 hold. Then (θ̂∗ , γ̂ ∗ ) = argmaxθ∈Θ̃,γ∈ΓM ln∗ (θ, γ) with2

2

probability approaching one, where Θ̃ := Θα × Θβ × Θλ × Θ̃σ with Θ̃σ := [lσ02 , uσ02 ] for some
0 < lσ02 < σ02 < uσ02 < ∞ and ΓM := {γ ∈ Γ : ∥γ∥1 ≤ M n/p} for some ﬁnite M > 0.

Proof of Lemma 1. We ﬁrst prove (θ̂∗ , γ̂ ∗ ) = argmaxθ∈Θ̃,γ∈Γ ln∗ (θ, γ) with probability approachingone. Our argument is based on Lemma 3.1 of Chen (2017). By a straightforward calculation,log f (y|w; θ, γ) ≤ −log(2π) log σ 2 (y − x′ α − d(β + λ))2 ∧ (y − x′ α − dβ)2
−
−
,
2
2
2σ 2

(29)

which implies that supα∈Θα ,β∈Θβ ,λ∈Θλ ,γ∈Γ n−1 ln∗ (θ, γ) − n−1 ln∗ (θ0 , 0) is bounded by−log(2π) + log σ 21l∗ (θ0 , 0)
− 2
inf
Pn (Y −X ′ α−D(β+λ))2 ∧(Y −X ′ α−Dβ)2 − n. (30)
22σ α∈Θα ,β∈Θβ ,λ∈Θλn
Let An = ∥(Pn − P )(Y − X ′ α − D(β + λ))2 ∧ (Y − X ′ α − Dβ)2 ∥Θα ×Θβ ×Θλ . Then, for (30),

inf α∈Θα ,β∈Θβ ,λ∈Θλ Pn (Y − X ′ α − D(β + λ))2 ∧ (Y − X ′ α − Dβ)2 is no smaller than
−An +

inf

α∈Θα ,β∈Θβ ,λ∈ΘλP (Y − X ′ α − D(β + λ))2 ∧ (Y − X ′ α − Dβ)2 .

(31)

We consider bounding the right side from below. For An , it follows from Lemma 2.6.15, Lemma
2.6.18(v), Theorem 2.6.7 and Theorem 2.4.3 of van der Vaart and Wellner (1996) and Assumption2(a) that each of {Yi −Xi′ α−Di (β +λ) : α ∈ Θα , β ∈ Θβ , λ ∈ Θλ } and {Yi −Xi′ α−Di β : α ∈ Θα , β ∈23

Θβ } is a Glivenko-Cantelli class. Then, by Theorem 3 of van der Vaart and Wellner (2000) andAssumption 2(a), An →p 0. For inf α∈Θα ,β∈Θβ ,λ∈Θλ P (Y −X ′ α−D(β +λ))2 ∧(Y −X ′ α−Dβ)2 , notethat Y −X ′ α−D(β +λ) = ε+X ′ (α0 −α)+D(β0 −β +δλ0 −λ). Then P(Y −X ′ α−D(β +λ) = 0) =E[P(ε+X ′ (α0 −α)+D(β0 −β +δλ0 −λ) = 0|X, D, δ)]. Because ε and (X, D, δ) are independent, theconditional probability inside the expectation on the right side is zero so that P(Y −X ′ α−D(β+λ) =0) = 0. Similarly, P(Y −X ′ α−Dβ = 0) = 0. Hence, P (Y −X ′ α−D(β +λ))2 ∧(Y −X ′ α−Dβ)2 > 0
over Θα × Θβ × Θλ . Because P (Y − X ′ α − D(β + λ))2 ∧ (Y − X ′ α − Dβ)2 is continuous in (α′ , β, λ)′from Assumption 2(a) and the dominated convergence theorem, and Θα × Θβ × Θλ is compact, itholds that inf α∈Θα ,β∈Θβ ,λ∈Θλ P (Y − X ′ α − D(β + λ))2 ∧ (Y − X ′ α − Dβ)2 > 0. Combining thisinequality with An →p 0 and (31) yields that there exists a ﬁnite positive constant M1 such thatinfα∈Θα ,β∈Θβ ,λ∈ΘλPn (Y − X ′ α − D(β + λ))2 ∧ (Y − X ′ α − Dβ)2 > M1

(32)

with probability approaching one.
For n−1 ln∗ (θ0 , 0) on the right side of (30), we ﬁrst note the following inequality: for any positive
real numbers a and b, | log(a/2+b/2)| ≤ | log(a∧b)|∨| log(a∨b)| = | log a|∨| log b| ≤ | log a|+| log b|.
Applying this inequality to | log f (Y |W ; θ0 , 0)|, P | log f (Y |W ; θ0 , 0)| is bounded by√
1log σ0 2π + 2 P (Y − X ′ α0 − D(β0 + λ0 ))2 + P (Y − X ′ α0 − Dβ0 )2 ,4σ0which is ﬁnite by Assumption 2(a). Hence, by the law of large numbers, there exists a ﬁnite positive
constant M2 such that |n−1 ln∗ (θ0 , 0)| ≤ M2 holds with probability approaching one.
In view of this bound, (30) and (32),supα∈Θα ,β∈Θβ ,λ∈Θλ ,γ∈Γ

n−1 ln∗ (θ, γ) − n−1 ln∗ (θ0 , 0) ≤ −M1log(2π) log σ 2−− 2 + M222
2σholds for any σ 2 with probability approaching one. Because the right side tends to minus inﬁnity as σ 2 → 0 or σ 2 → ∞, supα∈Θα ,β∈Θβ ,λ∈Θλ ,σ2 ∈(0,l)∪(u,∞),γ∈Γ n−1 ln∗ (θ, γ) − n−1 ln∗ (θ0 , 0) < 0
holds with probability approaching one for some 0 < l < σ02 < u < ∞. This proves (θ̂∗ , γ̂ ∗ ) =argmaxθ∈Θ̃,γ∈Γ ln∗ (θ, γ) with probability approaching one.

We move on to verify (θ̂∗ , γ̂ ∗ ) = argmaxθ∈Θ,γ∈Γl∗ (θ, γ) with probability approaching one. Dee
M nﬁne (θ̃∗ , γ̃ ∗ ) := argmaxθ∈Θ̃,γ∈Γ ln∗ (θ, γ). Then P(n−1 ln∗ (θ̃∗ , γ̃ ∗ ) ≥ −M2 ) ≥ P(n−1 ln∗ (θ0 , 0) ≥ −M2 ) →1 because |n−1 ln∗ (θ0 , 0)| ≤ M2 with probability approaching one. Furthermore, (29) implies thatn−1 ln (θ̃∗ , γ̃ ∗ ) ≤ − log(2π)/2 − log(lσ02 )/2. Consequently,
P n−1 ln∗ (θ̃∗ , γ̃ ∗ ) ≥ −M2 , n−1 ln (θ̃∗ , γ̃ ∗ ) ≤ − log(2π)/2 − log(lσ02 )/2≤ P p/n∥γ̃ ∗ ∥1 ≤ − log(2π)/2 − log(lσ02 )/2 + M2 .

24

Because the left side converges to one, the desired result follows by setting M = − log(2π)/2 −log(lσ02 )/2 + M2 .
The following lemma is the key to proving the consistency and the convergence rate in Proposition 1. This result is an adaptation of Lemma A1 of Andrews (1993) to our high-dimensionalsetting.
Lemma 2. Assume the assumption of Proposition 1 holds. Let {cn }n∈N be a sequence of positive real numbers converging to zero and an := E[supθ∈Θ̃,γ∈ΓM |n−1 ln∗ (θ, γ) − E[log f (Y |W ; θ, γ)] +pn /n∥γ∥1 |]. For ε > 0, deﬁne bε,n = E[log f (Y |W ; θ0 , 0)] − sup(θ′ ,γ ′ )′ ∈Ξε,n (E[log f (Y |W ; θ, γ)] −p/n∥γ∥1 ) with Ξε,n := {(θ, γ) ∈ Θ̃ × ΓM : ∥θ − θ0 ∥ + c−1n ∥γ∥1 ≥ ε}. Then if an = o(bε,n ) for eachε > 0, it holds that θ̂∗ →p θ0 and ∥γ̂ ∗ ∥1 = op (cn ).

Proof of Lemma 2. The proof is based on Lemma A1 of Andrews (1993). By the assumption onbε,n ,

P((θ̂∗ , γ̂ ∗ ) ∈ Ξε,n ) ≤ P E[log f (Y |W ; θ0 , 0)] − (E[log f (Y |W ; θ̂∗ , γ̂ ∗ )] − pn /n∥γ̂ ∗ ∥1 ) ≥ bε,n(33)

Now, for the term inside the probability on the right side, E[log f (Y |W ; θ0 , 0)]−(E[log f (Y |W ; θ̂∗ , γ̂ ∗ )]−pn /n∥γ̂ ∗ ∥1 ) is bounded by
E[log f (Y |W ; θ0 , 0)] − n−1 ln∗ (θ̂∗ , γ̂ ∗ ) + n−1 ln∗ (θ̂∗ , γ̂ ∗ ) − E[log f (Y |W ; θ̂∗ , γ̂ ∗ )] + pn /n∥γ̂ ∗ ∥1
≤ E[log f (Y |W ; θ0 , 0)] − n−1 ln∗ (θ0 , 0) + n−1 ln∗ (θ̂∗ , γ̂ ∗ ) − E[log f (Y |W ; θ̂∗ , γ̂ ∗ )] + pn /n∥γ̂ ∗ ∥1≤2

n−1 ln∗ (θ, γ) − E[log f (Y |W ; θ, γ)] + pn /n∥γ∥1 ,supθ∈Θ̃,γ∈ΓM

where the ﬁrst inequality follows from the deﬁnition of (θ̂∗ , γ̂ ∗ ). Combining this inequality with (33)and Markov’s inequality gives P((θ̂∗ , γ̂ ∗ ) ∈ Ξε,n ) ≤ an /bε,n = o(1). This completes the proof.
Lemma 3 is multivariate contraction principle, which is instrumental in handling the highdimensionality in the proof of Proposition 1. Similar but slightly diﬀerent results are obtained in
Theorem 4.1 of van de Geer (2013) and Theorem 16.2 of van de Geer (2016).
Lemma 3. Let {Xi }ni=1 be X -valued i.i.d. random variables for some measurable space (X , S) andF be a class of Rr -valued measurable functions on X . Consider L1 -Lipschitz functions ρi : Rr → Rsuch that |ρi (z)−ρi (z̃)| ≤ ∥z−z̃∥1 for all z, z̃ ∈ Rr and i = 1, . . . , n. Let {ξi }ni=1 be i.i.d. Rademacher
random variables and {ωi,k : 1 ≤ i ≤ n, 1 ≤ k ≤ r} be a collection of i.i.d. standard normal randomvariables, both of which are independent of each other and of {Xi }ni=1 . Then it holds that
"
E

sup

n
X

#
ξi (ρi (f (Xi )) − ρi (g(Xi )))f,g∈F i=1

with f := (f1 , . . . , fr )′ .
25"
≲ E supr XnX

f ∈F k=1 i=1#
ωi,k fk (Xi ) ,Proof of Lemma 3. We follow the proof of Theorem 4.1 of van de Geer (2013) and that of Theorem
16.2 of van de Geer (2016). Observe that
"
E

sup

n
X

#
ξi (ρi (f (Xi )) − ρi (g(Xi )))

" "
=E E

n
X

sup

f,g∈F i=1##
ξi (ρi (f (Xi )) − ρi (g(Xi ))) X(n),f,g∈F i=1

where X (n) := (X1 , . . . , Xn )′ . We investigate the tail behavior of a centered, symmetric stochastic
Pprocess ( ni=1 ξi ρi (f (Xi )))f ∈F with X (n) ﬁxed. Note that, for any f, g ∈ F ,nX(ρi (f (Xi )) − ρi (g(Xi )))2 ≤

i=1

n
X

∥f (Xi ) − g(Xi )∥21 ≤ r

i=1

n X
r
X

(fk (Xi ) − gk (Xi ))2 ,(34)

i=1 k=1

where g = (g1 , . . . , gr )′ and the last inequality follows from the Cauchy-Schwarz inequality for∥f (Xi ) − g(Xi )∥1 . For u > 0 and (ρ1 (f (Xi )), . . . , ρn (f (Xn )))′ ̸= (ρ1 (g(X1 )), . . . , ρn (g(Xn )))′ ,Lemma 2.2.7 of van der Vaart and Wellner (1996) yields that
P

n
X

!
ξi (ρi (f (Xi )) − ρi (g(Xi ))) ≥ u X (n)

i=1


 u2
≤ 2 exp − 2nX(ρi (f (Xi )) − ρi (g(Xi )))2

!−1 



i=1

.

(35)
It now follows from (34) and (35) that
P

r−1/2

n
X

!
ξi (ρi (f (Xi )) − ρi (g(Xi ))) ≥ u X (n)

i=1

Pn Pr

k=1 (fk (Xi ) − gk (Xi )) u2≤ 2 exp −
 2

n X
r
X

(fk (Xi ) − gk (Xi ))2

!−1 



i=1 k=1

(36)


2 1/2

. This e is the canonical semi-metric as inPP(2.113) of Talagrand (2021) for a centered Gaussian process ( rk=1 ni=1 ωi,k fk (Xi ))f ∈F with X (n)Let e(f, g) :=i=1

ﬁxed because, for any f, g ∈ F ,

E

r X
n
X



!2
ωi,k (fk (Xi ) − gk (Xi ))

X

(n) 

=

k=1 i=1

r X
nX(fk (Xi ) − gk (Xi ))2 ,k=1 i=1

by the assumption on {ωi,k : 1 ≤ i ≤ n, 1 ≤ k ≤ r}. In view of (36), Theorem 2.10.11 of Talagrand(2021) gives that
"
E

sup

n
X

#
ξi (ρi (f (Xi )) − ρi (g(Xi ))) X(n)f,g∈F i=1

"
≲r

1/2

E sup

r X
n
X

f ∈F k=1 i=1#
ωi,k fk (Xi ) X

Taking the expectation with respect to X (n) completes the proof.
The following lemma plays an important role in the proof of Proposition 1.

26

(n)

.

.

Lemmah 4. Assumei the√assumption of Proposition 1 holds. Then it holds that(a) E ∥Pn ωZ ′ γ∥ΓM ≲ n log d/p,hi′2(b) E Pn ω (Y −X α−D(β+λ))≲ n−1/2 ,
2
2σ
Θ̃
i
h
′
2
(c) E Pn ω (Y −X2σα−Dβ)≲ n−1/2 .
2Θ̃

Proof of Lemma 4. (a). Observe that

E

h

i

′

Pn ωZ γ Γ

M



n
1X

= E

n



!′
ωi Z i ≤ E max Pn ωZ(j) sup ∥γ∥1 .

γ

1≤j≤d

i=1

ΓM

(37)

γ∈ΓM

By Lemma 8 of Chernozhukov et al. (2015),

E max Pn ωZ(j)
1≤j≤d

v

s 
u

n
h
i
X
p
u

1
1
1
2
2
≲ √ t max
E Z(j),ilog d + √E
max
ωi Z(j),i
log d .
1≤j≤d n1≤i≤n,1≤j≤d
n
n
i=1

(38)
For the second term on the right side, the independence of {ωi }ni=1 and {Z(j),i }1≤i≤n,1≤j≤d implies
i



 h
2
that E max1≤i≤n,1≤j≤d (ωi Z(j),i )2 ≤ E max1≤i≤n ωi2 E max1≤i≤n,1≤j≤d Z(j),i. The right side is22further bounded by ∥max1≤i≤n |ωi |∥ψ2 ∥ max1≤i≤n,1≤j≤d |Z(j),i |∥ψ2 by page 95 of van der Vaart and2
Wellner (1996). Here, ∥ · ∥ψ2 is the Orlicz norm for a function ψ2 (x) = ex − 1 as deﬁned on page 95
of van der Vaart and Wellner (1996). Then, by Lemma 2.2.1 and 2.2.2 of van der Vaart and Wellner (1996) in conjunction with Assumption 2(c), ∥max1≤i≤n |ωi |∥2ψ2 ∥ max1≤i≤n,1≤j≤d |Z(j),i |∥2ψ2 ≲log(n + 1) log(nd + 1). Consequently, we obtain
s 
E

max

1≤i≤n,1≤j≤d

ωi Z(j),i

2


≲

p
p
log n log(nd).

(39)

h
i
2
Because E Z(j),iis bounded uniformly in i and j from Lemma 2.2.1 of van der Vaart and Wellner(1996) and Assumption 2(c), it follows from (38) and (39) thatr

E max Pn ωZ(j)
1≤j≤d

≲

√

log d
+
n

log n

r
p
log(n ∨ d) log d
log d
≲
,
n
n

(40)

where the last inequality follows from Assumption 3(b). We complete the proof by (37) and (40)in conjunction with supγ∈ΓM ∥γ∥1 ≲ n/p from Assumption 1′ (d).
(b). Observe that, by the assumption on ω,E(Y − X ′ α − D(β + λ))2
Pn ω
2σ 2




=E

Θ̃

(Y − X ′ α − D(β + λ))2
(Pn − P )ω
2σ 2


.

(41)

Θ̃

Let θ1 = (α1′ , β1 , λ1 , σ12 )′ and θ2 = (α2′ , β2 , λ2 , σ22 )′ ∈ Θ̃ be arbitrary. By the mean value theorem

27and the Cauchy-Schwarz inequality,(Y − X ′ α2 − D(β2 + λ2 ))(Y − X ′ α1 − D(β1 + λ1 ))
−ω
2
2σ1
2σ22
(Y − X ′ ᾱ − D(β̄ + λ̄))≤ ∇θ ω
∥θ1 − θ2 ∥,
2σ̄ 2
ω

(42)

where θ̄ lies on the path connecting θ1 and θ2 . By a straightforward derivative calculation in con′β̄+λ̄))
is bounded by F := C|ω|(|Y |+∥X∥+
junction with Assumption 2(d) and 1′ , ∇θ ω (Y −X ᾱ−D(2σ̄ 21)2 . Taking C suﬃciently large, this F can be an envelope functionandon (see page 84 of van der Vaart′: θ ∈ Θ̃ by AsWellner (1996) for the deﬁnition) for the functional class F := ω (Y −X α−D(β+λ))2σ 2sumption 1′ . Then, in view of (42), Lemma 26 of Kato (2019) yields supQ N (ε∥F ∥Q,2 , F, L2 (Q)) ≤(A/ε)ν for all 0 < ε < 1 with some A ≥ e and ν ≥ 1, where N (·, ·, ·) is a covering number (seepage 84 of van der Vaart and Wellner (1996) for the deﬁnition) and the supremum is taken overall discrete probability measures. Because E[F 2 ] is ﬁnite from Assumption 2(a), it follows from
Corollary 5.1 of Chernozhukov et al. (2014) thatE

(Pn − P )ω

(Y − X ′ α − D(β + λ))2
2σ 2

Θ̃

q 

2
E
max
F
1≤i≤n i1 ≲ n−1/2 ,√≲ √ 1 +nnwhere the second inequality follows from the fact that

(43)

q 
 √ p
E max1≤i≤n Fi2 ≤ n E[F 2 ]. (41) and(43) now complete the proof.
(c). The proof is similar to that of (b) and thus omitted.
Lemma 5 provides a simpliﬁed form of derivatives of the density function for the normal distribution and is cited multiple times in the proof of Proposition 2. This result is essentially due to
Proposition A of Kasahara and Shimotsu (2015).
Lemma 5. Let η = (η1 , . . . , ηq+1 )′ and U = (U(1) , . . . , U(q+1) )′ . Then the following equalities holdfor any nonnegative integer k1 , . . . , kq , kλ and l:∇kη11 . . . ∇ηqq ∇kλλ ∇lσ2 ϕσ (Y − U ′ η − Dλ/2) kλ  l  k+2l


q
′
Y
1
1
kj  Dk+2l Y − U η − Dλ/2

=
U(j)
H
ϕσ (Y − U ′ η − Dλ/2)
2
2
σ
σ
k

j=1

∇kη11 . . . ∇ηqq ∇kλλ ∇lσ2 ϕσ (Y − U ′ η + Dλ/2)    qY−D kλ 1 l 1 k+2l k+2l Y − U ′ η + Dλ/2k=U(j)j ϕσ (Y − U ′ η + Dλ/2),
H
2
2
σ
σ
k

j=1

where k := k1 + · · · + kq + kλ .

28

Proof of Lemma 5. The statement follows from a minor modiﬁcation of the proof of Proposition A
of Kasahara and Shimotsu (2015).
This lemma is the key to showing that the eﬀect of γ̂ ∗ on quadratic approximation for the
penalized log-likelihood function vanishes asymptotically in the proof of Proposition 2.
Lemma 6. Assume the assumption of Proposition 2 holds. Let {Vi }ni=1 be i.i.d. random variables
with ﬁnite second moment and ρ(εi ) be a polynomial of εi . Suppose that {Vi }ni=1 and {εi }ni=1 are
independent. Then, for any k ∈ N, it holds that(a) Pn (2π(Z ′ γ̂ ∗ ) − 1)DH 1 = op (n−3/4 ),(b) Pn V (π(Z ′ γ̂ ∗ ) − 1/2)k ρ(ε) = op (n−1/4 ),(c) Pn D2 (π(Z ′ γ̂ ∗ ) − 1/2)2 ρ(ε) = op (n−1/2 ).
Proof of Lemma 6. (a). By Proposition 1, there exists a sequence rn converging to zero such√that P(n1/4 log d log n∥γ̂ ∗ ∥1 ≥ rn ) → 0. Hence, it suﬃces to show ∥Pn (2π(Z ′ γ) − 1)DH 1 ∥Γn =
op (n−3/4 ), where Γn := γ ∈ Γ : ∥γ∥1 ≤ n−1/4 (log d log n)−1/2 rn . Because E[(2π(Z ′ γ) − 1)DH 1 ] =0, the symmetrization inequality gives that
E

h

i
i
h
Pn (2π(Z ′ γ) − 1)DH 1 Γn ≲ E Pn ξ(2π(Z ′ γ) − 1)DH 1 Γn ,""##n
1 X ξi (π(Z ′ γ) − 1/2)Di εi≲ E max |εi |E sup|D(n) , Z (n) , ε(n) ,1≤i≤nmax1≤i≤n |εi |γ∈Γn ni=1(44)where ξ is a Rademacher random variable independent of (D, Z, ε), and D(n) := (D1 , . . . , Dn ),Z (n) := (Z1 , . . . , Zn ) and ε(n) := (ε1 , . . . , εn ). From Assumption 2(d), we may assume |D| ≤ 1i εiwithout the loss of generality. Then a function φi (t) := (π(t)−1/2)D
max1≤i≤n |εi | is contraction with φi (0) = 0.

It follows from Theorem 4.12 of Ledoux and Talagrand (1991) that"E sup n−1γ∈ΓnnXξi (π(Z ′ γ) − 1/2)Di εii=1

max1≤i≤n |εi |#|D(n) , Z (n) , ε(n) ≤ 2E

hi
Pn ξZ ′ γ Γn |D(n) , Z (n) , ε(n) .
(45)

Combining (44) and (45), we obtainEh′Pn (2π(Z γ) − 1)DH

i

1
Γn











≲ E max |εi |∥Pn ξZ γ∥Γn = E max |εi | E ∥Pn ξZ ′ γ∥Γn , (46)
′

1≤i≤n

1≤i≤n

where the equality follows from the independence of ε and (ξ, Z). For the right side of the equal√
ity, E[max1≤i≤n |εi |] ≲ log n follows from Lemma 2.2.1 and Lemma 2.2.2 of van der Vaart′
andq Wellner (1996) in conjunction with sub-Gaussianity of ε. Additionally, E [∥Pn ξZ γ∥Γn ] ≲log d −1/4(log d log n)−1/2 rn from a similar argument to the proof of (a) in Lemma 4. Those twon nihinequality combined with (46) give that E Pn (2π(Z ′ γ) − 1)DH 1 Γn ≲ rn n−3/4 = o(n−3/4 ). We

now complete the proof by applying Markov’s inequality.
29

(b). Similarly to (a), it suﬃces to show that ∥Pn V (π(Z ′ γ) − 1/2)k ρ(ε)∥Γn = op (n−1/4 ). By themean value theorem and |π(z) − 1/2| ≤ 1 for any z ∈ R,hihi
E ∥Pn V (π(Z ′ γ) − 1/2)k ρ(ε)∥Γn ≲ E Pn |V ||Z ′ γ||ρ(ε)| Γn"#≤ E |V | sup |Z ′ γ||ρ(ε)|
γ∈Γn





≤ E |V | max |Z(j) | E[|ρ(ε)|] sup ∥γ∥1 ,
1≤j≤d

(47)

γ∈Γn

where the last inequality follows from the independence of V and ε. By the Cauchy-Schwarz in√equality, E |V | max1≤j≤d |Z(j) | ≤ (E[|V |2 ])1/2 (E[max1≤j≤d |Z(j) |2 ])1/2 ≲ log d, where the lastinequality follows from the assumption on the moment of V and the argument leading to (39).
Combining this inequality with (47), the ﬁniteness of the moment E[|ρ(ε)|] and supγ∈Γn ∥γ∥1 ≤√n−1/4 (log d log n)−1/2 rn yields E ∥Pn V (π(Z ′ γ) − 1/2)k ρ(ε)∥Γn ≲ rn n−1/4 / log n = o(n−1/4 ). Applying Markov’s inequality completes the proof.
(c). Similarly to (a), it suﬃces to show that ∥Pn D2 (π(Z ′ γ) − 1/2)2 ρ(ε)∥Γn = op (n−1/2 ). By themean value theorem in conjunction with Assumption 2(d),


′



"

#
′

E ∥Pn D (π(Z γ) − 1/2) ρ(ε)∥Γn ≲ E sup |Z γ| |ρ(ε)| ≲ E
2

2

2

γ∈Γn




2
max Z(j)1≤j≤dsup ∥γ∥21 ,γ∈Γnwhere the second inequality follows from the independence of ε and Z, and the ﬁniteness oftheFor the right side, a similar argument to the proof of (b) gives thath moment E[|ρ(ε)|].
i2
E max1≤j≤d Z(j) ≲ log d. Additionally, we have supγ∈Γn ∥γ∥21 ≤ n−1/2 (log d log n)−1 rn2 from
the choice of Γn . Therefore, we arrive at E ∥Pn D2 (π(Z ′ γ) − 1/2)2 ρ(ε)∥Γn ≲ rn2 n−1/2 / log n =o(n−1/2 ). We complete the proof by applying Markov’s inequality.

References
Abadir, K. M. and Magnus, J. R. (2005), Matrix Algebra, Cambridge University Press.
Andrews, D. W. (1993), “Tests for Parameter Instability and Structural Change with Unknown
Change Point,” Econometrica, 61, 821–856.
— (1999), “Estimation when a Parameter is on a Boundary,” Econometrica, 67, 1341–1383.
Andrews, D. W. and Cheng, X. (2012), “Estimation and Inference with Weak, Semi-Strong, andStrong Identiﬁcation,” Econometrica, 80, 2153–2211.
Andrews, D. W. and Ploberger, W. (1994), “Optimal Tests when a Nuisance Parameter is Present
Only under the Alternative,” Econometrica, 62, 1383–1414.

30

Chen, H., Chen, J., and Kalbﬂeisch, J. D. (2001), “A Modiﬁed Likelihood Ratio Test for Homogeneity in Finite Mixture Models,” Journal of the Royal Statistical Society: Series B, 63, 19–29.
Chen, J. (2017), “Consistency of the MLE under Mixture Models,” Statistical Science, 32, 47–63.
Chen, J. and Li, P. (2009), “Hypothesis Test for Normal Mixture Models: The EM Approach,”
Annals of Statistics, 37, 2523–2542.
Chen, J., Li, P., and Fu, Y. (2012), “Inference on the order of a normal mixture,” Journal of theAmerican Statistical Association, 107, 1096–1105.
Chernozhukov, V., Chetverikov, D., and Kato, K. (2014), “Gaussian Approximation of Suprema of
Empirical Processes,” Annals of Statistics, 42, 1564–1597.
— (2015), “Comparison and Anti-Concentration Bounds for Maxima of Gaussian Random Vectors,”
Probability Theory and Related Fields, 162, 47–70.
Davies, R. B. (1977), “Hypothesis Testing when a Nuisance Parameter is Present Only under theAlternative,” Biometrika, 64, 247–254.
— (1987), “Hypothesis Testing when a Nuisance Parameter is Present Only under the Alternative,”
Biometrika, 74, 33–43.
Fan, A., Song, R., and Lu, W. (2017), “Change-Plane Analysis for Subgroup Detection and Sample
Size Calculation,” Journal of the American Statistical Association, 112, 769–778.
Hansen, B. E. (1996), “Inference When a Nuisance Parameter is not Identiﬁed under the Null
Hypothesis,” Econometrica, 64, 413–430.
Huang, Y., Cho, J., and Fong, Y. (2021), “Threshold-Based Subgroup Testing in Logistic Regression
Models in Two-Phase Sampling Designs,” Journal of the Royal Statistical Society. Series C, 70,291.
Jiang, W. and Tanner, M. A. (1999), “On the Identiﬁability of Mixtures-of-Experts,” Neural Networks, 12, 1253–1258.
Jordan, M. I. and Jacobs, R. A. (1994), “Hierarchical Mixtures of Experts and the EM algorithm,”
Neural Computation, 6, 181–214.
Kang, S., Lu, W., and Song, R. (2017), “Subgroup Detection and Sample Size Calculation with
Proportional Hazards Regression for Survival Data,” Statistics in Medicine, 36, 4646–4659.
Kasahara, H., Okimoto, T., and Shimotsu, K. (2014), “Modiﬁed Quasi-Likelihood Ratio Test for
Regime Switching,” The Japanese Economic Review, 65, 25–41.

31

Kasahara, H. and Shimotsu, K. (2015), “Testing the Number of Components in Normal Mixture
Regression Models,” Journal of the American Statistical Association, 110, 1632–1645.
Kato, K. (2019), Lecture notes on empirical process theory, lecture notes available from the author’s
web-page: https://sites.google.com/site/kkatostat/home/research?authuser=0.
Ledoux, M. and Talagrand, M. (1991), Probability in Banach Spaces: Isoperimetry and Processes,
Springer Science & Business Media.
Li, P., Chen, J., and Marriott, P. (2009), “Non-ﬁnite Fisher Information and Homogeneity: an EM
Approach,” Biometrika, 96, 411–426.
Lu, W., Zhang, H. H., and Zeng, D. (2013), “Variable Selection for Optimal Treatment Decision,”
Statistical Methods in Medical Research, 22, 493–504.
R Core Team (2022), R: A Language and Environment for Statistical Computing, R Foundationfor Statistical Computing, Vienna, Austria.
Rotnitzky, A., Cox, D. R., Bottai, M., and Robins, J. (2000), “Likelihood-Based Inference with
Singular Information Matrix,” Bernoulli, 6, 243–284.
Shen, J. and He, X. (2015), “Inference for Subgroup Analysis with a Structured Logistic-Normal
Mixture Model,” Journal of the American Statistical Association, 110, 303–312.
Shen, J., Wang, Y., and He, X. (2017), “Penalized Likelihood for Logistic-Normal Mixture Models
with Unequal Variances,” Statistica Sinica, 27, 711–731.
Song, R., Kosorok, M. R., and Fine, J. P. (2009), “On Asymptotically Optimal Tests under Lossof Identiﬁability in Semiparametric Models,” Annals of Statistics, 37, 2409.
Talagrand, M. (2021), Upper and Lower Bounds for Stochastic Processes: Decomposition Theorems,
Springer, 2nd ed.
van de Geer, S. (2013), “Generic Chaining and the ℓ1-Penalty,” Journal of Statistical Planning andInference, 143, 1001–1012.
— (2016), Estimation and Testing under Sparsity, Springer.
van der Vaart, A. and Wellner, J. A. (1996), Weak Convergence and Empirical Processes: With
Applications to Statistics, Springer Science & Business Media.
— (2000), “Preservation Theorems for Glivenko-Cantelli and Uniform Glivenko-Cantelli Classes,”in High Dimensional Probability II, Springer, pp. 115–133.
Wang, R., Lagakos, S. W., Ware, J. H., Hunter, D. J., and Drazen, J. M. (2007), “Statistics in
Medicine―Reporting of Subgroup Analyses in Clinical Trials,” New England Journal of Medicine,
357, 2189–2194.
32

Wang, Y. (2016), “Logistic-Normal Mixtures with Heterogeneous Components and High Dimensional Covariates.” Ph.D. thesis, Department of Statistics, University of Michigan.
Wu, R.-f., Zheng, M., and Yu, W. (2016), “Subgroup Analysis with Time-to-Event Data under a
Logistic-Cox Mixture Model,” Scandinavian Journal of Statistics, 43, 863–878.
Yoshida, J. and Yoshida, N. (2023), “Penalized Estimation for Non-Identiﬁable Models,” arXiv
preprint arXiv:2301.09131.

33