# A Note on the Adaptive LASSO for Zero-Inflated Poisson Regression

Authors: Prithish Banerjee; Broti Garai; Himel Mallick; Shrabanti Chowdhury; Saptarshi Chatterjee
Journal: Journal of Probability and Statistics
 (2018)

---

## Abstract

We consider the problem of modelling count data with excess zeros using Zero-Inflated Poisson (ZIP) regression. Recently, various regularization methods have been developed for variable selection in ZIP models. Among these, EM LASSO is a popular method for simultaneous variable selection and parameter estimation. However, EM LASSO suffers from estimation inefficiency and selection inconsistency. To remedy these problems, we propose a set of EM adaptive LASSO methods using a variety of data-adaptive weights. We show theoretically that the new methods are able to identify the true model consistently, and the resulting estimators can be as efficient as oracle. The methods are further evaluated through extensive synthetic experiments and applied to a German health care demand dataset.


---

## Body

## 1. Introduction

Modern research studies routinely collect information on a broad array of outcomes including count measurements with excess amount of zeros. Modeling such zero-inflated count outcomes is challenging for several reasons. First, traditional count models such as Poisson and Negative Binomial are suboptimal in accounting for excess variability due to zero-inflation [1, 2]. Second, alternative zero-inflated models such as theZero-InflatedPoisson (ZIP) [2] andZero-InflatedNegativeBinomial (ZINB) [1] models are computationally prohibitive in the presence of high-dimensional and collinear variables.Regularization methods have been proposed as a powerful framework to mitigate these problems, which tend to exhibit significant advantages over traditional methods [3, 4]. Essentially all these methods enforce sparsity through a suitable penalty function and identify predictive features by means of a computationally efficient Expectation Maximization (EM) algorithm. Among these, EM LASSO is particularly attractive due to its capability to perform simultaneous model selection and stable effect estimation. However, recent research suggests that EM LASSO may not be fully efficient and its model selection result could be inconsistent [5, 6]. This led to a simple modification of the LASSO penalty, namely, the EM adaptive LASSO (EM AL). EM AL achieves “oracle selection consistency” by allowing different amounts of shrinkage for different regression coefficients.Previous studies have not, however, investigated the EM AL at sufficient depth to evaluate its properties under diversified and realistic scenarios. It is not yet clear, for example, how reliable the resulting parameter estimates are in the presence of multicollinearity. In particular, the actual variable selection performance of EM AL depends on the proper construction of the data-adaptive weight vector. When the features to be associated possess an inherent collinearity, EM AL is expected to produce suboptimal results, a phenomenon that is especially evident when the sample size is limited [7]. Several remedies have been suggested for linear and generalized linear models (GLMs) such as the standard error-adjusted adaptive LASSO (SEAL) [7, 8]. However, there is a lack of similar published methods for zero-inflated count regression models. In addition, complete software packages of these methods have not been made available to the community.We address these issues by providing a set of flexible variable selection approaches to efficiently identify correlated features associated with zero-inflated count outcomes in a ZIP regression framework. We have implemented this method as AMAZonn (A Multicollinearity-adjustedAdaptive LASSO forZero-inflated Count Regression). AMAZonn considers two data-adaptive weights: (i) the inverse of the maximum likelihood (ML) estimates (EM AL) and (ii) inverse of the ML estimates divided by their standard errors (EM SEAL). We show theoretically that AMAZonn is able to identify the true model consistently, and the resulting estimator is as efficient as oracle. Numerical studies confirmed our theoretical findings. The rest of the article is organized as follows. The AMAZonn method is proposed in the next section, and its theoretical properties are established in Section 3. Simulation results are reported in Section 4 and one real dataset is analyzed in Section 5. Then, the article concludes with a short discussion in Section 6. All technical details are presented in the Appendix.


## 2. Methods

### 2.1. Zero-Inflated Poisson (ZIP) Model

Zero-inflated count models assume that the observations originate either from a “susceptible” population that generates zero and positive counts according to a count distribution or from a “nonsusceptible” population, which produces additional zeros [1, 2]. Thus, while a subject with a positive count is considered to belong to the “susceptible” population, individuals with zero counts may belong to either of the two latent populations. We denote the observed values of the response variable as y
=
y
1
,
y
2
,
…
,
y
n
′. Following Lambert [2], a ZIP mixture distribution can be written as (1)
P
y
i
=
k
=
p
i
+
1
-
p
i
e
-
λ
i
if
k
=
0
,
1
-
p
i
e
-
λ
i
λ
i
k
k
!
if
k
=
1,2
,
…
,where p
i is the probability of belonging to the nonsusceptible population and λ
i is the Poisson mean corresponding to the susceptible population for the i
t
h individual (i
=
1
,
…
,
n). It can be seen from (1) that ZIP reduces to the standard Poisson model when p
i
=
0. Also, P
y
i
=
0
>
e
-
λ
i, indicating zero-inflation. The probability of belonging to the “nonsusceptible” population, p
i, and the Poisson mean, λ
i, are linked to the explanatory variables through the logit and log links as (2)
l
o
g
i
t
p
i
=
z
i
′
γ
and
(3)
log
⁡
λ
i
=
x
i
′
β
,where x
i and z
i are vectors of covariates for the ith subject (i
=
1
,
…
,
n) corresponding to the count and zero models, respectively, and γ
=
γ
0
,
γ
1
,
…
,
γ
q
′ and β
=
β
0
,
β
1
,
…
,
β
p
′ are the corresponding regression coefficients including the intercepts.Forn independent observations, the ZIP log-likelihood function can be written as (4)
L
β
,
γ
=
∑
y
i
=
0
log
⁡
e
z
i
′
γ
+
e
-
e
x
i
′
β
+
∑
y
i
>
0
y
i
x
i
′
β
+
e
-
x
i
′
β
-
∑
i
=
1
n
log
⁡
1
+
e
z
i
′
γ
-
∑
y
i
>
0
log
⁡
y
i
!
.


### 2.2. The AMAZonn Method

AMAZonn considers two data-adaptive weights in the EM adaptive LASSO framework: (i) the inverse of the maximum likelihood (ML) estimates (EM AL) and (ii) inverse of the ML estimates divided by their standard errors (EM SEAL). As defined by Tang et al. [6], the EM adaptive LASSO formulation for ZIP regression is given by (5)
θ
^
∗
=
arg
⁡
min
⁡
-
L
θ
+
ν
1
∑
j
=
1
p
w
1
j
β
j
+
ν
2
∑
j
=
1
p
w
2
j
γ
j
,where θ
=
β
,
γ is the parameter vector of interest with known weights w
1
=
w
11
,
…
,
w
1
p
′ and w
2
=
w
21
,
…
,
w
2
p
′. As noted by Qian and Yang [7], the inverse of the maximum likelihood (ML) estimates as weights may not always be stable, especially when the multicollinearity of the design matrix is a concern. In order to adjust for this instability, AMAZonn additionally considers the inverse of the ML estimates divided by their standard errors as weights. We refer to these two methods as AMAZonn - EM AL and AMAZonn - EM SEAL, respectively (Table 1).Table 1
The AMAZonn data-adaptive weights.β
^
M
L and γ
^
M
L denote the ML estimates based on the unpenalized ZIP model, corresponding to count and zero submodels, respectively. SE denotes the standard errors of the corresponding ML estimates.
Weighting Scheme
Count
Zero
AMAZonn - EM AL
1
β
^
j
M
L
1
γ
^
j
M
L
AMAZonn - EM SEAL
S
E
β
^
j
M
L
β
^
j
M
L
S
E
γ
^
j
M
L
γ
^
j
M
L


### 2.3. The EM Algorithm

In order to efficiently estimate the parameters in the above optimization problem (5), we resort to the EM algorithm. To this end, we define a set of latent variables z
i as follows: (6)
z
i
=
1
if
y
i
is  from  the  zero  state,  and
z
i
=
0
if
y
i
is  from  the  count  state,
i
=
1
,
…
,
n
.We consider the latent variables z
i’s as the “missing data" and rewrite the complete-data log-likelihood function in (4) as follows: (7)
L
θ
=
∑
i
=
1
n
z
i
X
i
γ
-
log
⁡
1
+
exp
⁡
X
i
γ
+
1
-
z
i
y
i
X
i
β
-
y
i
+
1
log
⁡
1
+
X
i
β
.With the above formulation, the objective function in (5) can be rewritten as (8)
Q
∗
θ
=
-
L
θ
+
ν
1
∑
j
=
1
p
w
1
j
β
j
+
ν
2
∑
j
=
1
p
w
2
j
γ
j
,which can be iteratively solved as follows: (1)
At iteration t, theE step computes the expectation of Q
∗
(
θ
) by substituting z
i with its conditional expectation given observed data and current parameter estimates (9)
z
^
i
t
=
1
+
exp
⁡
-
X
i
γ
^
t
1
+
exp
⁡
-
X
i
β
^
t
if
y
i
=
0
,
0
if
y
i
>
0
.
(2)
In theM step, the expected penalized complete-data log-likelihood (5) can be minimized the with respect to θ as (10)
Q
∗
θ
∣
θ
t
=
-
2
E
(
L
θ
∣
θ
t
+
ν
1
∑
j
=
1
p
w
1
j
β
j
+
ν
2
∑
j
=
1
p
w
2
j
γ
j
.
(3)
Continue this process until convergence,t
=
1,2
,
….It is to be noted that (10) can be further decomposed as (11)
Q
∗
θ
∣
θ
t
=
Q
1
∗
β
∣
θ
t
+
Q
2
∗
γ
∣
θ
t
,where Q
1
∗ is the weighted penalized Poisson log-likelihood defined as (12)
Q
1
∗
β
∣
θ
t
=
-
2
∑
i
=
1
n
1
-
z
^
i
t
y
i
X
i
β
-
y
i
+
1
log
⁡
1
+
X
i
β
+
ν
1
∑
j
=
1
p
w
1
j
β
j
,and Q
2
∗ is the penalized logistic log-likelihood defined as (13)
Q
2
∗
γ
∣
θ
t
=
-
2
∑
i
=
1
n
z
^
i
t
X
i
γ
-
log
⁡
1
+
exp
⁡
X
i
γ
+
ν
2
∑
j
=
1
p
w
2
j
γ
j
,both of which can be minimized separately using computationally efficient coordinate descent algorithms developed for GLMs [9].


### 2.4. Selection of Tuning Parameters

We select the tuning parameters based on the minimum BIC [10] criterion, which is known to provide better variable selection performance as compared to other information criteria [11]. This can be effortlessly incorporated in our formulation by using existing implementations for zero-inflated count models [3, 4, 6].


## 2.1. Zero-Inflated Poisson (ZIP) Model

Zero-inflated count models assume that the observations originate either from a “susceptible” population that generates zero and positive counts according to a count distribution or from a “nonsusceptible” population, which produces additional zeros [1, 2]. Thus, while a subject with a positive count is considered to belong to the “susceptible” population, individuals with zero counts may belong to either of the two latent populations. We denote the observed values of the response variable as y
=
y
1
,
y
2
,
…
,
y
n
′. Following Lambert [2], a ZIP mixture distribution can be written as (1)
P
y
i
=
k
=
p
i
+
1
-
p
i
e
-
λ
i
if
k
=
0
,
1
-
p
i
e
-
λ
i
λ
i
k
k
!
if
k
=
1,2
,
…
,where p
i is the probability of belonging to the nonsusceptible population and λ
i is the Poisson mean corresponding to the susceptible population for the i
t
h individual (i
=
1
,
…
,
n). It can be seen from (1) that ZIP reduces to the standard Poisson model when p
i
=
0. Also, P
y
i
=
0
>
e
-
λ
i, indicating zero-inflation. The probability of belonging to the “nonsusceptible” population, p
i, and the Poisson mean, λ
i, are linked to the explanatory variables through the logit and log links as (2)
l
o
g
i
t
p
i
=
z
i
′
γ
and
(3)
log
⁡
λ
i
=
x
i
′
β
,where x
i and z
i are vectors of covariates for the ith subject (i
=
1
,
…
,
n) corresponding to the count and zero models, respectively, and γ
=
γ
0
,
γ
1
,
…
,
γ
q
′ and β
=
β
0
,
β
1
,
…
,
β
p
′ are the corresponding regression coefficients including the intercepts.Forn independent observations, the ZIP log-likelihood function can be written as (4)
L
β
,
γ
=
∑
y
i
=
0
log
⁡
e
z
i
′
γ
+
e
-
e
x
i
′
β
+
∑
y
i
>
0
y
i
x
i
′
β
+
e
-
x
i
′
β
-
∑
i
=
1
n
log
⁡
1
+
e
z
i
′
γ
-
∑
y
i
>
0
log
⁡
y
i
!
.


## 2.2. The AMAZonn Method

AMAZonn considers two data-adaptive weights in the EM adaptive LASSO framework: (i) the inverse of the maximum likelihood (ML) estimates (EM AL) and (ii) inverse of the ML estimates divided by their standard errors (EM SEAL). As defined by Tang et al. [6], the EM adaptive LASSO formulation for ZIP regression is given by (5)
θ
^
∗
=
arg
⁡
min
⁡
-
L
θ
+
ν
1
∑
j
=
1
p
w
1
j
β
j
+
ν
2
∑
j
=
1
p
w
2
j
γ
j
,where θ
=
β
,
γ is the parameter vector of interest with known weights w
1
=
w
11
,
…
,
w
1
p
′ and w
2
=
w
21
,
…
,
w
2
p
′. As noted by Qian and Yang [7], the inverse of the maximum likelihood (ML) estimates as weights may not always be stable, especially when the multicollinearity of the design matrix is a concern. In order to adjust for this instability, AMAZonn additionally considers the inverse of the ML estimates divided by their standard errors as weights. We refer to these two methods as AMAZonn - EM AL and AMAZonn - EM SEAL, respectively (Table 1).Table 1
The AMAZonn data-adaptive weights.β
^
M
L and γ
^
M
L denote the ML estimates based on the unpenalized ZIP model, corresponding to count and zero submodels, respectively. SE denotes the standard errors of the corresponding ML estimates.
Weighting Scheme
Count
Zero
AMAZonn - EM AL
1
β
^
j
M
L
1
γ
^
j
M
L
AMAZonn - EM SEAL
S
E
β
^
j
M
L
β
^
j
M
L
S
E
γ
^
j
M
L
γ
^
j
M
L


## 2.3. The EM Algorithm

In order to efficiently estimate the parameters in the above optimization problem (5), we resort to the EM algorithm. To this end, we define a set of latent variables z
i as follows: (6)
z
i
=
1
if
y
i
is  from  the  zero  state,  and
z
i
=
0
if
y
i
is  from  the  count  state,
i
=
1
,
…
,
n
.We consider the latent variables z
i’s as the “missing data" and rewrite the complete-data log-likelihood function in (4) as follows: (7)
L
θ
=
∑
i
=
1
n
z
i
X
i
γ
-
log
⁡
1
+
exp
⁡
X
i
γ
+
1
-
z
i
y
i
X
i
β
-
y
i
+
1
log
⁡
1
+
X
i
β
.With the above formulation, the objective function in (5) can be rewritten as (8)
Q
∗
θ
=
-
L
θ
+
ν
1
∑
j
=
1
p
w
1
j
β
j
+
ν
2
∑
j
=
1
p
w
2
j
γ
j
,which can be iteratively solved as follows: (1)
At iteration t, theE step computes the expectation of Q
∗
(
θ
) by substituting z
i with its conditional expectation given observed data and current parameter estimates (9)
z
^
i
t
=
1
+
exp
⁡
-
X
i
γ
^
t
1
+
exp
⁡
-
X
i
β
^
t
if
y
i
=
0
,
0
if
y
i
>
0
.
(2)
In theM step, the expected penalized complete-data log-likelihood (5) can be minimized the with respect to θ as (10)
Q
∗
θ
∣
θ
t
=
-
2
E
(
L
θ
∣
θ
t
+
ν
1
∑
j
=
1
p
w
1
j
β
j
+
ν
2
∑
j
=
1
p
w
2
j
γ
j
.
(3)
Continue this process until convergence,t
=
1,2
,
….It is to be noted that (10) can be further decomposed as (11)
Q
∗
θ
∣
θ
t
=
Q
1
∗
β
∣
θ
t
+
Q
2
∗
γ
∣
θ
t
,where Q
1
∗ is the weighted penalized Poisson log-likelihood defined as (12)
Q
1
∗
β
∣
θ
t
=
-
2
∑
i
=
1
n
1
-
z
^
i
t
y
i
X
i
β
-
y
i
+
1
log
⁡
1
+
X
i
β
+
ν
1
∑
j
=
1
p
w
1
j
β
j
,and Q
2
∗ is the penalized logistic log-likelihood defined as (13)
Q
2
∗
γ
∣
θ
t
=
-
2
∑
i
=
1
n
z
^
i
t
X
i
γ
-
log
⁡
1
+
exp
⁡
X
i
γ
+
ν
2
∑
j
=
1
p
w
2
j
γ
j
,both of which can be minimized separately using computationally efficient coordinate descent algorithms developed for GLMs [9].


## 2.4. Selection of Tuning Parameters

We select the tuning parameters based on the minimum BIC [10] criterion, which is known to provide better variable selection performance as compared to other information criteria [11]. This can be effortlessly incorporated in our formulation by using existing implementations for zero-inflated count models [3, 4, 6].


## 3. Oracle Properties

Recently, Tang et al. [6] showed that the EM adaptive LASSO (i.e., AMAZonn - EM AL) enjoys the so-called oracle properties, i.e., the estimator is able to identify the true model consistently, and the resulting estimator is as efficient asoracle. Here we extend these results to the AMAZonn - EM SEAL estimator and show that the AMAZonn - EM SEAL estimator also maintains the same theoretical properties. For the sake of completeness, we provide a combined general proof for both AMAZonn estimators.Without being too rigorous mathematically, recall that the log-likelihood function for the ZIP regression model is given by(14)
L
θ
;
υ
i
=
∑
y
i
=
0
log
⁡
ψ
i
+
1
-
ψ
i
f
0
;
λ
i
+
∑
y
i
>
0
log
⁡
1
-
ψ
i
f
y
i
;
λ
i
,where υ
i’s are the observed data (i.i.d observations from the ZIP distribution), f
(
.
;
λ
i
) is the probability mass function of Poisson distribution with parameter λ
i
=
exp
⁡
(
X
i
β
) and ψ
i
=
e
x
p
(
X
i
γ
)
/
1
+
e
x
p
(
X
i
γ
), i
=
1
,
…
,
n. The corresponding penalized log-likelihood is given by (15)
Q
θ
=
-
L
θ
;
υ
i
+
ν
1
n
∑
j
=
1
p
w
1
j
β
j
+
ν
2
n
∑
j
=
1
p
w
2
j
γ
j
.Let us denote the true coefficient vector as θ
0
=
(
β
0
T
,
γ
0
T
)
T. Decompose θ
0
=
(
θ
10
T
,
θ
20
T
)
T and assume that θ
20
T contains all zero coefficients. Let us denote the subset of true nonzero coefficients as A
=
j
:
θ
j
0
≠
0 and the subset of selected nonzero coefficients as A
^
=
j
:
θ
^
j
≠
0. With this formulation, the Fisher information matrix can be written as (16)
I
θ
0
=
I
11
I
12
I
21
I
22
,where I
11 is the Fisher information corresponding the true nonzero submodel. The oracle property of AMAZonn may be developed based on certain mild regularity conditions which are as follows: (A1):
The Fisher information matrixI
(
θ
) is finite and positive definite for all values of θ.
(A2):
There exists functionsG
j
k
l such that (17)
∂
3
L
θ
;
υ
i
∂
θ
j
∂
θ
k
∂
θ
l
≤
G
j
k
l
υ
i
∀
θ
,
whereg
j
k
l
=
E
θ
0
(
G
j
k
l
(
υ
i
)
)
<
∞ for all j
,
k
,
l.Theorem 1.
Under (A1) and (A2), ifν
1
n
→
∞, ν
2
n
→
∞, ν
1
n
/
n
→
0, ν
2
n
/
n
→
0, then the AMAZonn estimators obey the following oracle properties: (1)
consistency in variable selection:l
i
m
n
P
(
A
^
=
A
)
=
1, and
(2)
asymptotic normality of the nonzero coefficients:n
(
θ
^
-
θ
0
)
→
d
N
(
0
,
I
11
-
1
).


## 4. Simulation Studies

In this section, we conduct simulation studies to evaluate the finite sample performance of AMAZonn. For comparison purposes, the performance of both AMAZonn and EM LASSO is evaluated. For each simulated dataset, the associated tuning parameters are selected by the minimum BIC criterion for all the methods under consideration. All the examples reported in this section are obtained from published papers with slight modifications within the scope of the current study [11, 12].Specially, three scenarios are considered: in the data generating models of Simulations1 and 2, we consider all continuous predictors, whereas in Simulation 3, both continuous and categorical variables are included. For each experimental instance, we randomly partition the data into training and test sets: models are fitted on the training set and prediction errors based on mean absolute scaled error (MASE) are calculated on the held-out samples in the test set. For an exhaustive comparison, we considered three sets of sample sizes {
n
T
,
n
P
}
=
{
200,200
}
,
{
500,500
}, and {
1000,1000
}, where n
T and n
P represent the size of the training and test data, respectively. The corresponding regression coefficients and intercepts are chosen so that a desired level of sparsity proportion ϕ is achieved. In order to remain as model-agnostic as possible, we consider the same set of predictors for both zero and count submodels (i.e., X
=
Z). Such models are common in many practical applications where no domain-specific prior information about the zero-inflation mechanism is available. Below we provide the detailed data generation steps for both simulation examples:Simulation 1.
(1)
Generate40 predictors from the multivariate normal distribution with mean vector 0, variance vector 1, and variance-covariance matrix V, where the elements of V are ρ
j
1
-
j
2
∀
j
1
≠
j
2
=
1
,
…
,
40. The values of pairwise correlation ρ varies from 0 (uncorrelated) to 0.4 (moderate collinearity) to 0.8 (high collinearity).
(2)
The count and zero regression parameters are chosen as follows:(18)
β
1
,
…
,
β
8
=
-
1
,
-
0.5
,
-
0.25
,
-
0.1,0.1,0.25,0.5,0.75
′
,
β
9
,
…
,
β
16
=
0.2
,
…
,
0.2
′
,
β
17
,
…
,
β
40
=
0
,
…
,
0
′
,
γ
1
,
…
,
γ
8
=
-
0.4
,
-
0.3
,
-
0.2
,
-
0.1,0.1,0.2,0.3,0.4
′
,
γ
9
,
…
,
γ
16
=
0.2
,
…
,
0.2
′
,
γ
17
,
…
,
γ
40
=
0
,
…
,
0
′
.
(3)
The zero-inflated count outcomey is simulated according to (1) with the above parameters and input data.Simulation 2.
It is similar to Simulation1 except that the count and zero regression parameters are chosen as follows: (19)
β
1
,
…
,
β
10
=
0.05
,
-
0.25,0.05,0.25
,
-
0.15,0.15,0.25
,
-
0.2,0.25
,
-
0.25
′
,
β
11
,
…
,
β
30
=
-
0.2,0.25,0.15
,
-
0.25,0.2,0
,
…
,
0
′
,
β
31
,
…
,
β
40
=
0.27
,
-
0.27,0.14,0.2
,
-
0.2,0.2,0
,
…
,
0
′
,
γ
1
,
…
,
γ
10
=
-
0.5
,
-
0.4
,
-
0.3
,
-
0.2
,
-
0.1,0.1,0.2,0.3,0.4,0.5
′
,
γ
11
,
…
,
γ
30
=
-
0.2,0.25,0.15
,
-
0.25,0.2,0
,
…
,
0
′
,
γ
31
,
…
,
γ
40
=
0.27
,
-
0.27
,
-
0.14
,
-
0.2
,
-
0.2,0.2,0
,
…
,
0
′
.Simulation 3.
(1)
First simulateX
1
,
…
,
X
6 independently from the standard normal distribution. Consider the following as the continuous predictors: {
X
1
}
,
{
X
2
}
,
{
X
3
,
X
3
2
,
X
3
3
}
,
{
X
4
}
,
{
X
5
} and {
X
6
,
X
6
2
,
X
6
3
}.
(2)
Simulate 5 continuous variables from the multivariate normal distribution with mean0, variance 1, and AR(ρ) correlation structure for varying ρ in {
0,0.4,0.8
} as before, and quantile-discretize each of them into 5 new variables based on their quantiles: -
∞
,
Φ
-
1
1
/
5, Φ
-
1
1
/
5
,
Φ
-
1
2
/
5, Φ
-
1
2
/
5
,
Φ
-
1
3
/
5, Φ
-
1
3
/
5
,
Φ
-
1
4
/
5, and Φ
-
1
4
/
5
,
∞, leading to a total of 20 categorical variables.
(3)
With the above input data and parameters, the zero-inflated count outcomey is simulated according to (1), where the two sets of regression parameters are chosen as follows: (20)
β
1
,
…
,
β
10
=
0,0
,
0.1,0.2,0.1,0
,
0
,
2
3
,
-
1
,
1
3
,
β
11
,
…
,
β
30
=
-
2
,
-
1,1
,
2,0
,
…
,
0
,
γ
1
,
…
,
γ
10
=
0,0
,
0.1,0.2,0.1,0
,
0
,
2
3
,
-
1
,
1
3
,
γ
11
,
…
,
γ
30
=
-
2
,
-
1,1
,
2,0
,
…
,
0
.The resulting performance measures iterated over 200 replications (Table 2) reveal that AMAZonn performs as well as or better than EM LASSO in most of the simulation scenarios. For highly collinear designs, AMAZonn - EM SEAL stands out to be the best estimator for almost every sample size and zero-inflation proportion, highlighting the benefit of incorporating data-adaptive weights based on both ML estimates and their standard errors. This phenomenon is also apparent in the analysis of German health care data in Section 5, where the parameter estimates from the AMAZonn - EM SEAL method appear to be more parsimonious than those from other methods.Table 2
Results of Simulations1–3. Average (over 200 replications) of Mean Absolute Scale Errors (MASEs) of AMAZonn and EM LASSO is reported.
ρ
ϕ
n
Simulation1
Simulation2
Simulation3
AMAZonn - EM SEAL
AMAZonn - EM AL
EM LASSO
AMAZonn - EM SEAL
AMAZonn - EM AL
EM LASSO
AMAZonn - EM SEAL
AMAZonn - EM AL
EM LASSO
0.0
0.3
200
0.91
0.92
0.91
0.60
0.61
0.62
0.97
1.03
1.00
500
0.90
0.90
0.91
0.60
0.60
0.61
0.97
0.99
1.00
1000
0.91
0.91
0.92
0.58
0.58
0.60
0.97
0.98
0.98
0.4
200
1.12
1.13
1.12
0.75
0.75
0.76
1.18
1.23
1.23
500
1.05
1.05
1.06
0.73
0.73
0.74
1.11
1.17
1.20
1000
1.03
1.03
1.04
0.71
0.71
0.72
1.11
1.16
1.16
0.5
200
1.28
1.28
1.27
0.87
0.87
0.87
1.40
1.46
1.46
500
1.16
1.16
1.17
0.84
0.84
0.85
1.28
1.33
1.36
1000
1.15
1.15
1.19
0.80
0.80
0.82
1.23
1.30
1.31
0.4
0.3
200
1.05
1.06
1.09
0.63
0.63
0.63
0.96
1.01
0.99
500
1.04
1.04
1.05
0.61
0.61
0.62
0.95
0.97
0.99
1000
0.96
0.96
0.98
0.58
0.58
0.59
0.97
0.98
0.98
0.4
200
1.21
1.22
1.22
0.75
0.75
0.76
1.19
1.22
1.23
500
1.18
1.18
1.21
0.71
0.71
0.72
1.14
1.19
1.22
1000
1.13
1.14
1.18
0.68
0.68
0.70
1.13
1.18
1.17
0.5
200
1.42
1.43
1.42
0.83
0.84
0.83
1.34
1.40
1.43
500
1.26
1.26
1.32
0.80
0.81
0.82
1.27
1.32
1.35
1000
1.23
1.23
1.30
0.75
0.75
0.77
1.27
1.34
1.33
0.8
0.3
200
1.32
1.31
1.36
0.62
0.63
0.63
0.96
1.00
1.01
500
1.13
1.13
1.23
0.59
0.59
0.61
0.97
0.99
1.01
1000
1.13
1.13
1.21
0.56
0.56
0.58
0.95
0.96
0.96
0.4
200
1.52
1.52
1.58
0.71
0.72
0.72
1.18
1.21
1.23
500
1.31
1.32
1.45
0.68
0.68
0.69
1.12
1.19
1.20
1000
1.24
1.24
1.37
0.64
0.64
0.64
1.12
1.17
1.16
0.5
200
1.56
1.58
1.61
0.78
0.78
0.78
1.37
1.42
1.44
500
1.44
1.45
1.65
0.73
0.73
0.76
1.29
1.34
1.39
1000
1.33
1.36
1.52
0.69
0.70
0.69
1.26
1.33
1.34


## 5. Application to German Health Care Demand Data

Next, we apply our method to the German health care demand data [3], a subset of the German Socioeconomic Panel (GSOEP) dataset [13], which has also been used for illustration purposes in previous studies [3, 14]. The original data contains number of doctor office visits for 1,812 West German men aged 25 to 65 years in the last three months of 1994 (response variable of interest), which is supplemented with complementary information on twelve annual waves from 1984 to 1995 including health care utilization, current employment status, and insurance arrangements under which subjects are protected [3]. The goal of the original study was to investigate how the employment characteristics of the German nationals are related to their health care demand. The distribution of the dependent variable (Figure 1) reveals that many doctor visits are zeros (41.2
%), confirming that classical methods such as Poisson regression are inappropriate for modeling this outcome.Figure 1
Number of doctor office visits in the German health care data.In the model fitting process, along with the original variables, the interactions between age groups and health condition are also considered, resulting in 28 candidate predictors (Table3). The fitting results from the full models indicate that both EM adaptive LASSO methods provide competitive model selection performance (Table 4), often leading to sparser model selection than EM LASSO (Table 5). In addition, the AMAZonn - EM SEAL method appears to choose even fewer numbers of variables. Such feature of AMAZonn - EM SEAL can be appealing in many practical situations, where data collinearity between variables is a concern and a more aggressive feature selection is desired. While the computational overheads of both EM adaptive LASSO methods are similar, they are an order of magnitude faster than EM LASSO (Table 4), further confirming that AMAZonn offers a viable alternative to existing methods.Table 3
Summary of predictors in German health care demand data.
Variables
Mean (sd) or Frequency
Description
health
6.84 (2.19)
health satisfaction:
0 (low) - 10 (high)
handicap
216 / 1596
1
: handicap, 0
: otherwise
hdegree
6.16 (18.49)
degree of handicap in percentage points
married
1257 / 555
1
: married, 0
: otherwise
schooling
11.83 (2.49)
years of schooling
hhincome
4.52 (2.13)
household income per month in German marks/1000
children
703 / 1109
1
: children under 16 in household, 0
: otherwise
self
153 / 1659
1
: self-employed, 0
: otherwise
civil
198 / 1614
1
: civil servant, 0
: otherwise
bluec
566 / 1246
1
: blue collar employee, 0
: otherwise
employed
1506 / 306
1
: employed, 0
: otherwise
public
1535 / 277
1
: public health insurance, 0
: otherwise
addon
33 / 1779
1
: addon insurance, 0
: otherwise
age30
1480 / 332
1 if age ≥ 30
age35
1176 / 636
1 if age ≥ 35
age40
919 / 893
1 if age ≥ 40
age45
716 / 1096
1 if age ≥ 45
age50
535 / 1227
1 if age ≥ 50
age55
351 / 1461
1 if age ≥ 55
age60
147 / 1665
1 if age ≥ 60Table 4
Model selection performance of EM LASSO and AMAZonn on German health care data.
Methods
BIC
Time (in seconds)
EM LASSO
9062.744
50.252
AMAZonn - EM AL
9002.487
26.215
AMAZonn - EM SEAL
8982
.
924
26.528Table 5
Estimated coefficients from the best-fitting ZIP models in German health care demand data analysis.
(a)
Methods
Count Coefficients
(Intercept)
hlth
handicap
ddegree
married
schooling
hhincome
children
self
civil
bluec
employed
public
addon
EM LASSO
2.322
-0.14
0.207
-0.002
-0.97
0.0
0.0
0.078
-0.178
-0.166
0.038
-0.106
0.089
0.205
AMAZonn - EM AL
2.305
-0.135
0.111
0.0
-0.947
0.0
0.0
0.079
-0.234
-0.245
0.0
-0.059
0.043
0.205
AMAZonn - EM SEAL
2.378
-0.142
0.098
0.0
-0.066
0.0
0.0
0.046
-0.189
-0.222
0.0
-0.055
0.0
0.14
Methods
Count Coefficients
ag30
ag35
ag40
ag45
ag50
ag55
ag60
ag30:hlth
ag35:hlth
ag40:hlth
ag45:hlth
ag50:hlth
ag55:hlth
ag60:hlth
EM LASSO
0.0
0.0
0.0
0.586
0.0
-0.27
0.081
0.0
0.0
-0.006
-0.076
0.006
0.082
-0.034
AMAZonn - EM AL
0.0
0.0
-0.047
0.769
0.0
-0.402
0.099
0.0
0.0
0.0
-0.101
0.0
0.106
-0.034
AMAZonn - EM SEAL
0.0
0.0
0.0
0.586
0.0
-0.25
0.0
0.0
0.0
0.0
-0.081
0.0
0.081
-0.017
(b)
Methods
Zero Coefficients
(Intercept)
hlth
handicap
ddegree
married
schooling
hhincome
children
self
civil
bluec
employed
public
addon
EM LASSO
-2.193
-0.262
-0.098
-0.003
-0.121
0.0
-0.012
0.253
0.112
0.134
0.0
0.0
-0.012
0.0
AMAZonn - EM AL
-2.226
-0.261
-0.162
0.0
0.0
0.0
0.0
0.163
0.0
0.0
0.0
0.0
0.0
0.0
AMAZonn - EM SEAL
-2.403
-0.283
0.0
0.0
-0.053
0.0
0.0
0.238
0.0
0.0
0.0
0.0
0.0
0.0
Methods
Zero Coefficients
ag30
ag35
ag40
ag45
ag50
ag55
ag60
ag30:hlth
ag35:hlth
ag40:hlth
ag45:hlth
ag50:hlth
ag55:hlth
ag60:hlth
EM LASSO
0.0
0.0
0.0
0.0
-0.459
0.0
-0.217
0.013
0.0
0.005
0.0
0.0
0.023
0.0
AMAZonn - EM AL
0.047
0.0
0.065
0.009
-0.527
0.0
-0.198
0.0
0.0
0.0
0.0
0.0
0.0
0.0
AMAZonn - EM SEAL
0.0
0.0
0.0
0.0
-0.443
0.0
0.0
0.009
0.0
0.0
0.0
0.0
0.0
0.0


## 6. Discussion

In recent years, there has been a huge influx of zero-inflated count measurements spanning several disciplines including biology, public health, and medicine. This has motivated the widespread use of zero-inflated count models in many practical applications such as metagenomics, single-cell RNA sequencing, and health care research. In this article, we propose the AMAZonn method for adaptive variable selection in ZIP regression models. Both our simulation and real data experience suggest that AMAZonn can outperform EM LASSO under a variety of regression settings while maintaining the desired theoretical properties and computational convenience. Our preliminary results are rather encouraging, and for practical purposes, we provide a publicly available R package implementing this method:https://github.com/himelmallick/AMAZonn.We envision a number of improvements that may further refine AMAZonn’s performance. While AMAZonn relies on ML estimates to construct the weight vector, these estimates may not be available in ultrahigh dimensions [7]. Alternative initialization schemes could further improve on this such as the ridge estimates [15]. Extension to other zero-inflated models such as marginalized zero-inflated count regression [16, 17], two-part and hurdle models [18], and multiple-inflation models [19] can form a useful basis for further investigations. Although we only focused on variable selection for fixed effects models, future work could include an extension to other regularization problems such as grouped variable selection [12, 20] as well as sparse mixed effects models [21].



---

*Source: 2834183-2018-12-30.xml*