# Error-Function-Based Penalized Quantile Regression in the Linear Mixed Model
## 2 Model and Algorithm  
Consider a linear mixed-effects model with random-effect coefficients as follows:  
\[Y_{ij}=x_{ij}^{\mathrm{T}}\beta+z_{ij}^{\mathrm{T}}\alpha_{i}+\epsilon_{ij}, \quad i=1,2,\ldots,n,\quad j=1,2,\ldots,m, \tag{1}\]  
where \(Y_{ij}\) and \(\epsilon_{ij}\) denote the response variable and the error at the \(j\)-th observation of the \(i\)-th individuals, respectively. The vector \(x_{ij}=(x_{ij1},x_{ij2},\ldots,x_{ijp})^{\mathrm{T}}\in\mathbb{R}^{p}\) is the fixed-effect variables, and \(\beta=(\beta_{1},\ldots,\beta_{p})^{\mathrm{T}}\) denotes the corresponding \(p\)-dimensional coefficients. Similarly, \(z_{ij}=(z_{ij1},z_{ij2},\ldots,z_{ijq})^{\mathrm{T}}\in\mathbb{R}^{q}\) and \(\alpha_{i}=(\alpha_{i1},\ldots,\alpha_{iq})^{\mathrm{T}}\) are the random-effects covariates and their corresponding \(q\)-dimensional coefficients, respectively. In this linear mixed-effects model, an excessive number of random-effect covariates may lead to overfitting and hinder accurate parameter estimation. To address this, we study a doubly penalized quantile regression method for the linear mixed-effects model, with regularization terms based on the error function applied to both fixed- and random-effects coefficients. For a given response variable \(\gamma_{ij}\) and quantile \(\tau\in(0,1)\), the conditional quantile regression function is expressed as follows:  
\[Q\big{(}\tau\mid x_{ij},z_{ij},\alpha_{i}\big{)}=x_{ij}^{\mathrm{T}}\beta+z_{ ij}^{\mathrm{T}}\alpha_{i}, \tag{2}\]  
where \(Q\big{(}\tau|x_{ij},z_{ij},\alpha_{i}\big{)}=\inf\{y:F\big{(}y|x_{ij},z_{ij}, \alpha_{i}\big{)}\geq\tau\}\) is the \(\tau\)-quantile of \(Y_{ij}\). The model can be transformed into the following minimization problem:  
\[\min_{\beta\cdot\alpha}\Bigg{\{}\sum_{i=1}^{n}\sum_{j=1}^{m}\rho_ {\tau}\Big{(}\Upsilon_{ij}-x_{ij}^{\mathrm{T}}\beta-z_{ij}^{\mathrm{T}}\alpha_ {i}\Big{)}+\lambda_{\beta}J_{\sigma}(\beta)+\sum_{i=1}^{n}\lambda_{\alpha}J_{ \sigma}(\alpha_{i})\Bigg{\}}\] \[J_{\sigma}(\beta)=\sum_{l=1}^{p}\Phi(|\beta_{l}|),\quad\Phi(| \beta_{l}|)=\int_{0}^{|\beta_{l}|}e^{-\frac{l^{2}}{\sigma^{2}}}dt, \tag{3}\]  
where \(\rho_{\tau}(u)=u(\tau-I(u\leq 0))\) is the quantile loss function, and \(\lambda_{\beta}\geq 0\), \(\lambda_{\alpha}\geq 0\) are the penalty parameters associated with the fixed-effects and random-effects coefficients, respectively. \(\Phi(x)\) is the error function and define \(J_{\sigma}(\beta)\) is the ERF(ERorr Function) regularization term [16]. The parameter \(\sigma>0\) controls the intensity of the penalty. For clarity of discussion, we reformulate Equations (3) as follows:  
\[\min_{\beta\cdot\alpha}\Bigg{\{} \sum_{i=1}^{n}\sum_{j=1}^{m}\rho_{\tau}\Big{(}\Upsilon_{ij}-x_{ ij}^{\mathrm{T}}\beta-z_{ij}^{\mathrm{T}}\alpha_{i}\Big{)}\] \[+\lambda_{\beta}\sum_{l=1}^{p}\Phi(|\beta_{l}|)+\lambda_{\alpha} \sum_{i=1}^{n}\sum_{k=1}^{q}\Phi(|\alpha_{ik}|)\Bigg{\}}. \tag{4}\]  
By imposing penalties on both the fixed- and random-effects coefficients in the linear mixed model, one can not only perform variable selection for the fixed effects but also eliminate redundant information in the random effects. [PERSON] et al. proved that, in doubly penalized quantile regression for linear mixed-effects models, the estimator under double Lasso penalties converges in distribution asymptotically, and the bias in estimating non-zero coefficients is controlled by the penalty parameters [13]. We implement doubly penalized regression for linear mixed-effects models by applying ERF regularization to both the fixed- and random-effects coefficients. The \(L_{1}\) is the most commonly used penalty function in quantile regression. However, it introduces estimation bias, particularly for large coefficients. To address this issue, several non-convex penalties such as SCAD and MCP have been proposed [11; 18]. These penalties adaptively apply varying degrees of shrinkage to different coefficient magnitudes, thereby reducing the bias introduced by \(L_{1}\).