# Smooth Information Criterion for Regularized Estimation of Item Response Models
## 1 Introduction
\tag{5}\]  
In many studies, the recommended value of \(a=3.7\) (see [15]) has been adopted (e.g., [16; 10]). Note that \(\mathcal{P}_{\text{SCAD}}\) has the property of the LASSO penalty around zero, but has zero derivatives for \(x\) values that strongly differ from zero. A parameter estimate \(\hat{\mathbf{\gamma}}\) of the regularized IRT model is defined as an estimator defined as the minimizer of \(l_{\text{reg}}\)  
\[\hat{\mathbf{\gamma}}(\lambda)=\operatorname*{arg\,min}_{\mathbf{\gamma}}\,l_{\text{ reg}}(\mathbf{\gamma};\lambda). \tag{6}\]  
Note that the penalty function \(\mathcal{P}\) involves a fixed tuning parameter \(\lambda\). Hence, the parameter estimate \(\hat{\mathbf{\gamma}}(\lambda)\) depends on \(\lambda\). A crucial issue of the LASSO and the SCAD penalty functions is that they are nondifferentiable functions because the function \(x\mapsto|x|\) is nondifferentiable. Hence, particular estimation techniques for nondifferentiable optimization problems must be applied [14; 17; 18]. As an alternative, the nondifferentiable optimization function can be replaced by a differentiable approximation [19; 20; 21; 22]. For example, the absolute value function \(x\mapsto|x|\) in the SCAD penalty can be replaced with \(x\mapsto\sqrt{x^{2}+\varepsilon}\) for a sufficiently small \(\varepsilon\) such as \(\varepsilon=0.001\). Using differentiable approximations has the advantage that ordinary gradient-based optimizers can be utilized. In practice, the estimation of the regularized IRT model is carried out on a grid of \(T\) values of \(\lambda\) in a grid \(\Lambda=\{\lambda_{1},\ldots,\lambda_{T}\}\). For each value of the tuning parameter \(\lambda_{t}\), a parameter estimate \(\widehat{\gamma}(\lambda_{t})\) is obtained. A final parameter estimate \(\widehat{\gamma}\) is obtained by minimizing an information criterion  
\[IC(\widehat{\gamma}(\lambda))=2l(\widehat{\gamma}(\lambda))+K_{N}\Bigg{(}H_{0 }+\sum_{h=1}^{H}u_{h}\mathbf{1}(\widehat{\gamma}_{h}(\lambda)\
eq 0)\Bigg{)}\, \tag{7}\]  
where the factor \(K_{N}\) is chosen as \(K_{N}=2\) for the Akaike information criterion (AIC; [23]) and \(K_{N}=\log N\) for the Bayesian information criterion (BIC; [24]) (see [25]). If the regularized likelihood function is evaluated with differentiable approximations, there are no regularized parameters that exactly equal zero (in contrast to special-purpose optimizers for regularized estimation; [17]). Hence, estimated parameters \(\gamma_{h}\) are counted as zero if they do not exceed a fixed threshold \(\tau\) (such as 0.001, 0.01, or 0.02) in its absolute value. Hence, the approximated information criterion is computed as  
\[IC(\widehat{\gamma}(\lambda))=2l(\widehat{\gamma}(\lambda))+K_{N}\Bigg{(}H_{0 }+\sum_{h=1}^{H}u_{h}\mathbf{1}(|\widehat{\gamma}_{h}(\lambda)|>\tau)\Bigg{)}. \tag{8}\]  
The final estimator of \(\gamma\) is defined as  
\[\widehat{\gamma}_{IC}=\widehat{\gamma}(\widehat{\lambda}_{\text{opt}})\ \text{with}\ \widehat{\lambda}_{\text{opt}}=\underset{\lambda\in\Lambda}{\arg\min}\ \ IC(\widehat{\gamma}(\lambda)). \tag{9}\]  
Depending on the chosen value of \(K_{N}\), the regularized parameter estimate can be based on the AIC and BIC. The ordinary estimation approach to regularized estimation described above has the computational disadvantage that it requires a sequential fitting of models on the grid \(\Lambda\) of the regularization parameter \(\lambda\). This approach is referred to as an indirect optimization approach because it first minimizes a criterion function (i.e., the regularized likelihood function) with respect to \(\gamma\) for a fixed value of \(\lambda\) and optimizes a second criterion (i.e., the AIC or BIC) in the second step. [PERSON] and [PERSON] [26] proposed an estimation approach to regularized estimation that directly minimizes a smooth version of the BIC (i.e., smooth Bayesian information criterion, SBIC) for regression models. This direct estimation approach has been successfully implemented for structural equation models [21; 27].