# Smooth Information Criterion for Regularized Estimation of Item Response Models
## 1 Introduction  
Item response theory (IRT; [1; 2; 3; 4; 5]) modeling is a class of statistical models that analyze discrete multivariate data. In these models, a vector \(\mathbf{X}=(X_{1},\ldots,X_{l})\) of \(I\) discrete variables \(X_{i}\) (\(i=1,\ldots,I\); also referred to as items) is summarized by a unidimensional or multidimensional factor variable \(\Theta\). In this article, we confine ourselves to dichotomous random variables \(X_{i}\in\{0,1\}\). The multivariate distribution for the vector \(\mathbf{X}\in\{0,1\}^{I}\) in the IRT model is defined as  
\[\mathsf{P}(\mathbf{X}=\mathbf{x};\mathbf{\gamma})=\int\prod_{i=1}^{l}\mathsf{P}(X_{i}=x_{i} |\Theta;\mathbf{\gamma}_{i})f(\Theta;\beta)\,\mathrm{d}\Theta\, \tag{1}\]  
where \(\mathbf{\gamma}=(\mathbf{\gamma}_{1},\ldots,\mathbf{\gamma}_{I},\beta)\) is the vector of model parameters. The vector \(\mathbf{\gamma}_{i}\) contains item parameters of item \(i\), while \(\beta\) parametrizes the density \(f\) of the factor variable \(\Theta\). Note that (1) includes a local independence assumption. That is, the items \(X_{i}\) are conditionally independent given the factor variable \(\Theta\). The function \(\Theta\mapsto\mathsf{P}(X_{i}=x_{i}|\Theta;\mathbf{\gamma}_{i})\) is also referred to as the item response function (IRF; [6; 7; 8]). The two-parameter logistic (2 PL) model [9] uses the IRF \(\Theta\mapsto\Psi(a_{i}\Theta-b_{i})\), where \(\Psi\) denotes the logistic distribution function. Now, assume that \(N\) independent replications of \(\mathbf{X}\) are available. The parameter vector \(\mathbf{\gamma}\) from these observations \(\mathbf{x}_{1},\ldots,\mathbf{x}_{N}\) can be estimated by minimizing the negative log-likelihood function  
\[l(\mathbf{\gamma})=-\sum_{n=1}^{N}\log\mathsf{P}(\mathbf{X}=\mathbf{x}_{n};\mathbf{\gamma})\, \tag{2}\]  
where the parameter vector \(\mathbf{\gamma}=(\gamma_{1},\ldots,\gamma_{H})\) contains \(H\) components that have to be estimated. In various applications, the IRT model (1) is not identified or includes too many parameters, making the interpretation difficult. To this end, some sparsity structure [10] on model parameters \(\mathbf{\gamma}\) is imposed. Regularized estimation as a machine learning technique is employed in IRT models to make estimation feasible [11; 12; 13]. More formally, sparsity structure on \(\mathbf{\gamma}\) is imposed by replacing the negative log-likelihood function with a negative regularized log-likelihood function  
\[l_{\text{reg}}(\mathbf{\gamma};\lambda)=l(\mathbf{\gamma})+N\sum_{h=1}^{H}u_{h} \mathcal{P}(\gamma_{h},\lambda)\, \tag{3}\]  
where \(u_{h}\) is an indicator variable for the parameter \(\gamma_{h}\) that takes values 0 or 1. The indicator \(u_{h}\) equals 1 if \(\gamma_{h}\) is regularized (i.e., the sparsity structure assumption applies to this parameter), while it is 0 if \(\gamma_{h}\) should not be regularized. Let \(H_{1}=\sum_{h=1}^{H}u_{h}\) be the number of regularized parameters and \(H_{0}=H-H_{1}\) is the number of nonregularized model parameters. The regularized negative log-likelihood function \(l_{\text{reg}}\) defined in (3) includes a penalty function \(\mathcal{P}\) that decodes the assumptions about sparsity. For a scalar parameter \(x\), the least absolute shrinkage and selection operator (LASSO; [14]) penalty is a popular penalty function used in regularization, and it is defined as  
\[\mathcal{P}_{\text{LASSO}}(x,\lambda)=\lambda|x|\, \tag{4}\]  
where \(\lambda\) is a nonnegative regularization parameter that controls the extent of sparsity in the obtained parameter estimate. It is well-known that the LASSO penalty introduces bias in estimated parameters. To circumvent this issue, the smoothly clipped absolute deviation (SCAD; [15]) penalty has been proposed. \[\mathcal{P}_{\text{SCAD}}(x,\lambda)=\left\{ \begin{array}{ll}\lambda|x|& \text{if }|x|<\lambda\\ -(x^{2}-2a\lambda|x|^{2}+\lambda^{2})(2(a-1))^{-1}&\text{if }\lambda\leq|x|\leq a \lambda\\ (a+1)\lambda^{2}&\text{if }|x|>a\lambda\end{array} \right.