# Implementation Aspects in Regularized Structural Equation Models
###### Abstract  
This article reviews several implementation aspects in estimating regularized single-group and multiple-group structural equation models (SEM). It is demonstrated that approximate estimation approaches that rely on a differentiable approximation of non-differentiable penalty functions perform similarly to the coordinate descent optimization approach of regularized SEMs. Furthermore, using a fixed regularization parameter can sometimes be superior to an optimal regularization parameter selected by the Bayesian information criterion when it comes to the estimation of structural parameters. Moreover, the widespread penalty functions of regularized SEM implemented in several R packages were compared with the estimation based on a recently proposed penalty function in the Mplus software. Finally, we also investigate the performance of a clever replacement of the optimization function in regularized SEM with a smoothed differentiable approximation of the Bayesian information criterion proposed by [PERSON] and [PERSON] in 2023. The findings were derived through two simulation studies and are intended to guide the practical implementation of regularized SEM in future software pieces. structural equation modeling; confirmatory factor analysis; regularized estimation; Bayesian information criterion 20231223231232  
In the general SEM approach, a matrix \(B\) of regression coefficients is specified such that  
\[\eta=B\eta+\xi\text{ with }\text{E}(\xi)=\mathbf{\alpha}\ \ \text{and}\ \ \text{Var}(\xi)=\mathbf{\Phi}. \tag{3}\]  
Note that (3) can be rewritten as  
\[\eta=(\mathbf{I}-\mathbf{B})^{-1}\xi\text{ with }\text{E}(\xi)=\mathbf{\alpha}\ \ \text{and}\ \ \text{Var}(\xi)=\mathbf{\Phi}\, \tag{4}\]  
where \(\mathbf{I}\) denotes the identity matrix. Hence, the mean vector and the covariance matrix are represented in SEM as  
\[\mu(\theta)=\mathbf{\
u}+\mathbf{\Lambda}(\mathbf{I}-\mathbf{B})^{-1}\mathbf{\alpha}\ \ \text{and}\ \ \mathbf{\Sigma}(\theta)=\mathbf{\Lambda}(\mathbf{I}-\mathbf{B})^{-1}\mathbf{\Phi}[(\mathbf{I}-\mathbf{B})^ {-1}]^{\top}\mathbf{\Lambda}^{\top}+\mathbf{\Psi}\, \tag{5}\]  
The estimation of SEM often follows an ideal measurement model. For example, a simple-structure factor loading matrix \(\mathbf{\Lambda}\) is desired in a multidimensional CFA. In this case, an item loads on one and only factor \(\mathbf{\eta}\), meaning that the number of non-zero entries in a row of \(\mathbf{\Lambda}\) is one. However, this assumption on the simple-structure loading matrix could be somewhat violated in practice. For this reason, some cross-loadings could be assumed to be different from zero. Such sparsity assumptions on SEM model parameters can be tackled with regularized SEM [11]. Moreover, deviations in the entries of the observed and modeled mean vector (i.e., \(\mu-\mu(\theta)\)) can be quantified in non-zero entries of the vector of item intercepts \(\mathbf{\
u}\) (see [12]). Again, model errors could be sparsely distributed, which would allow for the application of regularized SEM. In a similar manner, model deviations \(\mathbf{\Sigma}-\mathbf{\Sigma}(\theta)\) can be tackled by assuming sparsely distributed entries in the matrix of residual covariances \(\mathbf{\Psi}\). Notably, regularized SEM estimation is now becoming more popular in the social sciences and is recognized as an important approach in the machine learning literature [13]. In this article, we review several implementation aspects in estimating regularized SEMs with single and multiple groups. A recent article by [PERSON] et al. [14] recommended avoiding differentiable approximations for the non-differentiable optimization function in regularized SEM. We critically evaluate the credibility of this statement. Furthermore, we compare the currently used regularization estimation approach in most software, such as the regesem R package [15], with a recently proposed optimization function in the commercial SEM software package Mplus [16].