# Implementation Aspects in Regularized Structural Equation Models
## 5 Simulation Study 2: Two-Dimensional Factor Model with Cross-Loadings
### Results  
Figures 4 and 5 display the absolute bias and the RMSE of the factor correlation \(\phi_{2,11}\) of the two factors as a function of the regularization parameter \(\lambda\) for the two sample sizes, \(N=500\) and \(N=1000\), and the two powers of the penalty function \(p=1\) and \(p=0.5\), respectively. In contrast to Simulation Study 1, the two figures showed that using a regularization parameter \(\lambda\) that is smaller than the optimal \(\lambda\) selected by BIC resulted in parameter estimates with a smaller RMSE. This property somehow questions the standard procedure of searching for a parsimonious model according to the BIC if a structural parameter should be estimated with low variance. Furthermore, parameter estimates with the power \(p=0.5\) in the penalty function resulted in a lower RMSE than using the power \(p=1\). Table 4 presents the bias and the RMSE of the estimated factor correlation and the four factor loadings for the first two items. According to the findings in Figures 4 and 5, we also display the parameter estimates for the fixed regularization parameter \(\lambda=0.025\). The estimated parameters were unbiased. Hence, we focus on differences between the estimation approaches regarding the RMSE. Like in Simulation Study 1, the exact estimation approach (CD) performed similarly to the differentiable estimation approach (DA). The only exception was the case of \(p=0.5\) and the fixed regularization parameter \(\lambda=0.025\). In this case, the DA approach resulted in fewer variables estimated than the CD approach for the factor correlation \(\phi_{2,11}\). Moreover, the direct BIC minimization approach (DIR) was similar or superior to the CD and DA estimation approaches based on BIC. This is an interesting finding because repeatedly fitting the regularized SEM on a grid of regularization parameters is not required in DIR. \begin{table}
\begin{tabular}{c c c c c c c c c c c c} \hline  & & & & & \multicolumn{4}{c}{\(p=1\)} & \multicolumn{4}{c}{\(p=0.5\)} \\ \cline{4-13}  & & & \multicolumn{2}{c}{**BIC**} & \multicolumn{2}{c}{**BIC**} & \multicolumn{2}{c}{\(\lambda=0.025\)} & BIC & \multicolumn{2}{c}{\(\lambda=0.025\)} \\ \cline{4-13}
**Par** & \(N\) & \(\delta\) & **DIR** & **CD** & **DA** & **CD** & **DA** & **CD** & **DA** & **CD** & **DA** \\ \hline \multirow{4}{*}{\(\phi_{12}\)} & \multirow{4}{*}{500} & \multicolumn{4}{c}{_Bias_} & & & & & & & & & \\  & & 0.2 & 0.000 & 0.011 & 0.009 & \(-\)0.001 & \(-\)0.001 & 0.001 & 0.001 & 0.000 & 0.000 \\  & & 0.4 & 0.000 & 0.001 & 0.000 & 0.000 & \(-\)0.001 & 0.001 & 0.001 & 0.000 & 0.000 \\  & & 0.2 & 0.000 & 0.002 & 0.001 & \(-\)0.001 & \(-\)0.001 & 0.000 & 0.000 & 0.000 & 0.000 \\  & & 0.4 & \(-\)0.001 & 0.000 & \(-\)0.001 & \(-\)0.001 & 0.000 & 0.000 & 0.000 & 0.000 & 0.000 \\  & & 0.2 & \(-\)0.001 & \(-\)0.002 & \(-\)0.002 & \(-\)0.001 & \(-\)0.001 & 0.000 & \(-\)0.001 & \(-\)0.001 & \(-\)0.001 \\ \(\lambda_{11}\) & & 0.4 & 0.000 & \(-\)0.001 & \(-\)0.001 & 0.000 & 0.000 & 0.000 & 0.000 & \(-\)0.001 & \(-\)0.001 \\  & & 1000 & 0.2 & 0.000 & 0.000 & \(-\)0.001 & 0.000 & 0.000 & 0.000 & 0.000 & 0.000 & 0.000 \\  & & 0.4 & 0.000 & 0.000 & 0.000 & 0.000 & 0.000 & 0.000 & 0.000 & 0.000 & \(-\)0.001 \\ \hline \end{tabular}
\end{table} 
Table 4: Simulation Study 2: Bias and root mean square error (RMSE) of factor means as a function of sample size \(N\) and the size of cross-loadings \(\delta\).