# Coordinate Descent for Variance-Component Models
## 4 Numerical Results
### Simulations
#### 4.1.2 Full-Rank  
We now present the results when \(\text{rank}(\mathbf{V}_{i})=n=1000\). We implement the standard PX-CD, PXI-CD and MM algorithms, where either the Woodbury, or the QR method cannot be used. Initially, PX-CD is run until the relative change \(\left[L(\gamma^{(t+1)})-L(\gamma^{(t)})\right]/\left[\ |\ L(\gamma^{(t)})\ |\ +1\right]\) is less than \(10^{-10}\), and we store the final objective value as \(L^{*}\). The other algorithms terminate once \(L(\gamma^{(t+1)})>L^{*}\). For the following simulations, we consider one iteration of a CD algorithm as a single cycle of updates. Each simulation scenario is replicated 10 times. The mean running time and mean iteration number is reported with the standard error of the mean running time and mean iteration provided in parentheses.  
The results of the full-rank simulations are given in Tables 4-6. PX-CD and PXI-CD both significantly outperform the MM and the basic CD algorithms in these examples. We observe that as the number of components \(m\) increases the problem becomes increasingly difficult for the MM algorithm. An intuitive explanation for this performance gap is that the CD algorithms are able to identify the active constraint set \(\{k:\hat{\gamma}_{k}=0\}\) in only a few cycles.  
When \(\gamma_{0}=0.1\) and \(\gamma_{0}=1\) and \(m\) is large (\(m=50\), \(m=100\)), PXI-CD is the fastest algorithm, even though it is computationally the most expensive per cycle. When \(\gamma_{0}=10\) and \(m=100\), PXI-CD is the fastest algorithm. In fact, as the problem size grows, the number of iterations that PXI-CD requires to converge is less than that of the basic CD. This simulation indicates that PXI-CD is well-suited to problems with large \(m,n\) and when \(\mathbf{V}_{i}\) are full-rank. The basic CD algorithm, while numerically the inferior compared to the PX-CD and PXI-CD algorithms still outperforms the MM algorithm in these simulations.  
\begin{table}
\begin{tabular}{l c c c c} \hline \hline
**Method** & \(m\) & **Iterations** & **Time (s)** & **Objective** \\ \hline PX-CD & 25 & 89.00 (2.90) & 37.47 (1.71) & \(-1383.44\) \\ PXI-CD & & 146.90 (21.17) & 97.49 (13.58) & \(-1383.44\) \\ CD & & 109.40 (17.52) & 286.96 (19.83) & \(-1383.44\) \\ MM & & 3957.90 (300.77) & 281.53 (21.03) & \(-1383.44\) \\ PX-CD & 50 & 182.40 (11.34) & 147.21 (10.64) & \(-1831.40\) \\ PXI-CD & & 73.10 (2.97) & 104.42 (4.64) & \(-1831.40\) \\ CD & & 103.40 (6.18) & 852.41 (70.22) & \(-1831.40\) \\ MM & & 10,240.40 (2140.25) & 1557.79 (343.47) & \(-1831.40\) \\ PX-CD & 100 & 279.10 (15.09) & 376.18 (24.14) & \(-2143.93\) \\ PXI-CD & & 80.70 (2.97) & 211.19 (8.58) & \(-2143.93\) \\ CD & & 164.00 (8.84) & 2060.69 (155.37) & \(-2143.93\) \\ MM & & 12,171.90 (1526.30) & 3482.30 (465.51) & \(-2143.93\) \\ \hline \hline \end{tabular}
\end{table}
Table 4: The convergencceresults when \(\mathbf{V}_{i}\) are full-rank, \(n=1000\), and \(\gamma_{0}=0.1\).  
\begin{table}
\begin{tabular}{l c c c c c} \hline \hline  & & & \(m\) & \\ \hline
**Method** & \(r\) & **20** & **50** & **75** & **100** \\ \hline PXI-CD & 20 & 3.62 (0.10) & 8.79 (0.20) & 15.60 (0.41) & 23.13 (0.91) \\ MM & & 10.17 (0.84) & 318.53 (90.49) & 648.64 (132.62) & 839.24 (158.52) \\ PXI-CD & 50 & 4.36 (0.09) & 14.39 (0.4) & 24.65 (0.97) & 42.07 (1.13) \\ MM & & 184.03 (38.8) & 473.58 (80.82) & 648.33 (114.98) & 1230.56 (128.79) \\ PXI-CD & 100 & 6.53 (0.22) & 26.07(1.51) & 48.48 (2.48) & 60.72 (5.33) \\ MM & & 124.68 (19.93) & 511.66 (65.85) & 880.53 (69.53) & 1279.40 (81.93) \\ PXI-CD & 150 & 10.15 (0.37) & 35.34 (1.72) & 66.94 (2.28) & 103.89 (10.36) \\ MM & & 199.80 (33.17) & 512.35 (49.97) & 943.84 (84.24) & 1244.32 (72.13) \\ \hline \hline \end{tabular}
\end{table}
Table 3: The running times (s) when \(\mathbf{V}_{i}\) are low-rank, \(n=1000\), and \(\gamma_{0}=10\).