## 4 Applying ADMM to the Ordered Ridge Regression
### Experimental Results of Real Data  
Variable selection difficulty arises when the number of features (p) are greater than the number of instances (n). The proposed method genuinely handles these types of issues. The practical application of the ADMM method is in many domains such as computer vision and graphics [35], analysis of biological data [36; 37], and smart electric grids [38; 39]. A biological leukaemia dataset [40] was used to demonstrate the performance of the proposed method. Leukemia is a type of cancer that impairs the body's ability to build healthy blood cells. Leukemia begins in the bone marrow. There are many types of leukemia, such as acute lymphoblastic leukemia, acute myeloid leukemia, and chronic lymphocytic leukemia. The following two types of leukemia are used in this experiment: acute lymphoblastic leukemia and acute myeloid leukemia. The leukemia dataset consists of 7129 genes and 72 samples [41]. Randomly split the data into training and test sets. In the training set, there are 38 samples, among  
Figure 3: (**a**) Relative primal–dual gap versus dual infeasibility, respectively, and (**b**) primal and dual residual versus primal and dual feasibility, respectively: Input synthetic data. Figure 2: Primal and dual residual versus primal and dual feasibility, respectively: Input synthetic data. which 27 are type I ALL (acute lymphoblastic leukemia) and 11 are type II AML (acute myeloid leukemia). The remaining 34 samples allowed us to test the prediction accuracy. The test set contains 20 type I ALL and 14 type II AML. The data were labeled according to the type of leukemia (ALL or AML). Therefore, before applying an ordered elastic net, the type of leukemia (ALL = \(-1\), or AML = 1) is converted as a (\(-1\), 1) response \(y\). Predicted response \(\hat{y}\) is set to 1 if \(\hat{y}>0\); otherwise, it is set to \(-1\). \(\lambda\in\mathbb{R}^{p}\) is a non-increasing, ordered sequence generated according to Section 5.1 and Algorithm 2. For the regular lasso, \(\lambda\) is a single scalar value generated using Equation (14). \(\alpha=0.1\) is used for the leukemia dataset. All other settings are the same as experiment with synthetic data. Table 3 illustrates the experiment results of the leukemia dataset for different types of regularization. The lowest average mean square error (MSE) of regularization is the ordered \(\ell_{2}\), followed by the ordered \(\ell_{1,2}\) and lasso, while the highest average MSE can be seen in the ordered \(\ell_{1}\). Looking at Table 3 first, it is clear that the ordered \(\ell_{2}\) converges the fastest among all the regularizations. The second fastest converging regularization is the ordered \(\ell_{1,2}\), while the slowest converging regularization is the ordered \(\ell_{1}\). The ordered \(\ell_{2}\) takes an average iteration around 190 and an average time around 0.15 s to converge. On the other hand, the ordered \(\ell_{1,2}\), the ordered \(\ell_{1}\), and lasso take average iterations around 1381, 10,000, and 10,000, respectively, and average times around: 1.0, 14.0, and 5.0 s, respectively, to converge. It can also be seen from the data in Table 3 that the ordered \(\ell_{2}\) selected all the variables but that the goal is to select only the relevant variables from strongly correlated, high-dimensional dataset. Therefore, the ordered elastic net was proposed, which only selects relevant variables and discards irrelevant variables. As can be seen from Table 3, average MSE, time, and iteration in the ordered \(\ell_{1}\) regularization and lasso are significantly more than the ordered \(\ell_{1,2}\) regularization, although an average gene selection in the ordered \(\ell_{1,2}\) regularization is more than that of the ordered \(\ell_{1}\) regularization and lasso. The ordered \(\ell_{1}\) and lasso select averages around 84 and 7 variables, respectively, whereas the ordered \(\ell_{1,2}\) selects an average around 107 variables. The lasso performs poorly on the leukemia dataset.