## 3 Ridge Regression with the Ordered \(\ell_{2}\) Regularization  
### The Ordered \(\ell_{2}\) Regularization  
The proposed parameter estimation and variable selection method in this paper is computationally manageable and adaptive. This procedure depends on the ordered \(\ell_{2}\) regularization. Let \(\lambda=(\lambda_{1},\lambda_{2},\ldots,\lambda_{p})\) be a decreasing sequence of positive scalars that satisfy the following condition:  
\[\lambda_{1}\geq\lambda_{2}\geq\lambda_{3}\geq...\geq\lambda_{p}\geq 0 \tag{2}\]  
The ordered \(\ell_{2}\) regularization of a vector \(x\in\mathbb{R}^{p}\) when \(\lambda_{1}>0\) can be defined as follows:  
\[J_{\lambda}(x)=\lambda_{1}x_{(1)}^{2}+\lambda_{2}x_{(2)}^{2}+\cdots+\lambda_{ p}x_{(p)}^{2}=\sum_{k=1}^{p}\lambda_{BH^{(k)}}x_{(k)}^{2} \tag{3}\]  
where \(\lambda_{BH^{(k)}}\) is called a BHq method [27], which generates an adaptive and a non-increasing value for \(\lambda\) (Reference [2], Section 1.1). The details of \(\lambda_{BH^{(k)}}\) are available in Section 4.2. For ease of presentation, we have written \(\lambda_{k}\) in place of \(\lambda_{BH^{(k)}}\) in the rest of the paper. \(x_{(1)}^{2}\geq x_{(2)}^{2}\geq x_{(3)}^{2}\geq\cdots\geq x_{(p)}^{2}\) is the order statistic of the magnitudes of \(x\)[28]. The subscript \(k\) of \(x\) enclosed in parentheses indicates the \(k\)th-order statistic of a sample. Suppose that \(x\) is a sample size of 4. Hence, four numbers are observed in \(x\) if the sample values of \(x=(-2.1,\,-0.5,\,3.2,\,7.2)\). The order statistics of \(x\) would be \(x_{(1)}^{2}=7.2^{2},x_{(2)}^{2}=3.2^{2},x_{(3)}^{2}=2.1^{2},x_{(4)}^{2}=0.5^{2}\). The ordered \(\ell_{2}\) regularization is expressed as the first largest value of \(\lambda\) times the square of the first largest entry of \(x\), plus the second largest value of \(\lambda\) times the square of the second largest entry of \(x\), and so on. \(A\in\mathbb{R}^{n\times p}\) and \(b\in\mathbb{R}^{n}\) are a matrix and a vector, respectively. The ordered \(\ell_{2}\) regularized loss minimization can be expressed as follows:  
\[\underset{x\in\mathbb{R}^{p}}{\text{min}}\frac{1}{2}\|Ax-b\|_{2}^{2}+\frac{1} {2}\{\lambda_{1}x_{(1)}^{2}+\cdots+\lambda_{p}x_{(p)}^{2}\} \tag{4}\]  
**Theorem 1**.: _If the square root of \(J_{\lambda}(x)\) (Equation (3)) is a norm on \(\mathbb{R}^{p}\) and a function \(\|.\|:\mathbb{R}^{p}\to\mathbb{R}\) satisfying the following three properties, then the following Corollaries 1 and 2 are true._  
1. _(Positivity)_ \(\|x\|\geq 0\) _for any_ \(x\in\mathbb{R}^{p}\) _and_ \(\|x\|=0\) _if and only if_ \(x=0\)_._
2. _(Homogeneity)_ \(\|cx\|=|c|\|x\|\) _for any_ \(x\in\mathbb{R}^{p}\) _and_ \(c\in\mathbb{R}\)_._
3. _(Triangle inequality)_ \(\|x+y\|\leq\|x\|+\|y\|\) _for any_ \(x,y\in\mathbb{R}^{p}\)_._  
_Note: \(\|x\|\) and \(\|x\|_{2}\) are used interchangeably._  
**Corollary 1**.: _When all the \(\lambda_{k}^{\prime}\)s take on an equal positive value, \(J_{\lambda}(x)\) reduces to the square of the usual \(\ell_{2}\) norm._  
**Corollary 2**.: _When \(\lambda_{1}>0\) and \(\lambda_{2}=\ldots=\lambda_{p}=0\), the square root of \(J_{\lambda}(x)\) reduces to the \(\ell_{\infty}\) norm._  
Proofs of the theorem and corollaries are provided in Appendix A. Table 2 shows the notations used in this paper and their meanings.