## 4 Application to Classification Problems
### Experiments  
First, by setting \(\mathbf{W}=\mathbf{I}_{n}\), we conduct the experiments on a relatively simple dataset to investigate the influence by bias of \(\ell_{2,1}\) regularizer on the classification problem (especially when training set is unbalanced), and verify the performance improvement using \((\|\cdot\|_{2,1})_{\mathbf{B}}\) as the regularizer by conducting the experiments on a relatively simple dataset. The USPS handwritten digit database [42] has 11,000 samples of digits \"0\" through \"9\" (1100 samples per class). The dimension of each sample is \(16\times 16\). In our classification experiments, we vectorized them to 256-D vectors. The number of training samples for each class is not necessarily equal, which varies from 5 to 50 (the size of test set is fixed to 50 images per class). We set \(\mathbf{W}=\mathbf{I}_{n}\) (the initialization of \(\mathbf{W}\) should be modified in Algorithm 1) for the proposed model (20) and compared it with GSC (with \(\ell_{2,1}\) regularizer) [10]. We set \(\mathbf{B}=\sqrt{\theta/\lambda}\mathbf{A}\) and fix \(\theta=0.9\) to achieve the overall convexity of proposed method, and set \(\kappa=1.1\), \(\iota=\|(\kappa/2)\mathbf{A}^{\top}\mathbf{A}+\lambda\mathbf{I}_{n}\|_{\text{spec}}+(\kappa -1)\), \(\tau=(\kappa/2+2/\kappa)\lambda\|\mathbf{B}\|_{\text{spec}}^{2}+(\kappa-1)\). The initial estimate is set as \((\mathbf{x}^{(0)},\mathbf{u}^{(0)},\mathbf{w}^{(0)})=(\mathbf{O}_{n\times 1},\mathbf{O}_{n\times 1},)\), and the stopping criterion is set to either \(\|(\mathbf{x}^{(k)},\mathbf{u}^{(k)},\mathbf{w}^{(k)})-(\mathbf{x}^{(k+1)},\mathbf{u}^{(k+1)},\mathbf{ w}^{(k+1)})\|_{2}<10^{-4}\) or steps reaching 10,000. Figure 2 shows an example of unbalanced training set (digits \"0\" through \"4\" have 5 samples per class and \"5\" through \"9\" have 25 samples per class). The input (an image of  
digit \"0\") was misclassified (into digital \"6\") by GSC while classified correctly by proposed method. The obtained coefficient vectors by GSC and proposed method (both with \(\lambda=4\)) are illustrated respectively, and some samples corresponding to nonzero coefficients are also displayed in Figure 2. It can be seen that the samples from digit \"6\" made the greatest contribution to the representation in GSC, and samples from \"5\" and \"0\" also made small contribution. In our method, samples from the correct class \"0\" made the biggest contribution and led to correct result. It is reasonable, because our method did not suppress the high value coefficients too much whereas \(\ell_{2,1}\) did. The big suppression of \(\ell_{2,1}\) made the coefficients of the correct class cannot be large enough, and thus easily led to misclassification. **Input:** A matrix of training samples \(\mathbf{A}=[\mathbf{A}_{1},\mathbf{A}_{2},\cdots,\mathbf{A}_{\mathcal{G}}]\in\mathbb{R}^{m \times n}\) grouped by class information, a test sample vector \(\mathbf{y}\in\mathbb{R}^{m}\), parameters \(\lambda\), \(\sigma_{1}\) and \(\sigma_{2}\). **1. Initialization:** Let \((\mathbf{x}^{(0)},\mathbf{u}^{(0)},\mathbf{v}^{(0)})\in\mathbb{R}^{n}\times\mathbb{R}^{n}\). Compute the weight matrix \(\mathbf{W}\) by (9). Choose \(\mathbf{B}\) satisfying \(\mathbf{A}^{\top}\mathbf{A}-\lambda\mathbf{W}^{\top}\mathbf{B}^{\top}\mathbf{B}\mathbf{W}\succeq\mathbf{O }_{n\times n}\). Choose \((\iota,\tau,\kappa)\in\mathbb{R}_{++}\times\mathbb{R}_{++}\times(1,+\infty)\) satisfying  
\[\iota\mathbf{I}_{n}-\frac{\kappa}{2}\mathbf{A}^{\top}\mathbf{A}-\lambda\mathbf{W}^{\top}\mathbf{W }\succeq\mathbf{O}_{n\times n}\quad\text{and}\quad\tau\geq\Big{(}\frac{\kappa}{2 }+\frac{2}{\kappa}\Big{)}\lambda\|\mathbf{B}\|_{\text{spec}}^{2}. \tag{22}\]  
**2.