## 2 Materials and Methods
### Proposed Model: DNNs and l2,1-Norm Regularization Framework for Robust Features Selection
#### 2.3.2 Learning  
In the learning process, it is aimed to minimize the proposed cost function with respect to the projection matrix W, as in Equation (3).  
\[min_{W}^{E(y,x)}=\ \left\{\left(sign\Big{(}X^{T}W\Big{)}-Y\right)^{2}+\lambda||W||_{ 2,1}\right\} \tag{3}\]  
The cost function in Equation (2) is not easy to minimize with the presence of the \(l2\),\(1\)-norm. In [52,54,55], the half-quadratic minimization method was used to solve the minimization of the \(l2\),\(1\)-norm. One should know that the minimizer function of the \(l2\),\(1\)-norm is unpredictable near the origin. Therefore, according to \(l2\),\(1\)-norm analysis in [55], a \(\varnothing(x)=\sqrt{\epsilon+x^{2}}\) can be defined to solve this problem, where \(\epsilon\) is chosen to be a decreased value to ensure that the function in Equation (3) with the \(l2\),\(1\)-norm is converged. Additionally, \(\varnothing\) should satisfy all of the conditions in Equation (4).  
\[\begin{array}{l}x\rightarrow\varnothing(x)\mbox{ is convex on }R,\\ x\rightarrow\varnothing\big{(}\sqrt{x}\big{)}\mbox{ is concave on }R_{+},\\ \varnothing(x)=\varnothing(-x),\ \forall x\in R,\\ \varnothing(x)\mbox{ is }C^{1}\mbox{ on }R,\\ \varnothing^{n}\left(0^{+}\right)>0,\stackrel{{ lim}}{{x\to \infty}}\varnothing(x)\,/\,x^{2}=0.\end{array} \tag{4}\]  
**Lemma 1**.: _Let be a function satisfying all conditions in Equation (4). There exists a conjugate function,\(\varphi(.)\), as in Equation (5) such that_  
\[\varnothing\big{(}\Big{|}\Big{|}w^{i}\Big{|}\Big{|}_{2}=\inf_{P\in R}\left\{p \Big{|}\Big{|}w^{i}\Big{|}\Big{|}_{2}^{2}+\varphi(p)\right\} \tag{5}\]  
\(p\) is determined by the minimizer function \(\delta(.)\) with respect to \(\varnothing(.)\). Based on \(\varnothing(x)\), \(\lambda||W||_{2,1}\) is replaced with \(\lambda\sum_{i}^{f}\sqrt{\epsilon+\big{|}\big{|}w^{i}\big{|}_{2}^{2}}\), then the Equation (3) is reformulated, as in Equation (6).  
\[\begin{array}{l}\underset{W}{\min}E(y,x)\ \ =\left\{\frac{1}{2m}\Big{(} sign\Big{(}X^{T}W\Big{)}-Y\Big{)}^{2}+\lambda\sum_{i}^{f}\sqrt{\epsilon+ \big{|}\big{|}w^{i}\big{|}_{2}^{2}}\,\right\}\end{array} \tag{6}\]  
Figure 2: Robust feature selection process. The dimension of the resulted robust features for the training samples is mxc.  
According to Lemma 1, the function of \(\lambda\sum_{i}^{f}\sqrt{\epsilon+\big{|}\big{|}w^{i}\big{|}\big{|}_{2}^{2}}\) can be reformulated, as in Equation (7).  
\[\lambda Tr\Big{(}W^{T}QW\Big{)} \tag{7}\]  
\(q=\delta\big{(}\big{|}\big{|}w^{i}\big{|}\big{|}_{2}\big{)}\in\mathbb{R}^{\text {f}}\) is an auxiliary vector and \(Q=(q)\). The operator \((.)\) puts a vector q on the main diagonal of \(Q\). \(q\) is computed by using the optimizer function, as in Equation (8).  
\[q_{i}=\frac{1}{\sqrt{\big{|}\big{|}w^{i}\big{|}\big{|}_{2}^{2}+\epsilon}} \tag{8}\]  
According to Equation (7), the minimization function can be written, as in Equation (9).  
\[\begin{array}{c}\min E(y,x)\\ W\end{array}=\frac{d}{dW}\bigg{\{}\frac{1}{2m}\Big{(}sigma\Big{(}X^{T}W\Big{)} -Y\Big{)}^{2}+\lambda Tr\Big{(}W^{T}QW\Big{)}\bigg{\}} \tag{9}\]  
The analytic minimization solution of Equation (9) with respect to \(W\) is finally written by Equation (10).  
\[\begin{array}{c}\min E(y,x)\\ W\end{array}=\Big{(}sigma\Big{(}X^{T}W\Big{)}-Y\Big{)}\bigg{(}sigma\Big{(}X^{T }W\Big{)}\bigg{(}1-\frac{1}{sigma\Big{(}X^{T}W\Big{)}}\bigg{)}\bigg{)}+\lambda QW \end{array} \tag{10}\]