# Robust Face Recognition Based on a New Supervised Kernel Subspace Learning Method
## 3 Supervised Kernel Locality-Based Discriminant Neighborhood Embedding
### Mathematics  
Suppose \(X=[x_{1},\ x_{2},\ \ldots,\ x_{n}]\) is a set of d-dimensional input samples. This input data is projected onto a higher dimensional feature space \(F\) via nonlinear mapping \(\varnothing\): \(R^{n}\)\(\rightarrow\)\(F\). Then, manifold learning is then carried out on the projected samples \(\varnothing(X)=[\varnothing(x_{1}),\varnothing(x_{2}),\ldots,\varnothing(x_{n})]\). Now assume that we have to find a projection transformation \(V_{\varnothing}\) in \(F\). The optimization problem can be expressed as:  
\[\text{Maximize}\ \sum_{ij}^{n}\parallel z_{i}-z_{j}\parallel^{2}F_{ij} \tag{14}\]  
Subject to \(V_{\varnothing}\)\({}^{\text{T}}V_{\varnothing}=I\), where \(I\) denotes the identity matrix, \(z_{i}=v_{\varnothing}^{\text{T}}\varnothing(x_{i})\) and \(z_{j}=v_{\varnothing}^{\text{T}}\varnothing(x_{j})\) are the projection of \(\varnothing(x_{i})\) and \(\varnothing(x_{j})\) respectively onto \(V_{\varnothing}\), and \(F_{ij}\) represents the relationship between of \(x_{i}\) and \(x_{j}\). The optimization problem can be kernelized as:  
\[\sum_{ij}^{n}\parallel z_{i}-z_{j}\parallel^{2}F_{ij}=\sum_{ij}^{n}\parallel v _{\varnothing}^{\text{T}}\varnothing(x_{i})-v_{\varnothing}^{\text{T}} \varnothing(x_{j})\parallel^{2}F_{ij} \tag{15}\]  
This equation can be rewritten from the square of norm in Equation (15) into the form of trace as follows:  
\[\begin{split}\sum_{ij}^{n}\parallel v_{\varnothing}^{\text{T}} \varnothing(x_{i})-v_{\varnothing}^{\text{T}}\varnothing(x_{j})\parallel^{2}F_ {ij}\\ =\text{tr}\ \left\{\sum_{ij}^{n}\bigl{(}v_{\varnothing}^{\text{T}} \varnothing(x_{i})-v_{\varnothing}^{\text{T}}\varnothing(x_{j})\bigr{)}\bigl{(}v_ {\varnothing}^{\text{T}}\varnothing(x_{i})-v_{\varnothing}^{\text{T}}\varnothing (x_{j})\bigr{)}^{\text{T}}F_{ij}\right\}\\ =\text{tr}\ \{v_{\varnothing}^{\text{T}}\sum_{ij}^{n}\left(2 \varnothing(x_{i})F_{ij}\varnothing(x_{j})^{\text{T}}-\left(2\varnothing(x_{j} )F_{ij}\varnothing(x_{i})^{\text{T}}\ F_{ij}\right)V_{\varnothing}\right\}\end{split} \tag{16}\]The linear transformation should lie in the span of \(\varnothing(x_{1}),\varnothing(x_{2}),\dots,\varnothing(x_{n})\) and \(\alpha=[\alpha_{1},\alpha_{2},\dots,\ \alpha_{n}]\) are the expansion coefficient vectors and  
\[V_{\varnothing}=\sum_{i=1}^{n}\alpha_{i}\varnothing(x_{i})=\varnothing(X)\ \alpha \tag{17}\]  
Substituting (17) into (16) we obtain:  
\[\begin{array}{l}U_{\varnothing}=2\mathrm{tr}\left\{v_{\varnothing}^{\mathrm{ T}}\varnothing(X)(D-F)\varnothing(X)^{\mathrm{T}}V_{\varnothing}\right\}=2 \mathrm{tr}\left\{v_{\varnothing}^{\mathrm{T}}\varnothing(X)L\varnothing(X)^{ \mathrm{T}}V_{\varnothing}\right\}\\ =2\sum_{l=1}^{m}\{v_{\varnothing l}^{\mathrm{T}}\varnothing(X)L\varnothing(X)^{ \mathrm{T}}V_{\varnothing l}\}\end{array} \tag{18}\]  
where \(D_{ii}=\sum_{j}F(i,j)\) is diagonal matrix and \(L=D-F\), \(L\) and \(D\) are the symmetric matrix and represent the number of eigenvalues. The optimization problem can be rewritten as:  
\[\mathrm{Maximize}\ \mathrm{tr}\left\{v_{\varnothing}^{\mathrm{T}}\varnothing(X)L \varnothing(X)^{\mathrm{T}}V_{\varnothing}\right\}=\alpha^{\mathrm{T}}KLK\alpha \tag{19}\]  
Subject to \(\alpha^{\mathrm{T}}K\alpha=I\), where \(K\) is a kernel matrix with \(k(xi,xj)=[\ \varnothing(x_{i}).\varnothing(x_{j})]\), and a kernel in matrix form is:  
\[K=\varnothing(X)^{\mathrm{T}}\varnothing(X) \tag{20}\]  
The corresponding generalized eigenvalue problem can be solved through calculating the maximum eigenvalues in \(\varnothing(X)L\varnothing(X)^{\mathrm{T}}V_{\varnothing}=\lambda V_{\varnothing}\), where the generalized eigenvector equivalent to the biggest eigenvalue is the main interest. Finally, we need to compute the dot product through a kernel and find its nearest neighbor in the embedding space. It should be noted that, although SKLDNE and other methods like DNE consider geometrical information of both intra-class and inter-class, the construction of their contiguous graphs and weights are done in very diverse ways.