## 2 The Framework of Deep Uniform Kernel Mapping Support Vector Machine (_D_U_KMSVM)
In short, DMKMSVM is a SVM concerning \(\phi\left(\mathbf{x}^{l}\right)\), and its output is the classification result \(\phi\left(\mathbf{x}^{l}\right)\) utilizing SVM as a classifier. Figure 1: The architecture of deep uniform kernel mapping support vector machine (DUKMSVM). Figure 2: The architecture of multi-layer perceptron kernel mapping (DMKMSVM). In the course of training, the objective function of DMKMSVM is:  
\[ \begin{array}{l}L_{N}=\min\limits_{\begin{subarray}{c}W^{l}\psi_{j,l}^{l}\\ 1\leq j<r+1,1\leq l\leq N\end{subarray}}\frac{1}{2}||W^{r+1}||^{2}+C\sum\limits _{l=1}^{N}\xi_{l}^{2}\\ s.t.y^{l}(W^{r+1}\phi(x^{l})+b^{r+1})\geq 1-\xi_{l},\xi_{l}\geq 0,l=1,...,N\end{array}  \tag{2}\]  
where \(W^{r+1}\) and \(b^{r+1}\) refer to the weight and bias of the SVM layer, respectively. Hence, the issue of function constraint extremum can be equivalently converted to the following optimization problem without constraint:  
\[L_{N}=\operatorname*{arg\,min}_{\theta}\frac{1}{2}||W^{r+1}||^{2}+C\sum\limits _{l=1}^{N}\left[max(1-y^{l}\times(W^{r+1}\phi(x^{l}|\theta)+b^{r+1}),0)\right]^ {2} \tag{3}\]  
where \(\theta=\left\{W^{l},b^{l},1\leq j\leq r\right\}\cup\left\{W^{r+1},b^{r+1}\right\}\) are all the parameters to be learned, and \(y^{l}\) is the label of \(x^{l}\), whose value is \(+1\) or \(-1\). In order to train DMKMSVM effectively, we take advantage of the two-stage training method integrating unsupervised pre-training with Restricted Boltzman Machine (RBM) and supervised fine-tuning. The detailed description is illustrated in Algorithm 1. ```
Input:Training set \(S=\left\{(x^{l},y^{l}),1\leq l\leq N\right\}\), network architecture, max number of epoch Output: \(\theta=\left\{W^{l},b^{l},1\leq j\leq r+1\right\}\) Stage One: Pre-training 1. Randomly initialize \(W\approx 0,b^{l}\approx 0\), \(1\leq j\leq r+1\) ; 2. Learn \(W^{l}\) and \(b^{l}\) for every \(RBM\) of \(\textbf{h}_{j-1}\) and \(\textbf{h}_{j},1\leq j\leq r\) ; Stage Two: Fine-tuning 3. For\(epoch=1\) to maxepoch do
4. For\(l\)=1 to \(N\)do
5. Compute \(\textbf{h}_{0}=x^{l}\), \(\textbf{h}^{l}_{j}=W^{l}\textbf{h}_{j-1}+b^{l}\), \(\textbf{h}_{j}=\sigma\left(\textbf{u}^{l}_{j}\right),1\leq j\leq r\)
6. Compute \(\phi\left(x^{l}\right)=\sigma\left(W^{l}\textbf{h}_{r-1}+b^{r}\right)\)
7. Compute \(o^{l}=W^{r+1}\phi\left(x^{l}\right)+b^{r+1}\)
8. Compute \(\delta^{l}_{j}=-2 Cyl\left(\max\left(1-y^{l}o^{l},0\right)\right)\left(l \left\{1>y^{l}o^{l}\right\}\right)\)
9. Compute \(\delta^{l}_{j}=\left[\left(W^{l+1}\right)^{T}\delta^{l}_{j+1}\right]\circ \sigma\left(\textbf{u}^{l}_{j}\right),1\leq j\leq r\)
10. Compute \(\left\{ \begin{array}{l}\frac{\partial L_{N}}{\partial W}=W^{l}-\sum\limits _{l=1}^{N}\delta^{l}_{j-1}\left(\textbf{h}^{l}_{j-1}\right)^{T},j=r+1\\ \frac{\partial L_{N}}{\partial W}=\sum\limits_{l=1}^{N}\delta^{l}_{j}\left( \textbf{h}^{l}_{j-1}\right)^{T},1\leq j\leq r\\ \end{array} \right.\)
11. Compute \(\frac{\partial L_{N}}{\partial\textbf{b}^{l}}=\sum\limits_{l=1}^{N}\delta^{l }_{j},1\leq j\leq r+1\)
12. Update weights and biases: \(W^{l}\gets W^{l}-\eta\frac{\partial L_{N}}{\partial W^{l}},b^{l}\gets b ^{l}-\eta\frac{\partial L_{N}}{\partial W}\)
13. End For
14. End For
```  
**Algorithm 1** Joint training of DMKMSVM