## 3 Classifier
### Multilayer Perceptron  
Multilayer perceptron (MLP) is a back-propagation neural network with high learning accuracy and fast recall. It can handle complex sample discrimination and highly nonlinear function synthesis problems where the output values can be suspended values. It is a popular neural network that has a wide range of applications that include: sample identification, bifurcation problems, function simulation, prediction, system control, noise filtering, data compression, etc. MLP is a back-propagating supervised learning algorithm, through \(f(\cdot):R^{m}\to R^{o}\), \(m\) is the dimension at input and \(o\) is the dimension at output. By inputting the feature \(X=x_{1},x_{2},\ldots x_{m}\) and the target value \(Y\), this algorithm can classify the data using nonlinear approximation or perform regression. MLP can have many nonlinear layers inserted between the input and output layers. The stochastic gradient descent (SGD) method is used in MLP training. SGD uses the gradient of the loss function, relative to the parameter that needs to be adaptive for updating, see Equation (16), where \(\eta\) is the learning rate in the control parameter space search step, and Loss is the loss function used by the network. \[w\gets w-\eta\bigg{(}\alpha\frac{\partial R(w)}{\partial w}+\frac{ \partial\text{Loss}}{\partial w}\bigg{)} \tag{16}\]  
If a training sample set \((x_{1},y_{1}),(x_{2},y_{2}),\ldots\ldots,(x_{n},y_{n})\) is given, where \(x_{i}\in R^{n}\) with \(y_{i}\in\{0,1\}\), then the MLP learning function of one hidden layer and one hidden neuron is shown as in Equation (17)  
\[f(x)=W_{2}g(W_{1}^{T}x+b_{1})+b_{2} \tag{17}\]  
In Equation (17), \(W_{1}\in R^{m}\) and \(W_{2},b_{1},b_{2}\in R\), is the model parameter. \(W_{1}\) and \(W_{2}\) the weights of the input and hidden layers respectively, and \(b_{1}\) and \(b_{2}\) are the deviations added to the hidden and the output layers. \(g\) is the activate function, set here as the hyperbolic tangent (tanh), the equation is shown as (18)  
\[g(z)=\frac{\varepsilon^{z}-e^{-z}}{\varepsilon^{z}+e^{-z}} \tag{18}\]  
For binary classification, \(f(x)\) can have an output value between 0 and 1 through the logic function \(g(z)=1/(1+e^{-z})\). With the threshold set to 0.5, the output sample will be greater than or equal to 0.5 in the positive category, and the rest will be negative. If there are more than two categories, \(f(x)\) will be a vector of size n and will be a softmax function rather than a logical one. \(z_{i}\) represents the \(i\)th element input to softmax, which corresponds to the \(i\)th category, and K is the number of categories. The result is a probability vector that contains sample \(x\) for each category. The output category is the one with the highest probability, the mathematical Equation (19) is  
\[\text{softmax}(z)_{i}=\frac{\exp(z_{i})}{\sum_{l=1}^{k}\exp(z_{l})} \tag{19}\]  
As for the regression method, the output remains \(f(x)\), so the output start function is an identical function. MLP uses a different loss function, depending on the type of problem. The loss function of the classification has cross entropy, and in the binary case, its loss function is shown in Equation (20)  
\[Loss(\widehat{y},y,W)=-y\ln\widehat{y}-(1-y)\ln(1-\widehat{y})+\alpha\|w\|_{ 2}^{2} \tag{20}\]  
Starting with the initial random weight, the multilayer perceptron (MLP) reduces the loss function to the greatest extent by repeatedly updating these weights. After the loss has been calculated, it is passed back to propagate from the output layer to the previous layer, and each weight parameter is provided with an updated value to reduce error in the loss function. In the gradient descent, the update of the weight can be expressed as Equation (21)  
\[W^{i+1}=W^{i}-\varepsilon\
abla Loss_{w}^{t} \tag{21}\]  
In Equation (21), \(i\) is the iteration step and the learning rate \(\varepsilon\) is a value greater than zero.