# Data Sampling Methods to Deal With the Big Data Multi-Class Imbalance Problem
## 2 Deep Learning Multi-Layer Perceptron  
The Multi-Layer Perceptron (MLP) constitutes the most conventional Artificial Neural Network (ANN) architecture [54]. It is formed by a set of simple elements called computational nodes or neurons, where each node sends a signal that depends on its activation state [55]. Before arriving at the receiver neuron, the output is modified multiplying it by the corresponding weight of the link. The signals (_net_) are accumulated in the receiver node and the neurons are grouped forming layers. MLP is commonly based on three layers: input, output and one hidden layer [7]. The MLP has been translated into a deep neural network by incorporating two or more hidden layers within its architecture, making it a Deep Learning MLP (DL-MLP). This allows it to reduce the number of nodes per layer and it uses fewer parameters, but it also leads to a more complex optimization problem [8]; however, due to the availability of more efficient frameworks, such as Apache-Spark and Tensorflow, this disadvantage is less restrictive than before. MLP is a practical vehicle with which to perform a nonlinear input-output mapping of a general nature [56]. Given a set of input vectors \(\mathbf{x}\) and a set of wished answers \(\mathbf{d}\), learning systems must find parameters linking these specifications. The \(j\) output of an ANN: \(y_{j}=\hat{f}(\mathbf{x},\mathbf{w})\) depends on a set of parameters \(\mathbf{w}\) (weights), which can be modified to minimize the discrepancy between the system output \(\mathbf{z}\) and the answer desired \(\mathbf{d}\). The aim of a MLP is to represent the behavior of \(\hat{f}(\mathbf{x},\mathbf{w})\), in a region \(R\) of the input space, by means of a linear combination of functions \(\varphi_{j}(\mathbf{x})\) as follows:\[\hat{f}(\mathbf{x},\mathbf{W})=\sum_{h=1}w_{kh}^{L}\varphi_{h}^{(L-1)}(\mathbf{x}), \tag{1}\]  
\[\varphi^{(L-1)}(\mathbf{x})=\sum_{h=1}w_{h}^{(L-1)}\varphi_{h}^{(L-2)}(s_{i}), \tag{2}\]  
\[s_{i}=\sum_{h}w_{hi}^{l}\varphi_{h}^{(l-1)}(s_{(i-1)}). \tag{3}\]  
If \(l=1\) then \(s_{i}=\sum_{h}w_{hi}x_{h}\); i.e., \(s_{i}\) is the \(i\)-th input hidden neuron on the first hidden layer; \(w_{i}^{l}\) is the weight connected to the \(i\)-th input of the \(h\) hidden neuron on the \(l\)-th hidden layer with \((l-1)\)-th hidden layer; and \(\varphi_{h}^{l-1}\) is the \(h\)-th hidden node output on the \((l-1)\)-th hidden layer. \(L\) is the total number of hidden layers. Traditionally, MLP has been trained with the back-propagation algorithm (which is based in the stochastic gradient descent) and its weights randomly initialized [54; 57]. However, in the latest versions of DL-MLPs, the hidden layers are pretrained by an unsupervised algorithm and the weights are optimized by the back-propagation algorithm [7]. MLP uses sigmoid activation functions, such as the hyperbolic tangent or logistic function. In contrast, DL-MLP includes (commonly) the Rectified Linear Unit (ReLU) \(f(z)=\max\,(0,z)\) because it typically learns much faster in networks with many layers, allowing the training of a DL-MLP without unsupervised pretraining. There are three variants of the descending gradient that differ in how many data are used to process the gradient of the objective function [58]: (a) batch gradient descent calculates the gradient of the cost function to the parameters for the entire training dataset, (b) stochastic gradient descent performs an update of parameters for each training example and (c) mini-batch gradient descent takes the best of the previous two and performs the update for each mini-batch of a given a number of training examples.