## 3 MLP Neural Network and Training
### MLP Neural Networks  
The multilayer perceptron neural network consists of a determined number of neurons, organized in two or more layers, highly interconnected [39]. The neuron operation consists of applying a determined transfer function to a weighted sum of its inputs. The MLP topology can be seen in Figure 4. It is usual the choice of a \"tan-sigmoid\" transfer function for hidden layers and a \"linear\" function for output layer.  
The MLP network requires supervised learning. It uses a given set of input-output pairs for adjustment of the interconnection weights. The backpropagation learning algorithm is applied to adjust the weight array, \(W\), associated with the interconnections. For each learning cycle, the ANN receives the input vectors (organized in the input matrix array containing the patterns, \(P\)). The resulting output vectors are compared to the desired outputs contained in the learning patterns (organized in the output matrix array containing the targets, \(T\)), and the performance index is calculated. This index is the mean squared error, \"_mse_\" [40], given in (15). Figure 4a shows the MLP with the inputs and outputs used in the training process. Figure 4b shows the operation of a neuron in any of the  
Figure 4: MLP topology: (**a**) network containing \(n\) input units, two 6-neuron hidden layers and 2 neurons in the output layer, where the subindex “\(i\)” stays for the different training patterns; (**b**) detail of the neuron function, where \(p_{j}\) are the neuron input values, \(w_{j}\) are the connection weights, and the subindex “\(j\)” stays for the different neuron inputs.  
layers. The mathematical backpropagation algorithm determines the re-adjustment of the interconnection weights to reduce the _mse_. This cycle is iteratively repeated until the _mse_ reaches a small enough value, previously given as the training goal.  
The performance index, _mse_, taken as the accuracy goal whose value must be minimized, is defined as in (15):  
\[mse=\frac{1}{m}{\sum\limits_{i=1}^{m}{\|\mathbf{y}_{i}-\mathbf{t}_{i}\|^{2}}} \tag{15}\]  
where \(m\) is the number of training patterns, and \(\mathbf{y_{i}}\), \(\mathbf{t_{i}}\), are the output and target vectors corresponding to the pattern \(i\). The difference \(\mathbf{y_{i}}-\mathbf{t_{i}}\) is the error vector for that pattern, and \(\|\mathbf{y_{i}}-\mathbf{t_{i}}\|\) expresses the error vector length in the output space. In particular, with a two-neuron output layer, the output space is bidimensional. The sum of the squares of those error lengths, divided by \(m\), gives the mean squared error, which is calculated after every iteration or epoch.  
Among the different available backpropagation algorithms, the default algorithm applied in feedforward ANN training by the Neural Network Toolbox in MATLAB is the Levenberg-Marquardt algorithm, LM. It is also known as Damped Least-Squares, DLS. The authors have also tested other algorithms, such as the Scaled Conjugate Gradient, SCG. However, the best results were, in general, obtained with the LM algorithm.  
The required number of neurons and neuron layers is selected after some tests of convergence. We have experienced with one, two, and three hidden layers.  
The work stage after the training usually consists of some simulation tests to check the performance of the resulting network, evaluating its \"generalization\" capability. It consists of checking the ability of the ANN to give the correct output for input vectors different from those used for training.