## 4 Artificial Neural Network  
An artificial neural network (ANN) has two types of basic components, namely, neuron and link. A neuron is a processing element and a link is used to connect one neuron with another. Each link has its own weight. Each neuron receives stimulation from other neurons, processes the information, and produces an output. Neurons are organized into a sequence of layers. The first and the last layers are called input and output layers, respectively, and the middle layers are called hidden layers. The input layer is a buffer that presents data to the network. It is not a neural computing layer because it has no input weights and no activation functions. The hidden layer has no connections to the outside world. The output layer presents the output response to a given input. The activation coming into a neuron from other neurons is multiplied by the weights on the links over which it spreads, and then is added together with other incoming activations. A neural network in which activations spread only in a forward direction from the input layer through one or more hidden layers to the output layer is known as a multilayer feed-forward network. For a given set of data, a multi-layer feed-forward network can provide a good non-linear relationship. Studies have shown that a feed-forward network, even with only one hidden layer, can approximate any continuous function [28]. Therefore, a feed-forward network is an attractive approach [29]. Figure 3 shows an example of a feed-forward network with three layers. In Figure 3, _R, N,_ and \(S\) are the numbers of input, hidden neurons, and output, respectively; _iw_ and _hw_ are the input and hidden weights matrices, respectively; _hb_ and _ob_ are the bias vectors of the hidden and output layers, respectively; \(x\) is the input vector of the network; _ho_ is the output vector of the hidden layer; and \(y\) is the output vector of the network. The neural network in Figure 1 can be expressed through the following equations:  
\[hoi=f(\sum\limits_{j=1}^{R}iw_{i,j}\cdot x_{j}\ +\ hbi),\textit{for}\ i=1,..,N \tag{8}\]  
\[y_{i}=f(\sum\limits_{k=1}^{N}hw_{i,k}\cdot ho_{k}\ +\ obi),\textit{for}\ i=1,..,S \tag{9}\]  
Where \(f\) is an activation function.  
Figure 3: A feed-forward network with three layers.  
When implementing a neural network, it is necessary to determine the structure in terms of number of layers and number of neurons in the layers. The larger the number of hidden layers and nodes, the more complex the network will be. A network with a structure that is more complicated than necessary over fits the training data [30]. This means that it performs well on data included in the training dataset, but may perform poorly on that in a testing dataset.  
Once a network has been structured for a particular application, it is ready for training. Training a network means finding a set of weights and biases that will give desired values at the network's output when presented with different patterns at its input. When network training is initiated, the iterative process of presenting the training data set to the network's input continues until a given termination condition is satisfied. This usually happens based on a criterion indicating that the current achieved solution is good enough to stop training. Some of the common termination criteria are sum squared error (SSE) and mean squared error (MSE). Through continuous iterations, the optimal or near-optimal solution is finally achieved, which is regarded as the weights and biases of a neural network. Suppose that there are \(m\) input-target sets, \(x_{k}-t_{k}\) for \(k=1\), \(2\), \(...\), \(m\), for neural network training. Thus, network variables arranged as \(iw\), \(hw\), \(hb\), and \(ob\) are to be changed to minimize a cost function. \(E\), such as the SSE between network outputs, \(y_{k}\), and desired targets, \(t_{k}\), is as follows:  
\[E{=}\sum\limits_{k=1}^{m}\epsilon_{k}^{2}=\sum\limits_{k=1}^{m}(t_{k}-y_{k})^{2} \tag{10}\]