## 4 How to Experiment with the Database
### Bilayer Neural Network  
The ANN aims to associate a given input feature that represents the pattern to one of the classes of interest. The simplest ANN can be represented by the model given in Figure 5. The _inputs_\(X=x_{1},x_{2},\cdots,x_{n}\) are signals, samples, or features of the data. They are usually normalized to improve the learning algorithm. The _weights_\(W=w_{1},w_{2},\cdots,w_{n}\) are values used to weigh each one of the inputs. Their objective is the quantification of their relevance within neuron functionality. The _bias_ (\(b\)) is a variable used to specify an appropriate threshold, in other words, it is the way to generate a trigger value toward the neuron output by the linear aggregator. The _linear aggregator_ (\(\Sigma\)) assembles all input signals weighted by the weights to produce an activation of the neuron. The _activate function_ (\(f\)) limits the neuron output within a reasonable range of values given by the difference (\(\delta\)) between the linear aggregator and the activation threshold. An excitatory potential for a positive \(\delta\) value or inhibitory otherwise. The activation functions can be classified into two groups: (1) partially differentiable functions such as step function, signal function, and a symmetric ramp function. (2) fully differentiable functions such as logistic function, hyperbolic tangent function, and Gaussian function. The _outputs_\(Y=y_{1},y_{2},\cdots,y_{n}\) are the final values produced by the neuron for a particular set of input signals. The model for one output \(y\) can be expressed as  
\[y =f(\delta) \tag{10}\] \[\delta =\sum_{i=1}^{n}w_{i}.x_{i}-b \tag{11}\]  
A Rectified Linear Unit (ReLU) layer can be used to perform the threshold operation on each element of the input, where any value less than zero is set to zero as follows  
\[f(x)=\left\{\begin{array}{cc}x,&x\geq 0\\ 0,&x<0\end{array}\right. \tag{12}\]  
A bilayered neural network is a network with two hidden layers, see Figure 6. Each layer extracts patterns associated with the process of interest, where their learning algorithms architectures are based on the backpropagation algorithm and the competitive rules. During this process is when the network extracts discriminant characteristics from the input layer to make the feature map. Moreover, in the training stage, the forward (when the input is inserted into the network) and backward (when the weights are tuned) stages let the weights and thresholds of the neurons be adjusted automatically in each iteration, to reduce the final error between the network responses and the desired responses. For example, for a binary detection pattern, the first output may be for a healthy pattern, and the second output for a defective pattern.  
To use the bilayered neural network as a quality detector, the observations are classifiers according to their scores using the softmax activation function. These scores, which accompany the end fully connected layer in the network, are estimated through the decision rule or posterior probability, that an observation \(x\) is of class \(k\)  
\[\hat{P}(k|x)=\frac{P(x|k)P(k)}{\sum_{j=1}^{K}P(x|j)P(j)}=\frac{\exp(a_{k}(x))}{ \sum_{j=1}^{K}\exp(a_{j}(x))} \tag{13}\]  
where \(P(x|k)P(k)\) is the conditional probability of \(x\) given the class \(k\), \(K\) is the number of classes in the response variable, \(P(k)\) is the prior probability for class \(k\), it tells us how likely each of the classes is a prior, or before the data \(x\) is observed, and \(a_{k}(x)\) is the \(k\) output from the end fully connected layer for the observation \(x\). We refer the reader to [28; 29] for a comprehensive treatment of the mathematical properties of the neural networks.