# A Review of Convolutional Neural Network Applied to Fruit Image Processing
## 3 Background on Convolutional Neural Networks
### CNN Architecture
\] (4)
* _Sigmoid function_: its curve looks like a S-shape as it is shown in Figure 5b [9, 10]. The function varies between \([0,1]\), therefore it is used to predict a probability as an output. Mathematically it has the form: \[f(x)=\frac{1}{1+e^{-x}}\] (5)
* _Hyperbolic Tangent (tanh) function_: the _tanh_ function has similar form to Sigmoid function [9, 10], as it is depicted in Figure 5c, but the range is \([-1,1]\). The advantage is that the zero values will be mapped near zero, and negative values will be mapped strongly negative. Its mathematical definition is: \[f(x)=\tanh(x)=\frac{2}{1+e^{-2x}}-1\] (6)  
Figure 4: Example of the convolution operation with an input image (\(4\times 4\)) and a \(3\times 3\) kernel. **Pooling layer**: it reduces the number of parameters of the network by reducing the spatial size of convolutional outputs. Additionally, pooling operations contribute to obtaining an invariant representation to small translations of the input [9; 11; 47]. The two main pooling operations are explained following and Figure 6 depicts an example of pooling operations by using a \(2\times 2\) filter. * _Max pooling_: it calculates the maximum value for each patch of the input [48; 49]. The max-pooling layer preserves the maximum value of each patch by sliding the filter over the feature map. Mathematically it has the form: \[f_{max}\left(A\right)=max_{n\times m}\left(A_{n\times m}\right)\] (7) Commonly, in max pooling layer a \(2\times 2\) filters are applied with a stride of 2. It downsamples the input by 2 along its dimensions and discards the 75% of the convolutional outputs. * _Average pooling_: it computes the average value for each patch of the input [48; 49]. The average pooling layer downsamples the convolutional activation by dividing the input into pooling regions and computing their average values. It it matematically defined as follows: \[f_{ave}\left(A\right)=\frac{1}{n+m}\sum_{i=1}^{n}\sum_{k=1}^{m}\left(A_{i,k}\right)\] (8)  
**Dropout layer**: it is a regularization layer that randomly drops neuron units of the network, preventing the units from co-adapt too much. The dropout technique allows facing the overfitting problem, at the same time, it improves the performance of the network. It can be applied to any layer in the network. **Fully connected (FC) layer**: the final output of the convolutional stages is flattened to a 1D array and connected to a fully connected layer. FC layers take the results of the convolution/pooling process and use them to classify the image into a label (i.e., class), like a traditional neural network. Thus, the activation function of the last layer (i.e., output layer) computes the final probabilities for each class and it is selected according to the task. Typically, a multi-class classification task uses the Softmax function, where each class probability value ranges between \([0,1]\), and their total sum is  
Figure 5: Curve representations of most used activation functions: (**a**) ReLU, (**b**) Sigmoid, and (**c**) Hyperbolic Tangent. Figure 6: Examples of pooling operations by using a \(2\times 2\) filters applied with a stride of 2.  
equal to 1. Finally, each output neuron decides on each of the labels, and the greatest output value corresponds to the classification decision.