# A Review of Convolutional Neural Network Applied to Fruit Image Processing
## 3 Background on Convolutional Neural Networks
### CNN Architecture  
Contrary to traditional neural networks, CNNs use convolution operations in at least one of its layers [9; 19]. The CNN architecture includes multiple stages or blocks composed of four main components: a filter bank called kernels, a convolution layer, a non-linearity activation function, and a pooling or subsampling layer. Each stage aim to represent features as sets of arrays called feature maps (see Figure 2) [12; 19; 42; 43]. We depict a typical CNN architecture in Figure 3, comprising of a stack of several convolutional stages and one or more fully connected layers, which gives the final output as a classification module. Following we introduce the main components of a typical CNN architecture. **Filter bank or kernels:** each filter or kernel aims to detect a particular characteristic at each input location, therefore, the spatial translation of the input from a characteristic detection layer will be transferred to the output without changes [43]. As it is defined by [PERSON] [43], there is a bank of \(m_{1}\) filters in each convolutional layer and the output \(Y_{i}^{(l)}\) of the \(l^{th}\) layer consists of \(m_{1}^{(l)}\) feature maps of size \(m_{2}^{(l)}\times m_{3}^{(l)}\). The \(i^{th}\) feature map is computed as follows:  
\[Y_{i}^{(l)}=B_{i}^{(l)}+\sum_{j=1}^{m_{1}^{(l-1)}}K_{ij}^{(l)}*Y_{j}^{(l-1)} \tag{1}\]  
where \(B_{i}^{(l)}\) denotes the trainable bias parameters matrix, \(K_{ij}^{(l)}\) is the filter with dimensions \((2h_{1}^{(l)+1}\times 2h_{2}^{(l)+1})\) that connect the \(j^{th}\) feature map of \((l-1)\) layer with \(i^{th}\) feature map of \((l)\) layer, and \((*)\) is the 2D discrete convolution operator. **Convolution layer:** the convolution operation is widely used in digital image processing where the 2D matrix representing the image \((I)\) is convolved with the smaller 2D kernel matrix \((K)\), then the mathematical formulation with zero padding is given by [9]:  
\[S_{i,j}=(I*K)_{i,j}=\sum_{m}\sum_{n}I_{i,j}\cdot K_{i-m,j-n}. \tag{2}\]  
In the convolution process, a small sliding filter operates from left to right through the image from top to bottom. Figure 4 shows an example of the convolution operation with an input image \((4\times 4)\) and a convolution kernel \((3\times 3)\), obtaining an output convolved image. At each location, it is computed the sum of the products between each kernel element and the corresponding input element. This process is repeated using different kernels to form as many output feature maps as desired [44]. Figure 3: General architecture of a convolutional neural network. The dimensions of the output characteristics map are more reduced than the input. Alternatively, we can apply a padding technique to keep the same in-plane dimension by adding zeroes around the input and fitting the center of the kernel on outermost elements [44, 45]. Besides, the stride denotes the size of the passage between two successive positions of the kernel nucleus. Generally, a stride equal to 1 is chosen, but sometimes a stride greater than 1 is used to reduce the resolution of feature maps constituting to subsampling. **Nonlinear activation function:** after the filter bank produces the output, a nonlinear activation function is applied (Equation (1)) to produce the activation maps, where only the activated features are carried forward to the next layer. This function determines the behavior of the neuron output. Then, the operation of the activation function \(f(\cdot)\) is as follows:  
\[\phi(Y_{i}^{(l)})=f\left(B_{i}^{(l)}+\sum_{j=1}^{m_{1}^{(l-1)}}K_{ij}^{(l)}*Y_ {j}^{(l-1)}\right) \tag{3}\]  
There are different types of activation functions. Currently, the most widely used in CNN are:  
* _Rectified Linear Unit function (ReLU)_: ReLU is the most used activation function for convolution layers. It is a half rectified function [19, 46] (see Figure 5a). It is mathematically defined as: \[f(x)=max(0,x)= \begin{cases}0&\text{if}\quad x<0\\ x&\text{if}\quad x\geq 0\end{cases}