## 3 Methodology
### Convolutional Neural Network  
A deep neural network (DNN) is an artificial neural network with multiple hidden layers between the input and output layers, which can enhance classification accuracy. CNN is a special type of DNN designed for image classification [35]. The basic structure of CNN consists of 5 layers: the input layer, the convolutional layer, the pooling layer, the fully connected layer, and the output layer; detailed explanations of these layers are provided in the following [1; 12; 32; 36]:  
* Input layer The input layer receives the raw pixel values of the original images. This layer includes a three-dimensional matrix, in which each dimension denotes one of the dimensions of the image, for example, height, width, and the number of color channels. * Convolutional layer A convolution layer is also known as the feature extractor layer, since features are extracted from the image within this layer. To perform the convolution, a convolution filter is simply slid across the image while maintaining the spatial relationships between the pixels, and the sum is calculated by multiplying the filter's elements by the square of the image it covers. The input to the convolutional layer consists of the three-dimensional matrix, height, width, and the number of color channels. The convolutional process is shown in Equation (1) [36]:  
\[x_{i}^{l}=f\left(\sum_{j=1}^{m^{l-1}}x_{i}^{l-1}w_{i,j}^{l}+b_{i}^{l}\right), \left(i=1,2,\ldots,m^{l}\right) \tag{1}\]  
where \(x_{i}^{l}\) is the \(i\)th output feature on the \(l\)th layer, \(x_{i}^{l-1}\) is the \(j\)th output feature on the \((l-1)\)th layer, \(w_{i,j}^{l}\) is the weight vector of the convolutional kernel between the \(i\)th feature on the \(l\)th layer and the \(j\)th feature on the \((l-1)\)th layer, \(b_{i}^{l}\) is the bias, and \(m^{l}\) is the number of features on the \(l\)th layer. In addition, each component of the convolution feature map is treated as a nonlinear function \(f(x)\). In particular, the rectified linear unit (ReLU) [37] is a commonly used nonlinear function, which we refer to as the activation function. \[f(x)=\max(0,x) \tag{2}\]
* Pooling layer The pooling layer is applied to reduce the amount of computational power required to process the data by using dimensionality reduction. Two types of pooling method are commonly used: max-pooling and average-pooling. In the max-pooling method, the maximum value from the region covered by the filter is used, and in the average-pooling method, the average of the values is used. The mathematical equations for both methods are presented in Equations (3) and (4):  
\[y_{ij}=\text{Max }x_{i,j} \tag{3}\]  
\[y_{ij}=\text{Average }x_{i,j} \tag{4}\]  
These two common types of pooling method are shown in Figure 1.
* Fully connected layer The output of the final pooling or convolutional layer is flattened and fed as input to the fully connected layer. The fully connected layer includes weights, biases, and neurons, which are used to learn the possible nonlinear function in the space. * Output layer The output layer is also fully connected, and contains the class label that is the target for the classification tasks. In this layer, the Sigmoid function is applied as an activation function if the target is a binary classification task, and the Softmax activation function activates each neuron for multi-class logistic regression and compresses its output to valuesbetween 0 and 1 for multi-classification tasks [1; 12; 35; 38]. The mathematical equations for the two activation functions are shown in Equations (5) and (6):  
\[\text{Sigmoid}:\ f(x)=\frac{1}{1+e^{-x}} \tag{5}\]  
\[\text{Softmax}:\ f(z)_{i}=\frac{e^{z_{i}}}{\sum_{j=1}^{K}e^{z_{j}}}\,(i=1,2, \ldots,K) \tag{6}\]  
where all \(z_{i}\) values are the elements of the input vector to the final output layer and can take any real value. A simple CNN architecture is shown in Figure 2. The input layer receives the original images as the input. The convolutional layer is used to extract high-level features from the input images.