## 4 Experiment
### Convolution Neural Network
#### 4.1.2 Convolutional Neural Network Architecture  
A CNN is a special case of ANNs, as shown in Figure 12. CNN has been proven excellent performance in recent various image processing tasks [31; 32]. Some representative CNN architectures include AlexNet [10], VGGNet [11], Inception [12], ResNet [13], and DenseNet [14]. The basic building blocks of a CNN are a convolution layer and pooling layer. Contrary to the basic ANN where the previous layer is fully connected to the next layer, a small region of the previous layer (\(m\) by \(n\) by \(c^{\prime}\)), known as the receptive field, is connected to the next layer by the convolution operation in CNN. This convolution operation is repeated starting from the top-most left receptive field to the right-most bottom receptive field with the same convolution weights. Once the convolution operation is completed, another convolution operation with different weights is performed in the same manner. The number of different sets of convolution weights is known as the number of convolution filters between two layers. Since the same weights are used for a specified convolution filter, the number of weights to be learned is much smaller with the CNN than fully connected ANNs. In addition, a single convolution filter can find a significant local pattern over all regions in the previous layer using the same weights for the corresponding convolution filter. Once the convolution step is completed, the size of the resulting tensor is reduced by pooling operation to improve the learning process efficiency. The pooling result depends on the size and method of pooling: the former decides how a large area is summarized while the latter decides how to represent the targeted area. Max pooling and average pooling are two common pooling methods: the former uses the largest value while the latter uses the average values of the targeted area. After the set of convolution and pooling layers are processed, many commonly used CNN architectures place fully connected layers between the end of convolution building blocks and the output layer. For a classification task, the number of nodes in the output layer is set to the number of classes, and the output values of the CNN are computed using the softmax function to ensure that every output node value is between 0 and 1, and the sum of all output node values is 1. By doing so, the output node value can be interpreted as the probability of the input image belonging to the corresponding class. Because the number of weights to be learned is still quite large, some  
Figure 11: Convolution and max pooling operations in CNN. practical techniques are used to improve training efficiency [33], such as adopting a rectified linear unit (ReLU) [10] as an activation function and batch normalization [34] during the weight learning by the gradient descent algorithm. The most commonly used input image size for CNN architecture is \(224\times 224\), which is the input size of the dataset provided by ImageNet [35], one of the largest labeled datasets for image processing. The maximum input image size in this study is \(47\times 62\). Hence, we used a simpler CNN architecture, i.e., the number of convolution and pooling layers is less than that of other well-known CNN architectures. Figure 12 shows the CNN architecture used in our study. Four convolution and pooling layers were used followed by two fully connected layers. For all convolution layers, \(3\times 3\) convolution operation with stride size of 1 was performed, followed by \(2\times 2\) max pooling with stride size of 2. The number of convolution filters are 64, 128, 256, and 256 for the first, second, third, and fourth convolution layers, respectively. The number of nodes used for the first and second fully connected layers are 1048 and 1024, respectively. The number of output nodes was set to two: good and bad. We used ReLU and batch normalization after all of the convolution layers. The batchsize was set to 32.