## 2 Materials and Methods
### Transfer Learning Using Pre-Trained Neural Networks  
Transfer learning is a neural network learning method that utilizes a similar problem solved by the network model, but then retrains it with the training data using only a specific part of the trained model [48]. There are currently two main methods to implement transfer learning. The first method retrains only the last layer, while the original model remains as a tool-defined feature. In the second method, other layers are also trained. This method is called fine-tuning. These are currently some of the most popular transfer models, such as: AlexNet, VGG, ResNet, and GoogleLeNet models.  
The AlexNet neural network consists of 11 layers, of which there are five convolution layers and three layers of maximum pooling [49]. The architecture of this network is depicted in Figure 8. The network input is a \(224\times 224\times 3\) pixel-sized vector that matches the size of the image, and the network output receives a 1000-value vector that indicates the class to which the object was assigned.  
The VGG-16 architecture consists of 21 layers, of which 13 layers are convolution layers, five are maximum pooling layers, and three are fully connected layers [50]. A feature of this network is that all layers of the fold are \(3\times 3\) pixels in size. The VGG16 architecture is illustrated in Figure 9. The input to  
Figure 8: Architecture of the AlexNet neural network. CL is the convolution layer, MPL is the maximum pooling layer, and FCL is the fully connected layer.  
this architecture, like AlexNet, is a \(224\times 224\times 3\)-sized vector, and the output is a 1000-value vector that indicates the class to which the image belongs.  
A residual network (ResNet) [51] addresses the vanishing gradient problem. This problem occurs when the network is too deep and the gradient value after the loss function is reduced, and later it becomes too small to influence the layer multipliers, so the network no longer learns. The ResNet architecture solves this problem by creating additional connections between different layers, where gradient information is transmitted directly by skipping the intermediate layers. Figure 10 shows the ResNet simplified layer link.  
The GoogleNet or inception network architecture has 22 neural layers [52]. The main features of this network were \(1\times 1\) size convolution operations and the inception module. The \(1\times 1\) convolution operations were first introduced on the NIN (network in network) [53]. NIN was used to distinguish attributes instead of linear filters before classification, which increases the accuracy. Also, NINs were  
Figure 10: ResNet unit. CL is the convolution layer; reLu is a rectified linear unit.  
Figure 9: Architecture of the VGG16 neural network. CL is the convolution layer, and MPL is the maximum pooling layer.  
used instead of maximum pooling layers, and activation layers were created, the number of which is proportional to the number of classes. Applying the average operation and the soft peak layer results in class prediction. This application of NIN increased the speed and accuracy of the method. The second feature of the GoogleNet network is the inception module. In this module, the convolution filters of different size are applied in parallel to the input layer and are assembled to obtain a new vector of the output attributes. The diagram of the inception module is shown in Figure 11.  
In the connection layer, all results are combined into one feature map. The BNInception method uses batch normalization in the input layer, but also in the hidden layers, where each batch of data in the new iteration is adapted to the mean and distribution values learned [54].  
Batch normalization makes it possible to train models with higher training speeds and reduced distribution, and it also reduces learning when the model adapts too much to the training data.