# Adenocarcinoma Recognition in Endoscopy Images Using Optimized Convolutional Neural Networks
## 2 Image Classification Using Deep Learning  
In the late 1990s, the LeNet [8] architecture was used for image classification using deep learning. LeNet's architecture consists of a convolution layer, a pooling layer, and a fully-connected layer. The architecture of most deep image classification methods is inspired by LeNet. The operation performed in the convolution layer is given by Equation (1).  
\[F_{i}=b_{i}+\sum_{i}(W_{ij}*X_{i}) \tag{1}\]where \(X_{i}\) refers to the ith pixel, \(W_{ij}\) corresponds to the weight value used for convolution, * denotes the convolution operation, and \(b_{i}\) is the bias. The kernel \(W\) slides across the image and performs the convolution operation between \(W_{ij}\) and \(X_{i}\). The convolution output for all pixels of an image results in a feature map \(F_{i}\). Multiple kernels are used to generate multiple feature maps.  
To reduce the image size, the pooling layer selects the maximum value in k-sized kernels at row r and column h in feature map F as shown in Equation (2). This procedure is known as max-pooling. If the average value is extracted instead of the maximum value, it is known as average pooling.  
\[P_{i,h}=maxF_{r,h},F_{r+1,h+1},....,F_{r+k,h+k} \tag{2}\]  
AlexNet [25], published in 2012, consists of five convolution and three fully-connected layers as shown in Figure 1. To solve the problem of the vanishing gradient, AlexNet replaces the existing sigmoid function or hyperbolic tangent activation function with the Rectified Linear Units (ReLU) [26] activation function as shown in Equation (3). To reduce overfitting, the dropout [27] method was applied for neural network learning. The AlexNet structure showed excellent results with a 15.4% test error rate in the image recognition part of ImageNet.  
\[f(x)=max(0,x) \tag{3}\]  
Two years after AlexNet was published, VGGNet [28] was developed by the University of Oxford. VGGNet consists of up to sixteen convolution layers. Unlike AlexNet, which uses a variety of kernel sizes, VGGNet reduces the number of parameters by using a fixed kernel size of 3 \(\times\) 3. The nonlinearity in a network increases with the increase in the number of convolution layers. This aids in extracting more unique features.  
A limitation with deep learning is that it may be trained poorly due to the slope loss as the architecture gets deeper. To solve this problem, ResNet [29] introduced the method of a block structure as shown in Figure 2 in which the input layer and the output layer are connected to each other. This structure is called the residual structure, and it is a structure that learns to minimize the difference between the input and output. Through this structure, the problem of slope disappearance is solved, and the training result is compared and evaluated by increasing the layers of the neural network.  
Figure 1: An illustration of the architecture of AlexNet.  
Figure 2: Residual learning: a building block in ResNet.  
Further, DenseNet [30] was introduced in 2017, where each layer takes all preceding feature maps as the input. Unlike ResNet, DenseNet has a structure in which an input layer and an output layer having the same feature map size are directly connected to each other and transmitted as the input values of the next layer as shown in Figure 3. It has small parameters and fewer computations with better performance than the state-of-the-art.