# Improvement of a Subpixel Convolutional Neural Network for a Super-Resolution Image
## 2 Materials and Methods
### R-ESPCN Structure  
In the first stage of the feature extraction process of the network structure, there are 2 convolutional layers. Starting from the first 2 layers, 32 convolution kernels of 3 \(\times\) 3 size were determined to keep the size low due to computational costs. In order to reduce the computational complexity in these layers, 32 convolution kernels of 3 \(\times\) 3 size are preferred instead of 64 convolution kernels of 5 \(\times\) 5 size in the previous known ESPCN models. The network takes a low-resolution image as input to convolutional layer 1, and the feature extraction process is carried out according to Equation (2). \[f^{1}\ (I^{\text{LR}};W_{1},b_{1})=\varphi\ W_{i}\times I^{\text{LR}}+b_{1} \tag{2}\]  
In the expressions used in Equation (2), the low-resolution image input is shown as \(I^{\text{LR}}\), the weight of the network is Wi, and the bias is Bi. The size of the weight, Wi, of the network is found by multiplying the number of channels in the input image, the number of convolution kernels of the output channel, and the kernel size. Tanh is used as the activation function in all convolution layers of the network. The activation function is used here to control the Fi value, that is, to decide whether a neuron will be active. Activation functions are an important feature for deep neural networks in this respect. The feature extraction processes of the R-ESPCN model we proposed are modeled as shown in Figure 6.  
Figure 5: Flowchart of the proposed R-ESPCN. Many activation functions, such as sigmoid, tanh, ReLu, and leaky Relu, are used in deep neural networks. The most popular of these is the sigmoid function, which confines the values it receives between 0 and 1. When a high value arrives, it is close to 1, while when a low value arrives, it is close to 0. Therefore, the activation result is not in any value range between 0 and 1. The disadvantages of the sigmoid function are that it has a high gradient measurement, is not a 0-oriented function, and exp() calculations are quite slow [23]. The hyperbolic tangent function, known as tanh in short, is known for its similarity to sigmoid. Both have compression in a certain range, but the tanh activation function confines the incoming values between \(-1\) and 1. Better results can be obtained compared to the sigmoid function because it is 0-oriented. The other most popular activation function, ReLu, was first used in 2012, and its popularity has increased to the present day. In addition, the ReLu function is known for its similarity to biological neurons. The working logic of the ReLu activation function looks at whether the incoming values are positive or negative and then decides that the result of the operation is 0 if the incoming value is negative [24]. However, if the incoming value is positive, it does not apply any compression or modification process and passes as is. While computers perform complex calculations in tanh and sigmoid, ReLu only looks at the positivity and negativity status. Therefore, the computer calculates this equation much faster. However, we cannot say that it is a magnificent function, because some neurons may die since this function is not 0-oriented. The reason for using tanh as the activation function in the model we proposed is to be able to perform complex calculations. There are two different pooling methods commonly used in convolutional neural networks. Local pooling is a method used to display feature maps by bringing together 3 \(\times\) 3 data from small local regions as examples. The other method, global pooling, is a scalar value representing the image from the feature vector for each feature along the feature map [25]. The 3 \(\times\) 3 local pooling method was used in this study. In the R-ESPCN model given in Figure 6, convolutional layer 1 accepts a low-resolution image as input, while convolutional layer 2 takes the F\({}_{1}\) feature set as input. Convolutional layer 3 takes  
Figure 6: Structure of the proposed R-ESPCN model.