# Improving Medical Image Quality Using a Super-Resolution Technique with Attention Mechanism
## 2 Related Work
### ResNet  
ResNet is a neural network architecture that is widely employed in deep learning for image classification and computer vision tasks [15]. Residual learning is a methodology that addresses the increasing difficulty in learning as the network becomes deeper. This involves adding an input value to the output value of the network. This approach enables the effective training of deep networks while ensuring network stability. The ResNet architecture exhibits the following characteristics:  
Figure 1: ISIC 2020 Challenge dataset. 1. Expansion to deep networks: ResNet introduces residual connections to enable the stacking of networks to deeper levels, thereby alleviating the gradient vanishing that occurs in deep networks. Residual connections construct a network by learning the residuals between the input and output and adding them to the output. This allows for a better performance as the network depth increases. 2. Performance: With its ability to construct deep networks, ResNet can learn and extract more complex and abstract image features, leading to high classification accuracy. 3. Efficient learning: Residual connections preserve information from the previous layers, ensuring stable gradients and a smooth learning process. 4. Reusable block structure: The block structure is composed of stacked basic blocks, constituting a reusable form that facilitates model construction. 5. Neural network architecture: Generally, ResNet is built by stacking basic blocks consisting of convolution layers and rectified linear units (ReLUs) as activation functions. These basic blocks are connected by residual connections, enabling the neural network to learn the residuals between the inputs and outputs. Convolution layers with various kernel sizes and strides are employed for each layer, enabling feature extraction at different scales. As the number of layers in a neural network increases, higher-level features are expected to be extracted. However, when the neural network becomes excessively deep, gradient vanishing occurs, leading to training difficulties. Additionally, several disadvantages have been observed (Figure 2):  
1. Computational and memory requirements: Deep networks have high computational and memory requirements during training and inference. Training with larger models requires significant resources and increases memory usage. 2. Model Size: Deep structures result in relatively large models. Therefore, models with extensive structures may encounter storage space and transmission speed constraints in certain scenarios. 3. Overfitting Issue: Constructing models with deep structures requires more parameters. Consequently, overfitting might arise in cases with limited training data. Thus, caution is required regarding computational resources and model size. Therefore, proper data management and the prevention of overfitting must be considered. Hence, a residual block that employs skip connections is proposed to solve these problems. In Figure 3, (a) represents the conventional neural network, and (b) represents the residual block structure. The primary difference lies in the configuration of residual connections. In a conventional neural network, the input is transformed into features, and multiple layers are traversed to compute the final output. Each layer learns a transformation function between the input and output and maps the input into the desired form. However, as the network deepens, traditional neural networks must progressively learn the transformations between the input and output, which can lead to vanishing or exploding gradients. Figure 2: ResNet architecture. In contrast, residual blocks explicitly learn the residuals between the input and output. After passing through the input, a residual block learns the difference between the input and output through a residual connection and adds this difference to the output.