# Improving Medical Image Quality Using a Super-Resolution Technique with Attention Mechanism
## 2 Related Work
### Attention Mechanism  
The attention mechanism can be applied to machine translation, question answering, and natural language processing tasks [20]. This technique allows the model to focus more on specific parts and assign weights, allowing it to concentrate more on the relevant information within the input sequence. It has been used in the sequence-to-sequence (seq2 seq) model to apply features to previous inputs differentially. It improves the traditional encoder-decoder structure and regulates the relevance between the input and output. This effectively mitigates the loss and gradient vanishing issues that occur during the compression of fixed-size vectors. The mechanism consists of the following stages:  
Figure 3: Network model: (**a**) existing CNN; (**b**) CNN-based ResNet residual block. 1. Encoder stage This involves embedding each element (e.g., word or token) of the input sequence to represent its meaning in a vector space. These embeddings contribute to extracting and representing input features. Encoder structures are typically implemented using recurrent neural networks (RNNs) or modified functions (e.g., transformers). The output of the encoder serves as a context vector and is used in the subsequent attention stage. 2. Attention stage Herein, a weighted average is computed based on the output from the encoder and the previously generated state from the decoder. This process focuses on the relevant parts of the input sequence. The query, key, and value concepts are commonly used. The query is generated based on the previous state or output of the decoder, whereas the key and value are generated from the output of the encoder. Attention weights are used to calculate the relevance between the query and key, with the attention-weighted sum summarizing the information of specific parts of the input. 3. Decoder stage The decoder utilizes the computed weighted average from the attention stage and combines it with the previous state to generate the subsequent output. This process is iteratively repeated with the decoder, using the previous output as the input to complete the sequence. Similarly to the encoder, the decoder is implemented using RNNs or structures such as transformers. The output of the decoder serves as the final prediction, and can be used in various ways, depending on the objective of the model. This attention mechanism dynamically allocates weights to each element in the input sequence, thereby assigning relevance. It can efficiently recognize relevant information by focusing on specific parts of the input sequence and adjusting the relevance between the input and output. Recent attention mechanisms include seq2 seq and mechanisms such as Reformer [21], LinFormer [22], and Swin Transformer [23], which utilize transformer structures [24]. These attention mechanisms play a role in efficient attention operations that reduce memory usage and optimize computational costs. They also demonstrate superior performance in natural language processing tasks using self-attention instead of RNNs. Moreover, attention in computer vision is applied in object detection and semantic segmentation. In such tasks, the objects of interest are often distributed in specific regions rather than uniformly across the entire image. Therefore, attention is employed to focus on crucial object regions and extract features by disregarding unnecessary regions. This approach emphasizes spatial and semantic information and improves network performance by leveraging the meaning and positional characteristics of objects. In addition, this method even demonstrated good performance with noisy inputs. In the convolutional neural network (CNN), attention mechanisms such as the convolutional block attention module (CBAM) are used [25]. CBAM provides an effective attention mechanism for computer vision tasks, such as image classification and object detection.