# Attention Score Enhancement Model Through Pairwise Image Comparison
###### Abstract  
This study proposes the Pairwise Attention Enhancement (PAE) model to address the limitations of the Vision Transformer (ViT). While the ViT effectively models global relationships between image patches, it encounters challenges in medical image analysis where fine-grained local features are crucial. Although the ViT excels at capturing global interactions within the entire image, it may potentially underperform due to its inadequate representation of local features such as color, texture, and edges. The proposed PAE model enhances local features by calculating cosine similarity between the attention maps of training and reference images and integrating attention maps in regions with high similarity. This approach complements the ViT's global capture capability, allowing for a more accurate reflection of subtle visual differences. Experiments using Clock Drawing Test data demonstrated that the PAE model achieved a precision of 0.9383, recall of 0.8916, F1-Score of 0.9133, and accuracy of 92.69%, showing a 12% improvement over API-Net and a 1% improvement over the ViT. This study suggests that the PAE model can enhance performance in computer vision fields where local features are crucial by overcoming the limitations of the ViT. vision transformer; clock drawing test; attention mechanism; dementia classification; image processing +
Footnote †: journal: Medical Image Analysis  
1
Footnote 1: email: [EMAIL_ADDRESS]  
Footnote †: journal: Medical Image Analysis  
Therefore, the primary motivation of this study is to improve the classification accuracy of CDT data by correctly classifying borderline images. In particular, the aim of this study is to reduce such classification errors by accurately recognizing subtle differences between images with similar visual features [2; 3; 4; 5]. To address these challenges, the Transformer architecture has recently gained attention [7; 16; 17; 18; 19; 20]. Originally excelling in natural language processing, Transformers were later adapted for computer vision through the Vision Transformer (ViT) model [16; 17; 19]. The ViT divides an image into patches and learns the long-range dependencies between them using a self-attention mechanism [16; 19]. This makes the ViT highly effective at learning the global structure and patterns within an image, making it suitable for analyzing complex visual data such as the CDT [2; 3; 4; 19]. However, the ViT also has its limitations. While it is advantageous in learning global relationships, it struggles to detect subtle differences between visually similar images [3; 4; 19]. In datasets with variations in angle, lighting, and background, the ViT's performance can degrade [3; 19]. CDT data often contain such variability, making it difficult for the ViT to capture fine local features [2; 3; 4]. This can lead to classification errors, reducing the accuracy of CDT analysis [3; 4; 5; 6; 7; 8; 9; 10; 11; 12; 13; 14; 15; 16; 17; 18; 19]. To overcome these challenges, recent studies have explored various approaches to enhance the performance of the ViT. Data augmentation methods have been proposed to improve model generalization [8; 9; 12; 13; 14; 15], and composite loss functions have been used to encourage the model to learn finer details [21; 22; 23; 24; 25]. Additionally, some studies have focused on modeling interactions between images rather than treating them independently, in an effort to improve the model's ability to capture complex relationships [16; 21; 23; 24; 26]. While these approaches improve the ability of ViT to learn global relationships, they are still limited in their ability to detect finer details [2; 3; 4; 23]. In this study, we propose the Pairwise Attention Enhancement (PAE) model to address the limitations of the ViT model [7]. While the ViT excels at capturing global relationships within images [17; 18], it struggles to detect subtle differences in visually similar images [22].