# Automated Chest X-Ray Diagnosis Report Generation with Cross-Attention Mechanism
###### Abstract
[7] adopted the pretrained BERT model for text encoding, making full use of Bidirectional Encoder Representations from Transformers (BERT)'s advantages in capturing text context information, so as to more accurately extract key information in the diagnostic report. The Knowledge-driven Encode, Retrieve, Paraphrase (KERP) [8] project designed an anomaly-based knowledge graph that combined image features extracted by a deep convolutional network and a rich template library to innovatively generate text reports with high accuracy and relevance. Equally compelling work has been conducted on modality alignment; R2 GenCMN [9] uses a cross-modal memory network to not only strengthen the mutual understanding between images and texts, but also to improve the quality of generated reports, demonstrating the importance of cross-modal understanding in improving the performance of report generation systems. ConVIRT [10] intelligently identifies the relationship between images and their corresponding text by adopting a contrastive learning method. This method effectively achieves precise alignment between modalities and significantly improves the overall performance of the system. MedClip [11] uses text entity labels as weak labels to decouple image-text pairs, effectively improving the robustness of the model in dealing with a large number of samples that are easily misclassified as negative samples. TieNet [12] combines image features with text encoding weighted by an attention mechanism and achieves a significant improvement in the quality of diagnostic reports, which demonstrates the high capabilities of deep learning models in fine-grained feature extraction and semantic understanding. In the processing and analysis of chest X-ray datasets, a significant challenge faced by researchers is the uneven distribution of normal and abnormal areas in the data. This imbalance, in which the proportion of normal areas is much larger than the abnormal areas, inevitably leads to data bias problems. To address this series of challenges, this study proposes a method to introduce a convolutional block attention module on the visual encoder side. CBAM [13] is an efficient attention mechanism that can dynamically adjust the network's attention to different areas of the image, thereby paying special attention to abnormal areas of the image. We integrate CBAM into the visual encoder. The method in this paper redefines the attention distribution of the image, so that the model's attention weight on abnormal areas is increased, while the weight distribution on normal areas is relatively reduced. This adjustment effectively alleviates the deviation problem caused by data imbalance and provides more favorable conditions for accurately identifying and analyzing abnormal areas. In addition, chest X-ray datasets usually contain X-ray images and corresponding diagnostic reports, which constitute the main modalities of research. However, due to the lack of pretrained object detectors for specific medical image data, the alignment between the two modalities becomes a difficult problem that increases the complexity of data processing. In order to further strengthen the mutual understanding and alignment between images and texts, this paper adopts the cross-attention mechanism. This mechanism can establish a closer connection between the two modalities and achieve more accurate modality alignment by learning each other's feature representations. This improves the accuracy and relevance of automatically generated diagnostic reports.