# PneumoNet: Artificial Intelligence Assistance for Pneumonia Detection on X-Rays
## 1 Introduction  
Pneumonia is also a common cause of death among older adults and people with weakened immune systems. It can be caused by bacteria, viruses, fungi, or other microorganisms, and risk factors for pneumonia include malnutrition, smoking, air pollution, and other respiratory infections [1]. It is important to seek medical attention if you suspect you may have Pneumonia, as early diagnosis and treatment can improve outcomes and reduce the risk of complications. It has been verified that around 150 million people are affected by pneumonia in one year. According to the World Health Organisation (WHO), pneumonia is a leading cause of death among children under the age of five, and an estimated 156 million cases of childhood pneumonia occur each year worldwide, leading to around 111 million clinical episodes and 922,000 deaths [2]. The detection process of pneumonia is problematic as it is very labor-intensive and time consuming [3] and depends on the speed of development of the pathological agent [4]; to reduce human error, artificial intelligence (AI) technologies can be more assertive, cheaper and efficient; although these technologies are not yet perfect, they can avoid \"mistakes\" made by doctors or can help them in the diagnosis. Deep learning (DL) models and AI algorithms still have some limitations concerning the classification accuracy of an X-Ray; see, e.g., [5,6]. The previous work performed by the authors [7] shows an innovative approach for detecting COVID-19 from CT scans by changing the last layers of the ResNet-50, which obtained accuracy rates ranging from 97.0% to 99.8% across three well-established and extensively documented datasets focused on COVID-19 detection. To integrate the data and provide medical reports, we present a framework called PneumoNet that combines (i) a state-of-the-art DL model for the classification of X-ray images with a (ii) large language model (LLM), GPT-Neo, to produce the medical report. PneumoNet is built over an AlexNet architecture modified by removing its last three layers and replacing them with custom layers suited for pneumonia classification. The modifications involved adding three convolutional layers, each followed by a batch normalisation layer, to effectively process the X-ray images. A rectified linear unit (ReLU) activation layer was introduced to provide non-linearity, followed by a fully connected layer, a SoftMax layer, and a classification layer to perform the final classification. These adjustments optimised the network for pneumonia detection while maintaining high classification accuracy, tailoring the model to the specific task of analysing X-ray images. Also important to stress is that the model was trained and tested using X-ray images from three large labelled datasets containing both normal and pneumonia-affected cases, namely: the chest X-ray (COVID-19 and pneumonia)--Dataset #1 [8], the labelled optical coherence tomography and chest X-ray images for classification--Dataset #2 [9], and the chest X-ray images (pneumonia)--Dataset #3 [10]. The method for identifying pneumonia on X-rays using a DL architecture in conjunction with explainable AI (XAI) models to aid in the doctor's diagnosis process and with the LLM via chatbot to address any questions regarding the prediction process is the paper's primary contribution. The second contribution is a DL architecture to classify X-rays, which achieved accuracy above state-of-the-art results. The present section introduces the theme and mentions the contributions of the paper; the second section shows the theoretical background and related work. The third section describes the PneumoNet framework and its respective modules: pneumonia classification, XAI, and the Medical Report Module with the respective user interface (UI). The last section provides conclusions and future work. To further contextualise PneumoNet's contributions, a theoretical comparison can be drawn with frameworks in medical imaging that integrate multimodal data to generate domain-specific diagnostic reports.