# PneumoNet: Artificial Intelligence Assistance for Pneumonia Detection on X-Rays
## 3 Pneumonet Framework
### XAI Module
Semantic coherence was assessed via cosine similarity between report sections and ground-truth radiological descriptions, yielding an average score of 0.78, reflecting strong alignment with clinical terminology. The integration of XAI metrics (e.g., pixel counts for lung regions) and vital signs in prompts ensured reports correlated with diagnostic confidence (e.g., 0.85 for positive cases), with 95% of reports accurately reflecting prediction probabilities. Future work could enhance evaluation by incorporating clinician feedback loops or BLEU scores against reference reports, further refining MRM's outputs. These assessments, grounded in the code's structured prompt design and error handling, confirm the MRM's ability to produce high-quality, clinically actionable reports within the PneumoNet framework. As already mentioned above, the MRM utilises GPT-Neo, a 1.3B-parameter transformer model, to generate detailed medical reports and interactive chatbot responses, with its implementation detailed in the Flask application code. GPT-Neo is configured as shown at the beginning of Section 3.2 to ensure coherent and diverse responses, which are truncated to 200 words for clarity, generating reports through four already mentioned structured prompts, which produce narratives that combine radiological descriptions (e.g.,ground-glass opacities), XAI-driven feature importance (e.g., red regions in LungFocusXAI indicating high focus), and clinical context from vital signs, ensuring reports are both technically precise and clinically relevant for physicians. The MRM's robustness is enhanced by error handling mechanisms and fallback responses, which log failures (e.g., LLM generation errors) and return default outputs if GPT-Neo fails. The chatbot functionality uses similar prompt engineering, tailoring responses to user queries by incorporating prediction confidence, XAI metrics, and vital signs, with instructions to use simple language and provide practical advice (e.g., consulting a doctor). The pneumonia classification module via ONNX ensures seamless data flow, where preprocessed X-ray images (224 \(\times\) 224, normalised) feed into GPT-Neo's prompts. Future improvements could involve fine-tuning GPT-Neo on medical corpora to enhance domain-specific accuracy or upgrading to larger models (e.g., GPT-4) for improved coherence, as suggested in the code's modularity. These enhancements, combined with the detailed prompt structure and XAI integration, position the MRM as a scalable and interpretable component of PneumoNet, supporting clinicians with comprehensive diagnostic insights. Finally, the application delivers the results through a user-friendly interface. The classification outcome, medical report, and uploaded image are displayed on the results page, offering a seamless experience. This integration of machine learning and natural language processing creates a robust tool for pneumonia detection and report generation, enhancing diagnostic support for medical professionals and patients alike. ## 4 Conclusions

The developed PneumoNet framework demonstrates remarkable effectiveness in the detection of pneumonia from X-ray images, as evidenced by its performance across multiple datasets. With an accuracy of 98.70% and an AUC of 99.70% on Dataset #2 [9], PneumoNet (PCM) outperforms existing state-of-the-art methods in the field. Similarly, its application to Dataset #1 [8] achieved an accuracy of 96.70%, with an F1 score of 93.12% and specificity of 99.30%. These results reflect the model's robustness and adaptability to diverse datasets. The integration of Grad-CAM and LIME visualisations further enhances the interpretability of PneumoNet's predictions. These techniques provide clear insights into the regions of X-ray images influencing the model's decisions, fostering transparency and trustworthiness essential for clinical applications. The ability to visualise and understand model reasoning is a significant step forward in making deep learning systems more acceptable in healthcare.