# PneumoNet: Artificial Intelligence Assistance for Pneumonia Detection on X-Rays
## 3 Pneumonet Framework
### XAI Module
The generation of medical reports using transformer-based models such as GPT-Neo facilitates the interpretation of the results by the physician, and the MRM allows for clarifying further doubts. A critical limitation of the PneumoNet framework is its exclusive reliance on chest X-ray images for pneumonia detection, without incorporating clinical information such as fever, oxygen saturation, respiratory rate, or other vital signs. These clinical parameters are essential in real-world diagnostics, as they provide contextual data that enhances diagnostic accuracy and supports clinical decision-making. For instance, fever is a hallmark symptom of pneumonia, while low oxygen saturation often indicates severe cases, both of which can refine the interpretation of imaging findings. The absence of such data limits PneumoNet's ability to replicate the comprehensive diagnostic approach used by radiologists, who integrate imaging with patient history and symptoms. This constraint may reduce the framework's effectiveness in complex cases, such as early-stage pneumonia or atypical presentations, where imaging alone may be insufficient. To overcome this limitation, future iterations of PneumoNet could incorporate a multimodal approach by integrating clinical metadata into the model's pipeline. For example, vital signs such as fever and oxygen saturation could be encoded as numerical or categorical features and fused with image-based features during training and inference. Alternatively, clinical data could weigh the model's predictions, prioritising cases with corroborating symptoms. Such enhancements would align PneumoNet more closely with clinical workflows, improving its diagnostic precision and generalisability. Additionally, incorporating clinical information could enrich the Medical Report Module's outputs, enabling GPT-Neo to generate more comprehensive reports that reflect both imaging and clinical findings, thus enhancing PneumoNet's utility as a diagnostic support tool. At this phase of prototyping/development, there is in the PneumoNet framework the absence of prospective validation, which is critical for establishing its external validity in real-world clinical settings, and this will be part of the future work to be developed. Currently, PneumoNet has been evaluated using retrospective datasets [8; 9; 10], which, while diverse, do not reflect the dynamic and variable conditions of live clinical environments, such as differences in imaging protocols, patient demographics, or disease prevalence. Prospective validation, involving real-time data collection from ongoing clinical practice, is essential to confirm the model's generalisability and robustness across varied healthcare settings. Without such validation, the framework's performance in detecting pneumonia under real-world conditions remains untested, potentially limiting its reliability and acceptance in clinical practice. Additionally, the lack of prospective studies restricts the ability to assess how PneumoNet handles evolving patient presentations or integrates with clinical decision-making processes. Furthermore, PneumoNet has not been integrated with Picture Archiving and Communication Systems (PACSs) or Electronic Health Records (EHRs), which are critical for practical deployment. Real-time testing would evaluate the model's performance on live X-ray data as it is acquired, ensuring timely and accurate diagnostic support. Integration with PACS would enable seamless access to imaging data within hospital workflows, while EHR integration could incorporate clinical data (e.g., vital signs, patient history) to enhance diagnostic accuracy. The absence of such integration limits PneumoNet's current utility as a clinical tool. As mentioned, future work should focus on conducting prospective validation studies in real-world settings and developing interfaces for PACS and EHR integration to align PneumoNet with standard clinical workflows, thereby improving its practical applicability and external validity.