# PneumoNet: Artificial Intelligence Assistance for Pneumonia Detection on X-Rays
## 3 Pneumonet Framework
### XAI Module
The system also includes a chatbot functionality that leverages GPT-Neo to generate coherent responses to user queries; see Figure 5, bottom image. This provides an interactive component where users can ask questions or seek additional explanations related to their diagnosis or other medical topics. The algorithm's structure ensures modularity and scalability, making it adaptable for future enhancements. Each component (image preprocessing, model prediction, and text generation) is implemented as a separate function, allowing for independent updates or replacements. For instance, the ONNX model can be substituted with a more advanced deep learning model, or the GPT-Neo model can be upgraded to a larger version for improved visual language generation (examples already presented in the UI are GPT-4, Llma 2, and PaLM2). ```
0: An uploaded chest X-ray image in JPG/PNG format via a web interface. User text query (optional) for chatbot interaction. 0: A classification result: \"Normal\" or \"Pneumonia\". A detailed medical report explaining the implications of the classification. Optional chatbot-generated medical responses based on user input. Steps of the Algorithm:
0: Initialize the Flask application and configure the upload folder. Load the ONNX model using the ONNX Runtime. Load the GPT-Neo language model and tokenizer. Define the image preprocessing function: If the image is provided: Convert it to RGB format. Resize to (227, 227). Normalize to the range (0, 1). Convert to NCHW format and add a batch dimension. End If Define the prediction function: If a pre-processed image is given: Pass the image to the ONNX model. Get the predicted class with the highest probability. If the class is 1: Return \"Pneumonia.\" Else: Return \"Normal.\" End If End If Define the report generation function: If a prediction result is available: Create a prompt based on the prediction. Generate a medical explanation using GPT-Neo. Postprocess the response to ensure clarity. End If Create the home route: If the method is GET: Render the index.html page. Else If the method is POST:
```

**Algorithm 2**Application with artificial intelligence assistance
Save the uploaded image. Preprocess the image. Predict the result using the model. Generate the report. Render the result.html page with the result, report, and image. **End Else If End If** Create the upload route: **If** an image is uploaded: **Save the image. Preprocess and predict. Generate the medical report. Render the result.html page. **End If** Create the chatbot route: **If** a user query is received: **Generate a coherent response using GPT-Neo. Postprocess the response. Return the response as a JSON object. To evaluate the quality of medical reports generated by the GPT-Neo-based MRM, a dual approach combining qualitative and quantitative assessments was implemented. Qualitatively, reports were assessed for coherence, clinical relevance, and comprehensiveness by comparing their content--generated from four structured prompts (introduction, findings, clinical interpretation, conclusion)--against radiological standards for pneumonia (e.g., descriptions of opacities, consolidations). Each prompt, as explained before, integrates prediction outcomes (e.g., \"Pneumonia Positive\" with 0.85 confidence), XAI metrics (e.g., LIME superpixel importance, LungFocusXAI red regions for high focus), and vital signs (e.g., heart rate, oxygen saturation), ensuring reports exceed 15 lines with detailed narratives. A sample of 50 reports was reviewed by domain experts, who rated 92% as clinically relevant, citing accurate integration of XAI-driven features and vital signs. In the code's logging mechanism (logging.basicConfig), tracked generation errors revealed a 98% success rate across 1000 test runs, with fallback responses handling failures effectively, ensuring reliability in report production.