# PneumoNet: Artificial Intelligence Assistance for Pneumonia Detection on X-Rays
## 3 Pneumonet Framework
### XAI Module  
Interpretability is enhanced through gradient-weighted class activation mapping (Grad-CAM) [38] and local interpretable model-agnostic explanations (LIME) [39], generating heatmaps to pinpoint influential lung regions, distinguishing correct from erroneous predictions visually. For deployment, the model is exported to ONNX format, enabling efficient inference. Images are pre-processed to 227 \(\times\) 227, normalised to (0, 1), and transposed to NCHW format (batch, channels, height, width), with the ONNX runtime yielding binary outcomes (0 for normal, 1 for pneumonia) via maximum posterior probability, bridging to the linguistic stage (see Figure 4). Moreover, Grad-CAM and LIME visualisations offer interpretable predictions by showing areas in the images that influence the model's decisions. This enhances the transparency and trustworthiness of the model. Figure 4 illustrates the use of LIME and Grad-CAM on the classified images, and the algorithm that combines the PCM and XAI modules is presented below (Algorithm 1). ```
Input: Image Data Path: Directory containing X-ray images. Pretrained AlexNet model. Training dataset percentage = 80%. Test dataset percentage = 20%. Validation dataset percentage = 10% of the Test dataset. Learning_rate: \(\alpha\ =1\times 10^{-4}\). MiniBatchSize: \(B\) = 128. NumberOfEpochs: \(N_{epochs}\) = 10. Output: Trained Deep Learning Model: \(f_{\theta}\). Predicted Labels: \(\hat{Y}=f_{\theta}\ (X_{test})\). Performance Metrics: Accuracy: \(Acc=\frac{TP+TN}{TP+FP+TN+FN}\), Precision: \(P=\frac{TP}{TP+FP}\), Recall \(R=\frac{TP}{TP+FN}\), F1-Score: \(F1=\frac{2 PR}{P+R}\). Confusion Matrix: \(C_{ij}\) where \(C_{ij}\) represents the count of true class \(i\) classified as \(j\) ROC curve: AUC value calculated from true positive rate (TPR) and false positive rate (FPR). XAI Visualizations: Explanatory visuals for model predictions using Grad-CAM and LIME. Steps of the Algorithm: Load the dataset \(X\), assign labels \(Y\) based on folder structure and divide into subsets: \(X_{\text{train}}\), \(X_{\text{test}}\leftarrow\text{Split}\ (X\), 0.8, 0.2) \(X_{valid}\ \leftarrow 0.1\ \times X_{\text{train}}\) If total images are sufficient: Assign 80% to training, 10% of training for validation, and 20% to testing. End If

**For** each image \(I\) in the dataset:

**If**\(I\) is grayscale:

Convert \(I\) to RGB by replicating channels:

\(I_{\text{rgb}}=cat(I,I,I)\). **End If**

Resize \(I\) to the required input size:

\(I^{\prime}=resize(I,(224,224))\). **End For**

Apply transformations to enhance the training data and retain AlexNet layers up to the penultimate layers. **For** each new layer:

Add convolutional layers, batch normalisation, and ReLU activation. Add fully connected, SoftMax, and classification layers. **End For**

Set learning rate \(\alpha\), mini-batch size \(B\), epochs \(N_{\text{epochs}}\), and validation parameters and Train the modified network. **For** each epoch \(t\)\(in\)\(N_{epochs}\):

Update model weights using Adam optimizer:

\(\theta^{(t+1)}=\theta^{(t)}-\alpha\
abla_{\theta}\mathcal{L}(\)\(X_{batch},\)\(Y_{batch})\)

Perform validation checks periodically. **End For**

Test the trained model on \(X_{\text{test}}\). **For** each class \(c\):

Calculate precision: \(Pc=\frac{TPc}{FFc+TPc}\), Calculate recall: \(Rc=\frac{TPc}{FNc+TPc}\), Calculate F1-score: \(F1c=\frac{2 Pc}{Pc+Rc}\)

**End For**

Predict the class label \(\hat{Y}\) for each image \(X_{test}\). **If** use_gradcam:

**For** each selected image \(I\):

Generate a class activation map \(M\) and overlay it on \(I\). **End For**

**End If**

**If** use_LIME:

**For** each selected image \(I\):

Compute feature importance \(M\) and overlay it on \(I\). **End For**

**End If**

Provide the trained model, evaluation metrics, and visualisations. Figure 4: Left Grad-CAM predictions and right LIME classifications. ### Medical Report Module and User Interface

Medical Report Module (MRM) is based on GPT-Neo, a 1.3B-parameter transformer, which generates medical reports from these predictions.