# Perturbation-Based Explainable AI for ECG Sensor Data
## 4 Model Training and Evaluation
### Evaluation  
During training, the algorithm was set to monitor the accuracy, recall, and precision. These metrics were tracked over ten epochs. For evaluation of the model, a confusion matrix was used to display the resulting predictions on the validation set. One ECG record could be marked with several arrhythmias. A neural network prediction contains a probability for each target attribute value. The key is to learn a threshold that accurately matches the target attribute's value.  
The threshold was calculated using the same method that [PERSON] and [PERSON] [10] presented in their work. The classifier ran on the training data and received a score between 0 and 1 for each class. The Nelder-Mead downhill simplex method [11; 12] was used to find the optimal threshold for all 24 classes. A prediction value for a target attribute greater than threshold = 0.4899078 was considered true, while a value less than or equal to the threshold was considered false. The confusion matrix is shown in Figure 3.  
Model: \"model\"  
Layer (type) Output Shape Param s ========================  
input_l (InputLayer) [(None, 5000, 12)] 0 convid (Conv1D) (None, 5000, 128) 12416 batch_normalization (BatchN (None, 5000, 128) 512 normalization) activation (Activation) (None, 5000, 128) 0 convid_l (Conv1D) (None, 5000, 256) 164096 batch_normalization_l (Sato (None, 5000, 256) 1024 hNormalization) activation_l (Activation) (None, 5000, 256) 0 convid_2 (Conv1D) (None, 5000, 128) 98432 batch_normaliztion_2 (Sato (None, 5000, 128) 512 hNormalization) activation_2 (Activation) (None, 5000, 128) 0 global_average_pooling(G (None, 128) 0 global_averagepoolingD) dense (Dense) (None, 24) 3096 In addition to the confusion matrix, the target attribute values were evaluated using the accuracy, recall, and F1-score metrics. In Figure 3, the resulting metrics on the validation set are displayed for individual target classes. Summarizing metrics are labeled according to the type of averaging used as \"micro\" and \"macro\". Macro-averaging assigns equal weights to each class, while micro-averaging assigns equal weights to each sample. When the number of examples in particular classes is approximately the same, both are acceptable. However, for imbalanced data, as in our case, which approach is preferable depends on the application. In our case, all diagnoses are important, regardless of whether they are frequent or rare; therefore, macro-averaging is preferable. The recall metric is very important if true positive cases need to be correctly predicted. According to the recall metric, the model best classified the values of the target attributes SB, STach, and LBBB. Our primary goal, however, was not to produce the best model; rather, it was to evaluate our new perturbation-based explanation method on a model of acceptable quality. Alternatively, the more sophisticated models presented in [10] achieved a macro-average F1-score of 0.3-0.4. Our simple convolutional neural network achieved an F1 score of 0.27. Compared to current models, although our model achieved slightly worse results, it saved significant computing time with the implemented explainability method, which was our primary goal.