## 4 Experimental Setup and Results
### Evaluation Metrics  
In the SNNtrainer3D application, after each training session, the evaluation is extended beyond the traditional accuracy and loss metrics to include a confusion matrix and other more comprehensive sets of metrics such as the Precision, Recall, and F1 score [51]. These metrics offer deeper insights into the model's performance, particularly in scenarios where class imbalance might affect the overall accuracy. A Confusion Matrix is a table that describes the performance of a classification model on a set of test data for which the true values are known. It breaks down predictions into four categories:  
* True Positives (_TP_): Correctly predicted positive observations. * True Negatives (_TN_): Correctly predicted negative observations. * False Positives (_FP_): Incorrectly predicted positive observations (Type I error). * False Negatives (_FN_): Incorrectly predicted negative observations (Type II error). The Confusion Matrix does not have a singular equation but is the foundation from which Precision, Recall, and the F1 score are calculated. Precision measures the accuracy of positive predictions. It calculates the ratio of correctly predicted positive observations to the total predicted positive observations. High precision indicates a low rate of false positives. It can be seen in Equation (9) [51]:  
\[\text{Precision}=\frac{TP}{TP+FP} \tag{9}\]  
Recall, or Sensitivity, measures the model's ability to capture all relevant instances within the dataset. It calculates the ratio of correctly predicted positive observations to all observations in the actual class. It can be seen in Equation (10) [51]:  
\[\text{Recall}=\frac{TP}{TP+FN} \tag{10}\]  
The F1 Score is the harmonic mean of Precision and Recall, providing a balance between them. It is particularly useful when you need a single metric to evaluate models with imbalanced classes. It can be seen in Equation (11) [51]:  
\[\text{F1 Score}=2\times\frac{\text{Precision}\times\text{Recall}}{\text{ Precision}+\text{Recall}} \tag{11}\]  
By incorporating these metrics, SNNtrainer3D provides a nuanced understanding of the model's performance, especially in distinguishing between the errors made (Type I vs. Type II) and evaluating the model in the context of class imbalances. This comprehensive analysis allows for targeted improvements to the model, focusing on areas most needing refinement. Including these metrics in the application enables users to make more informed decisions when selecting and optimizing SNN architectures and hyperparameters for their specific tasks. The confusion matrix for all SNN architectures trained can be seen in the link mentioned under this paper's \"Data Availability Statement\" section. Table 2 shows the Precision, Recall, and F1 Score metrics across the models for the final epoch (index 9). Here, we can see that the 100-neuron Alpha model achieves the highest F1 Score of 0.9067, with well-balanced Recall (0.8738) and Precision (0.9421). The 100n LIF and Synaptic models follow closely with F1 Scores of 0.8918 and 0.8861, respectively. The 100n Lapicque model lags significantly behind with an F1 of only 0.6628. As we move down to 80 neurons, the Alpha model remains the top performer with an F1 Score of 0.9057, closely followed by the Synaptic at 0.9011. The LIF model's performance drops notably to an F1 of 0.8239, while the Lapicque continues to underperform at 0.6628. At 50 neurons, the Synaptic model takes the lead with an F1 Score of 0.8796. The LIF model achieves a more balanced Recall and Precision for an F1 of 0.8380. Alpha is in the middle at 0.8717, while Lapicque remains the weakest at 0.6498. Interestingly, the 30-neuron LIF model outperforms all the 50n models with an F1 Score of 0.8887. The 30n Synaptic and Alpha follow with 0.8476 and 0.8450, respectively. The Lapicque variant struggles with low Recall (0.4543) and F1 (0.5652). The 25-neuron LIF model performs strongly with an F1 of 0.8886, while the 25n Alpha drops to 0.8114. Among 20 neuron models, the LIF is the clear winner with an F1 Score of 0.8922.