## 4 Experimental Setup and Results
### Training Dynamics  
The final set of experiments (Figures 20 and 21) focuses on training stability and convergence. They illustrate the benefits of the integrated backpropagation algorithm and the potential for future enhancements with more biologically plausible learning methods.  
Figure 20: Performance Comparison regarding Accuracy and Loss for each of the four types of neurons after training different SNN architectures with the same number of neurons on the MNIST dataset. More exactly, for 80 neurons in one hidden layer.  
Here, SNN represents the trained SNN model; 1L represents one hidden layer; n represents the number of neurons; LIF represents the Leaky Integrate-and-Fire neurons; Lapicque represents the Lapicque neurons; syn represents the Synaptic neurons; and alfa represents the Alpha neurons.  
Concerning the experiments in Figures 20 and 21, the data indicates that SNN models with LIF neurons consistently outperform those with Lapicque, Synaptic, and Alpha neurons regarding accuracy and loss metrics. This superiority is most pronounced in smaller networks (30 and 50 neurons) and starts to converge as the network complexity increases (80 and 100 neurons).  
Specifically, LIF neurons exhibit rapid learning capabilities, as indicated by the steep initial descent in loss and rapid ascent in accuracy. This trend is sustained across all network sizes, though the marginal improvement diminishes as network complexity increases. In contrast, Synaptic neurons demonstrate a competitive rate of learning and generalization, particularly in more extensive networks. They closely approach the performance of LIF neurons in networks with 80 and 100 neurons, suggesting that Synaptic neurons may benefit more proportionally from increased network capacity. Lapicque and Alpha neurons show a parallel performance pattern, with less effective learning rates and lower overall accuracy. However, their performance improves with network size, indicating that these neuron models may require more neurons to reach optimal performance. In networks with 100 neurons, all neuron types converge in performance metrics, suggesting a saturation point in learning capability for the MNIST dataset. This is particularly relevant in neuromorphic computing, where hardware constraints may limit network size [50].  
The LIF neuron's performance can be attributed to its ability to capture temporal dynamics more effectively, which may be crucial for tasks involving temporal pattern recognition, such as those presented by the MNIST dataset. The Synaptic neuron model's increasing efficacy in more extensive networks could be due to its potential to model complex synaptic interactions, which become more prominent as network size increases. The similar performance trajectory of Lapicque and Alpha neurons could suggest inherent limitations in the expressiveness of these models for the task at hand. Their improvement  
Figure 21: Performance Comparison regarding Accuracy and Loss for each of the four types of neurons after training different SNN architectures with the same number of neurons on the MNIST dataset. More exactly, for 100 neurons in one hidden layer.  
with network size raises questions about the scalability of these neuron models and their potential efficacy in larger, more complex tasks.  
Experiments were also done by increasing the number of hidden layers; however, they are not presented in this paper due to the lower loss and accuracy performance by keeping the hyperparameters the same as was done for one hidden layer.