## 5 Towards Neuromorphic 3D Circuits Using Printed Memristors  
In this chapter, the implications of the earlier experimental results for designing neuromorphic hardware and developing algorithms for SNNs, more precisely, for creating neuromorphic 3D circuits using printed memristors [53], are discussed. These experiments can provide valuable insights into the design, training, and application of SNNs in neuromorphic computing, enhancing the understanding of complex neural network behaviors and architectures [50; 54]. One of the benefits of the proposed SNNtrainer3D application is that users can run various experiments regarding hyperparameter tuning, where different hyperparameters, such as the number of epochs, learning rate, beta value, training steps, and batch size, can be applied. This can investigate the hyperparameter's effects on the convergence speed and accuracy of the SNN on the task of digit classification using the MNIST dataset. Also, after training, users can do a weight visualization experiment to visualize the weights in 3D and gain insights into the connections they have learned and their strengths. This can help understand the network's decision-making process. The proposed application lays the groundwork for possible integration with physical memristor technology, such as neuromorphic circuits [50; 53]. This can be done using the saved weights, which can be used as input in circuit simulation tools such as LTSpice. By training and visualizing SNNs in software, researchers can better understand how to implement these networks using physical memristors in circuit simulators. However, based on the experiments in training SNN in software with the proposed SNNtrainer3D application, creating a neuromorphic 3D circuit using printed memristors [53] for solving the MNIST problem with a single or just a few neurons per layer (i.e., printed paper) would be challenging. The main reason is that a single neuron per layer cannot learn or extract enough features to predict the MNIST problem accurately. This is because neural networks learn by extracting relevant features from the input data across multiple layers, and a single of just a few neurons in a layer cannot capture enough information or features to make accurate predictions. Suppose any layer has only one neuron. In that case, it creates an information bottleneck, where the subsequent layers will not receive enough information to learn additional features, rendering the deeper layers ineffective. It has been for a long time mathematically proven that a simple problem, like learning the XOR gate, cannot be solved with a single neuron at any point in the network, with recent works solving it using the Growing Cosine Unit (GCU) [55]. However, GCU works well for many convolution layers with many neurons, and it wouldn't solve the issue of having a single neuron layer because it cannot learn enough features. Each layer must have at least five neurons to stack many layers. For this, one would have to figure out how to 3D print those 25 connections between layers, raising feasibility concerns. Another challenge is the physical connectivity between neurons in different layers when printing the neural network with memristors [53]. To avoid crossing connections, a 3D structure is necessary [54]. In a 2D plane, it becomes impossible to connect, for example, 10 neurons from one layer to 10 neurons in the next layer without any connections crossing each other. Given the challenges, some possible solutions can be applied. Firstly, instead of tackling MNIST directly, one can start with more straightforward problems that require hidden layers, such as the XOR problem or a small-scale image classification task. Secondly, rather than using a single neuron per layer, one can increase the number of neurons per layer to allow for better feature extraction and learning. This must be carefully analyzed according to the machine's capabilities to print large paper areas of printed memristors [53].