title: Design Space Exploration for Analog Spiking Neural Networks
title: Design Space-Exploration für analoge gepulste neuronale Netze
titleType: AlternativeTitle
Creators: name: Elmasry, Moamen
Publisher: name: Friedrich-Alexander-Universität Erlangen-Nürnberg (FAU)
Year: 2024
Subjects: subject: Brain-inspired computing
subject: Spiking Neural Networks
subject: non-volatile memory
subject: resistive-RAM
subject: near-memory computing

description: The human brain will always remain one of the greatest creations of all time. The development of brain-inspired neuromorphic computing has become mainstream nowadays due to its high parallelism and energy efficiency. Neuromorphic computing is an emerging field that aims to mimic the functionality of the human brain using electronic circuits. One of the most important aspects of this field is the implementation of neural models and the evaluation of their performance. Developing and deploying neuromorphic computing systems is an exceptionally challenging undertaking, requiring the collaboration of researchers from a variety of disciplines. Collaboration is necessary to bring together the development of data processing techniques, computational structures and the fundamental technologies on which these systems are based. The development of neuromorphic computing systems requires the simultaneous processing of large amounts of data, adaptation to dynamic environmental changes and replication of the complex functionality of the human brain. Artificial intelligence plays a role both in large data centers and in end devices such as smartphones, drones and networked household appliances. Data processing directly on these devices is called edge computing. Neuromorphic computing systems for edge computing offer promising prospects. Using these systems, data can be processed and analyzed in real time at the source, rather than in centralized data centers, thereby increasing responsiveness and reducing latencies. Furthermore, local computation satisfies the security requirements of potentially sensitive data by eliminating the need to communicate data over wireless networks to central computing systems. Additionally, neuromorphic systems are well-suited for a range of cognitive applications due to their intrinsic learning capabilities and ability to process complex data. Their versatility and efficiency can significantly advance areas such as pattern recognition, natural language processing and robotics. Moreover, neuromorphic systems are an attractive choice for energy-efficient computing applications, which are increasingly important in today’s world of sustainable technologies, due to their ability to achieve significant energy savings compared to traditional processors (Christensen et al., 2022). The efficient and effective execution of AI on conventional hardware is hindered by the so-called von Neumann architecture. This is characterized by the fact that a processor is responsible for calculating the data and a memory for storing the results. Both are connected by a data bus, which leads to the so-called "memory wall" during the execution of artificial neural networks. In this case, data transfer to and from external memory devices becomes a major obstacle to energy efficiency. In addition, current hardware accelerators struggle with performance and power consumption issues. To address these challenges, non-volatile memory devices such as ReRAM, CBRAM, PCM, MRAM and FeFET offer promising possibilities, as they can partially handle the computation directly in memory, thus helping to reduce extensive data movement. In addition, optimization of device parameters such as memory capacity, lifetime, cycle stability, variation, and speed is crucial for an effective use in neuromorphic applications. Material innovations to enhance analog switching capabilities and three-dimensional component architectures are also being investigated to advance the design of neuromorphic hardware. In this work, a comprehensive design for a spike-based neural network platform has been presented to integrate a full ReRAM (Resistive Random Access Memory) IP implementation into a dynamic Spiking Neural Network architecture. A fully integrated System-on-Module (SoM) has been implemented, enabling a comprehensive exploration of the design space for neurosynaptic behavior in Spiking Neural Networks. This platform includes several state-of the-art components, including a ReRAM array, buffers, word line/bit line (WL/BL) drive and a digital interface, all working in synergy with replaceable non-volatile memory (NVM) modules. These components are designed to accommodate different types of spike-based modules. The interconnectivity of the platform has been a key driver for the introduction of the latest generation NVM to keep pace with ReRAM technology available in the market. This platform provides a comprehensive evaluation of ReRAM technology compared to other non-volatile memory (NVM) technologies. The integration not only increases versatility, but also expands the platform’s capabilities and applications. One of the key innovations in our platform is the emergence of a hybrid Memristor CMOS multi IP architecture. Within this framework, a ReRAM IP module coexists harmoniously with a neural network composed of spiking components. A novel weighted synaptic structure that combines weight detection and synaptic current provisioning is introduced as an interface structure between the NVM macro and the evolved SNN components. This innovative architecture provides practical solutions to real-world problems while providing a research platform for computational neuroscience. Our fully integrated system-on-module, realized in 28 nm CMOS technology, enables enhanced evaluation of the spatio-temporal properties of Spiking Neural Networks (SNNs). This platform provides a reconfigurable and interchangeable environment for modular testing of spike-based components. Integrated non-volatile memory in the form of a 128x128 ReRAM cell array enables storage of synaptic weights, while fully traceable signals ensure reliable data acquisition and analysis. The integration of a considerable number of I/O pins improves the testability of the system. Furthermore, the integrated neurons show competitive performance in terms of parameter sets, reconfigurability and energy efficiency, with activation requiring only about 0.2 pJ per spike.