Compute- and Process-in-Memory Modules for Machine Learning and Neuromorphic Computing
Creators: name: Wu, Yuting
Publisher: name: My University
Year: 2023
Subjects: subject: Memristor
subject: Process in memory
subject: Neuromorphic computing
subject: Machine learning acceleration
subject: Compute in memory
subject: Electrical Engineering
subject: Engineering

description: The conventional von Neumann architecture increasingly faces performance and energy bottlenecks imposed by constant data movement. Compute-in-Memory (CIM) and Process-in-Memory (PIM) are considered as promising candidates to provide efficient processing for data-intensive applications. They are designed to process at the data location, i.e. directly in the memory cells (CIM), or near the data location, i.e. on the same memory chip (typically DRAM, PIM). By performing the computation in the analog domain, RRAM crossbar arrays can perform vector-matrix multiplication (VMM), the core computation of machine learning, with high parallelism and throughput. However, hardware limitations caused by device non-idealities and limited precision in analog computing nature hamper practical implementations. In this work, two types of memristors that demonstrate good multi-cell switching capability were integrated with CMOS transistors in a standalone modular design. Specifically, the bulk-type resistive random-access memory (b-RRAM) exhibits analog switching with low operation current. A custom simulator was developed that models all levels of hardware constraints. The simulator can retrain NN models to alleviate accuracy drop in the CIM systems. Software-comparable accuracy was obtained for MNIST classification and keyword spotting tasks on chip. Training of deep neural networks was also explored on the CIM system in a mixed-precision approach, where the low-precision CIM modules accelerates the VMM operation and the high-precision digital system computes and accumulates weight updates. Weight updates will xvii only be propagated to RRAM devices when exceeding a predefined threshold, thus relaxing the requirement for programming endurance and granularity. The efficacy of the proposed method was validated with experimental implementation of LeNet and hardware-aware simulation of larger DNN models. On-chip trained models are inherently robust to hardware non-idealities and can be directly transferred to other CIM chips without retraining. Large language models (LLMs) like GPT have gained tremendous interest recently. LLMs, based on Transformer architectures, require constant access to large amounts of data, making CPUs and GPUs inefficient for GPT inference tasks. Inspired by a recently announced GDDR6-PIM technology from SK Hynix, we proposed a PIM-GPT architecture to address this. PIM-GPT consists of GDDR6-PIM chips that perform VMM operations locally, and a custom ASIC for the rest of model and data distribution. The proposed system achieves 41×~137× (639×~1074×) speedup compared to GPU (CPU) on 8 tested GPT models. Besides machine learning applications, memristive devices also possess rich internal dynamics that can be exploited to implement brain-inspired algorithms for spatio-temporal data processing. We proposed a second-order memristor-based system for neural functional connectivity detection. The system directly takes spikes from a biological network and detects the correlations of the spikes between pairs of neurons. It shows good performance under a wide range of network settings and device non-idealities and captures the dynamic connectivity evolutions. We further presented a reservoir computing system using dynamic memristors that exhibit short-term memory effects. The proposed RC system can reliably decode the running speed from spikes recorded from the mouse brain with 84.9% accuracy over 2 weeks. Device-to-device variations were employed to construct a high-dimension feature space. Looking ahead, tiled chiplets will be developed for larger CNN applications and the mapping scheme and dataflow will be investigated to accelerate general Transformer models with PIM. By performing computation in or near memory, we develop computing systems that provide efficient processing capabilities for data-intensive applications.