# A Survey on Memory Subsystems for Deep Neural Network Accelerators
## 4 ReRAM Technology
Authors in [25] mention that in compared with PRIME [22], the ability of the FPSA can improve the computational density by about \(31\times\)  
Figure 15:2 Crossbar-based pruning algorithm proposed in [24]. Every NN possesses two stages, training and testing. In the testing phases, the network's weights have been established and data drifts via the layers in a forward path. Before using neural networks in any application, it is required to move into the training stage in which the weights are generated by the model. The data moves bidirectionally to update the weights. In [26], training and testing are aided in a ReRAM-based accelerator called Pipelayer. Interlayer parallelism is carried out with an effective pipeline system in which data constantly drifts in the accelerator at successive cycles. In addition to it, the pipelayer architecture utilizes a spike-based scheme for data input instead of voltage level. The overhead of ADC and DACs is removed in such a manner. Pipelayer configuration immediately utilizes the ReRAM cells in order to carry out computations regardless of the processing units. Figure 24 shows the pipelayer architecture to process the training phase of a neural network. The main memory is spilt into two categories, morphable and memory subarrays. The memory subarrays can store data while the morphable subarrays can both store data and carry out computations. In Figure 24, the memory subarrays are represented by circles and rectangles represent morphable subarrays. The results of computations carried out at morphable subarrays are stored in memory subarrays. 3D-ReG is another ReRAM-based heterogeneous manycore system that has been proposed for accelerating of the training phase of deep neural networks [27]. 3D integration and ReRAM are two technologies that have been applied for designing the 3D-ReG. A  
Figure 15:3 Pipelayer architecture to process the training phase of a neural network [26]. Figure 15:4 Overview of the FPSA architecture and reconfigurable wiring network [25]. processing-in-memory (PIM) design based on the ReRAM technology has been introduced in [27] to accelerate the DNNs. The designed PIM possesses two distinct array types, memory and morphable arrays. Memory arrays are used as standard data storage whereas morphable arrays are enabled to jump between the data storage and computational data. A ReRAM layer has been stacked on the GPU layer in the 3D-ReG architecture as shown in Figure 15:5  
Another ReRAM-oriented deconvolution architecture (RED) design of neural accelerator has been proposed in [28]. The performance is analyzed for deconvolution execution for pre-existing PIMs. Extra operations are disregarded to improve the utilization. Figure 26 illustrates the overall presented ReRAM deconvolution architecture (RED). A local input buffer is common for various processing elements to omit the inserted zeroes. The single functional (SFB) and multifunctional (MFB) registers in the input buffer allow the reuse of data with shift and buffer modes. When in shift mode, the buffers shift the data required for carrying out the computations. On the other hand, the buffer mode is used to load the data from memory or supply to the processing elements. Each processing element (PE) consists of an array of ReRAM crossbars and peripheral devices as decoders, multiplexers, and read circuits. For a computational result, the outputs from a PE cluster are selected via multiplexer and are added to compute the final result. Firstly, a pixel-wise mapping scheme is carried out which maps the kernel weights to resistive RAM arrays at a very fine level. Parallel execution of various computation modes is facilitated through this technique. Secondly, in order to facilitate the reuse of input data, a  
Figure 15:6 Block diagram depicting the RED architecture proposed in [28]. Figure 15:7 An overview of the 3D-ReG architecture [27]. zero-skipping data flow technique is also presented.