# A Survey on Memory Subsystems for Deep Neural Network Accelerators
## 6 Processing-in-Memory (PIM)  
Unlike 3D memory and ReRAM-based memory architectures, processing-in-memory is another rich memory technology that has wide scope in design of efficient neural networks. Yet, PIM architectures have certain challenges that need to be addressed. Firstly, it is difficult to obtain computations of high precision with PIM. Also, PIM architectures face the limitation of nonscalability of analog/mixed circuits designs. These challenges are catered in FloatPIM architecture proposed in [13]. In the presented architecture, out of 32 tiles in PIM, every tile embraces a memory block with 256 crossbars with row and column drivers. The switches allow simultaneous flow of data between different blocks of memory. The operations of switches and row and column drivers are synchronized by a controller. Float PIM architecture incorporates digital data storage features which enables block-size scalability with need of lesser memory subblocks for computations. The memory blocks are arranged in such a manner that each of the blocks is connected to two of its adjacent neighboring blocks. This arrangement allows effective row-parallel communication in between the blocks. At first cycle of the clock (T\({}_{0}\)), all the memory blocks work parallelly in computing mode. In the other two cycles, T\({}_{1}\) and T\({}_{2}\), the blocks operate in data transfer mode. In T\({}_{1}\), the odd-numbered blocks send data to neighboring blocks, while in T\({}_{2}\), data is transferred by even-numbered blocks. In this manner, the complete data transfer is carried out in only two consecutive cycles. In [14], another PIM-based architecture is a proposal that focuses on an energy efficient design possessing high-performance binary weights in deep neural networks. A presented architecture, ParaPIM, aims at achieving parallel access of spin-orbit torque magnetic random access memory subarrays for efficient and high-speed BWNNs. The structural block diagram of presented ParaPIM as depicted in Figure 28, broadly involves image and kernel banks, memory subarrays, and a digital processing unit. Before the mapping of kernels into memory subarrays, they are binarized by binary unit of DPU. The obtained results are further drawn to corresponding subarrays. The ParaPIM subarray structure is also depicted in Figure 28. It can be operated in double mode for both read/write memory operations and bit-line computations. The comparison results of different memory architectures on the BWNN model are gathered in [14] for a common task considering batch sizes of 8 and 32 as shown in Figure 29. It is witnessed that ParaPIM is at least 3.9 times a more energy-efficient model than DRAM for batch size 8. Also, when considering a comparison with a ReRAM-based model with batch size 32, ParaPIM comes out as being 13 times more energy efficient. Likewise, the super-fast and in-situ operations place it ahead of other technologies in terms of performance as well. Paper [15] presents a comparator-based PIM memory architecture accelerator design for a fast and efficient comparator-based neural network. The presented architecture utilizes parallel SOT-MRAM-based memory subarrays as processing units. The pre-existing CNN algorithms are altered by substituting convolution layer multiplications with lesser complex additions and comparisons as depicted in Figure 30.  
Figure 28: Proposed ParaPIM architecture [14]. Figure 29: Comparison results for (**a**) normalized energy efficiency; (**b**) normalized performance of different memory architecture accelerators [14]. The basic block comprises a set of operating functions as batch normalization, depthwise and pointwise convolution. Figure 31 depicts the presented CMP-PIM architecture, and it comprises image and kernel banks, memory subarrays, and a digital processing unit. Before the mapping of kernels into memory subarrays, they are binarized by binary unit of DPU. The obtained results are further drawn to corresponding subarrays. [PERSON] et al.