# A Survey on Memory Subsystems for Deep Neural Network Accelerators
## 3 3D Memory Integration  
In order to integrate computational layers inside a three-dimensional topology of a memory package possessing very high levels of density, the hybrid memory cube (HMC) is a new concept introduced in [4]. DRAM dice are layered upon a logic die vertically in the  
Figure 1: Classification of the memory architectures discussed in this survey. Figure 2: PRISMA flow chart followed for paper selection. HMC. The HMC offers an efficient near-data processing with respect to its large memory bandwidth and capacity, in addition to low latency. In the proposed HMC-based accelerator, named Neurocube, in [4], there is a logic die in the lowest layer and DRAM layers stacked on it, shown in Figure 3. The logic die in the lowest layer includes logic and computation components. As illustrated in Figure 3, the implementation of data storage for each PE is achieved by the DRAM dies in the vault. Since every vault possesses its own vault controller in the logic layer, the PEs can access multiple vaults in parallel. HMC in the Neurocube accelerator has 16 vaults. As demonstrated in Figure 3 every vault is considered to be a subsystem, consisting of a router, and stacked DRAM dice. PEs perform multiply-accumulate (MAC) operations. Routers perform data transfer between the DRAM and logic layers. A source vault has the ability to transmit data packets to a target vault via the routers. Neurocube programs the implementation of a single layer of an NN individually. Figure 4 examines the operation of a PNG, the key component in memory-centric neural computing in the accelerator, for each layer. The important mission of the PNG is to generate the weights and addresses of the neurons residing in the previous layer in the memory. In addition to Neurocube, which is based on the HMC concept, TETRIS [5] is the other DNN accelerator based on the HMC. The performance of NN accelerators is limited by off-chip main memory and on-chip buffers. Although sizable on-chip SRAM buffers amounts in higher memory bandwidths, and DRAM access is also reduced, it also leads to a significant amount of power consump  
Figure 4: Proposed programming and execution of a NN in Neurocube [4]. Figure 3: Neurocube accelerator with detailed PE architecture. TSV: through silicon via [4]. tion (Figure 5). The 3D memory technology promises an efficient memory system for the NN accelerator. DRAM memory dice are integrated vertically over the logic layer as a single package. This improves the bandwidth and lowers the access energy significantly. In order to completely utilize 3D memory benefits, there are certain challenges that are required to be addressed. Firstly, 3D memory technology allows different use of smaller on-chip memory buffers. Secondly, 3D technology gives the opportunity to decide where the computations will be carried out. Some operations can be moved close to memory locations. Thirdly, there is a wide scope to frame new approaches for dataflow scheduling of NN computations. Last but not least, an efficient partitioning scheme is required for highly parallelized 3D memory stacks. Addressing the above stated challenges in [5], the scalability of neural networks is improved by the proposed TETRIS architecture in which the hardware includes 3D stacking of memory with processing elements. In addition to this, for the software side, an analytical representation is developed for optimized data flow schedules. At the hardware level, more area is utilized in processing elements in addition to minor area utilization with respect to SRAM buffers. In the proposed TETRIS architecture, the HMC stack is alienated into 16 32-bit vaults. TSVs get utilized by the vault channel bus in order to connect to the DRAM memory dice to base logic die. This results in two banks per vault in each DRAM die (Figure 6). Two-dimensional mesh network is incorporated for the NN accelerator. All processing elements share a global buffer for storage and reuse of data from memory. In this manner, significant bandwidth of memory is saved, and performance is improved.