# A Survey on Memory Subsystems for Deep Neural Network Accelerators
## 6 Processing-in-Memory (PIM)
in [17] have proposed a processing-in-memory architecture derived from a runtime system considering a system gleaned from a runtime design. According to the authors' observations in [17], the most time-consuming operations, which dominate in the training phases of the DNNs, are the most memory intensive. Based on their observations, simply executing pure multiplication and summation are inadequate for cumbersome memory exhaustive computations. Relu, MaxPool, ApplyAdam, Conv2 DBackpropFilter, and Conv2 DBackpropInputs are the best example functions in DNNs' training phases, which are composed of operations and logic beyond the capacity of simple summation and product computations [16]. These comprehensive operations have the capability to exhaust on average 40% of the total implementation time of the DNNs based on the study in [17]. This observation shows a heterogeneous computation requirement in DNNs. Therefore, a heterogenous PIM design which intertwines micropro  
Figure 31: (**a**) Presented CMP-PIM architecture; (**b**) detailed structure of P-sum [15]. Figure 30: Block representation of comparator-based neural network [15]. cessor based/programmable cores and fixed function logic has been proposed in [17]. In the presented architecture, a runtime framework oversees time-based computations for CPU, fixed function PIM, and programmable PIM. The runtime framework includes two steps, profiling and scheduling. In the profiling phase, the runtime framework for each clock cycle tracks every computation with a hardware counter accumulating the amount of main memory access-level cache misses and the execution time shown in Figure 32. The runtime framework makes a decision derived from the profiling results about determining the eligible functions, which can be outsourced to the CPUs and PIMs. In making the decision, the runtime framework organizes operations into two categories determined by the amount of main memory accesses and execution times. These categorical lists are in descending order and every computation in each list has one index. The runtime framework generates a global index by the summation of the two respective indexes. A global list is generated by organizing all operations evolved from their respective global indexes. Finally, from this global list the top operations are chosen to unburden the PIMs via the runtime framework. In the scheduling phase, the runtime framework schedules the given candidate operations to offload to CPU, fixed function PIMs, and programmable PIM based on a set of principles mentioned in the paper. A heterogeneous PIM architecture is proposed in [17]. This particular software/hardware scheme encourages fast data communication between memory and processors (Figure 33). The presented design broadly has three key mechanisms. Firstly, fixed function logic blocks and programmable hubs are stacked in a 3D network to form heterogeneous architecture. Secondly, the programming challenges are catered by the OpenCL model. Then, a runtime system is presented to maximize hardware utilization and operation parallelism. Figure 32: The proposed runtime framework in [17]. The runtime system proposed in [17] includes two main features. Firstly, it includes a pipeline scheme to simultaneously run multiple neural network operations. Secondly, it also includes a recursive method to call the fixed function PIMs. Hardware utilization is considerably improved through these topographies. To achieve a bit precision reconfigurable digital in memory computing macro, smaller components called bitcells are used in [18]. A macro consists of a 128 by 128 bitcell array. Each bitcell is comprised of three parts:  
1. 6T SRAM cell: stores binary values for weight. 2. Bitwise multiplier replicated by an XNOR gate. 3. Full adder: carries out bitwise addition. An implementation of parallel configured neurons can be represented by bitcell arrays. Furthermore, it was found that bitcell arrays have the capability to be reconstructed into parallel row neurons.