## 3 Energy-Efficient Architectures
### Spatial Architectures: Fpgas and Asics
#### 3.2.3 Reconfigurable Accelerators  
Given the increasing interest in deep learning, a wide variety of models with very different features and layers have emerged, as seen in Section 2.3. However, the majority of ASIC/FPGA accelerators for DNNs are designed and optimized to support only one type of dataflow. It can be complex to map different layers on these accelerators equally efficiently. To allow for more widespread and mass deployment of ASIC/FPGA accelerators, flexible and easily reconfigurable designs are required to support different types of layers and models.  
FlexFlow [87] and DNA [88] are two accelerators that support a flexible dataflow to exploit the different types of reuse and parallelism of Conv layers. However, they only target CNNs. On the other hand, MPNA [89] supports dedicated PE array units for Conv and FC layers. In [90], an ASIC  
\begin{table}
\begin{tabular}{l c c c c c} \hline \hline
**Accelerator** & **Ref.** & **Weight Bits** & **Activation Bits** & **Features** & **Target** & **Year** \\ \hline Stripes & [82] & 16-bit & 1-bit to 16-bit & Bit-serial & ASIC & 2016 \\ \hline UNPU & [83] & 1-bit to 16-bit & 16-bit & Bit-serial & ASIC & 2019 \\ \hline Loom & [84] & 1-bit to 16-bit & 1-bit to 16-bit & Bit-serial & ASIC & 2018 \\ \hline Bit Fusion & [85] & 1,2,4,8,16-bit & 1,2,4,8,16-bit & Spatial combination & ASIC & 2018 \\ \hline BitBlade & [86] & 1,2,4,8,16-bit & 1,2,4,8,16-bit & Spatial combination & ASIC & 2019 \\ \hline \hline \end{tabular}
\end{table}
Table 3: Comparison of accelerators that support variable bitwidth operations.  
reconfigurable processor targeting hybrid NNs, i.e., networks with different layers, is presented. The PEs are organized into two 16 \(\times\) 16 arrays, and they are divided into general PEs and super PEs. The former supports Conv and FC layers, while the latter supports Pooling layers, RNN layers, and non-linear activation functions. Each PE has two 8-bit multipliers that can be used separately or jointly to form a 16-bit multiplier, allowing for a variable bitwidth. The arrays can be arbitrarily partitioned into sub-arrays to process multiple layers or networks in parallel. Project Brainwave [91] is an FPGA platform used in Microsoft servers for real-time AI. The core of Project Brainwave is the NPU, a spatially distributed microarchitecture with up to 96 thousand MACs. The architecture achieves flexibility working with vectors and matrices as data-types, using efficient matrix-vector multipliers and multifunction units that can be programmed to implement a function chosen in a broad set. For easy programming, the architecture has a custom SIMD Instruction Set. MAERI [92] has a set of PEs, each containing a Register File and a multiplier. The reconfigurability is obtained with the interconnections. The activations and weight are delivered to the PEs with a distribution tree fully configurable. Similarly, the outputs of the multipliers are collected by a configurable adder tree. SIGMA [93] introduces the Flexible Dot Product Engine (Flex-DPE), which has a structure similar to MAERI. In the Flex-DPE, the multipliers are in fact arranged in a 1D structure. Thanks to highly flexible distribution and reduction networks, multiple variable-sized dot-products can be performed in parallel. Thanks to the flexible distribution network, SIGMA also supports the acceleration of sparse networks. To maximize energy efficiency, DNPU [94] proposes a heterogeneous architecture with two processors, one optimized for Conv layers and CNNs, the other targeting FC layers and Recurrent Neural Networks. Cerebras has recently announced the Cerebras Wafer Scale Engine (WSE), the largest chip ever built and specialized for DL computing only. The WSE has a huge numbers of flexible cores that support general operations (e.g., arithmetic, logical, load/store operations) but are optimized in particular for tensor operations. The memory, in the order of gygabytes, is on-chip and distributed.