Efficient Systolic Array Based on Decomposable MAC for Quantized Deep Neural Networks

Abstract: Deep Neural Networks (DNNs) have achieved high accuracy in various machine learning applications in recent years. As the recognition accuracy of deep learning applications increases, reducing the complexity of these neural networks and performing the DNN computation on embedded systems or mobile devices become an emerging and crucial challenge. Quantization has been presented to reduce the utilization of computational resources by compressing the input data and weights from floating-point numbers to integers with shorter bit-width. For practical power reduction, it is necessary to operate these DNNs with quantized parameters on appropriate hardware. Therefore, systolic arrays are adopted to be the major computation units for matrix multiplication in DNN accelerators. To obtain a better tradeoff between the precision/accuracy and power consumption, using parameters with various bit-widths among different layers within a DNN is an advanced quantization method. In this paper, we propose a novel decomposition strategy to construct a low-power decomposable multiplier-accumulator (MAC) for the energy efficiency of quantized DNNs. In the experiments, when 65% multiplication operations of VGG-16 are operated in shorter bit-width with at most 1% accuracy loss on the CIFAR-10 dataset, our decomposable MAC has 50% energy reduction compared with a non-decomposable MAC.

Reviews and author responses:

This paper proposes to shorten the shift-addition operations in the straightforward configurable MACs (Sharma et al., 2018), to an addition-shift style. The authors claim that the new design is able to lower the energy consumption in the matrix multiplication. In the experimental analysis, the authors demonstrate the effectiveness of the proposed method.

This paper should be rejected since it proposes exactly the same architecture with the following published work:
BitBlade: Area and Energy-Efficient Precision-Scalable Neural Network Accelerator with Bitwise Summation

Also, the authors do not provide a valid approach for the auto-selection of quantization bits, which is more significant in my opinion.

This paper presents a decomposable MAC unit for low-power computation for matrix-multiplication operation for neural networks.

Although I believe I understood the idea of this technique, I strongly believe this paper should be submitted to an architecture of design automation conference instead of ICLR. I am also not in the position to assess the experiments, which were conducted with synthesis of Synopsis 28nm library. The paper discussed systolic array, MAC in detail, without too many algorithmic elements inside, therefore may be a paper not toward the audience of this conference.

Summary: 

This paper proposes a novel decomposition strategy for matrix multiplication to aim for less energy consumption. They demonstrate that energy consumption is reduced on several CNNs. It is an interesting work but it is not sure that there is a difference in contribution when compared to previous work.

The main argument to impact to the score:

    1. A similar idea is addressed in [1]. It is highly recommended to show the difference in contribution.
    2. In Figure 7,  the performance and energy consumption of systolic arrays of decomposable MAC  is shown. However, the only energy consumption is shown in section 4.2 when the method is applied to DNNs. It is better to show both performance and energy consumption.
    3. It is inappropriate to quantize VGG16, VGG19, DenseNet-121, DenseNet-169 and MobileNet based on CIFAR10 dataset since all models are designed for ImageNet dataset. It is appropriate to quantize these models on ImageNet dataset.


[1]     Sungju Ryu, Hyungjun Kim, Wooseok Yi and Jae-Joon Kim. BitBlade: Area and Energy-Efficient Precision-Scalable Neural Network Accelerator with Bitwise Summation. DAC. 2019.

Minor comments not to impact the score: 
    1. "Very Deep Convolutional Networks for Large-Scale Image Recognition" has been accepted by ICLR 2014. It is better not to use arXiv preprint on citations unless there is a reason.
    2. It is recommended to put citations on CIFAR-10 dataset.

This paper proposes decomposable MAC and a systolic array-based accelerator for varible bitwidth of weights/activations in CNN. The core insight is that the partial results of a multiplication which have the same shift distance can be composed together before shifted so as to reduce area and energy consumption.

However, exactly the same idea has been seen in previous work BitBlade [1], which was also an incremental work based on Bit Fusion [2]. The difference is that BitBlade adopted a 2D-mesh architecture instead of systolic array. 

I think the authors should cite BitBlade in the paper and reconsider the contributions. 

Thanks.



[1] Sungju Ryu, Hyungjun Kim, Wooseok Yi, Jae-Joon Kim. BitBlade: Area and Energy-Efficient Precision-Scalable Neural Network Accelerator with Bitwise Summation. DAC'2019
https://dl.acm.org/citation.cfm?id=3317784

[2] Hardik Sharma et al. Bit Fusion: Bit-Level Dynamically Composable Architecture for Accelerating Deep Neural Networks. ISCA'2018

Meta-review: This paper presents an energy-efficient architecture for quantized deep neural networks based on decomposable multiplication using MACs. Although the proposed approach is shown to be somehow effective, two reviewers pointed out that the very similar idea was already proposed in the previous work, BitBlade [1]. As the authors did not submit a rebuttal to defend this critical point, I’d like to recommend rejection. I recommend authors to discuss and clarify the difference from [1] in the future version of the paper. 

[1] Sungju Ryu, Hyungjun Kim, Wooseok Yi, Jae-Joon Kim. BitBlade: Area and Energy-Efficient Precision-Scalable Neural Network Accelerator with Bitwise Summation. DAC'2019


Decision: reject