# A Survey of Convolutional Neural Networks on Edge with Reconfigurable Computing
## 3 Reduction of Complexity of CNN Models  
The large number of parameters of a CNN model increases the design space but also introduces a high redundancy and increases the number of design points corresponding to a good solution. This simplifies the learning process of the network at the cost of increasing the number of parameters. On the other side, it leaves space for a significant simplification of the model. Different simplifications \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Model** & **\# Layers** & **\# Parameters** & **\# MACCs** & **Error-5(\%)** \\ \hline AlexNet & 8 & 60 & 650 & 19.7 \\ ZefNet & 8 & 60 & 650 & 11.2 \\ VGG16 & 16 & 138 & 7800 & 10.4 \\ SqueezeNet [24] & 18 & 1.2 & 860 & 19.7 \\ GoogleNet & 22 & 5 & 750 & 6.7 \\ Inception-v3 & 48 & 23.6 & 5700 & 5.6 \\ Inception-v4 & 70 & 35 & 6250 & 5 \\ ResNet-101 & 101 & 40 & 3800 & 6.8 \\ ResNet-152 & 152 & 55 & 5650 & 6.7 \\ ResNet-200 & 200 & 65 & 6850 & 5.8 \\ ResNeXt [25] & 101 & 68 & 4000 & 5.3 \\ DenseNet-201 & 201 & 16.5 & 1500 & 6.3 \\ SENet-154 & 154 & 100 & 10,500 & 4.5 \\ MobileNet-v1 & 28 & 4.2 & 569 & 10 \\ MobileNet-v2 & 28 & 3.5 & 300 & 9 \\ ShuffleNet & 11 & 5.3 & 260 & 10 \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Complexity and accuracy of known convolutional neural network (CNN) networks. or optimization methods have been proposed that take advantage of this redundancy. The objective of these simplifications is to reduce the memory required to store weights and the computational complexity during network inference. The complexity reduction of CNN models is fundamental for deploying large CNNs in edge devices with limited hardware resources and energy. Two main classes of optimizations have been considered: (1) Data quantization and (2) data reduction. Data quantization is the process of reducing the arithmetic complexity and the number of bits (bitwidth) used to represent weights and activations (neurons outputs after activation function). Reducing the arithmetic complexity is done by converting single precision floating-point operations to fixed-point, integer or even power of two to convert multiplications to logic shifts. In any of these type of arithmetic, the number of bits used to represent data can be reduced. Custom floating-point representations consider less bits to represent floating-point data. A total of 16- [26] and 8-bit [27] floating-point are commonly used representations. Reconfigurable devices can take advantage of its hardware flexibility to implement optimized arithmetic units with custom bitwidths. Several authors have explored data quantization for the design of optimized implementations for CNN inference. In [28], the author has shown that fixed-point data representations with 8 bits guarantee accuracies close to those obtained with 32-bit floating point. Several other works [29; 30; 31] have shown that the same inference accuracy can be achieved with reduced precision of weights and activations. The size of data can be fixed for all layers or optimized for each layer [32]. In [33], the authors propose different bit sizes for different layers: Hybrid quantization. An important aspect is that the first and last layers are the most sensitive to size of data. Therefore, they use 8 bits for the first and last layers and reduce the size in the hidden layers. The authors also explore data reduction for activations and weights, concluding that activations are more sensible to bit width reduction. Reduced hybrid quantization improves performance, memory, and energy efficiency. The optimization of quantization can be efficiently achieved with reconfigurable devices, in particular FPGAs where each operand can be tailored for best quantization. Even coarse-grained arithmetic blocks of FPGA--DSP (Digital Signal Processing) slices--take advantage of quantization since multiple multiplications and additions can be implemented in a single DSP if reduced precision is considered [34]. Data quantization also consider very small representations.