# FPGA-QNN: Quantized Neural Network Hardware Acceleration on FPGAs
## 4 Definition of Models in FPGA-QNN
### The Accelerator System Design  
This paper makes use of a general WxAx formation for defining the quantization levels. The terms of W and A in WxAx symbolize the weights and activations. The bit width for the actual quantization level is expressed by the term of x. For MLP and LeNet models at W1A1, W1A2, and W2A2 quantization levels using Pytorch and Brevitas, 12 accelerators have been designed according to high (H) and low (L) folding levels. The properties of the developed accelerators are given in Table 1.  
The selection of quantization levels (W1A1: 1-bit Weight, 1-bit Activation, W1A2: 1-bit Weight, 2-bit Activation, W2A2: 2-bit Weight, 2-bit Activation) is based on a methodology aimed at optimizing the balance between accuracy, performance, and resource utilization. Low bit-width significantly reduces the usage of resources such as flip-flops (FF), look-up tables (LUT), digital signal processing (DSP) units, and block RAM (BRAM) on platforms like FPGA or ASIC. This provides a critical advantage in terms of reducing energy consumption and hardware costs. While low bit-width levels like W1A1 generally improve resource efficiency, they may have a negative impact on accuracy. The selected parameters have been optimized to minimize accuracy loss. Additionally, increasing bit-width raises latency, thereby reducing the number of operations performed per second. Moreover, as the bit-width increases, the hardware requirements grow, leading to higher energy consumption. In this context, the chosen quantization levels have been determined to align with values selected in similar studies in the literature, while maintaining a focus on balancing performance and cost efficiency. In the first stage, a deep learning network is designed and trained in MLP and LeNet models using Pytorch and Brevitas. The training of the models was carried out on the Google Colab platform with Nvidia Tesla K80 GPU, 12 GB memory and 4.1 TFLOPs running on Linux operating system. When the deep learning models used are created quantitatively with Brevitas, they are divided into many blocks such as multiplication (Mul), subtraction (Sub), multiple thresholding (MultiTreshold), matrix multiplication  
\begin{table}
\begin{tabular}{c c c} \hline \hline
**Model** & **Precision** & **Folding** \\ \hline \multirow{4}{*}{MLP} & W1A1 & L \\  & W1A1 & H \\  & W1A2 & L \\  & W1A2 & H \\  & W2A2 & L \\  & W2A2 & H \\ \hline \multirow{4}{*}{LeNet-5} & W1A1 & L \\  & W1A1 & H \\ \cline{1-1}  & W1A2 & L \\ \cline{1-1}  & W1A2 & H \\  
(MatMul) and normalization (BatchNorm Normalization). While the model created using Pytorch is simpler and consists of only layers, the model created with Brevitas is divided into many components. The present model automatically determines the quantization type based on the weight and activation bit quantization number. For instance, if the weight is 1 bit and the activation is 1 bit (W1A1), the quantization type is binary, while when the quantization parameter is selected as W2A2, the quantization type is integer at the specified bit width. For binary quantization type, the sign(x) activation function is used with thresholding, while the ReLu activation function is used in the integer quantization type. Models created using Brevitas with quantization parameters W1A1, W1A2, and W2A2 were trained with 100 training epochs. In this training phase, accuracy and loss data were recorded and presented in Figure 12. During the backward propagation phase, it is observed that the MLP model consistently increases its accuracy across all three quantization levels, whereas significant spikes occur in the accuracy levels of the LeNet model at the W1a1 quantization level. This phenomenon underscores the need for a higher number of training iterations in the training phase of BNN networks. The models trained on with the Brevitas library and Pytorch framework are converted to ONNX format and imported into the FINN framework. Using FINN converters with these six models, 12 deep learning accelerator hardwares are obtained at H and L folding levels.