# FPGA-QNN: Quantized Neural Network Hardware Acceleration on FPGAs
## 4 Definition of Models in FPGA-QNN
### The Accelerator System Design
After transferring these models into FINN, a series of graph transformations are applied, as outlined in Figure 13. In the transformation block of TidyUp, the nodes in the graph are initially given names with unique numbers, and then readable and meaningful names are given according to the node names. Then, the shapes and data types of the tensors are derived using model features. The next transformation is to add folding properties to the nodes. After TidyUp, a new block is attached to the beginning of the model, which divides the 8-bit image data by 255 to normalize it to the range of (0, 1). The purpose of the StreamLine transformation is to convert floating point operations into integer operations and then collapse them into multiple threshold nodes in a single operation. Contrary to MLP model, The LeNet model transforms the convolution nodes into sliding window arrays after the StreamLine transformation. Thus, instead of performing convolution, simple matrix multiplication is carried out. In the bitstream model, each row of the input matrix is multiplied by the flattened form of the core matrix and added together with parallel adders to obtain the feature matrix. To lower high-level operations, the upcoming transformation converts the nodes into synthesizable HLS layers. The most important transformation takes place in the folding block as this section contains the parallelization process that determines the resource usage and acceleration level of the network. FINN transformations automatically deter  
Figure 12: Training phase accuracy and loss plots for MLP and LeNet quantized models. mine the quantization type based on the weight and number of activation bits selected in both models. In this study, two different folding types for MLP and LeNet models were determined by calculating the total folding values of PE and SIMD parameters as H and L. For the MLP model, at the L folding level, the PE and SIMD parameters were set to 1 for all quantization types, and no folding was performed. In this study, two different folding types for MLP and LeNet models were determined by calculating the total folding values of PE and SIMD parameters as H and L. For the MLP model, at the L folding level, the PE and SIMD parameters were set to 1 for all quantization types, and no folding was performed. For the H folding level, the folding parameters were calculated using the equations and formulas below. **NHWC**\(\Rightarrow\)**N** = Batch size, **H** = Height, **W** = Width, **C** = Channels  
\[C=F\times P \tag{1}\]  
where \(C\) is the number of channels, \(F\) is the folding factor, and \(P\) is the parallelism level. \[H\mod PE=0,\quad W\mod SIMD=0 \tag{2}\]  
Here, \(PE\) must be a divisor of the input matrix height, and \(SIMD\) must be a divisor of the input matrix width. \[F_{N}=\frac{H}{PE}\quad\text{(Neuron folding level)} \tag{3}\]  
\[F_{S}=\frac{W}{SIMD}\quad\text{(Synapse folding level)} \tag{4}\]  
\[T_{F}=F_{N}\times F_{S}\quad\text{(Total folding level)} \tag{5}\]  
\[L_{FPS}=\frac{T_{F}}{F_{clk}}\quad\text{(Frames per second)} \tag{6}\]  
The \(PE\) value represents the number of output features, while \(SIMD\) corresponds to the number of input features, which must also be divisible without a remainder. When evaluating the performance of the accelerator, the \(L_{FPS}\) value, which indicates the number of frames processed per second, is compared. Figure 12.0 Transformations applied to models in FINN. At the H folding level, to maximize the utilization of FPGA resources, the folding values were set to 64 for the W1A1 quantization type, 128 for W1A2, and 256 for W2A2. These values were chosen to optimize the use of FPGA resources on the PYNQ Z1 board (Digilent, Pullman, WA, USA). Higher SIMD and PE values can provide higher performance but also mean greater resource usage. This balance is generally found through an experimental approach and using resource estimation tools. The values of these parameters were determined through experimental evaluations within the constraints of PYNQ Z1 resources.