# Configurable Hardware Core for IoT Object Detection
## 2 Background and Related Work
### Convolutional Neural Network Accelerators in FPGA  
One of the advantages of using FPGAs is the capacity to design parallel architectures that explore the available parallelism of the algorithm. CNN models have many levels of parallelism to explore [28]:  
* intra-convolution: multiplications in 2D convolutions are implemented concurrently;
* inter-convolution: multiple 2D convolutions are computed concurrently;
* intra-FM: multiple pixels of a single output FM are processed concurrently;
* inter-FM: multiple output FM are processed concurrently. Different implementations explore some or all these forms of parallelism [29; 30; 31; 32; 33] and different memory hierarchies to buffer data on-chip to reduce external memory accesses. Recent accelerators, like [33], have on-chip buffers to store feature maps and weights. Data access and computation are executed in parallel so that a continuous stream of data is fed into configurable cores that execute the basic multiply and accumulate (MAC) operations. For devices with limited on-chip memory, the output feature maps (OFM) are sent to external memory and retrieved later for the next layer. High throughput is achieved with a pipelined implementation. Loop tiling is used if the input data in deep CNNs are too large to fit in the on-chip memory at the same time [34]. Loop tiling divides the data into blocks placed in the on-chip memory. The main goal of this technique is to assign the tile size in a way that leverages the data locality of the convolution and minimizes the data transfers from and to external memory. Ideally, each input and weight is only transferred once from external memory to the on-chip buffers. The tiling factors set the lower bound for the size of the on-chip buffer. A few CNN accelerators have been proposed in the context of YOLO. [PERSON] et al. [35] proposed an FPGA-based architecture for the acceleration of Tiny-YOLOv2. The hardware module implemented in a ZYNOZ7035 achieved a performance of 19 frames per second (FPS). [PERSON] et al. [36] also proposed an accelerator of Tiny-YOLOv2 with a 16-bit fixed-point quantization. The system achieved 69 FPS in an Arria 10 GX1150 FPGA. In [37], a hybrid solution with a CNN and a support vector machine was implemented in a Zynq XCZU9 EG FPGA device. With a 1.5-pp accuracy drop, it processed 40 FPS. A hardware accelerator for the Tiny-YOLOv3 was proposed by [PERSON] et al. [38] and implemented in a Zynq XCZU9 EG. The weights and activations were quantized with an 8-bit fixed-point format. The authors reported a throughput of 104 FPS, but the precision was about 15% lower compared to a model with a floating-point format. [PERSON] et al. [39] also proposed a hardware accelerator of Tiny-YOLOv3 layers. Data were quantized with 16 bits with a consequent reduction in mAP\({}_{50}\) of 2.5 pp. The system achieved 2 FPS in a ZYNOZ7020. The solution does not apply to real-time applications but provides a YOLO solution in a low-cost FPGA. Recently, another implementation of Tiny-YOLOv3 [40] with a 16-bit fixed-point format achieved 32 FPS in a UltraScale XCKU040 FPGA. The accelerator runs the CNN and pre- and post-processing tasks with the same architecture. Recently, another hardware/software architecture [41] was proposed to execute the Tiny-YOLOv3 in FPGA. The solution targets high-density FPGAs with high utilization of DSPs and LUTs. The work only reports the peak performance. This study proposes a configurable hardware core for the execution of object detectors based on Tiny-YOLOv3. Contrary to almost all previous solutions for Tiny-YOLOv3 that target high-density FPGAs, one of the objectives of the proposed work was to target low-cost FPGA devices. The main challenge of deploying CNNs on low-density FPGAs is the scarce on-chip memory resources. Therefore, we cannot assume ping-pong memories in all cases, enough on-chip memory storage for full feature maps, nor enough buffer for the full output map.