Partitioning FPGAOptimized Systolic Arrays for Fun and Profit We can improve the inference throughput of deep convolutional networks mapped to FPGAoptimized systolic arrays, at the expense of latency, with array partitioning and layer pipelining. Modern convolutional networks have a growing number of layers, such as the 58 separable layer GoogleNetv1, with varying compute, storage, and data movement requirements. At the same time, modern highend FPGAs, such as the Xilinx UltraScale VU37P, can accommodate highperformance, 650 MHz, layouts of large 1920x9 systolic arrays. These can stay underutilized if the network layer requirements do not match the array size. We formulate an optimization problem, for improving array utilization, and boosting inference throughput, that determines how to partition the systolic array on the FPGA chip, and how to slice the network layers across the array partitions in a pipelined fashion. We adopt a two phase approach where (1) we identify layer assignment for each partition using an Evolutionary Strategy, and (2) we adopt a greedybutoptimal approach for resource allocation to select the systolic array dimensions of each partition. When compared to stateoftheart systolic architectures, we show throughput improvements in the range 1.31.5x and latency improvements in the range 0.51.8x against MultiCLP and Xilinx SuperTile.

A Tilebased FusedLayer Approach to Accelerate DCNNs on LowDensity FPGAs Accelerating Deep Convolutional Neural Networks on FPGAs is achieving a lot of interest across a wide range of applications such as image recognition and classification. Although memory requirement still represents a challenge for fast and efficient inference workloads, fusedaccelerated layers have been recently proposed to mitigate memory bandwidth problems. In this scenario, we propose a tiledbased fusedlayer approach to exploit temporal and spatial locality in mapping a DCNN to a specialized lowdensity FPGA. In our tilebased approach, we propose applying a fusing depth of two convolutional layers in order to fit into a lowdensity FPGA without affecting the performance, while reducing the memory bandwidth. We demonstrated the effectiveness of our twofused layer approach to accelerate the VGG16 network on a Zynq7020. We achieved an average speedup of 1.44x times, while saving up to 61% of memory transactions with respect to a baseline represented by tiledonly (notfused) version. A tiledbased fullyfused version of VGG16 would not have been feasible to be mapped on a lowdensity Zynq7020.