  Custom hardware accelerators for Deep Neural Networks are increasingly
popular: in fact, the flexibility and performance offered by FPGAs are
well-suited to the computational effort and low latency constraints required by
many image recognition and natural language processing tasks. The gap between
high-level Machine Learning frameworks (e.g., Tensorflow, Pytorch) and
low-level hardware design in Verilog/VHDL creates a barrier to widespread
adoption of FPGAs, which can be overcome with the help of High-Level Synthesis.
hls4ml is a framework that translates Deep Neural Networks into annotated C++
code for High-Level Synthesis, offering a complete and user-friendly design
process that has been enthusiastically adopted in physics research. We analyze
the strengths and weaknesses of hls4ml, drafting a plan to enhance its core
library of components in order to allow more advanced optimizations, target a
wider selection of FPGAs, and support larger Neural Network models.
