  In order to handle modern convolutional neural networks (CNNs) efficiently, a
hardware architecture of CNN inference accelerator is proposed to handle
depthwise convolutions and regular convolutions, which are both essential
building blocks for embedded-computer-vision algorithms. Different from related
works, the proposed architecture can support filter kernels with different
sizes with high flexibility since it does not require extra costs for
intra-kernel parallelism, and it can generate convolution results faster than
the architecture of the related works. The experimental results show the
importance of supporting depthwise convolutions and dilated convolutions with
the proposed hardware architecture. In addition to depthwise convolutions with
large-kernels, a new structure called DDC layer, which includes the combination
of depthwise convolutions and dilated convolutions, is also analyzed in this
paper. For face detection, the computational costs decrease by 30%, and the
model size decreases by 20% when the DDC layers are applied to the network. For
image classification, the accuracy is increased by 1% by simply replacing $3
\times 3$ filters with $5 \times 5$ filters in depthwise convolutions.
