  Semantic segmentation is the task to cluster pixels on an image belonging to
the same class. It is widely used in the real-world applications including
autonomous driving, medical imaging analysis, industrial inspection, smartphone
camera for person segmentation and so on. Accelerating the semantic
segmentation models on the mobile and edge devices are practical needs for the
industry. Recent years have witnessed the wide availability of CNN
(Convolutional Neural Networks) accelerators. They have the advantages on power
efficiency, inference speed, which are ideal for accelerating the semantic
segmentation models on the edge devices. However, the CNN accelerator chips
also have the limitations on flexibility and memory. In addition, the CPU load
is very critical because the CNN accelerator chip works as a co-processor with
a host CPU. In this paper, we optimize the semantic segmentation model in order
to fully utilize the limited memory and the supported operators on the CNN
accelerator chips, and at the same time reduce the CPU load of the CNN model to
zero. The resulting model is called GnetSeg. Furthermore, we propose the
integer encoding for the mask of the GnetSeg model, which minimizes the latency
of data transfer between the CNN accelerator and the host CPU. The experimental
result shows that the model running on the 224mW chip achieves the speed of
318FPS with excellent accuracy for applications such as person segmentation.
