Optimizing Deep Learning Based Semantic Video Segmentation on Embedded GPUs Decision making in many industries today is being improved drastically thanks to artificial intelligence and deep learning. New algorithms address challenges such as genome mapping, medical diagnostics, selfdriving cars, autonomous robots and more. Deep learning in embedded systems requires high optimization due to the high computational demand, given that power, heat dissipation, size and price constraints are numerous. In this paper we analyze several acceleration methods which include utilization of GPUs for most complex variants of deep learning, such as semantic video segmentation operating in real time. Specifically, we propose mapping of acceleration routines commonly present within deep learning SDKs to different network layers in semantic segmentation. Finally, we evaluate one implementation utilizing the enumerated techniques for semantic segmentation of front camera in autonomous driving front view.

A Distributed Scheme for Accelerating Semantic Video Segmentation on An Embedded Cluster We present a methodology for enhancing the throughput of semantic video segmentation tasks on an embedded cluster containing multiple embedded processing elements (ePEs). The methodology embraces a scalable masterslave hierarchy and features a global and local key management scheme for allocating video frames to different ePEs. The master ePE divides each video frame into frame regions, and dynamically distributes different regions to different slave ePEs. Each slave ePE executes either a segmentation path or a flow path: the former is highly accurate but slower, while the latter is faster but less accurate. A lightweight decision network is employed to determine the execution path for each slave ePE. We propose a global and local key management scheme to facilitate the execution of the embedded cluster, such that the average processing latency of each frame is significantly reduced. We evaluate the performance of our methodology on a real embedded cluster in terms of accuracy and frame rate, and validate its effectiveness and efficiency for various ePE configurations. We further provide a detailed latency analysis for different configurations of ePEs.