# Real-Time Semantic Image Segmentation with Deep Learning for Autonomous Driving: A Survey
## 3 State-of-the-Art Deep-Learning Models
The parallel factorized convolution unit module with multiple branch parallel convolutions manage to learn imparallel feature representations in a powerful manner, without increasing the computational complexity. ShelfNet18 [33] structure is presented in Figure 14. More specifically, it is composed of multiple encoder-decoder branches, uses shared-weights and a residual block. To decrease inference time, ShelfNet18 proposes channel reduction to efficiently reduce to computation burden. The use of different encoder-decoder branches ameliorates the computational process, increasing the segmentation accuracy. Weights are being shared between convolutional layers of the same residual block, in order to decrease the number of network parameters without decreasing accuracy. Figure 12: Network architecture of FasterSeg (adapted from [27]). Figure 13: Overall symmetric architecture of ESNet (adapted from [32]). ICNet [34] proposes a framework for saving operations in multiple resolutions and features a cascade feature fusion unit. Its architecture is shown in Figure 7.0 It processes, simultaneously, semantic information from low resolution branches, along with details from high-resolution images in an efficiently manner. In the case of using factorized convolutions to reduce latency the efficiency is achieved by modifying the way convolutions work in the network architecture to boost speed [2,35]. In the literature, several models employ such an approach. First, ESPNet [36] introduces a convolutional module, called efficient spatial pyramid, which functions efficiently in terms of computational cost, memory usage and power. ESPNet depends on a principle of convolution factorization. More analytically, convolutions are resolved into two steps: (1) pointwise convolutions and (2) spatial pyramid of dilated convolutions. ESPNet's network structure, shown in Figure 16, is fast, small, capable of handling low power, and low latency while preserving semantic segmentation accuracy. Figure 7.1 Network architecture of ShelfNet (adapted from [33]). Figure 7.2 Network architecture of ICNet (adapted from [34]). ESPNetv2 [37] based on ESPNet, outperforms the latter by 4-5% and has 2-4\(\times\) fewer FLOPs on the PASCAL VOC and the \"Cityscapes\" dataset. ESPNetv2 modifies efficient spatial pyramid module and introduces an Extremely Efficient Spatial Pyramid unit by replacing pointwise convolutions with group pointwise convolutions, computationally expensive 33 dilated convolutions with depthwise dilated separable convolution. Lastly, it fuses the feature maps employing a computationally efficient hierarchical feature fusion method. According to [37], pruning and quantization are complementary methods for ESPNetv2. A structural unit of its architecture is presented in Figure 7.3  
ESSGG [38] improves runtime of ERFNet by over a factor of 5\(X\), by replacing its modules with more efficient ones, such as the aforementioned depthwise separable convolutions, grouped convolutions and channel shuffling. In this manner the inference time is reduced efficiently. Another contribution of this work is the training method, called gradual grouping where dense convolutions are transformed to grouped convolutions optimizing the function of gradient descent. Furthermore, it is important to refer that ESSGG uses a small decoder's size. Finally, pruning and quantization are proposed as secondary options used in the later stages of the network to improve efficiency. The detailed network's architecture is shown in Figure 7.4  
Figure 7.5 Network architecture of ESPNet (adapted from [36]). Figure 7.6 Strided Extremely Efficient Spatial Pyramid unit with shortcut connection to an input image (highlighted in red) for downsampling (adapted from [37]). DABNet [39] introduces a Depthwise Asymmetric Bottleneck module, which extracts combined local and contextual information and reduces the number of parameters of the network. This specific module uses depthwise asymmetric convolution and dilated convolution.