Poster 738

Fast and Accurate Real-Time Semantic Segmentation
with Dilated Asymmetric Convolutions
Leonel Rosas-Arias1, Gibran Benitez-Garcia2, Jose Portillo-Portillo1,
Gabriel Sanchez-Perez1 and Keiji Yanai2
1Instituto Politecnico Nacional, Mexico City, Mexico
2The University of Electro-Communications, Tokyo, Japan

1

UEC Yanai Lab. Tokyo.

ⓒ 20192020 UEC Tokyo.

Real-time semantic segmentation
Problem:• High-accuracy semantic segmentation
is extremely expensive to compute.
• Networks for real-time semanticsegmentation sacrifice a lot ofaccuracy.

Objective:• Reduce the accuracy gap between
real-time and non-real-time forsemantic segmentation.
Find and classify pixels belonging to each objects in the scene.
Image from: Janai, Joel & Guney, Fatma & Behl, Aseem & Geiger, Andreas. Computer Vision for Autonomous Vehicles: Problems, Datasets and State-of-the-Art, 2017.

2UEC Yanai Lab. Tokyo.

ⓒ 2020 UEC Tokyo.

Contributions• FASSD-Net:– SOTA performance of speed and accuracy with high resolution images (1024x2048).
– Two additional variations to balance the speed and accuracy trade-offs.
• Dilated Asymmetric Pyramidal Fusion module (DAPF):– Obtains feature maps rich in contextual information.
– Requires considerably fewer floating-point operations compared with similar method,such as ASPP [1].
• Multi-resolution Dilated Asymmetric module (MDA):– Offers an improved way to fuse two set of feature maps of different resolution.
– Simultaneously refines spatial and contextual information from input feature maps.
– Can be used in all decoder stages.
UEC Yanai Lab. Tokyo.

ⓒ 2020 UEC Tokyo.
3 [1] Chen, L.C, et.al. Encoder-Decoder with Atrous Separable Convolution
for Semantic Image Segmentation. ECCV’18

FASSD-Net architecture

FASSD-Net:• Encoder.- HarDNet [2] (custom version)• Decoder.- DAPF + MDA

UEC Yanai Lab. Tokyo.

2020 2019.
UEC Tokyo.
4 [2] P. Chao, et. al. HarDNet: A Low Memory Traffic Network, ⓒICCVFASSD-Net architecture

Factorized
convolutions

With α=2, our proposed
module requires 50% fewer
floating-point operations
compared to ASPP.

The number of intermediate
feature maps is controlled by
the α parameter.
5UEC Yanai Lab. Tokyo.

ⓒ 2020 UEC Tokyo.

FASSD-Net architecture

Asymmetric branch: Context

Refines both contextual anddetail information

Symmetric
Multi-resolution Dilated Asymmetric module
branch: Details(MDA)
6

UEC Yanai Lab. Tokyo.

ⓒ 2020 UEC Tokyo.

Ablation study on the Cityscapes dataset

FASSD-Net

Experimental setup:• All networks pretrained with ImageNet.
• Training during 90k iterations.
• Batch size = 16.
• GFLOPs measured for high resolution images at 1024x2048.
• Speed in FPS measured with an Nvidia GTX 1080Ti card.
7

UEC Yanai Lab. Tokyo.

ⓒ 2020 UEC Tokyo.

Quantitative results

1) For fair comparison, the speed ofmethods marked by (*) areapproximated without TensorRT
acceleration [3].
2) PSPNet and FC-HarDNet-L2
speeds are placed on the x-axisedges for the sake of bettervisualization.

UEC Yanai Lab. Tokyo.

ⓒ 2020UEC Tokyo.
8 [3] NVIDIA, “TensorRT,” [Online; accessed December 1, 2020]
https://developer.nvidia.com/tensorrt

Qualitative results

9

UEC Yanai Lab. Tokyo.

ⓒ 2020 UEC Tokyo.

Conclusion• Conclusions:– We proposed two modules (DAPF & MDA) for reducing the accuracy gap
between real-time and non-real-time semantic segmentation networks.
– With FASSD-Net, we set a new SOTA accuracy for real-time semanticsegmentation on the Cityscapes validation set.

• Future work:– Include more backbone networks and different datasets for evaluation.
– Evaluate on different scenarios, such as indoor parsing and medical images.

10

UEC Yanai Lab. Tokyo.

ⓒ 2020 UEC Tokyo.

Poster 738

Thank You

Code & Models

11

UEC Yanai Lab. Tokyo.

ⓒ 2020 UEC Tokyo.