nxp/deeplabv3-imx

DeepLabV3

Introduction

DeepLabV3-MobileNetV2 is a lightweight deep learning model for semantic image segmentation based on the DeepLabV3 architecture [2] and a MobileNetV2 backbone [3] with depth multiplier 0.5. It classifies each pixel of the input image into 20 object classes plus a background class, using the PASCAL VOC label set.

Key Features

• Fully quantized int8 model ready for edge deployment
• Lightweight MobileNetV2 backbone (depth multiplier 0.5) for fast inference
• Outputs per-pixel class scores for 21 classes (20 PASCAL VOC + background)
• Compatible with i.MX 8M Plus (CPU), i.MX 93 (Ethos-U65 NPU), i.MX 95 and i.MX 952 (Neutron NPU)

Model Description

Modifications

No modifications were made to the original model weights or quantization. The int8 TFLite model is downloaded directly from TensorFlow Hub and compiled for each target platform.

Model Information

Information	Value
Input shape	RGB image (1, 513, 513, 3)
Input value range	[-1.0, 1.0] (normalized as (pixel - 127.5) / 127.5)
Output shape	(1, 513, 513, 21) - per-pixel scores for 21 classes
Output example
FLOPS	1.76B OPS
File size (int8)	~983 KB
Source framework	TensorFlow / TensorFlow Lite
Target platform	MPUs

Tested Configurations

The int8 model has been tested on the following configurations using benchmark_model:

Platform	BSP / SDK	Accelerator
i.MX 8M Plus	LF6.1.22_2.0.0	CPU, NPU
i.MX 93	lf-6.18.20_2.0.0	CPU, NPU
i.MX 95	eIQ Neutron SDK 3.1.3	CPU, NPU
i.MX 952	eIQ Neutron SDK 3.1.3	CPU, NPU

Training and Evaluation

The model was trained and evaluated on the PASCAL VOC dataset [1], which contains 20 object classes. It achieved 70.19% mIoU on the test set according to the TensorFlow Model Zoo.

Training code: TensorFlow Models - DeepLab train.py

Conversion / Quantization

The model is converted and quantized to int8 by the original authors and published on TensorFlow Hub. No additional quantization steps are required.

Quantization details: TensorFlow Models - DeepLab quantize.md

Download and Run

To download and compile the model for all supported platforms, run:

bash recipe.sh

This will:

1. Download the pre-quantized int8 TFLite model from TensorFlow Hub
2. Compile it with Vela for i.MX 93 (Ethos-U65 NPU)
3. Convert it with the eIQ Neutron SDK for i.MX 95 and i.MX 952

An example of running inference is in example.py:

python3 example.py -m original_model/deeplabv3_quant.tflite -i example_input.jpg -o example_output.jpg

Full on-device examples leveraging GStreamer/NNStreamer and this model can be found in nxp-nnstreamer-examples on GitHub

Origin

Model implementation: https://github.com/tensorflow/models/blob/master/research/deeplab/g3doc/model_zoo.md

TensorFlow Hub source: https://tfhub.dev/sayakpaul/lite-model/deeplabv3-mobilenetv2_dm05-int8/1/default/2

[1] Everingham, Mark, et al. "The pascal visual object classes (voc) challenge." International journal of computer vision 88 (2010): 303-338.

[2] Chen, Liang-Chieh, et al. "Encoder-decoder with atrous separable convolution for semantic image segmentation." Proceedings of the European conference on computer vision (ECCV). 2018.

[3] Sandler, Mark, et al. "Mobilenetv2: Inverted residuals and linear bottlenecks." Proceedings of the IEEE conference on computer vision and pattern recognition. 2018.