nxp/yolov4-tiny-imx

YOLOv4-tiny

Introduction

YOLO (You Only Look Once) is a series of object detection models designed for fast inference, which makes them well suited for edge devices.

YOLOv4 [2] was released in 2020 and provides many small improvements over YOLOv3 [3]. These improvements add up to create a more precise network at the same speed.

The model regresses bounding boxes (4 coordinates) and a confidence score for each box. The bounding box decoding and non-maximum suppression (NMS) steps are NOT included in the model. Please look at example.py for an example of implementation of box decoding and NMS.

Model Description

This repository contains pre-compiled model files optimized for NXP i.MX 93, 95, and 952 processors.

• Base Model: YOLOv4-tiny
• Original Model Authors: Alexey Bochkovskiy et al.
• Original License: MIT
• Modified by: NXP

Modifications

This model is a derivative work with the following changes from the original:

• Conversion: Converted from DarkNet framework to TensorFlow Lite with int8 quantization
• Compilation: Compiled for i.MX 93 with Ethos-U65 NPU using Vela tool
• Compilation: Compiled for i.MX 95 and i.MX 952 with eIQ Neutron NPU using eIQ Neutron SDK

Model Information

Information	Value
Input shape	RGB image (416, 416, 3)
Input example	(Image source, Public domain)
Output shape	Tensors of size (26, 26, 255) and (13, 13, 255) containing bounding box coordinates (not decoded) and class scores for two resolution levels and 3 anchor boxes per cell. More information in example.py.
FLOPS	6.9G
Number of parameters	6.05M
File size (int8)	5.9M
Source framework	DarkNet
Target platform	MPUs

Version and changelog

Initial release of quantized int8 and float32 models for i.MX 93, i.MX 95, i.MX 952, and i.MX 8M Plus.

Tested configurations

The int8 model has been tested on i.MX 8M Plus, i.MX 93, i.MX 95, and i.MX 952 using benchmark-model (see i.MX Machine Learning User Guide).

Training and evaluation

The model has been trained and evaluated on the COCO dataset [1], which features 80 classes. The floating point model achieved a score of 40mAP@0.5IoU on the test set, according to the source of the model. Using the evaluate.py script, we evaluate the int8 quantized model on the validation set and obtain 33mAP@0.5IoU.

Instructions to re-train the network can be found in the original repository.

Conversion/Quantization

The original model is converted from the DarkNet framework to TensorFlow Lite.

The export_model.py conversion script performs this conversion and outputs the int8 quantized model and float32 model. 100 random images from the COCO 2017 validation dataset are used as calibration for the quantization.

Use case and limitations

This model can be used for fast object detection on 416x416 pixel images. It is not the most accurate model, but it is enough for many applications. We noticed that the model performs well for large objects but has issues with small objects. This is probably due to the fact that it only features two output levels instead of three for larger models.

Download and run

To create the TensorFlow Lite model fully quantized in int8 with int8 input and float32 output and the float32 model, run:

bash recipe.sh

The TFLite model file for i.MX 8M Plus is original_model/yolov4-tiny_416_quant.tflite. The files for i.MX 93 are in the imx93 directory. The files for i.MX 95 are in the imx95 directory. The files for i.MX 952 are in the imx952 directory.

An example of how to use the model is in example.py. Please refer to i.MX Machine Learning User Guide for details on how to run the models on each platform.

Full on-device examples leveraging GStreamer/NNStreamer and this model can be found in nxp-nnstreamer-examples on GitHub

Origin

Model implementation: https://github.com/AlexeyAB/darknet/

[1] Lin, Tsung-Yi, et al. "Microsoft coco: Common objects in context." European conference on computer vision. Springer, Cham, 2014.

[2] Bochkovskiy, Alexey, Chien-Yao Wang, and Hong-Yuan Mark Liao. "Yolov4: Optimal speed and accuracy of object detection." arXiv preprint arXiv:2004.10934 (2020).

[3] Redmon, Joseph, and Ali Farhadi. "Yolov3: An incremental improvement." arXiv preprint arXiv:1804.02767 (2018).