nxp/ultraface-slim-imx

Ultraface slim

Introduction

Ultraface is a lightweight face detection model designed for edge computing devices. It regresses bounding boxes (4 coordinates) and a confidence score for each box. The bounding box decoding and non-maximum suppression steps are included in the model graph.

Note: This is only a face detection model. It does not perform face recognition/identification.

Key Features

• Very small (~400 KB int8) and fast face detector suitable for 320x240 inputs.
• Bounding box decoding and NMS are baked into the model, so the output is a ready-to-use list of faces.
• Fully quantized int8 model with uint8 input and float32 output.
• Compiled for i.MX 93 Ethos-U65 NPU and converted for i.MX 95 / i.MX 952 Neutron NPU.

Model Description

Modifications

The original PyTorch model is converted to a TensorFlow saved model and then to TensorFlow Lite. A patch (slim.patch) adapts the post-processing so that the box decoding and non-maximum suppression are folded into the exported graph, producing a fixed-size (100, 6) output tensor. The model is then fully int8 quantized using 100 calibration images from the WIDER FACE test set. The input type is uint8 (values 0-255) and the output type is float32.

Model Information

Information	Value
Input shape	RGB image (1, 240, 320, 3), uint8
Input example	(Image source: NASA, Public domain)
Output shape	Tensor of size (100, 6) containing up to 100 detected faces (class scores + box coordinates)
Output example
FLOPS	168,707,432
Number of parameters	264,732
File size (int8)	403 KB
Source framework	PyTorch
Target platform	MPUs (i.MX 8M Plus, i.MX 93, i.MX 95, i.MX 952)

Tested Configurations

The int8 model has been tested on i.MX 8M Plus and i.MX 93 using benchmark-model (see i.MX Machine Learning User Guide).

Training and Evaluation

The model was trained on the WIDER FACE dataset. According to the original source the float model achieves easy/medium/hard scores of 0.77 / 0.671 / 0.395 on the WIDER FACE validation set.

We re-evaluated the quantized int8 model on the WIDER FACE validation set (3226 images) using the official WIDER FACE evaluation protocol (Average Precision at IoU 0.5 for the easy/medium/hard subsets):

Model	Easy AP	Medium AP	Hard AP
Ultraface slim (reference, float32)	0.770	0.671	0.395
Ultraface slim TensorFlow Lite int8	0.755	0.642	0.346

The evaluation script is evaluate.py.

Conversion/Quantization

The original PyTorch model is converted to a TensorFlow saved model, then to TensorFlow Lite with full int8 post-training quantization. 100 random images from the WIDER FACE test set are used as calibration data. The conversion is performed with:

• inference_input_type = tf.uint8
• inference_output_type = tf.float32
• OpsSet.TFLITE_BUILTINS_INT8

The i.MX 93 model is compiled from the int8 TFLite model using Vela tag lf-6.18.20_2.0.0 with --accelerator-config ethos-u65-512 --optimise Performance.

The i.MX 95 and i.MX 952 models are converted using eIQ Neutron SDK version 3.1.3.

Download and Run

To regenerate the TFLite model fully quantized in int8 (uint8 input, float32 output) and compile it for all supported platforms, run:

bash recipe.sh

The generated model files are:

• original_model/ultraface_slim_uint8_float32.tflite - int8 model for i.MX 8M Plus
• imx93/lf-6.18.20_2.0.0/ultraface_slim_uint8_float32_vela.tflite - Vela-compiled for i.MX 93
• imx95/3.1.3/ultraface_slim_uint8_float32_converted.tflite - Neutron-converted for i.MX 95
• imx952/3.1.3/ultraface_slim_uint8_float32_converted.tflite - Neutron-converted for i.MX 952

An example showing how to run detection on an image is in example.py:

python example.py -m original_model/ultraface_slim_uint8_float32.tflite -i example_input.jpg -o example_output.jpg

To evaluate the model on the WIDER FACE validation set, download the validation images and run:

# WIDER_val/ should contain the event sub-folders (e.g. 0--Parade/)
python evaluate.py -m original_model/ultraface_slim_uint8_float32.tflite -d WIDER_val -g ground_truth

The WIDER FACE ground-truth .mat files are provided in the ground_truth/ directory.

Full on-device examples leveraging GStreamer/NNStreamer and this model can be found in nxp-nnstreamer-examples on GitHub

Origin

Model implementation: https://github.com/Linzaer/Ultra-Light-Fast-Generic-Face-Detector-1MB

[1] WIDER FACE dataset: Yang, Shuo, et al. "WIDER FACE: A Face Detection Benchmark." Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2016.