nxp/movenet-imx

MoveNet Single-Pose Lightning for i.MX

Introduction

MoveNet is a lightweight human pose estimation model designed to run in real time on mobile devices. Specifically, this repo uses the single-pose "Lightning" variant, which is optimized for speed. The model is based on a CenterNet architecture with a MobileNetV2 backbone and a Feature Pyramid Network (FPN).

It regresses the position and confidence score of 17 human body keypoints. See example.py for an example of how to interpret the model output.

Key Features

• Single-person pose estimation with 17 keypoints
• Ultra-low latency, suitable for real-time applications
• Pre-quantized int8 model available directly from TensorFlow Hub
• Deployable on i.MX 8M Plus (NPU), i.MX 93 (Ethos-U65 NPU), i.MX 95 and i.MX 952 (Neutron NPU)

Model Description

Modifications

No modifications were made to the model weights or architecture. The int8 quantized TFLite model is downloaded directly from TensorFlow Hub. Platform-specific compiled variants are generated using NXP's Vela and Neutron SDK tools.

Model Information

Information	Value
Input shape	RGB image (1, 192, 192, 3)
Input value range	[0, 255] uint8
Output shape	Tensor of size (1, 1, 17, 3): keypoint y, x coordinates (normalized) and confidence score
Output example
FLOPS	N/A
Number of parameters	N/A
File size (int8)	2.8 MB
Source framework	TensorFlow Lite
Target platform	i.MX 8M Plus, i.MX 93, i.MX 95, i.MX 952

Tested Configurations

The int8 model has been tested on i.MX 8M Plus, i.MX 93 (BSP lf-6.18.20_2.0.0), i.MX 95, and i.MX 952 (eIQ Neutron SDK 3.1.3).

Training and Evaluation

The model was trained and evaluated by Google on the COCO dataset and Google's proprietary Active dataset.

More information about the training and evaluation is available in the MoveNet Model Card.

Conversion/Quantization

The original model is converted from TensorFlow to TensorFlow Lite and quantized to int8 by Google. The int8 model is downloaded directly from TensorFlow Hub. No additional quantization steps are needed.

Download and Run

To download and compile the model, run:

bash recipe.sh

This generates:

• original_model/movenet_quant.tflite — int8 model for i.MX 8M Plus and CPU inference
• imx93/lf-6.18.20_2.0.0/movenet_quant_vela.tflite — Vela-compiled model for i.MX 93 NPU
• imx95/3.1.3/movenet_quant_converted.tflite — Neutron-converted model for i.MX 95 NPU
• imx952/3.1.3/movenet_quant_converted.tflite — Neutron-converted model for i.MX 952 NPU

To run the example:

pip install opencv-python numpy
python3 example.py -m original_model/movenet_quant.tflite -i example_input.jpg -o example_output.jpg

Full on-device examples leveraging GStreamer/NNStreamer and this model can be found in nxp-nnstreamer-examples on GitHub

Origin

Model source: https://tfhub.dev/google/lite-model/movenet/singlepose/lightning/tflite/int8/4

Model card: https://storage.googleapis.com/movenet/MoveNet.SinglePose%20Model%20Card.pdf

[1] Lin, Tsung-Yi, et al. "Microsoft COCO: Common Objects in Context." European Conference on Computer Vision. Springer, Cham, 2014.

[2] Sandler, Mark, et al. "MobileNetV2: Inverted Residuals and Linear Bottlenecks." CVPR, 2018.

[3] Duan, Kaiwen, et al. "CenterNet: Keypoint Triplets for Object Detection." ICCV, 2019.

[4] Lin, Tsung-Yi, et al. "Feature Pyramid Networks for Object Detection." CVPR, 2017.