nxp/deepface-emotion-imx

Deepface Emotion

Introduction

Deepface-emotion is a simple Convolutional Neural Network (CNN) for classification of facial expressions into 7 emotion categories. We use the pretrained weights from the DeepFace library.

Note: This is not a face detection model. Faces must be detected and cropped before being passed to this model.

Key Features

• Lightweight CNN optimized for edge deployment
• 7 emotion classes: angry, disgust, fear, happy, sad, surprise, neutral
• Grayscale 48x48 input - low memory footprint
• int8 quantized for accelerated NPU inference on NXP i.MX platforms

Model Description

This repository contains the Deepface emotion model quantized to int8 and compiled for NXP i.MX 8M Plus, i.MX 93, i.MX 95, and i.MX 952 processors.

• Base Model: serengil/deepface (facial_expression_model_weights.h5)
• Original Model Authors: Sefik Ilkin Serengil
• Original License: MIT
• Modified by: NXP

Modifications

• Quantization: Converted from Keras float32 to TensorFlow Lite int8 (uint8 input, float32 output)
• Compilation: Compiled for i.MX 93 with Ethos-U65 NPU using Vela (lf-6.18.20_2.0.0)
• Conversion: Converted for i.MX 95 and i.MX 952 with Neutron NPU using eIQ Neutron SDK 3.1.3

Model Information

Information	Value
Input shape	Grayscale face crop (1, 48, 48, 1), dtype uint8
Input example	(Image from FER2013 dataset [1])
Output shape	Vector of probabilities shape (1, 7), dtype float32
Output labels	angry, disgust, fear, happy, sad, surprise, neutral
Output example	[[0., 0., 0., 0.996, 0., 0., 0.]] -> Recognized emotion: happy
FLOPS	58.5 MOPS
Parameters	23,497,424
Source framework	TensorFlow / Keras

Version and Changelog

Initial release of quantized int8 model with i.MX 93, i.MX 95, and i.MX 952 compiled variants.

Tested Configurations

The quantized int8 model has been tested on i.MX 8M Plus and i.MX 93 using benchmark-model (see i.MX Machine Learning User Guide).

Training and Evaluation

The model was trained and evaluated on the FER2013 dataset [1]. The original training procedure is described here.

Accuracy on FER2013 PrivateTest split

Evaluation was performed on all 3,589 FER2013 PrivateTest images using the evaluate.py script included in this repository. The FER2013 dataset is a challenging benchmark; a score around 57% is considered strong for this split.

Model	Accuracy
Deepface emotion float32 (Keras)	56.5%
Deepface emotion int8 (TFLite)	56.76%

Conversion/Quantization

The model is converted directly from Keras to TensorFlow Lite int8 using the TFLite converter. 100 random images from the FER2013 PrivateTest split are used as the representative calibration dataset. The input type is uint8 and the output type is float32.

Download and Run

To recreate the TFLite model (quantized int8 with uint8 input and float32 output):

1. Create a Kaggle account and generate an API key.
2. Accept the competition terms at Challenges in Representation Learning: Facial Expression Recognition Challenge.
3. Run:

export KAGGLE_USERNAME='your_username'
export KAGGLE_KEY='your_api_key'
bash recipe.sh

The TFLite model for i.MX 8M Plus is at original_model/emotion_uint8_float32.tflite. The Vela-compiled model for i.MX 93 is in imx93/lf-6.18.20_2.0.0/. The Neutron-converted models for i.MX 95 and i.MX 952 are in imx95/3.1.3/ and imx952/3.1.3/.

An example of how to use the model is in example.py:

python example.py -m original_model/emotion_uint8_float32.tflite -i example_input.jpg

Please refer to the i.MX Machine Learning User Guide for details on how to run the models on each platform.

Full on-device examples leveraging GStreamer/NNStreamer and this model can be found in nxp-nnstreamer-examples on GitHub

Origin

Model implementation: https://github.com/serengil/deepface/

[1] Goodfellow, Ian J., et al. "Challenges in representation learning: A report on three machine learning contests." International Conference on Neural Information Processing. Springer, Berlin, Heidelberg, 2013.