Optimize neural networks
on STM32
with STM32Cube.AI

Introduction to Edge AI

Signals turning into data
Embedded applications will collect more data in the future

Growing demand for data-driven insights

Increasing use of sensors

Proliferation of IoT devices

AI is offering the best approachto process this growing amount of data

Traditional approaches show their limitations:• when dealing with large datasets

Algorithms and predefined models toanalyze data and make predictions or decisions

Machine learning algorithms to automaticallylearn patterns and relationships from the data• when the phenomena are too complex

AI-based data processing offers a more
flexible and powerful approach to analyzing
and making decisions from large data collection

The raise of Edge AI
Edge AI will benefit many application domains:
Ultra-low latency
Real-time applications

01
10

Reduced data transmission
Generate meaningful information
Enhanced privacy and security
No data sharing in the cloudPower efficiency
Low-data / Low-power
Improved accuracy
analyze data from a wide rangeof sensors and sources

Industrial maintenance
Condition monitoring
Predictive maintenanceControl systems
From home heating systems
to industrial machines

Internet of Things (IoT)
smart cities, smart buildings,
connected homes, andindustrial automation

AI development workflow – ST software offering

1 Data preparation

Dataacquisition

Dataprocessing

2 Data scienceModelselection andtraining

Modelvalidation

3 Model implementation

Model librarycreationModelinference

Automated Edge AI software

Edge AI toolkit

All STM32
MCUs

ST ecosystem ease your AI to reach production level

Edge AI toolkit
for model optimization on STM32

Automated ML softwarefor end-to-end Edge AI solution design onSTM32✓ Get optimized C-code from your trained model✓ Desktop and online versions✓ Benchmark service on remote hardware (onlineversion)✓ On-device performance validation✓ The easiest way to integrate AI into your system✓ Save resources and development cost
✓ Reach the highest performance with theautomated model finder embedded in the tool

Application
domainAllTime series (except voice and speech)

BusinessmodelFree of charge

Free for prototyping on STM32 dev boards
Production requires right of use

Key benefits8A proven technology
adopted by multiple clients

STM32Cube.AI
AI optimization tool for STM32 portfolio

STM32Cube.AI overview
STM32Cube.AI Developer Cloud
The brand-new online AI services
front end for STM32

STM32Cube.AI
The original desktop front end AI
optimizer for STM32

X-CUBE-AI
for STM32Cube.MX

X-CUBE-AI
Command Line Interface

ST model zoo

Core engine technology

Web GUI+ REST API

Board farmOne tool – two versions to deploy AI on STM32

Load your trained neural network model

Optimize and validate
your NN model

Generate optimized
code for STM32

or pick one from STM32 model zoo(AI models library)

STM32Cube.AI for desktop

STM32Cubeecosystem

Command LineInterface

STM32Cube.AI Developer Cloud

Onlineplatform

RESTAPI

Optimized model
code for STM32

Benchmarking
tool12

STM32Cube.AI
Get the best AI performance on STM32
Image Classif v1.0 MLPerf Tiny

Visual Wake Word v1.0 MLPerf Tiny

UP TO

60 %
faster inference time*

378
155

UP TO

20 %
space freed-up in
flash and RAM*

283

121

100

56505510272

5851
Latency (ms)Flash (Kbytes)

RAM (Kbytes)

Latency (ms)Flash (Kbytes)

HW Target: STM32H7A3Flash: 2 Mbytes
RAM: 1.4 Kbytes
Freq: 280 MHz
SW Version:
X-Cube.AI v 7.3.0
TFLm v2.10.0

RAM (Kbytes)* versus TensorFlow Lite for microcontroller
13

The 3 pillars of STM32Cube.AI
Graph optimizer

Quantized model support

Memory optimizer

Automatically improve performance throughgraph simplifications & optimizations that
benefit STM32 target HW architectures

Import your quantized ANN to be compatible
with STM32 embedded architectures while
keeping their performance

Optimize memory allocation to get the best
performance while respecting the constraintsof your embedded design•••••• From FP32 to Int8 or mixed-precision• Minimum loss of accuracy• Code validation on target
o Latencyo Accuracy
o Memory footprint• Memory allocation• Internal/external memory repartition• Model-only update option

Auto graph rewrite
Node/operator fusion
Layout optimization
Constant-folding…
Operator-level info to fine-tune
memory footprint and computation

STM32Cube.AI is free of charge, available both in graphical interface and in command line.
14

Graph optimizer
Squeeze your graph to fit into an MCU!

Fully automated process in theSTM32Cube.AI workflow• Your original graph is optimized at thevery early stage for optimal integration
into STM32 MCU/MPU• Loss-less conversion

15Quantized model support
Simply use quantized networks to reduce memory footprint andinference time
STM32Cube.AI support quantized Neural Networkmodels with all parameter formats:• FP32• Int8• Mixed binary Int1 to Int8 (Qkeras*, Larq.dev*)

LATENCY & MEMORY COMPARISON FOR
QUANTIZED MODELS
FP32800

Flash (kB)

700600500*Please contact edge.ai@st.com to request
the relevant version of STM32Cube.AI400300

Int8200100

Int 1 + Int80020406080

HW Target: NUCLEO-STM32H743ZI2
Model: Low complexity handwritten digit reading
Freq: 480 MHz
Accuracy: >97% for all quantized models

Latency (ms)Tested database: MNIST dataset

MNIST dataset
16

Memory optimizer
Optimize performance easily with the memory allocation tool
Model memory allocation▪ Set your external memory▪ Map in non-contiguous internal
flash section▪ Partition internal vs external flash
memories

Re-use model input buffer tostore activation data*▪ Minimize RAM requirements

Model RAM
consumption per layer▪ Easily identify most criticallayers

Relocatable network▪ A separate binary is generated forthe library and the network toenable standalone model upgrade* Requires input and activation buffers in same memoryPerformance benchmarking made simple
STM32Cube.AI Developer Cloud
The unique possibility to evaluate the performance of models
remotely, on real STM32 boards

Get the real inference time from optimized
models running on STM32
Benchmark models on a large variety ofSTM32 boards• Find the most appropriate board for yourapplication
Get access to the most recent devices• A board farm is constantly updated with thelatest available boards

Start with edge AI optimized models
STM32 model zoo
A collection of application-oriented models optimized for STM32
Human activity
recognition

Image classification

Motion Sensing

Computer vision

Audio event detection

Object detection

Audio classification

Computer vision

Hosted on Github<I>

Model training scripts• Scripts to generate and validate
Application code example• Designed to host optimized NN models• Automatically generated from the trainedmodels• Easy to deploy for end-to-end evaluation

We provide everything to kick off your project
Design documentation

Hardware and software tools

Support & Updates• Wiki by ST is a great forum to learn andstart developing AI on STM32!

• Evaluation platforms for STM32
MCU/MPU• ST Community: STM32 ML & AI group• Videos of application examples• Extra sensor boards• Support center• Massive Open Online Course (MOOC)• Full software suite• Newsletter

Getting started

Development zone

Be guided step-bystep to learn STM32
ecosystem

Get started onapplication
development andproject sharing• Distributor certified FAE

20

What’s new in STM32Cube.AI v8.0.0?
Bringing a higher degree of versatility with
STM32Cube.AI#ONNX quantized models' support#Up-to-date and improved code
generation

Introducing the support of ONNX Tensororiented file format (QDQ):• Support for TensorFlow 2.11 models• ONNX models quantized with ONNX
runtime post-training quantization.

• Support ONNX Runtime 1.13.1• Support Keras.io 2.11
• New kernel performance improvements.

21

Making Edge AI accessible to all STM32 portfolio
Take advantage of STM32Cube.AI on all STM32 seriesHigh Perf
MCUs

Mainstream
MCUs

Ultra-low Power
MCUs
Wireless
MCUs

STM32F2

STM32F4

STM32F7

STM32H7

Up to 398 CoreMark
120 MHz Cortex-M3

Up to 608 CoreMark
180 MHz Cortex-M4

1082 CoreMark
216 MHz Cortex-M7

Up to 3224 CoreMark
Up to 550 MHz Cortex -M7
240 MHz Cortex -M4

STM32F3

STM32G4245 CoreMark
72 MHz Cortex-M4569 CoreMark
170 MHz Cortex-M4

Mixed-signal MCUs

STM32C0

STM32F0

STM32G0

STM32F1

114 CoreMark
48MHz Cortex M0+

106 CoreMark
48 MHz Cortex-M0

142 CoreMark
64 MHz Cortex-M0+

177 CoreMark
72 MHz Cortex-M3

STM32L0

STM32L1

STM32L4

STM32L4+

STM32L5

STM32U575 CoreMark
32 MHz Cortex-M0+

93 CoreMark
32 MHz Cortex-M3

273 CoreMark
80 MHz Cortex-M4

409 CoreMark
120 MHz Cortex-M4

443 CoreMark
110 MHz Cortex-M33

651 CoreMark
160 MHz Cortex-M33

STM32WL

STM32WB

162 CoreMark
48 MHz Cortex-M4
48 MHz Cortex-M0+

216 CoreMark
64 MHz Cortex-M4
32 MHz Cortex-M0+

Latest product generation
22

Don't go alone
We have created a network of companies to support you

Trust our authorized partners to ensure the success of yourproject. Learn more at st.com/stm32ai

Wish to discuss a co-development partnership for ML/AI
projects? Contact us at edge.ai@st.com

23

Releasing your creativity/STM32@ST_World

community.st.com
stm32ai.st.com
wiki.st.com/stm32
github.com/STMicroelectronics
Videos

STM32Cube.AI blog articles

Find out more at stm32ai.st.com© STMicroelectronics - All rights reserved.
ST logo is a trademark or a registered trademark of STMicroelectronics International NV or its affiliates in the EU and/or other countries.
For additional information about ST trademarks, please refer to www.st.com/trademarks.
All other product or service names are the property of their respective owners.