Embedded AI Learning Diary 2 |How to Run the MNIST Demo on the Embedded System?
Published: 2020-07-30 10:29:31

This article will show you how to run the MNIST Demo (handwriting number recognition) on the [RT-Thread opensource operating system](https://www.rt-thread.io/) to support your handwriting digital verification.

##**Preparation**
- System: Windows | Ubuntu 18.04
- Board: STM32 H743ZI NUCLEO
- RT-Thread running environment
- MDK 5
- [GitHub](https://github.com/Lebhoryi/Edge_AI/tree/master/Project2-Mnist)

The MNIST (Modified National Institute of Standards and Technology) is a large dataset of handwritten digits recognition.
This experiment will focus on MNIST, to further learning the mysterious connection between artificial intelligence and embedded.

This experiment is going to achieve:

Reproducing neural networks through the `CMSIS NN` library, importing `int` type weight files, then to successfully implementing MNIST inference in RT-Thread systems.

**How to run MNIST on RT-Thread**

1. Pull `Mnist_CMSIS` or `Mnist_CMSIS.7z` from `GitHub` to local, `Mnist_CMSIS` has 520M, it is recommended to download the compression package with the only 66.4M.
2. Running options:
 - Scons
 - MDK5
3. CMSIS and RT-Thread Inference Success Interface.

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/ta045qoetauo183hnztu.png)

The CMSIS packages are already included on GitHub, download it and run, if you are generating a new project then you need to open the `CMSIS` package in the `Menuconfig` of RT-Thread.

##**1. PC Training Model**

File: mnist.ipynb

**1.1 PC Environment**

- Tensorflow: 2.3.0-dev20200515
- Numpy: 1.16.4
- Keras: 2.2.4-tf

**1.2 Data Set**

File: ./data/mnist.npz

The MNIST dataset consists of 60000 (Training Set) and 10000 (Test Set) handwritten characters, the size of each picture is 28*28, here’s the address to download the dataset: http://yann.lecun.com/exdb/mnist/.

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/3d7emich4yrks2942rw6.png)

**1.3 Network Structure**

Two layers of convolution plus one full layer of connection.


![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/833trfisqu0o310u5307.png)

**1.4 Training Model & Verification Model**

File: ./model/mnist.h5
- Training Model

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/0bwh2z7f8bg11kuu3k42.png)
- Verify the accuracy of the training model.
![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/n2rweg8x2j02gwkzk1de.png)

- Save the weight file.

```
1# save weights
 2model.save_weights(model_path / 'model_weights.h5')
 3
 4# load weights
 5model.load_weights(model_path / 'model_weights.h5')
 6
 7model.compile(optimizer='adam',
 8             loss='sparse_categorical_crossentropy',
 9             metrics=['accuracy',])
10loss, acc = model.evaluate(x_test, y_test)
11print("Restored model, accuracy: {:5.2f}%".format(100*acc))
1 313/313 [==============================] - 1s 2ms/step - loss: 0.1226 - accuracy: 0.9651
2 Restored model, accuracy: 96.51%
```

At this point. The model has been trained, but the preparation is not finished yet. Let’s keep going.

**1.5 Others**

Our ultimate goal is to have the trained Model to successfully reasoning (test) on the RT-Thread system.

The method used in this experiment is `CMISIS + RT-Thread` the taken steps are as follows:

1. Change weight data to `int` type and saved.
2. Using the `CMSIS NN` library to reproduce the neural network, the file is a `.c` class file.
3. Import weight files and test samples.
4. Reasoning success.

##**2. Choose CMSIS Software Package**

Note: Install the running environment required by RT-Thread

```
1# windows
 2> pkgs --upgrade
 3# Enable CMSIS
 4> menuconfig
 5> pkgs --update
 6> scons --target=mdk5
 7
 8# linux
 9# Enable failed, pelase execute
10# (base) Mnist_CMSIS[master*] % source ~/.env/env.sh
11(base) Mnist_CMSIS[master] % pkgs --upgrade 
12(base) Mnist_CMSIS[master] % scons --menuconfig
13(base) Mnist_CMSIS[master*] % pkgs --update
```

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/715gfl3ji1pmohyi2ccq.png)

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/4g5ymynx0792qn791geh.png)

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/fpf4f7nt2wye9kcyyi72.png)

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/a1dmku8yi07vgcrfugc6.png)

##**3. Build the Network Structure by Using the CMSIS Package**

By calling the CMSIS API to build the network structure, this step requires a certain depth of background knowledge.
![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/ddbze9ogvs0y3caeyj34.png)

In addition, in the process of refactoring, the `int` type is used instead of the `float` type.

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/ddbze9ogvs0y3caeyj34.png)

The way the data is passed in this project is a basic approach, with only an array of 784 sizes defined in the main.c file, and a handwritten digital picture of a size of 28 x 28, in an Int-type format. The code that reads the picture data specifically is implemented in mnist.ipynb.

As the input is a simple 28x28 x 784 (one-dimensional) array so the custom handwriting digital recognition verification can be supported here. It is recommended to start with custom handwriting digital recognition verification in mnist.ipynb.

The saved picture of the custom handwritten number is as similar as the picture in the training set, if the saved picture is not in a size of 28 x 28, you can refer to the code in mnist.ipynb, change its Resize into 28 x 28 and make sure that the one-dimensional array of input has a size of 784, consistent with the network input.

You can read the source file for more information, other files stay the same.

- ./Mnist_CMSIS/applications/main.c.
- ./Mnist_CMSIS/applications/mnist_parameters.h.

##**4. Compile and Burn**

- Windows (Recommend to use MDK)
By clicking only one key, MDK compiles the whole project and downloads the generated binary file to the board. Users could observe the output messages through Putty or other serial-port tools.

- Linux (Recommend to use Scons)
Compile all the source code by invoking `scons`, download the executable file to the board through `STM32 Cube Programmer` and observe the output through `minicom`.

Success Interface

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/lu0z29k18t803q2rykws.png)

##**5. FAQ**

**5.1 CMSIS + RT-Thread can’t find `arm_math.h`**

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/3036jtc4lsqg4cmoic7x.png)

**Solution:**

windows:
1、Check DSP switch
![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/wj8s6hnrcg8o5tqmhr5x.png)

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/je9dxsfhb8hyjqa3afjp.png)

2、Add Macro Definition
USE_STDPERIPH_DRIVER,ARM_MATH_CM4,CC_ARM,FPU_PRESENT, ARM_MATH_DSP
- linux:
- In `./Mnist_CMSIS/packages/CMSIS-latest/SConscript` , line 15, manually add `DSP`, add:
`CPPPATH = CPPPATH + [cwd + ‘/CMSIS_5/CMSIS/DSP/Include’]`

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/fg0jz670wx2qwavl0pnt.png)

After `Scons` will then report such an error:

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/zaxar1iwhwrdrbx4f71q.png)

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/i8pg3af1d6gwslxjp5ai.png)

**Solution:**

In `./Mnist_CMSIS/board/SConscript` , line 22, change it into:
`CPPDEFINES = [‘STM32H743xx’,’ARM_MATH_CM7',’__FPU_PRESENT’]`

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/mrfbwj2rs03xkkj1bljs.png)

**5.2 Scons Reports an Error**

The files already exist.
![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/zz6df8rgfh2dqklo9apo.png)

**Solution:**

In `./Mnist_CMSIS/SConscript`, change it as follows:

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/fo6zxgimknfba1lrpk9p.png)

**RT-Thread Contact Info:**

 [Website](https://www.rt-thread.io/)  |  [Github](https://github.com/RT-Thread)  | [Twitter](https://twitter.com/rt_thread) |  [Facebook](https://www.facebook.com/RT-Thread-IoT-OS-110395723808463)   |   [Youtube](https://www.youtube.com/channel/UCdDHtIfSYPq4002r27ffqPw
)