Train a Neural Network to classify images and optimize CPU inferencing in 10mins | by Ojas Dileep Sawant | Towards Data Science

There are tons of resources out there on simplified Training and optimized pre-trained Inferencing models. However, training something custom to optimize performance on readily available hardware with minimal effort still seemed far fetched!
In this article we will leverage the concept of transfer learning where a model trained to classify images is used to train our custom use-case (e.g. items in your pantry) in your device browser with Teachable Machine (GUI) and optimize CPU inferencing with Intel® OpenVINOTM Toolkit without any painful SW installation (in 10mins of-course!).
Is optimization necessary? Skip to the performance comparison at the end.
What setup do I need beforehand?
6th to 10th generation Intel Core or Intel Xeon processors (i.e. if you purchased an Intel device released 2016 onwards)
Docker installed on a Linux System with internet access to hub.docker.com
Access to a camera device (e.g. webcam)
On your camera-enabled laptop or desktop device navigate to https://teachablemachine.withgoogle.com/train/imageEdit Class labels (e.g. cereal box, cookies) and add as many as you wishUse Hold to Record button to capture a few frames from the live previewRepeat this for each class, and finally hit the Train Model buttonNote: Don’t switch your browser Tabs and let the training finishTest in the Preview Panel on the far right and hit Export ModelOn the Export your model.. pop-up click the second tab TensorflowWith Keras already selected, click the Download my model
On your camera-enabled laptop or desktop device navigate to https://teachablemachine.withgoogle.com/train/image
Edit Class labels (e.g. cereal box, cookies) and add as many as you wish
Use Hold to Record button to capture a few frames from the live preview
Repeat this for each class, and finally hit the Train Model buttonNote: Don’t switch your browser Tabs and let the training finish
Test in the Preview Panel on the far right and hit Export Model
On the Export your model.. pop-up click the second tab Tensorflow
With Keras already selected, click the Download my model
It might take some time but a file named converted_keras.zip will be downloaded eventually.
I have created a repo with utilities to automate this, so you don’t have to! Make sure to have docker, internet access, unzip, and git installed on the system.
Clone or download/extract the repo:
git clone https://github.com/ojjsaw/teachable-machine-openvino.git
Replace your custom converted_keras.zip file in the repo directory and run the below script from the repo root directory. It might take a few minutes while downloading docker images the very first time.
./util_conv_teachable_to_openvino.sh
Make sure to capture your test images pertaining to your custom trained model and run the code below.
Just 4ms with OpenVINO inference on the test.jpg after the TF Keras v1.15.0 model to OpenVINO IR conversion (includes removal of training nodes).
docker run --rm -it -v ${PWD}:/workdir openvino/ubuntu18_dev:latest /bin/bashcd /workdirpython3 teachable_img_openvino_classify.py frozen_model.xml frozen_model.bin labels.txt test.jpg
Teachable Machine website provides TF Keras python code for inferencing locally.
In my case, it took 804ms (~1.2fps) for the predict function with the default example code and model, while it took mere 4ms (~250fps) with OpenVINO python code on the same test.jpg image from previous section.
docker run --rm -it -v ${PWD}:/workdir tensorflow/tensorflow:1.15.0 bashcd /workdirpip install Pillowpython teachable_img_keras_orig_classify.py keras_model.h5 test.jpg
The first command enables rendering the OpenCV preview window from docker.
xhost + docker run --rm -it --privileged -v ${PWD}:/workdir -e DISPLAY=$DISPLAY -v /tmp/.X11-unix:/tmp/.X11-unix -v /dev/video0:/dev/video0 openvino/ubuntu18_dev:latest /bin/bash cd /workdir python3 teachable_livecam_openvino_classify.py frozen_model.xml frozen_model.bin labels.txt
There does exist a flow to quickly train custom image classification model with everyday hardware locally.
No code modification was required, just a few clicks away.
It is also totally worth going that extra mile for cutting down (800ms to 4ms) with OpenVINO inferencing on Intel CPU considering there was no extra effort required what-so-ever!