How to train a Tensorflow face object detection model | by Dion van Velde | Towards Data Science

Artificial Intelligence makes it possible to analyse images. In this blogpost I will focus on training a object detector with customized classes. The first thing you will have to do is the setup. In the Tensorflow documentation is written how to setup on your local machine. We are going to train a real-time object recognition application using Tensorflow object detection. The trained models are available in this repository
This is a translation of ‘Train een tensorflow gezicht object detectie model’ and Objectherkenning met de Computer Vision library Tensorflow
You can auto install OpenCV on Ubuntu in /usr/local. with the following script. This script installs OpenCV 3.2 and works with Ubuntu 16.04.
$ curl -L https://raw.githubusercontent.com/qdraw/tensorflow-object-detection-tutorial/master/install.opencv.ubuntu.sh | bash
On my Mac I use OpenCV 3.3.0 en Python 2.7.13. I’ve tried it with OpenCV 3.2 and 3.3 but this fails with Python 3.6. However, on Ubuntu Linux this combination does works.
$ brew install homebrew/science/opencv
The first step is cloning the Tensorflow-models repository. For this tutorial we use only the slim and object_detection module.
$ nano .profileexport PYTHONPATH=$PYTHONPATH:/home/dion/models/research:/home/dion/models/research/slim
You also need to compile the protobuf libraries
$ protoc object_detection/protos/*.proto --python_out=.
The example code is available in the tensorflow-face-object-detector-tutorial repository. You can clone this repo.
$ git clone https://github.com/qdraw/tensorflow-face-object-detector-tutorial.git
Go to the subfolder:
 $ cd tensorflow-face-object-detector-tutorial/
Install the dependencies using PIP: I use Python 3.6 and OpenCV is installed with Python bindings.
$ pip install -r requirements.txt
The Chinese University of Hong Kong has a large dataset of labelled images. The WIDER FACE dataset is a face detection benchmark dataset. I have used labelImg to show the bounding boxes. The selected text are the face annotations.
The script 001_down_data.py will be used to download WIDERFace and ssd_mobilenet_v1_coco_11_06_2017. I will use a pre trained model to speed up training time.
$ python 001_down_data.py
First we need to convert the dataset to Pascal XML. Tensorflow and labelImg use a different format. The images are downloaded in the WIDER_train folder. With 002_data-to-pascal-xml.py we convert the WIDERFace data and copy it to a different subfolder. It takes on my computer 5 minutes to process 9263 images.
$ python 002_data-to-pascal-xml.py
When the data is converted to Pascal XML, an index is created. By training and validating the dataset, we use these files as input to make TFRecords. However, it is also possible to label images with a tool like labelImg manually and use this step to create an index here.
$ python 003_xml-to-csv.py
A TFRecords file is a large binary file that can be read to train the Machine Learning model. The file is sequentially read by Tensorflow in the next step. The training and validation data will be converted into binary files.
Trainings data to TFRecord (847.6 MB)
$ python 004_generate_tfrecord.py --images_path=data/tf_wider_train/images --csv_input=data/tf_wider_train/train.csv  --output_path=data/train.record
Validation data to TFRecord (213.1MB)
$ python 004_generate_tfrecord.py --images_path=data/tf_wider_val/images --csv_input=data/tf_wider_val/val.csv  --output_path=data/val.record
In the repository, ssd_mobilenet_v1_face.config is a configuration file that is used to train an Artificial Neural Network. This file is based on a pet detector.
In this case, the number of num_classes remains one because only faces will be recognized.
The variable fine_tune_checkpoint is used to indicate the path to a previous model to get learning. This location will fit you in this file. The fine tune checkpoint file is used to apply transfer learning. Transfer learning is a method in Machine Learning that is focused on applying knowledge gained from one problem to another problem.
In the class train_input_reader, a link is made with the TFRecord files for training the model. In the config file, you need to customize it to the correct location.
The variable label_map_path contains index IDs and names. With this file, zero is used as a placeholder, so we start with numbers from one.
item {  id: 1  name: 'face'}
For validation, two variables are important. The variable num_examples within the class eval_config are used to set the number of examples.  The eval_input_reader class describes the location of the validation data. There is also a path in this location. Furthermore, it is still possible to change learning rate, batch size and other settings. For now, I have kept the default settings.
Now it’s going to start real work. The computer is going to learn from the dataset and make a Neural Network here. As I model the train on a CPU, this will take several days to get a good result. With powerful Nvidia graphics card it is possible to shorten this to a few hours.
$ python ~/tensorflow_models/object_detection/train.py --logtostderr --pipeline_config_path=ssd_mobilenet_v1_face.config  --train_dir=model_output
Tensorboard gives insight into the learning process. The tool is part of Tensorflow and is automatically installed.
$ tensorboard --logdir= model_output
To use the model in Object Recognition with the Computer Vision library Tensorflow. The command below provides a location to the models repository and to the last checkpoint. The folder folder will contain frozen_inference_graph.pb.
$ python ~/tensorflow_models/object_detection/export_inference_graph.py \--input_type image_tensor \--pipeline_config_path ssd_mobilenet_v1_face.config \--trained_checkpoint_prefix model_output/model.ckpt-12262 \--output_directory model/
TL; DR;In the model/frozen_inference_graph.pb folder on the github repository is a frozen model of the Artificial Neural Network. The Chinese University of Hong Kong has WIDERFace and this dataset has been used to train model.
In addition to the data used for training, there is also an evaluation dataset. Based on this evaluation dataset, it is possible to calculate the accuracy. For my model I calculated the accuracy (Mean Average Precision). I came to a score of 83.80% at 14337 steps (epochs). For this process, Tensorflow has a script and makes it possible to see in Tensorboard what the score is. It is recommended that you run an evaluation process in addition to training.
python ~/tensorflow_models/object_detection/eval.py --logtostderr --pipeline_config_path=ssd_mobilenet_v1_face.config  --checkpoint_dir=model_output --eval_dir=eval
You can then monitor the process with Tensorboard.
tensorboard --logdir=eval --port=6010
It has been possible to train a face recognition model. The frozen model model / frozen_inference_graph.pb can be deployed in, for example, Object Recognition with the Computer Vision Library Tensorflow.
Should the world of Computer Vision interest you, but you still do not know how to apply this and have the necessary questions? Send me an email then we can have a cup of coffee.