Object Detection- Tensorflow. Using the tensorflow Object Detection... | by Sahil Gupta | Towards Data Science

The model is trained using the Tensorflow Object Detection API for training the model for 20 classes on the Pascal VOC 2012 dataset.
There has been a buzz all around, about Machine Learning and Deep Neural Networks since, their popularity has revived after they were shelved for quite a few years. Lately, the communities around the world have started to contribute to the field of Machine Learning. Since its advancement, Object detection and Object Recognition have gained immense popularity and researchers around the world have contributed with various state-of-the-art architectures.
Tensorflow, by Google, has contributed with a dedicated API ( Object Detection API ) which provides with a bunch of models that have been trained on a large corpus. These models are freely available and also allow user contribution. Further in this post, I am going to train a model using the pre-trained model for detecting the objects in the PASCAL VOC dataset.
The dataset used in this blog is available freely here (2012 version). However, the server is mostly down so you may find the dataset here.
The dataset that we are using represent the data in the XML files and the corresponding images. Each XML file in the data contains the information of all the bounding boxes. The Object Detection API consumes these XML files and converts them into the csv format which can further be converted into the tf record format that is required to train the model. In our case, we are using the PASCAL VOC dataset, for which tensorflow has already provided various utilities to make our life easier. We will get into the details as we move ahead.
The API abstracts the whole load of tricky stuff for us. However, there are a few requirements that are to be satisfied by the user before using the API.
The API uses the dataset in the tf record format. It is a binary format for representing the data. The API uses this format to speed up the training process. Tf record internally represents the data in a format that allows for parallel processing.
Let’s begin by cloning the tensorflow models which is a collection of models that are under research and the officially released models.
Now we need to install the protocol buffer files inside of the object detection folder in models/research. Also, we need to install the all the models in the research folder. Let’s do that...
After installing all the models in the research folder we are good to go for converting the data in the tf record format. Now, in our case as we are using the PASCAL VOC dataset, for that tensorflow provides us with a script which we can use to convert the data in the tf record format. We also require the labels data that contains the number of classes for which the model is to be trained. The label file looks something like this...
item {  id: 0  name: 'businesscard'} 
Here each id maps to a unique number starting from and the name is the class name. In our case we have 20 classes and will have 20 unique item JSON objects in the same file. For our purpose, tensorflow provides us with a label file for our dataset which can be found at the following path in your colab notebook /content/models/research/object_detection/data/pascal_label_map.pbtxt
Similarly, you need to run the create_pascal_tf_record_test.py to convert the test data in the required format.
While dealing with the API, there are various compatibility issues that one has to deal with. This is due to the initial releases of the API. I spent around 3 days solving a major issue which most of us face, which is the slim module. Even after providing correct paths and after reading a whole lot of blogs and comments, I was not able to solve this initially. But, I will guide you the steps to help you with it as per the current version. The Slim module is a requirement when you run the training process. To import the slim module you firstly need to make a new notebook and import and download all the libraries and clone the models in there.
At the beginning of the colab notebook you must first convert the tensorflow version to 1.x using the following command %tensorflow_version 1.x
For our training purposes we will be using the ssd_mobilenet_v1_coco model. I tried to use other models also but somehow, I couldn’t get rid of errors in there eve after following the same process. But everything worked just fine for ssd_mobilenet_v1 model. It is a model trained on the COCO dataset and hence can detect 90 classes. But for out purpose we need to train it for only 20 classes in the VOC dataset.
Firstly, download the model from the official website of tensorflow (here). Now, we will extract the model tar file.
Now, I am providing with you some of the commands that you can run to get rid of the slim issue. Copy and paste the following commands in your notebook to install the slim library, once inside the research/slim folder and second, copy all the slim files in object_detection folder and install them there also.
Sometimes, directly installing tf-slim package using pip install tf-slim also works(mostly in the case of kaggle notebooks)
Next, you need to download the config file for the model from the models/research/object_detection/samples/configs/ folder and change the path of the input tf-record file, label map file, test tf record file and a directory to save the model in. Now, run the command to start the training process.
Note: In colab notebooks, even after modifying the fine_tune_checkpoint path the checkpoints were getting saved inside of the tmp folder in root of colab notebooks. Do consider the case if it still persists. Also, if there are errors in training then switch to tensorflow 2.x by restarting the session. This way slim error will not come now.
After getting the model trained for 20 classes, we need to convert the checkpoint file in a reusable format. For converting the model in the required format, tensorflow provides with a script that can be used.
Note: If export_inference_graph.py script throws an error which says that the path must be provided using output_file then you need to replace this script with another script which can downloaded found here and then run the above command
For getting the predictions we will user the tensorflow utilities to get the probabilities and the bounding box values and then display the bounding box on the pictures. Whole code for the predictions is quite simple and folows the same structure. you can find the code below...
You can find the complete working notebooks on my github repo:
github.com
I hope this tutorial helps you with every aspect of the training and testing. If you have any doubts you can ask them in the comments section.