Object Detection in 6 steps using Detectron2 | by Aakarsh Yelisetty | Towards Data Science

Have you ever tried training an object detection model using a custom dataset of your own choice from scratch?
If yes, you’d know how tedious the process would be. We need to start with building a model using a Feature Pyramid Network combined with a Region Proposal Network if we opt for region proposal based methods such as Faster R-CNN or we can also use one-shot detector algorithms like SSD and YOLO.
Either of them is a bit complicated to work with if we want to implement it from scratch. We need a framework where we can use state-of-the-art models such as Fast, Faster, and Mask R-CNNs with ease. Nevertheless, it is important to try building a model at least once from scratch to understand the math behind it.
Detectron 2 comes to the rescue if we want to train an object detection model in a snap with a custom dataset. All the models present in the model zoo of the Detectron 2 library are pre-trained on COCO Dataset. We just need to fine-tune our custom dataset on the pre-trained model.
Detectron 2 is a complete rewrite of the first Detectron which was released in the year 2018. The predecessor was written on Caffe2, a deep learning framework that is also backed by Facebook. Both the Caffe2 and Detectron are now deprecated. Caffe2 is now a part of PyTorch and the successor, Detectron 2 is completely written on PyTorch.
Detectron2 is meant to advance machine learning by offering speedy training and addressing the issues companies face when making the step from research to production.
These are the various types of Object Detection models that the Detectron 2 offers.
Let’s dive into Instance Detection directly.
Instance Detection refers to the classification and localization of an object with a bounding box around it. In this article, We are going to deal with identifying the language of text from images using the Faster RCNN model from the Detectron 2’s model zoo.
Note that we are going to limit our languages by 2.
We identify Hindi and English Text and we include a class labeled Others for other languages.
We’ll implement a model where the output would be in this way.
Let’s get started!
Using Detectron 2, Object Detection can be performed on any custom dataset using seven steps. All the steps are readily available in this Google Colab Notebook and you can run it straight away!
Using Google Colab for this would be an easy task as we can use a GPU for faster training.
Kickstart with installing a few dependencies such as Torch Vision and COCO API and check whether CUDA is available. CUDA helps in keeping track of the currently selected GPU. And then install Detectron2.
# install dependencies: !pip install -U torch==1.5 torchvision==0.6 -f https://download.pytorch.org/whl/cu101/torch_stable.html!pip install cython pyyaml==5.1!pip install -U 'git+https://github.com/cocodataset/cocoapi.git#subdirectory=PythonAPI'import torch, torchvisionprint(torch.__version__, torch.cuda.is_available())!gcc --version# install detectron2:!pip install detectron2==0.1.3 -f https://dl.fbaipublicfiles.com/detectron2/wheels/cu101/torch1.5/index.html
Import a few necessary packages.
Datasets that have builtin support in detectron2 are listed in builtin datasets. If you want to use a custom dataset while also reusing detectron2’s data loaders, you will need to Register your dataset (i.e., tell detectron2 how to obtain your dataset).
We use our Text Detection Dataset which has three classes:
EnglishHindiOthers
English
Hindi
Others
We’ll train a text detection model from an existing model pre-trained on the COCO dataset, available in detectron2’s model zoo.
If you’re interested to know the conversion from the raw format of the dataset to the format that Detectron 2 accepts, checkout this Colab Notebook.
There are certain formats in how data can be fed to a model such as a YOLO format, PASCAL VOC format, COCO format, etc. Detectron2 accepts the COCO Format of the dataset. COCO format of the dataset consists of a JSON file which includes all the details of an image such as size, annotations (i.e., bounding box coordinates), labels corresponding to it’s bounding box, etc. For example,
This is how the JSON looks like for one image. There are different types of formats for the bounding box representation. It must be a member of structures.BoxMode for Detectron2. There are 5 such formats. But currently, it supports BoxMode.XYXY_ABS, BoxMode.XYWH_ABS. We use the second format. The (X, Y) represents one coordinate of the bounding box, and W, H represents Width and Height of that box. The category_id refers to the class the box belongs to.
Then, we need to register our dataset.
To verify the data loading is correct, let’s visualize the annotations of randomly selected samples in the training set.
We’ll randomly pick 3 pictures from the train folder of our dataset and see how the bounding boxes look like.
The output looks in this way,
The big step. This is the step where we give configurations and set the model ready to get trained. Technically, we just fine-tune our model on the dataset as the model is already pre-trained on COCO Dataset.
There are a ton of models available for object detection in the Detectron2’s Model Zoo. Here, we use the faster_rcnn_R_50_FPN_3x model which looks in this way on a high level.
There’d be a Backbone Network (Resnet in this case) which is used to extract features from the image followed by a Region Proposal Network for proposing region proposals and a Box Head for tightening the bounding box.
You can read more about how Faster R-CNN works in my previous article.
Let’s set the configuration for training.
I wouldn’t say this is the best configuration. Surely, the accuracy may get improved for other configs as well. After all, it depends on choosing the right hyperparameters.
Note that, here we also calculate the accuracy for every 500 iterations on the Validation Set.
It’s time to infer the results by testing the model on the Validation Set.
An output folder gets saved in the local storage after successful completion of training in which the final weights are stored. You can save this folder for inferencing from this model in the future.
Results:
Usually, the model is evaluated following the COCO Standards of evaluation. Mean Average Precision (mAP) is used to evaluate the performance of the model. Here’s an article that precisely gives an idea on the mAP.
We get an accuracy of around 79.4% for an IoU of 0.5 which is not that bad. It can be increased certainly by tweaking the parameters a bit and increasing the number of iterations. But keep an eye on training extensively as the model might overfit the training set.
Go through this Colab Notebook if you need inference from a saved model.
In this article, I emphasized on the procedure of objection detection using a custom dataset using detectron 2 rather than focusing on gaining higher accuracy.
Although this seems to be a pretty straight forward process, there’s a lot to explore in Detectron 2’s library. We have a good amount of optimization parameters that can be further tweaked to attain higher accuracy, which completely depends on one’s custom dataset.
Hope you’ve learned something new today.
You can download the notebook from my Github repository and try running it on Google Colab or Jupyter Notebooks.
You could read more about Object Detection and the legacy R-CNN's in my previous article.
If you’d like to get in touch, connect with me on LinkedIn.