Detectron2 : The bare basic end to end tutorial | by Yousry Mohamed | Towards Data Science

In 2018, Facebook AI developed an object detection library called Detectron. It was an amazing library but a bit hard to use. An earlier version of myself wrote a blog post on just how to install Detectron and run the sample demos, nothing more. Detectron was a bit hard to install and use and it was powered by Caffe2. Since 2018, there was lots of code changes bringing Caffe2 & PyTorch into a single repository and that adds to the difficulty of using Detectron.
It seems there was some constructive feedback from the community and Facebook has come up with a v2.
According to GitHub page of Detectron2:
Detectron2 is Facebook AI Research’s next generation software system that implements state-of-the-art object detection algorithms.
Detectron2 is built using PyTorch which has much more active community now to the extent of competing with TensorFlow itself. Also the setup instructions are much easier plus a very easy to use API to extract scoring results. Other frameworks like YOLO have very obscure format of their scoring results which are delivered as a plain multi dimensional array. Anyone worked with YOLO can imagine the effort needed to parse the scoring result and get it right in the first place.
Enough history, in this post I will walk you through an end to end exercise on how to prepare a Detectron2 docker image hosting a web API for object detection and use it from a small web application acting as a service consumer.
I will start first with installation. Unfortunately, Windows is not supported but I will bypass that using containers. To be quick, we wouldn’t mind installing a CPU version as the GPU one needs (guess what) a GPU, CUDA and all that lovely stuff from NVIDIA.
So using your preferred text editor, create a Dockerfile with the this content.
It starts first by picking base image which has a Python version ≥ 3.6 as requested by Detectron2 setup instruction. Next a few prerequisites are installed then a copy of same setup instructions on Detectron2 installation page. The version installed is a CPU version, it won’t be super fast but good enough for a tutorial. Last, Flask is included as this docker image will host a web API website to accept scoring calls from clients.
Let’s build the image first.
docker build --rm . -t ylashin/detectron2:latest
You can use any name/tag you want and adjust accordingly for the next steps. Next, let’s give it a go.
docker run -it ylashin/detectron2:latest bin/bash
Hopefully all the above works fine and now we have a container up and running with Detectron2 installed. So we need to verify that nothing is broken and we have unicorns and and rainbows.
The following test.py script has to be copied into the container using docker CLI or by using nano which is installed on the container.
The main tasks involved are:
Printing Detectron2 version.Downloading an image for Lionel Messi and a soccer ball.Creating a predictor instance using one of the pre-trained architectures.Scoring the downloaded image using the predictor instance.
Printing Detectron2 version.
Downloading an image for Lionel Messi and a soccer ball.
Creating a predictor instance using one of the pre-trained architectures.
Scoring the downloaded image using the predictor instance.
Next we will run that test script as shown in the next screenshot.
Detectron2 seems to be at version 0.1.1 at the time of writing. Also, scoring is done correctly and the result is not a plain tensor like YOLO. It’s a typed class with easy to use members like bounding boxes and predicted classes. We will come to mapping class codes to human readable names later. What we have here is a couple bounding boxes and a couple high scores plus predicted classes of 0 (person) and 32 (sports ball).
P.S. Based on the problem you are solving, you need to pick the correct architecture and see if it was trained against similar dataset or else you might need to do custom training.
The next step is to make this scoring functionality accessible from outside the container. A Flask web API needs to be hosted in that container to accept an image and return scoring result. For simplicity, the API will accept an image URL and do the download itself. It will not accept binary payload which would be the natural thing to do but it’s just easy to use the same test script with slight modifications.
Also the response will be a dictionary serialized as a JSON object with enough information needed by the caller without any complicated parsing.
Now as the container will host a Flask web server, it has to be started with web API port published . So exit the current container and start it again with port publishing.
docker run -p 8181:5000 -it ylashin/detectron2:latest bin/bash
Flask port 5000 is mapped to host port 8181.
Now, create a file inside the container namedweb_api.py and paste the following GitHub gist inside it.
This is nearly the same test script used before with the following differences:
Some refactoring to have a global predictor instance ready to be used per each request.
A function mapped to a web API endpoint is used to receive HTTP request with JSON object which has image URL to be scored.
This function does the scoring and also some post-processing to prepare a final response object.
To start the web server run:
python web_api.py
The URL shown has a port of 5000 but that’s because it’s just inside the container. Any tool like Postman, curl or PowerShell can be used to test the API works as expected. For whatever tool you use, please remember to include a request header with key content-type and value application/json .
If we scroll down the response in Postman, we will find the expected scoring fields such as bounding boxes, scores and class predictions.
Now the web API server seems to be working fine, it’s better to be included as part of the definition of Dockerfile. That would allow us to push the built image to docker hub or any other registry and share it with others or use it without worrying about building it each time.
The updated Dockerfile can be found in the repo linked to this post but for the sake of brevity, the delta added to Docker file is:
WORKDIR /appCOPY web_api.py web_api.pyENTRYPOINT ["python", "/app/web_api.py"]
Once the modified docker image is built, we can start the container once again but this time we don’t need to copy the web API file nor start it ourselves. Actually we can start the container as a daemon.
docker run -p 8181:5000 -d ylashin/detectron2:latest
The web API could be tested using Postman again for verification but we need to give the container a few seconds to download weights file part of predictor preparation.
Once we are happy with the running container, we can shut it down and publish the image.
docker push ylashin/detectron2:latest
I published the image to my own Docker Hub account so that I can show you later how to use it with Azure container instances. The web app I will use to consume the scoring service will be hosted locally, so it can easily access a local running container. But I prefer to go to next level and host the container somewhere in the cloud to simulate a more realistic implementation.
Phew, nearly there. Last stage is to consume that web API from any RESTful client. But first, I will start the container in Azure using Azure CLI and Azure container instances service. If you want to keep using the container locally, you can skip this Azure bit but adjust the web API URL in the web app that will come later.
az group create --name detectron2 --location eastusaz container create --resource-group detectron2 --name predictor --image ylashin/detectron2:latest --cpu 2 --memory 4  --dns-name-label detectron2api --ports 5000az container show --resource-group detectron2 --name predictor --query "{FQDN:ipAddress.fqdn,ProvisioningState:provisioningState}" --out table
Last command above will show the FQDN of the container which we can use to test the cloud hosted API using Postman. Please note that the port used here is the plain Flask port without any mapping to a different port.
We have a web API endpoint up and running in the cloud that can be consumed from any REST client anywhere. I have built a small JavaScript app. The app is pretty simple:
It has a text box and a button.
Put a public image URL in the text box and click the button.
The app will call our API, render the requested image and draw the bounding boxes.
To try the app, clone it first and install the required npm packages:
git clone https://github.com/ylashin/detectron2-tutorialcd '.\3. client\'npm installnpm install -g browserifynpm install -g watchifywatchify index.js -o bundle.js
Open index.html file in a text editor and update the URL of the API inside the function scoreImage to match the URL of your local or cloud hosted API. Then double click index.html file which will open the app in your default browser. Play with different image URLs, but just notice that some websites hosting images have a bit strict CORS policies so not all images will work fine. It’s not related to our web app or API but more about how the web works.
The image used in the installation section of this post is tried above. We can see the bounding boxes, scores and labels on Messi and the ball. Developer console pane is shown with the JSON response of the API as well.
The main part to consider in this web app is parsing and using the API response which is very simple and straightforward.
All the remaining code is plumbing and HTML canvas rendering via an npm package.
We barely scratched the surface of Detectron2, it has many more amazing features to explore. Also lots of shortcuts were made to make the post as concise as possible. Al in all, Detectron2 is a very welcome addition to any developer toolbox because of its features but more importantly ease of use.
If you happen to create the container in Azure, please remember delete the resource group or stop the container once done experimenting otherwise it will keep burning your credit.