Split overlapping bounding boxes in Python
By: Miguel Pinto
Tags: Python, Software Development, Tutorial, Data Science, Programming

Split overlapping bounding boxes in Python

Complete tutorial in 5 steps from problem formulation up to creating a PyPI package, tested using data from Global Wheat Detection Kaggle competition.

In object detection, it is usual to have bounding box targets to identify objects in images. These bounding boxes may sometimes overlap. In some models like Mask RCNN the bounding boxes are predicted directly and the overlap of the bounding boxes in not a problem. Another possible approach is to convert the bounding boxes to masks and use a semantic segmentation model like U-Net. It that case, overlapping masks may be a problem if at the end you want to separate the individual objects.

In this story, I cover how to develop an algorithm in Python to separate overlapping bounding boxes and setting a margin between them. I will be using data from the Global Wheat Detection Kaggle competition.

Note that I’m not arguing that this method is the best for the Wheat Detection competition. In the future, I may write another story covering such topic.

Table of content

Problem formulation Writing the math Coding the algorithm in Python Testing the code in a real-world problem Create a Python package with fastai nbdev

1. Problem formulation

Let’s start by considering the case for two overlapping bounding boxes. We want to separate the bounding boxes leaving some margin in between them as suggested in the image below.

Example of splitting two bounding boxes with some margin. Image by the author.

To define the margin the procedure will be the following:

Consider the line defined by points A and B — the centroids of each bounding box — let’s call it AB.

Then consider a line perpendicular to that at the median point between A and B — let it be ABp.

Finally, the two lines in the figure above are parallel to ABp at a distance given by the margin value at our choice.

Later in section 4, I will show that the code can be easily applied for the case of multiple intersecting bounding boxes.

2. Writing the math

Consider the image and equations bellow where the bold notation denotes vectors. The vector AB is simply the vector that goes from A to B defined in the first equation. Then we can consider a vector perpendicular to AB — that I call ABp — using the second equation. Finally, the point M on the margin line for box A can be defined by the third equation.

Formulation of the margin lines. Image by the author.

The rationale is that: 1) you start in A; 2) you move in the direction of AB but only halfway to reach the median point; 3) you move slightly backwards in the same direction by a factor m multiplied by the unit vector in the direction of AB; 4) the point you reach is what I define as point M.

Using the point M and the vector ABp it is straight forward to define the margin line as I will show in the next section.

3. Coding the algorithm in Python

The function below receives two bounding boxes and returns the sliced region for the box A. The input boxes are shapely Polygons as well as the output returned by the function. Let’s now dive into the code line by line.

Line 1: The inputs are two bounding boxes — box_A and box_B of type shapely Polygon. The margin sets how big the distance between the boxes should be and line_mult just needs to be high enough to guarantee that the line crosses the polygon completely.

and of type shapely Polygon. The sets how big the distance between the boxes should be and just needs to be high enough to guarantee that the line crosses the polygon completely. Line 3: The vector AB ( vec_AB ) is defined using the centroids of the boxes.

( ) is defined using the centroids of the boxes. Line 4: Similar to line 3 but for the perpendicular vector ( vec_ABp ) following the equation in section 2.

) following the equation in section 2. Line 5: The norm of vector AB is computed as it will be needed later.

Line 6: The split_point (point M) is defined according to the equation of section 2.

(point M) is defined according to the equation of section 2. Line 7: The line is defined using shapely LineString class that allows defining the line given two points. Notice that the points are also shapely geometries. The line is therefore defined from the point M minus a multiple of vector ABp up to the point M plus a multiple of vector ABp.

Line 8: A shapely utility function is used to cut the polygon into two using the line.

Line 10: For each polygon, obtained on line 8, check if contains the centre — point A.

Line 11–15: In these lines, I separate the polygon containing the centre point (the one that matters for this purpose) and return it together with the other polygon (not containing the centre) and the line used for the slice. The extra returned objects are just in case it can be useful for future applications.

4. Testing the code in a real-world problem

The code defined in section 3 only applies to two bounding boxes. In order to apply it for several bounding boxes, I defined the following code that I will now explain briefly.

intersection_list — a function computes the intersection of all polygons in a list.

— a function computes the intersection of all polygons in a list. slice_all — this function receives as input a GeoDataFrame (see table below) containing all bounding boxes for an image and calls slice_one for each bounding box. slice_one simply applies the slice box function for a given box_A and considering all boxes that intersect with it. When there are several intersecting boxes, intersection_list function is used to obtain the final polygon.

Sample of the input GeoDataFrame. Image by the author.

The result of slice_all is a similar GeoDataFrame but with the sliced boxes. The image below shows the original bounding boxes (left) and the result of slice_all (right). As you can see there are several overlapping regions in the original data but none after applying the methodology just developed.

Original bounding boxes (left) and result after applying the method described (right). Image by the author.

5. Create a Python package with fastai nbdev

fastai nbdev is arguably the easiest and most user-friendly way to create a Python package and upload it to PyPI.

When you start a new project go to nbdev instructions and use the link to create a repository for the template. This will get you started easily.

Then you should clone the repository for your local machine, pip install nbdev and on the project directory run nbdev_install_git_hooks .

and on the project directory run . Now you can open jupyter notebook as usual. The settings.ini file contains setup information that you need to fill such as the project name, your github username and the requirements for your package.

file contains setup information that you need to fill such as the project name, your github username and the requirements for your package. The nbdev template includes a index.ipynb and a 00_core.ipynb . The index will be the GitHub README.md file.

and a . The index will be the GitHub file. Then on the 00_core.ipynb you can develop your code as usual but remember to add #export comment at top of the cells that should be in your package core.py file — generated by nbdev from the notebooks. You can read in detail how to use nbdev in their documentation . I highly recommend it! It will change the way you code.

you can develop your code as usual but remember to add #export comment at top of the cells that should be in your package file — generated by nbdev from the notebooks. After you are ready you can run nbdev_build_lib and nbdev_build_docs on the terminal. Then you can commit your changes and push to the github repo. Check the repo to see if all tests passed. A common error is ModuleNotFoundError when you import external packages. You need to include them under the requirements in settings.ini so that the package will be installed together with all required dependencies.

and on the terminal. Then you can commit your changes and push to the github repo. Check the repo to see if all tests passed. A common error is ModuleNotFoundError when you import external packages. You need to include them under the requirements in so that the package will be installed together with all required dependencies. When everything is green you can upload your package to PyPI by running the command make pypi. You need however, if you have not done it before, to create an account in PyPI and set up a configuration file (the detailed instructions are on nbdev page here).

And that’s it! My package is now on PyPI at https://pypi.org/project/splitbbox and can be pip installed.

Final remarks