Mask RCNN Implementation for Image Segmentation | Tutorial
Tags: deeplearning, computerscience, machinelearning, python
Published: 2021-04-09 13:01:10

In This article, we will try image segmentation using Mask RCNN. It's the successor of Faster-RCNN. We will use `tensorflow-gpu==1.15` for training purposes. Check the [Mask_RCNN](https://github.com/matterport/Mask_RCNN) Github repository. It's implemented in the TensorFlow framework using `Resnet101` as the default backbone.

## What is Image Segmentation

Image segmentation is like an advanced form of classification. In Classification, we used to classify pictures into classes. 
In the case of image segmentation, we classify each pixel of the image into different classes. The goal of segmentation is to simplify and/or change the representation of an image into something that is more meaningful and easier to analyze.

Self Driving cars has some concept of image segmentation for driving.

![Image Segmentation](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/taezf6anad149xdw1jld.png)

## Mask_RCNN Module

This is an implementation of Mask R-CNN on Python 3, Keras, and TensorFlow. The model generates bounding boxes and segmentation masks for each instance of an object in the image. It's based on Feature Pyramid Network (FPN) and a ResNet101 backbone.


## How to Annotate Data

[LabelMe](https://github.com/wkentaro/labelme) is open-source tool for polygen image annotations inspired by [MIT Label Me](http://labelme.csail.mit.edu/Release3.0/)

```bash
# Python3 on Ubuntu
sudo apt-get install python3-pyqt5  # PyQt5
sudo pip3 install labelme
```

Check Labelme Documentation for installtion on [winodws & Mac](https://github.com/wkentaro/labelme)

![Label me car](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/h4ncxh26b3r63epr5uh2.png)

## Training Part

1.Git clone the [Mask-RCNN-Implementation](https://github.com/codePerfectPlus/Mask-RCNN-Implementation)
2.Install the Mask_Rcnn module.
```bash
python -m pip install git+https://github.com/matterport/Mask_RCNN
```
3.Create Data_folder in Root Directory and arrange the folder in the below manner. Divide the Data into 70/30 for `Train` and `Val` respectively.

```
- Data_folder
    - train
        - img1.jpg
        - img1.json
        - img2.jpg
        - img2.json
        ...
    - val
        - img3.jpg
        - img2.json
        - img4.jpg
        - img4.json
        ...
```
4.Change the configuration according to your Data and System specifications.

```python
# Configuration
# Adjust according to your Dataset and GPU

IMAGES_PER_GPU = 2 # 1
 
# Number of classes (including background)
NUM_CLASSES = 1 + 1 # Background

# typically after labeled, class can be set from Dataset class
# if you want to test your model, better set it correctly based on your training dataset

# Number of training steps per epoch
STEPS_PER_EPOCH = 100
```

### Training the model on Custom Data

```bash
python customTrain.py train --dataset=path_to_Data_folder --weights=coco
```

### ReTraining from the Last Checkpoint

```bash
python customTrain.py train --dataset=path_to_Data_folder --weights=last
```

### Evaluation of the Mask_RCNN Model

```bash
python customTrain.py evaluate --dataset=path_to_Data_folder --weights=last
```

### Test the model for segmentation

```bash
pip install pixellib
```
```python
import pixellib
from pixellib.instance import custom_segmentation


model_path = "Trained_Model_path"
image_path = "Image_path"
output_path = "output_path"

segment_image = custom_segmentation()
segment_image.inferConfig(num_classes= 4, class_names= ["BG", "Arduino Nano", "ESP8266", "Raspberry Pi 3", "Heltec ESP32 Lora"])
segment_image.load_model(model_path)
segment_image.segmentImage(image_path, show_bboxes=True, output_image_name=output_path)
```

```python
from PIL import Image
from matplotlib import pyplot as plt

img = Image.open(output_path)
plt.figure(figsize=(12, 12))
plt.imshow(img)
```

Colab Notebook for Example training on [Microcontroller Segmentatiom](https://colab.research.google.com/drive/142qQPGuzz7AemMVDl8iKw0fEe-hnIL1p?usp=sharing)