Data Augmentation techniques in python | by Renu Khandelwal | Towards Data Science

In this article, we will explore different data augmentation techniques in Python using imgaug library
Image augmentation is a very powerful technique used to artificially create variations in existing images to expand an existing image data set. This creates new and different images from the existing image data set that represents a comprehensive set of possible images.
This is done by applying different transformation techniques like zooming the existing image, rotating the existing image by a few degrees, shearing or cropping the existing set of images etc.
Deep learning Convolutional Neural Networks(CNN) need a huge number of images for the model to be trained effectively. This helps to increase the performance of the model by generalizing better and thereby reducing overfitting. Most popular data sets for classification and object detection data sets have a few thousand to millions of images.
Generalizing refers to a model’s performance when evaluated on previously seen data during training against the test or validation data it has never seen before.
A CNN, due to its in-variance property, can classify objects even when visible in different sizes, orientations or different illumination. Hence, we can take the small dataset of images and transform the objects to different sizes by zooming in or zooming out, flipping them vertically or horizontally or changing the brightness whatever makes sense for the object. This way we create a rich, diverse data set with variations.
Image Augmentation creates a rich, diverse set of images from a small set of images for image classification, object detection or image segmentation.
Augmentation strategies to increase the size of the training data set need to be applied after carefully understanding the problem domain.
If we are classifying different types of cars, then applying image augmentation by flipping a car vertically is not useful for the problem domain or when we are doing an image classification of handwritten digits flipping digit 6 vertically does not add value
Image augmentation can be applied as a pre-processing step before we train the model or can be applied in real time.
Augmentation is applied as a pre-processing step to increase the size of the data set. This is usually done when we have a small training data set that we want to expand.
Generating augmentation on smaller data set is helpful but we need to consider the disk space when applying on larger data sets
As the name suggests, augmentation is applied in real time. This is usually applied for larger data sets as we do not need to save the augmented images on the disk.
In this case, we apply transformations in mini-batches and then feed it to the model.
Online augmentation model will see different images at each epoch. In Offline augmentation, augmented image is part of the training set, it views the augmented image multiple times depending on the number of epochs.
The model generalizes better with online augmentation as it sees more samples during training with online data augmentation.
We will be using imgaug class for demonstrating Image Augmentation.
imgaug supports a wide range of data augmentation techniques
Flipping: flipping the image vertically or horizontally
Rotation: rotates the image by a specified degree.
Shearing: shifts one part of the image like a parallelogram
Cropping: object appear in different positions in different proportions in the image
Zoom in, Zoom out
Changing brightness or contrast
We will now explore these data augmentation techniques using imgaug library
imgaug is a library for image augmentation along with key points/landmarks, bounding boxes, heat maps and segmentation maps.
pip install imgaug
In some cases we get an error with Shapely, in that case, we can try the below command
pip install imgaug — upgrade — no-deps
We will take an image and transform it using basic data augmentation techniques.
import imageioimport imgaug as iaimport imgaug.augmenters as iaaimport numpy as npimport pandas as pdimport matplotlib.pyplot as pltimport matplotlib.patches as patchesimport matplotlib%matplotlib inline
We display the original image using imageio
image = imageio.imread(“.\\car2.jpeg”)ia.imshow(image)
We can rotate the image by specifying the rotation in degrees. We are rotating the image by -50 degrees to 30 degrees
rotate=iaa.Affine(rotate=(-50, 30))rotated_image=rotate.augment_image(image)ia.imshow(rotated_image)
We add different noise values sampled from Gaussian distributions element wise to images.
gaussian_noise=iaa.AdditiveGaussianNoise(10,20)noise_image=gaussian_noise.augment_image(image)ia.imshow(noise_image)
Cropping removes columns/rows of pixels at the sides of images. In the example below, we are cropping one side of the image by 30%
crop = iaa.Crop(percent=(0, 0.3)) # crop imagecorp_image=crop.augment_image(image)ia.imshow(corp_image)
Shearing the image by 0 to 40 degrees
shear = iaa.Affine(shear=(0,40))shear_image=shear.augment_image(image)ia.imshow(shear_image)
We can flip the image vertically or horizontally. Fliplr flips image horizontally
#flipping image horizontallyflip_hr=iaa.Fliplr(p=1.0)flip_hr_image= flip_hr.augment_image(image)ia.imshow(flip_hr_image)
Flipup flips image vertically
flip_vr=iaa.Flipud(p=1.0)flip_vr_image= flip_vr.augment_image(image)ia.imshow(flip_vr_image)
We adjust the image brightness using GammaContrast by scaling pixel values. Values in the range gamma=(0.5, 2.0) seem to be sensible. We can use SigmoidContrast or LinearContrast also for changing the brightness of the image
image = imageio.imread(“.\\img Aug\\car2.jpeg”)contrast=iaa.GammaContrast(gamma=2.0)contrast_image =contrast.augment_image(image)ia.imshow(contrast_image)
We can zoom in or zoom out images using scale. We have scaled the image below to 150% to 80% of the image height/width. We can scale each axis independently
image = imageio.imread("C:\\Data\\img Aug\\car2.jpeg")scale_im=iaa.Affine(scale={"x": (1.5, 1.0), "y": (1.5, 1.0)})scale_image =scale_im.augment_image(image)ia.imshow(scale_image)
We draw bounding boxes for object detection. When we augment the image we want the bounding boxes also to be updated accordingly.
imgaug offers support for bounding boxes. When we rotate, shear or crop the image, bounding box around the object is also updated accordingly.
importing bounding boxes from imgaug
from imgaug.augmentables.bbs import BoundingBox, BoundingBoxesOnImage
Initializing the bounding box around the original image
bbs = BoundingBoxesOnImage([ BoundingBox(x1=10, x2=520, y1=10, y2=300)], shape=image.shape)
Displaying the bounding box on top of the original image
ia.imshow(bbs.draw_on_image(image, size=2))
In the code below, we move the image using translate_percentage, augment the bounding box and apply it over the image
move=iaa.Affine(translate_percent={"x": 0.1}, scale=0.8)image_aug, bbs_aug = move(image=image, bounding_boxes=bbs)ia.imshow(bbs_aug.draw_on_image(image_aug, size=2))
The bounding box may sometime go outside the image and we need additional code to handle such a scenario
We rotate the image and try to draw the bounding box around the object
rotate_bb=iaa.Affine(rotate=(-50, 30))image_aug, bbs_aug = rotate_bb(image=image, bounding_boxes=bbs)ia.imshow(bbs_aug.draw_on_image(image_aug, size=2))
Portions of the bounding box are outside the image. In the code below we will
Remove the bounding box fully or partially outside the image
Clip the bounding boxes that are partially outside so that they are fully inside the image
We create a padding function to pad the image with a 1 pixel white and 1 pixel black borders:
def pad(image, by):    image_border1 = ia.pad(image, top=1, right=1, bottom=1, left=1,                           mode="constant", cval=255)    image_border2 = ia.pad(image_border1, top=by-1, right=by-1,                           bottom=by-1, left=by-1,                           mode="constant", cval=0)    return image_border2
We then draw the bounding boxes on the image. We first extend the image plane by BORDER pixels and then mark the bounding boxes inside the image plane
def draw_bbs(image, bbs, border):    GREEN = [0, 255, 0]    ORANGE = [255, 140, 0]    RED = [255, 0, 0]    image_border = pad(image, border)    for bb in bbs.bounding_boxes:        if bb.is_fully_within_image(image.shape):            color = GREEN        elif bb.is_partly_within_image(image.shape):            color = ORANGE        else:            color = RED        image_border = bb.shift(left=border, top=border)\                         .draw_on_image(image_border, size=2, color=color)return image_border
We now apply the same rotation to the image and draw the bounding box
rotate=iaa.Affine(rotate=(-50, 30))image_aug, bbs_aug = rotate(image=image, bounding_boxes=bbs)image_after = draw_bbs(image_aug, bbs_aug.remove_out_of_image().clip_out_of_image(), 100)ia.imshow(image_after)
In the next article, we will explore Keras ImageDataGenerator for image augmentation