Common Image Processing Techniques in Python | by Renu Khandelwal | Towards Data Science

In this article, you will learn
Common image processing techniques using PIL and OpenCV like converting the RGB image to the grayscale image, rotating the images, de-noising the images, detecting edges in an image and cropping the region of interest in an image
Searching objects in an image using Template Match of OpenCV
PIL, OpenCV, and imutils
Why do you need to learn image processing techniques?
Deep learning is used to analyze, identify, and generate an intelligent understanding of the objects in an image. Some of the common applications are Image Classification, Object Detection, Instance Segmentation, etc. Hence, it is imperative to have a strong knowledge of image processing techniques like image enhancements that include cropping the image, removing the noise present in the image, or rotating the image, etc., to build a better training dataset.
Image processing techniques are equally helpful for Optical Character Recognition(OCR)
Image processing techniques increase the interpretability of the image to classify or detect objects present in the image by helping to identify key features or to read text from images.
Code and images available here
Importing Required Libraries
import cv2from PIL import Image
Let’s first display the image using OpenCV and PIL
Reading and Displaying the Image using OpenCV
# Reading and Displaying the fileimage = cv2.imread(r'love.jpg')cv2.imshow("Image", image)cv2.waitKey(0)
If the image is too big, then the window in which the image is displayed will not fit the screen.
So how do I display the full image on the screen?
By default, you get a cropped display when displaying an oversized image. To view the full image, you will use OpenCV’s namedWindow(name, flag) to create a new window. The first parameter, name, is the caption for the window and will be used as an identifier. When you set the flag to cv2.WINDOW_NORMAL, then the full image is displayed, and you can resize the window. There are other options for the flag parameter.
# Resizing the image for the user to resize the window image = cv2.imread(r'love.jpg')cv2.namedWindow('Normal Window', cv2.WINDOW_NORMAL)cv2.imshow('Normal Window', image)cv2.waitKey(0)
Resizing image
When we resize the image, we change the height or width or both of the image and maintain the aspect ratio. The aspect ratio of the image is the ratio of the image’s width to its height.
image= cv2.imread(r'taj.jpg')scale_percent =200 # percent of original sizewidth = int(image.shape[1] * scale_percent / 100)height = int(image.shape[0] * scale_percent / 100)dim = (width, height)resized = cv2.resize(image, dim, interpolation = cv2.INTER_AREA)cv2.imshow("Resize", resized)cv2.waitKey(0)
Reading and Displaying the Image using PIL
You will load the image using open() and then use show() to display.
image. show() creates a temporary file.
pil_image= Image.open(r'love.jpg')pil_image.show("PIL Image")
What if you are interested in identifying edges or other features in an image?
A grayscale image is used to identify edges. Grayscale images are also helpful to understand contrast or shading gradients present in an image and understand their role to analyze the features of an image.
RGB images have three channels Red, Green, and Blue, compared to a 2D channel for the grayscale image. There will be less information per pixel for the grayscale image compared to a color image, and hence the processing time for the grayscale image will be faster.
We are converting a color image to grayscale using cvtColor(), which will save an image from one color space to another.
#Converting image to grayscaleimage = cv2.imread(r'love.jpg')gray_image= cv2.cvtColor(image, cv2.COLOR_RGB2GRAY)cv2.namedWindow('Gray Image', cv2.WINDOW_NORMAL)cv2.imshow('Gray Image', gray_image)cv2.waitKey(0)
convert() returns a converted copy of this image; you use an “L” mode for grayscale and “RGB” mode for color.
# Gray image using PILpil_image= Image.open(r'love.jpg')gray_pil=pil_image.convert('L')gray_pil.show()
Edge in an image is detected using a canny edge detector. A Canny edge detector uses a grayscale image using a multi-stage algorithm.
Canny(): the first argument is the input image, and the second and third arguments are threshold1 and threshold2 values.
Edges with intensity gradient more than threshold2 are considered as the edge and those below threshold1 as non-edges. Non-edges will be removed. Any gradient intensity value between the two thresholds is classified as edges or non-edges based on their connectivity.
image= cv2.imread(r'taj.jpg')cv2.namedWindow("Edge", cv2.WINDOW_NORMAL)denoised_image = cv2.Canny(image, 100,200 )cv2.imshow("Edge", denoised_image)cv2.waitKey(0)
What if the image is skewed or slightly rotated, and you want to deskew the image?
OCR on skewed text does not work well, and hence we need to deskew. Images can be deskewed using rotate() of OpenCV and PIL.
rotate() will rotate the image based on the rotationCode, which is specified as the second parameter to the rotate function.
Values for rotationCode are
cv2.ROTATE_90_CLOCKWISE,
cv2. ROTATE_90_COUNTERCLOCKWISE
cv2.ROTATE_180
#Rotating the imageimage = cv2.imread(r'love.jpg')cv2.namedWindow("Rotated Image", cv2.WINDOW_NORMAL)rotated_img= cv2.rotate(image,cv2.ROTATE_90_CLOCKWISE )cv2.imshow("Rotated Image", rotated_img)cv2.waitKey(0)
What if I want to rotate the image by only a specific angle?
Rotating image based on a specific angle
In the code below, the image is rotated in increments of 60-degrees using rotate() of imutils
import imutilsimport numpy as npimage = cv2.imread(r'love.jpg')# loop over the rotation anglesfor angle in np.arange(0, 360, 60):    cv2.namedWindow("Rotated", cv2.WINDOW_NORMAL)    rotated = imutils.rotate(image, angle)    cv2.imshow("Rotated", rotated)    cv2.waitKey(0)
Here the image is rotated by 110 degrees using PIL
# Rotate image using PILpil_image= Image.open(r'love.jpg')rotate_img_pil=pil_image.rotate(110)rotate_img_pil.show()
How to enhance the image quality when it is deteriorated by the presence of noise and impacts the image analysis?
Noise is an unwanted signal, and in terms of an image, it is a random variation of color.
Minimizing the noise present in an image using OpenCV.
image= cv2.imread(r'taj.jpg')cv2.namedWindow("Noised Image", cv2.WINDOW_NORMAL)cv2.imshow("Noised Image", image)cv2.waitKey(0)
There are different methods in OpenCV to remove noise from images. The one used below is cv.fastNlMeansDenoisingColored(), which is to remove noise from a color image.
Common arguments for the fastNIMeansDenoising methods are
src: source image
dst: output image with the same size and type as src
h: regulates filter strength. A higher value for h perfectly removes noise along with image details, a smaller value for h will preserve the image details along with some noise.
hForColorComponents: same as h, but for color images only and is normally same as h
templateWindowSize: should be odd. (recommended 7)
searchWindowSize: should be odd. (recommended 21)
image= cv2.imread(r'taj.jpg')cv2.namedWindow("Denoised Image", cv2.WINDOW_NORMAL)denoised_image = cv2.fastNlMeansDenoisingColored(image,None, h=5)cv2.imshow("Denoised Image", denoised_image)cv2.waitKey(0)
How to extract certain regions of interest from an image?
Cropping the image lets you extract the area of interest in an image.
We will crop the image of TajMahal and only keep the TajMahal and remove other details from the image.
Cropping image using OpenCV
Cropping in OpenCV is done by slicing the image array where we pass the start and end of y co-ordinates followed by the start and end of x-coordinates.
image[y_start:y_end, x_start:x_end]
image= cv2.imread(r'taj.jpg')resized_img= image[15:170, 20:200]cv2.imshow("Resize", resized_img)cv2.waitKey(0)
Cropping image using PIL
crop() of PIL allows us to crop a rectangular area of the image, The parameter to the crop() is left co-ordinate, top co-ordinate, right co-ordinate and bottom pixel co-ordinate.
# Opens a image in RGB mode pil_image = Image.open(r'taj.jpg')   # Get the Size of the image in pixels width, height = pil_image.size   # Setting the cropped image co-ordinatesleft = 3top = height /25right = 200bottom = 3 * height / 4  # Crop the  image based on the above dimension cropped_image = pil_image.crop((left, top, right, bottom))  # Shows the image in image viewer cropped_image.show()
You can provide a template and search that template in an image and extract its location using matchTemplate() of OpenCV.
The template slides over the entire image like a convolution neural network and tries to match the template to the input image.
minMaxLoc() is used to get the maximum/minimum value, which will be the top left corner of the rectangle along with the width and height.
There are 6 methods available for the template match
cv2.TM_SQDIFF
cv2.TM_SQDIFF_NORMED
cv2.TM_CCORR
cv2.TM_CCORR_NORMED
cv2.TM_CCOEFF
cv2.TM_CCOEFF_NORMED
In the example below, I have cropped a small portion from the main image for creating the template.
The method used for the template match is TM_CCOEFF_NORMED. The threshold for the match is set to 0.95. It localizes the location with a higher matching probability greater than 95% and then draws a rectangle around the area corresponding to the match.
import cv2import numpy as npfrom matplotlib import pyplot as pltimg = cv2.imread(r'love.jpg',0)cv2.imshow("main",img)cv2.waitKey(0)template = cv2.imread(r'template1.png',0)cv2.imshow("Template",template)cv2.waitKey(0)w, h = template.shape[::-1]methods = [ 'cv2.TM_CCOEFF_NORMED']for meth in methods:        method = eval(meth)# Apply template Matching    res = cv2.matchTemplate(img,template,method)    min_val, max_val, min_loc, max_loc = cv2.minMaxLoc(res)    threshold=0.95    loc=np.where(res>threshold)    if len(loc[0])>0:# If the method is TM_SQDIFF or TM_SQDIFF_NORMED, take minimum        if method in [ cv2.TM_SQDIFF_NORMED]:            top_left = min_loc        bottom_right = (top_left[0] + w, top_left[1] + h)cv2.rectangle(img,top_left, bottom_right,100,20)plt.subplot(121),plt.imshow(res,cmap = 'gray')        plt.title('Matching Result'), plt.xticks([]), plt.yticks([])        plt.subplot(122),plt.imshow(img,cmap = 'gray')        plt.title('Detected Point'), plt.xticks([]), plt.yticks([])        plt.suptitle(meth)plt.show()    else:        print("Template not matched")
We have discussed the most common image processing techniques that can be used to analyze images for Image classification, Object detection, as well as OCR.