Understanding the Basics of Digital Image Processing and Computer Vision using OpenCV | by Hucker Marius | Towards Data Science

This article is about the basic concepts behind a digital image, the processing of it, and hence, also the fundaments of CV. At the end you can find a simple code implementation with Python using OpenCV.
You’ll be able to answer the following questions, after you’ve read this article:
How does a computer see an image?
What are the typical types of images?
What is binarization in the context of image processing?
What is thresholding?
How is thresholding and binarization applied?
Every image consists of pixels that are little squares filled with one color. Setting all different squares together and zooming out to see all pixels leads to an image as we know it. In the image below the little squares can be perfectly seen. It consists of 119 horizontal pixel columns and 82 vertical pixel rows. In total, this leads to 119 x 82 = 9758 Pixels and a really small data size of 16 KB.
How smooth and clear the image is, depends on the size and thus on the number of pixels. Zooming in an image consisting only of approximately 10000 pixels (as the one above) leads to unclear structures that seem more like Tetris-Blocks than smooth lines. Increasing this size to 904 horizontal pixel columns and 604 vertical pixel rows lead to 904x604=546.016 pixels and a data size of 209 KB. The image is now already clearer but still grained and a bit crispy.
Zooming further out to see the whole image which consists of 4928 × 3264 = 16.084.992 pixels, leads to a smooth and nice image. The image has now a data size of 6,7 MB, thus 6700 KB.
This is how we see a digital image. But how does it look like for a computer? What does it get and how does it process the input? For a computer, an image is a two-dimensional function in the shape
It’s a function of two continuous spatial variables x and y, with x being the row values of the pixels and y being the column values. To process it a computer samples it and quantizes it to turn the input into a matrix as seen above. Every single pixel of the image is processed with f(x,y) (Da Silva & Mendonca, 2005).
In typical digital images every pixel consists of an 8-bit integer, leading to 256 possible states:
For colored digital images exist for every pixel three functions in the form
Colored digital pixels consist of a 1x3-matrix, representing the three primary color channels Red, Green, and Blue with each being an 8-bit integer. The 256 states of each pixel refer to the brightness of each color channel, with 0 being the darkest version of the color and 255 being the brightest version of the primary color (Fisher et al. a, 2003).
Followingly a vector with R = 0, G = 0 and B = 0 would lead to a black pixel because all three color channels consist of the darkest shade. A constellation of R = 3, G = 168, B = 124 would lead to a light green pixel, with a really light red value (only 3) and thus only little influence of red on the resulting pixel color, but balanced values of Green and Blue with green being the strongest value. Both resulting colors can be seen in the pictures below.
Basically there exist multiple types of images, but we will contemplate four typical types:
Binary image: Consisting only of black and white pixels, which are either 0 for white or 1 for black. This type is often used in image processing, e.g. in Optical Character Recognition to recognize letters and text in images.Black&White Image: Images in black and white, especially in photography, are typically grayscaled images. This means the pixels do not only consist of black and white color, but every pixel is a shade of 256 different shades of grey.8-bit Color Image: Most of the digital color images you encounter are 8-bit color images. It is the most popular type and also the one which has been used for the explanation in the previous section Let’s talk about pixels. As explained there the 8-bit leads to 256 different shades of a color.16-bit Color Image: It is a more complex color image, that is not as popular as its little brother, the 8-bit color image. Taken 2 to the power of 16 leads to65536 different shades of a color for every colour channel of a pixel.
Binary image: Consisting only of black and white pixels, which are either 0 for white or 1 for black. This type is often used in image processing, e.g. in Optical Character Recognition to recognize letters and text in images.
Black&White Image: Images in black and white, especially in photography, are typically grayscaled images. This means the pixels do not only consist of black and white color, but every pixel is a shade of 256 different shades of grey.
8-bit Color Image: Most of the digital color images you encounter are 8-bit color images. It is the most popular type and also the one which has been used for the explanation in the previous section Let’s talk about pixels. As explained there the 8-bit leads to 256 different shades of a color.
16-bit Color Image: It is a more complex color image, that is not as popular as its little brother, the 8-bit color image. Taken 2 to the power of 16 leads to65536 different shades of a color for every colour channel of a pixel.
Grayscaling. In image processing grayscaling is a commonly used technique to standardize and compromise digital color images. While the three-dimensional RGB-space in color images leads to
possible combinations, a grayscale image has distinctively less states with
Since every state of the 256 possible states corresponds to different brightness, grayscaling is relevant to enhance the speed of an automatic brightness and contrast correction program. In the preprocessing of thousands of images, grayscaled images can be handled faster than colored images due to their one-channel structure. Hence, color images have to be converted to grayscaled images. For this step, the 1x3-vector has to be reduced to a 1x1-vector. The simplest method which comes to mind would be to assign every color space the same weight, being 0,33. The method is well-known and a typical approach:
However, the wavelengths of the three primary colors vary (Choudhury, p. 10 ff, 2014). This leads to a diverse contribution of the colors to the human vision. Due to this diverse contribution of the primary colors to human color perception, the proportions should not be considered equal and should rather be adapted to the human eye. The transformation formula of Open CV is based on the principle of constant luminance /BT601 (Open CV c, n.d.):
Using this formula a pixel consisting of the three values (RGB) is turned to one value (Y), being a shade of grey within the range of 256 different grey shades (0–255). Followingly this reduces also the information every pixel comprises and followingly the data size of an image.
Taking the example of the green tone above with R=3, G=168, and B=124 and filling it into the grayscaling formula leads to Y = 114, which is a grey tone in the middle range considering 256 shades of grey.
The input image in grayscale:
Turning the input image into a grayscaled image and plotting the histogram of its pixel values, shows how often each of the shades of grey is represented in the image. Pixels having a dark shade of grey within 200 to 250 have the highest frequency as can be seen on the right side of the histogram below. Taking a look at the picture reveals that the dark pixels at the bottom and some of the pixels of the bubbles are part of this peak in the histogram.
The conversion to a binary image is called thresholding. The idea behind thresholding is the segmentation of all pixels to either 1 or 0. All pixels of the grayscaled image surpassing a defined threshold are 1, while the values below are converted to 0. The output is an image consisting only of black and white pixels, with 0 being black and 1 being white. For defining the threshold value there exist diverse techniques. The algorithm presented in this work consists of two steps. It combines the simple binary thresholding and OTSU thresholding. It calculates the t-value for which the weighted within-class variance of both classes, 1 and 0, are minimized.
The OTSU thresholding (OpenCV, f, n.d.)
The retrieved value is then used for binary thresholding via (OpenCV, f, n.d.)
After thresholding, the image is segmented in fore- and background. All pixels unequal to 0 and hence being 1 can be defined as content and foreground of the document. The thresholding for the image below was done with Open CV and both methods explained above.
Thanks for reading!
medium.com
Below you find a short code snippet to apply the methods explained:
#import the OpenCV libraryimport cv2#read your filefile=r’/Users/marius/Desktop/color.jpg’img = cv2.imread(file)#Grayscaling the image via OpenCVgray = cv2.cvtColor(img, cv2.COLOR_RGB2GRAY)#Thresholding the imagegray,img_bin = cv2.threshold(gray,127,255,cv2.THRESH_BINARY | cv2.THRESH_OTSU)#Showing the imagecv2.imshow(‘image’,img_bin)cv2.waitKey(0)
References
Da Silva, E. A. B., & Mendonça, G. V. (2005). Digital Image Processing. The Electrical Engineering Handbook, 891–910.
Fisher, R., Perkins, S., Walker, A., & Wolfart, E. (2003). Pixel Values. https://homepages.inf.ed.ac.uk/rbf/HIPR2/value.htm
Open CV c. (n.d.). Color conversions.
Open CV d. (n.d.). Operations on Arrays. https://docs.opencv.org/2.4/modules/core/doc/operations_on_arrays.html
Open CV e. (n.d.). Image Thresholding. https://docs.opencv.org/master/d7/d4d/tutorial_py_thresholding.html
Open CV f. (n.d.). Miscellaneous Image Transformations. https://docs.opencv.org/master/d7/d1b/group__imgproc__misc.html#gae8a4a146d1ca78c626a53577199e9c57