Image Processing with Python. Bytes, sampling, and filters — an... | by James Briggs | Towards Data Science

Image processing is often a reasonably infrequent task. Of course, this will depend heavily on our roles — but even in heavy analytical roles such as data science or machine learning — image manipulation is pretty common.
In machine learning, we may need to take a number of images and classify them based on what their contents — we’ll likely need to downsample the images — taking them from an original 2, 4, 8K resolution image to a more manageable size.
There are many other disciplines that require a small amount of image processing, such as web development — or significantly more, as for image processing engineers.
We will touch upon the basics of how image data is organized in a computer, and then explore a few different methods for image manipulation:
Images as Arrays - intro to images in code and their representation in bits and bytesResize and Scaling - resizing our images, save on compute with downsamplingFilters - how to build an overly complicated Photoshop with significantly less features
We will use Numpy throughout the article — for quick and easy image manipulation, I would suggest another library called PIL.
PIL code will be included too, but we won’t focus on it — as it abstracts everything to a level where we can’t focus on the actual processes.
Throughout the article, we’ll work with the article cover image.
In the original format, this image consists of 7207 x 4801 pixels.
We will be representing this image as an array. In this array, we will have three ‘channels’ (layers) — representing red, green, and blue.
All of that gives us a total of 7207*4801*3 = 103,802,421 numeric values contained within our image array.
Now, let’s write some code to convert our image into a Numpy array:
from PIL import Imageimport numpy as npimg = Image.open(PATH)img.load()img_array = np.asarray(img, dtype='int32')
We can also take a look at the values that make up our array:
All values in our array vary from zero, the minimum — to 255, the maximum. This range is used because each of our colors is a 8-bit byte. Each bit can be either 0 or 1.
That gives us a total of 2^8 = 256 possible combinations, a range of 0–255.
Where —
0 = 000000001 = 000000012 = 000000103 = 00000011...254 = 01111111255 = 11111111
As we can see, 0 shows all bytes to be off, whereas for 255 all bytes are on — this makes sense as 0 corresponds to the minimum of that respective color in the image, and 255 the maximum. Let’s try turning our colors off-and-on.
red1 = np.copy(img_array)red1[:,:,0] = 255 # this is on (top row below)plt.imshow(red0)blue0 = np.copy(img_array)blue0[:,:,2] = 0  # this is off (bottom row below)plt.imshow(blue0)
There are two major reasons to do this in machine learning applications:
Downsampling to save on compute — training an image classifier with 8K resolution images will take an impressive setup — 360p is a little more realistic.Increasing the size of our dataset — (we have to stick to the same image size in classification, but that doesn’t mean we can’t use images with different resolutions)
Downsampling to save on compute — training an image classifier with 8K resolution images will take an impressive setup — 360p is a little more realistic.
Increasing the size of our dataset — (we have to stick to the same image size in classification, but that doesn’t mean we can’t use images with different resolutions)
Both reasons require the same approach, we take our original array and compress groups of pixels into single pixels in a way that maximizes the information retained.
Here, we will downsample our image by taking the average value of multiple pixels to create one new pixel.
We use sklearn.measure.block_reduce to perform this operation and np.mean to specify the reduction operation using averages.
To see a clear difference, we downsample our image by a factor of 20.
downsample = 20# first, change to 0-1ds_array = img_array/255r = skimage.measure.block_reduce(ds_array[:, :, 0],                                 (downsample, downsample),                                 np.mean)g = skimage.measure.block_reduce(ds_array[:, :, 1],                                 (downsample, downsample),                                 np.mean)b = skimage.measure.block_reduce(ds_array[:, :, 2],                                 (downsample, downsample),                                 np.mean)ds_array = np.stack((r, g, b), axis=-1)
We can compare the original and downsampled images using imshow, which gives us:
Again — speaking from a ML/data science perspective — it is impossible (for any normal computer) to cope with training a classifier on big sets of 100M+ value arrays.
But after downsampling by a factor of 20, our array has a total of just 241 * 361 * 3 = 261,003 values — an easily manageable size.
This is the fun part — we’re going to write some code to modify our picture — like Instagram filters.
The vignette is essentially a darkening of pixels towards the outer edges of the image — it looks like this:
A while back, this was the go-to effect for edgy teenagers trying to cover up their lackluster photography skills.
It’s a fun effect to get started with. All we need is a function that we can apply to our array that will reduce values towards the outer edges.
To achieve this effect, we start with a Gaussian function — we will feed it a single value for every integer row-wise and column-wise.
This code produces two Gaussian distributions that look like this:
We then normalize these distributions and apply them to our image. Visualizing just the two distributions applied to a white square gives us this:
This effect can be applied to an array easily:
Taking this code and applying it to our Unsplash image gives us that edgy teen/hipster Instagram effect:
Another cool thing we can do is modify the color layers in our image individually.
Simple balancing is easily achieved by accessing each color layer individually and increasing/decreasing layer values.
As we do this, we will inadvertently create numbers outside of our range of 0–255. To deal with these we use a simple clamp function to squeeze our array back into the correct range, shown above.
Playing around with the r, g, b values in col_bal can produce some interesting effects:
We use the exact same approach to modify image brightness. All we need to do is modify our RGB values by the exact same value:
Finally, we can put all of these together. Some color-balancing with the vignette effect can build some cool effects really easily.
One thing that we have avoided throughout this article is the use of the PIL library. PIL is the go-to for image processing in Python — so this article wouldn’t be complete without mentioning it.
PIL is an excellent library, purpose-made for image processing in Python. With it, we can compress what would take us several lines of Numpy code — into a single function.
However, PIL abstracts everything to a point where understanding the mechanics is not possible — hence our heavy use of Numpy.
We’ve covered the basics behind images in bits/bytes, how we can downsample images, and how to edit images with vignettes and color-balancing.
If you’re interested in learning more advanced concepts, I would highly recommend this course on Datacamp (the first part is free).
I hope you have learned something from the article, if you have any questions or suggestions, get in touch on Twitter or in the comments below!
Thanks for reading!
What Image Processing Techniques Are Actually Used in the ML Industry (2020), Neptune.ai