Image Processing: Image Scaling Algorithms | by Bilal Himite | Towards Data Science

Image scaling is an essential part of image processing. Images need to be scaled up or down for multiple reasons. In this article, we will take a look into different image scaling methods and we will implement them in Python.
This article is a part of a series of articles discussing concepts of image processing. Check out the previous articles:
Image Processing and Pixel Manipulation: Photo FiltersHow to Create a Photo Mosaic in Python
Image Processing and Pixel Manipulation: Photo Filters
How to Create a Photo Mosaic in Python
We will assume we have an image with a resolution of width×height that we want to resize to new_width×new_height . First, we will introduce the scaling factors scale_x and scale_y defined as follows:
scale_x = new_width / widthscale_y = new_height / height
A scale factor <1 indicates shrinking while a scale factor >1 indicates stretching.
We will have to import the image into an array, I will use PIL for this.
From now on, I will only talk about the magic part.
We will use these two photos (The first by John-Mark Smith from Unsplash and the second by Hiếu Hoàng from Pexels) as an example. The original pictures have a resolution of 1920×1080 (~2 Megapixels).
We will scale these images down to 500×281 and scale it up to 7373×4147 , a scale factor of 1/3.84 and 3.84. It’s important not to use nice numbers, like a scale factor of 2, 4, 0.5, as they can be edge cases where the algorithms give out appealing results that do not reflect the general case.
This is probably the simplest image scaling method. Every output pixel is replaced by its nearest pixel in the input.
In 1 dimension, the value at x is the value of its nearest point.
In 2D, a pixel with coordinates (x, y) in the output image has coordinate (x/scale_x, y/scale_y) in the input image. Since these coordinates don’t always exist (if they have a decimal part), we will round the coordinates to the nearest integer, thus rounding to the nearest neighbor.
An implementation of this would look like this:
Results:
You can use, instead, PILLOW ‘s resize method:
im.resize(size, Image.NEAREST)
Linear interpolation (a.k.a lerp) is equivalent to drawing a line between every two consecutive points. This can also be thought of as taking the average of the neighboring points weighted by their distance.
In 2 dimensions, this interpolation involves the 4 neighboring points. Linear interpolation is done on points the top 2 points (Q11 and Q12) and the bottom 2 points (Q21 and Q22) to obtain two new points (P1 and P2). Then, linear interpolation is applied to the new points to get the interpolated point P.
Implementing this in python:
You can use, instead, PILLOW ‘s resize method:
im.resize(size, Image.BILINEAR)
The 1-dimensional equivalent of bicubic interpolation is cubic spline interpolation.
Instead of drawing lines between the points, cubic spline interpolation draws smooth curves. These smooth curves are 3rd-degree polynomials.
Cubic spline interpolation requires 4 neighboring points. And the 4 coefficients are calculated under the conditions:
The derivative at the 2 inner neighboring points is the gradient between the inner neighbors and the outer neighbors.
The inner neighbors must be part of the curve.
The conditions could be translated into formulas by calculating the derivatives and setting them equal to the gradients, and by evaluating the cubic formula at the inner neighboring points.
We can calculate this using scipy’s interpolate.CubicSpline function.
In 2D, this will involve 16 points. We will perform cubic spline interpolation on 4 rows of points, then perform one last cubic spline interpolation on the new 4 interpolated points:
Implementing this algorithm efficiently is very difficult. Instead, you can use, PILLOW ‘s resize method:
im.resize(size, Image.BICUBIC)
This method is used specifically for downscaling. When using one of the algorithms above for downscaling some pixels are completely ignored. This results in poor downscaling quality. To overcome this, box sampling considers every pixel in the downscaled image as a box in the original. And its color is the average of the colors inside the box.
You can use, instead, PILLOW ‘s resize method:
im.resize(size, Image.BOX)
There are other image scaling algorithms like Sinc and Lanczos resampling or Deep convolutional neural networks based algorithms, but they are a bit more complicated.
Image scaling is an important part of image processing. In this article, I only discussed simple scaling algorithms and the principle behind them for images. Real-time video scaling algorithms are usually done on hardware (GPU) for faster performance.