Cartoonize Images with Python
By: Robert Van De Vlasakker
Tags: Python, Cartoon, Edge Detection, Photoshop

Import statements + opening an image

We start with importing both libraries:

# Import necessary libraries

import cv2

import numpy as np

We can use the cv2 library to directly open an image:

# Opens an image with cv2

img = cv2.imread('DanielGomez.jpg')

Cv2 uses numpy to open images, the 'img' variable now contains a numpy array of 3 dimensions. A little note, cv2 open the image as BGR (order of the spectrum) and not RGB as most people are used to. If you play around with the Notebook (see the end of this article), you see that for the plotting I converted the image from BGR to RGB.

In this tutorial I will use a laplacian filter to detect image edges. The Laplacian filter is a very nice filter (read more about it here). It is however, very prone to noise. That is why we will apply some image filtering before applying the laplace filter:

Filtering

# Apply some Gaussian blur on the image

img_gb = cv2.GaussianBlur(img, (7, 7) ,0)

# Apply some Median blur on the image

img_mb = cv2.medianBlur(img_gb, 5)

# Apply a bilateral filer on the image

img_bf = cv2.bilateralFilter(img_mb, 5, 80, 80)

We can of course also visualise the results to see the differences.

Photo by Daniel Gomez on Unsplash — Edited by author

Don't see the difference between the images? Neither do I that well, you can spot the differences if you know what each filter does! The Laplacian filter will detect the differences.

Edge detection

Next we can apply the laplacian filter, for comparison sake, we apply the filter on all the 4 different images and visualise the result. For each of the Laplacian filters we use a kernel size of 5. You can of course play around a bit, but in my opinion this will give the best result. 'CV_8U' means that we are using 8 bit values (0–255).

# Use the laplace filter to detect edges

img_lp_im = cv2.Laplacian(img, cv2.CV_8U, ksize=5)

img_lp_gb = cv2.Laplacian(img_gb, cv2.CV_8U, ksize=5)

img_lp_mb = cv2.Laplacian(img_mb, cv2.CV_8U, ksize=5)

img_lp_al = cv2.Laplacian(img_bf, cv2.CV_8U, ksize=5)

Again, we visualise the results:

Photo by Daniel Gomez on Unsplash — Edited by author

You can now see that Laplacian of the original image detected a lot of noise. The image with all the filters is the sharpest, which comes in handy in a bit. This is however not yet what we want. We need an image preferably black and white that we can use as a mask. So lets continue to edit this image. Let us first convert it to a greyscale image:

# Convert the image to greyscale (1D)

img_lp_im_grey = cv2.cvtColor(img_lp_im, cv2.COLOR_BGR2GRAY)

img_lp_gb_grey = cv2.cvtColor(img_lp_gb, cv2.COLOR_BGR2GRAY)

img_lp_mb_grey = cv2.cvtColor(img_lp_mb, cv2.COLOR_BGR2GRAY)

img_lp_al_grey = cv2.cvtColor(img_lp_al, cv2.COLOR_BGR2GRAY)

Each variable now contains a 1-dimensional array instead of a 3-dimensional array. Next we are going to use image thresholding to set values that are near black to black and set values that are near white to white. We can create a manual threshold with:

# Manual image thresholding

_, EdgeImage = cv2.threshold(img, 127, 255, cv2.THRESH_BINARY)

There are a lot of different ways to use image thresholding, try them and find out which works best in your case, you can read more about it here. In this tutorial we use Otsu thresholding. Let’s just remove some additional noise before applaying the Otsu threshold.

# Remove some additional noise

blur_im = cv2.GaussianBlur(img_lp_im_grey, (5, 5), 0)

blur_gb = cv2.GaussianBlur(img_lp_gb_grey, (5, 5), 0)

blur_mb = cv2.GaussianBlur(img_lp_mb_grey, (5, 5), 0)

blur_al = cv2.GaussianBlur(img_lp_al_grey, (5, 5), 0) # Apply a threshold (Otsu)

_, tresh_im = cv2.threshold(blur_im, 245, 255,cv2.THRESH_BINARY + cv2.THRESH_OTSU)

_, tresh_gb = cv2.threshold(blur_gb, 245, 255,cv2.THRESH_BINARY + cv2.THRESH_OTSU)

_, tresh_mb = cv2.threshold(blur_mb, 245, 255,cv2.THRESH_BINARY + cv2.THRESH_OTSU)

_, tresh_al = cv2.threshold(blur_al, 245, 255,cv2.THRESH_BINARY + cv2.THRESH_OTSU)

We now have a black image with white edges, we only have to invert the black and the white for our mask.

# Invert the black and the white

inverted_original = cv2.subtract(255, tresh_im)

inverted_GaussianBlur = cv2.subtract(255, tresh_gb)

inverted_MedianBlur = cv2.subtract(255, tresh_mb)

inverted_Bilateral = cv2.subtract(255, tresh_al)

Let's visualise the results:

Photo by Daniel Gomez on Unsplash — Edited by author

The last one looks like a great image to use for our mask!

Color Quantization

Now we have our image for the masking, we have to reduce the amount of color for our cartoon image. We can do this two ways, with the Kmeans algorithm or with a simple floor division. The Kmeans algorithm is ment for explanatory data analysis, but quite useful in this case. You can read more about the algorithm and how it works over here. For the Kmeans we need to some alteration before we run it, the code is provided below:

# Reshape the image

img_reshaped = img.reshape((-1,3))

# convert to np.float32

img_reshaped = np.float32(img_reshaped)

# Set the Kmeans criteria

criteria = (cv2.TERM_CRITERIA_EPS + cv2.TERM_CRITERIA_MAX_ITER, 10, 1.0)

# Set the amount of K (colors)

K = 8

# Apply Kmeans

_, label, center = cv2.kmeans(img_reshaped, K, None, criteria, 10, cv2.KMEANS_RANDOM_CENTERS) # Covert it back to np.int8

center = np.uint8(center)

res = center[label.flatten()]

# Reshape it back to an image

img_Kmeans = res.reshape((img.shape))

As you can see it takes quite a bit of work to make the Kmeans run. It also can take some time depending on the iteration, amount of clusters and of course the image size. A better approach for reducing the amount of colours of an image might be a simple floor division.

# Reduce the colors of the original image

div = 64

img_bins = img // div * div + div // 2

There as simple as that, and it is done in the blink of an eye! Because we work with a numpy array this formula is applied quick and fast on every number in our 3D numpy array. By applying this formula each number return 32, 96, 160 or 224. This means that we end up with 4 * 4 * 4 = 64 colours. Let's make the comparison between the two:

Photo by Daniel Gomez on Unsplash — Edited by author

I personally like the right one a bit more, it keeps the colours of the tattoos a bit better.

Combine the images

Now all that is left to do is combine both images and save the result:

# Convert the mask image back to color

inverted_Bilateral = cv2.cvtColor(inverted_Bilateral, cv2.COLOR_GRAY2RGB)

# Combine the edge image and the binned image

cartoon_Bilateral = cv2.bitwise_and(inverted_Bilateral, img_bins) # Save the image

cv2.imwrite('CartoonImage.png', cartoon_Bilateral)