Crack those CAPTCHAs with opencv and python (part 1)
By: Islem Bouzenia
Tags: Image Processing, Python, Captcha, Opencv

Have you ever felt like a bot after failing a captcha? Because I have!

Captchas are sometimes so annoying and waste a lot of time. I have been thinking about a way to solve a captcha automatically. The first thing that came to mind is to use deep learning. However, I didn’t find a data set which is large enough to train a model such as a CNN which requires a big amount of data. So, I decided to use opencv with python to solve this problem. Here are some examples of the captchas that we will be solving:

1. PREPARE THE ENVIRONMENT

To be able to follow the steps of this tutorial, you need to have python installed on your machine and also opencv-python. You can install the latter by running the following command:

Also, I will be using cv2 in this tutorial, for more details visit :

we will also need numpy and matplotlib, so please install them!!!

2. STARTING WITH A SIMPLE EXAMPLE

To be able to crack the the types of captchas I showed you before, I gathered some of them and split each one of them into letter. Thus, for each letter I got a bunch of pictures, here is an example:

Of course, I stored each picture of each letter in a separate file. Doing that, I would be able to use opencv feature matching. for example, with this captcha, I can look for the letters that matches with its letters.

Let’s try to match two variants of the letter ‘a’ with the letters of the previous captcha, the results are shown in figures below:

We looked for a match of the letter in the left, notice there are two matches in this captcha but it only took the nearest one.

In this one, there isn’t a strong match between the letter in the left and the letters of the captcha. Matching at one point isn’t enough to conclude an overall match.

So basically we will be using this technique to look for letters inside a captcha and the letter with the highest match will win.

3. IMPLEMENTATION

It’s time to write some code. First of all, import needed libraries:

Opencv uses different algorithms to detect edges, curves and key points of an object, those are called features. After computing the features of the letter we are looking for and computing all the features of the objects inside the captcha, the algorithm tries to match similar ones. For more details about this, visit the documentation page of opencv:

For our example we will use an algorithm called SIFT, and we will look for all matches between a specific letter and a whole captcha then keep only those who have a number of matching features greater than the minimum we fixed. The following code resumes all these steps.

The code might look complicated but this is a very typical code that will be used each time doing this task. With that, you are now able to reproduce all the experimentation we have done in the example.

4. WHAT’S NEXT?

In this first part we have seen how to match a letter with different letters of a captcha, we also gave a hint about the strategy we will be using to crack the entire captcha. Till now, this looks simple but a lot of work is yet to be done in order to create an effective captcha solver. We need to look for all possible letters, handle the case of rotation and scaling and also handle the case of multiple matches or no matches at all and that would be the goal of the next part.

See you in part 2 where we enhance this and developed it more!