Solve sudokus automatically with Python and PyAutoGUI | Towards Data Science

In this article, I will introduce a walkthrough on making your very own script that solves sudoku puzzles. Furthermore, this script is enhanced with PyAutoGUI to make sudoku solving quite automatic (you only have to execute the code).
I have divided the article in the following manner:
Introduction to sudoku — this brief section covers the basic rules that must be kept in mind;Sudoku solving algorithm — this section starts with defining necessary data structures and we will create an algorithm that would solve any given puzzle;Automation — it’s cool if your script knows the step-by-step solution to a puzzle. It’s even cooler if your code interacts with a web page and solves the sudoku automatically.
Introduction to sudoku — this brief section covers the basic rules that must be kept in mind;
Sudoku solving algorithm — this section starts with defining necessary data structures and we will create an algorithm that would solve any given puzzle;
Automation — it’s cool if your script knows the step-by-step solution to a puzzle. It’s even cooler if your code interacts with a web page and solves the sudoku automatically.
I hope you have installed Python, Pillow, and PyAutoGUI module. If you don’t have the PyAutoGUI yet then you should follow the installation steps HERE.
First things first. If you are not familiar with the rules of a sudoku puzzle, don’t be frightened. They are quite simple:
You must fill the boxes of a 9x9 grid with numbers from 1 to 9 in a way that there are no recurring numbers in each row, column, and 3x3 block.
If you take a look at the image, the only number that fits into the top left box is two since it’s the only one that doesn’t appear in this row, column and box.
Let’s take a random sudoku puzzle from the Internet and try to solve it with Python. I propose that we handle this puzzle as a two-dimensional Python array where empty boxes are represented with zeros and other boxes with corresponding numbers. We should obtain the following:
sudoku =    [[8, 1, 0, 0, 3, 0, 0, 2, 7],             [0, 6, 2, 0, 5, 0, 0, 9, 0],             [0, 7, 0, 0, 0, 0, 0, 0, 0],             [0, 9, 0, 6, 0, 0, 1, 0, 0],             [1, 0, 0, 0, 2, 0, 0, 0, 4],             [0, 0, 8, 0, 0, 5, 0, 7, 0],             [0, 0, 0, 0, 0, 0, 0, 8, 0],             [0, 2, 0, 0, 1, 0, 7, 5, 0],             [3, 8, 0, 0, 7, 0, 0, 4, 2]]
When trying to access a certain element from the i-th row and j-th column, you simply need to call
sudoku[i][j]
Sure, this two-dimensional python array does not look like real sudoku without any gridlines. Hence, it would be great to print out the current state of the puzzle. Below is the code example along with its output.
def printsudoku():    print("\n\n\n\n\n")    for i in range(len(sudoku)):        line = ""        if i == 3 or i == 6:            print("---------------------")        for j in range(len(sudoku[i])):            if j == 3 or j == 6:                line += "| "            line += str(sudoku[i][j])+" "        print(line)
Let’s get to it! Now we are ready for a step-by-step algorithm-creation.
Disclaimer: I did not create this sudoku-solving algorithm entirely myself. The idea and implementation were copied from MIT OpenCourseWare Youtube channel, more specifically from this video. Yet, it’s licensed as Creative Commons BY-NC-SA which gives me a green light for sharing.
To solve a certain cell, we must first find the row and column number of a cell that’s empty. The following function does the trick.
def findNextCellToFill(sudoku):    for x in range(9):        for y in range(9):            if sudoku[x][y] == 0:                return x, y    return -1, -1
This iterates through all the columns in the first row, then second row and so on. Whenever it encounters an empty cell, the function returns the corresponding indexes. Simple!
Suppose we have an entry e. We need a function that checks whether it violates the three main rules of sudoku when placed to the i-th row and j-th column. The following function returns True if none of the three rules are violated. Otherwise, the function will return False.
def isValid(sudoku, i, j, e):    rowOk = all([e != sudoku[i][x] for x in range(9)])    if rowOk:        columnOk = all([e != sudoku[x][j] for x in range(9)])        if columnOk:            secTopX, secTopY = 3*(i//3), 3*(j//3)            for x in range(secTopX, secTopX+3):                for y in range(secTopY, secTopY+3):                    if sudoku[x][y] == e:                        return False            return True    return False
Three rules are examined:
rowOk: this one checks if there are no repeating numbers in the i-th row;columnOk: this one makes sure if there are no repeating numbers in the j-th column;If both rowOk and columnOk are True then the lines 6–10 check if the entry fits a certain block.
rowOk: this one checks if there are no repeating numbers in the i-th row;
columnOk: this one makes sure if there are no repeating numbers in the j-th column;
If both rowOk and columnOk are True then the lines 6–10 check if the entry fits a certain block.
Now it’s all about solving the given sudoku. We’ll do it with the following function:
def solveSudoku(sudoku, i=0, j=0):    i, j = findNextCellToFill(sudoku)    if i == -1:        return True    for e in range(1, 10):        if isValid(sudoku, i, j, e):            sudoku[i][j] = e            if solveSudoku(sudoku, i, j):                return True            sudoku[i][j] = 0    return False
The first three lines of the function make sure that we have an empty cell in our puzzle. If i is equal to -1 then we have completed solving our puzzle.
If there is an empty cell at i-th row and j-th column then it will try to fit all the possible entries from 1 to 9 into this box. If the entry is valid (verified by the isValid function) then tries to solve the sudoku recursively under the assumption that this is a correct entry.
If our assumption was incorrect then the 9-th line would return False and that sets the value of a cell in i-th row and j-th column to zero.
To solve our sudoku, make the following function calls:
solveSudoku(sudoku)printsudoku()
The first one solves our puzzle and the latter shows the solution. Great!
The automation part is divided into two sections:
Locating the numbers on a sudoku grid. This way we can solve any sudoku without having to fill the sudoku variable manually.Filling the empty cells with numbers.
Locating the numbers on a sudoku grid. This way we can solve any sudoku without having to fill the sudoku variable manually.
Filling the empty cells with numbers.
We are doing our automation by using the PyAutoGUI module. Please import it in your code.
import pyautogui as pag
We are about to solve a random sudoku puzzle from Websudoku. Open a random puzzle and save image snippets from every number. They all should be saved to the same folder as your Python script. Make sure that the file format is .png.
The snippets may have different dimensions, as long as they all feature only a number on a white background. Let’s name them as “1.png”, “2.png”, etc.
The PyAutoGUI module has a function that locates the given image on your screen. In my case
for pos in pag.locateOnScreen("3.png"):    print(pos)
Returns all locations of number three on our screen:
Box(left=1124, top=510, width=32, height=33)Box(left=959, top=845, width=32, height=33)
The first two parameters (left and top) give us the pixel coordinates of the top-left corner of our image. The third and fourth give us the dimensions of our image but at the moment we are not going to need them.
To get the locations of all the numbers, use the following:
for i in range(1, 10):    for pos in pag.locateAllOnScreen(str(i)+'.png'):        print(pos)
Since we are not going to use our previous sudoku array, let’s change all the numbers in this array to zero:
sudoku =    [[0, 0, 0, 0, 0, 0, 0, 0, 0],             [0, 0, 0, 0, 0, 0, 0, 0, 0],             [0, 0, 0, 0, 0, 0, 0, 0, 0],             [0, 0, 0, 0, 0, 0, 0, 0, 0],             [0, 0, 0, 0, 0, 0, 0, 0, 0],             [0, 0, 0, 0, 0, 0, 0, 0, 0],             [0, 0, 0, 0, 0, 0, 0, 0, 0],             [0, 0, 0, 0, 0, 0, 0, 0, 0],             [0, 0, 0, 0, 0, 0, 0, 0, 0]]
Now let’s define the pixel coordinates of two cells on our puzzle: top-left and bottom-right. In my case the numbers were:
topleftx = 975toplefty = 507bottomrightx = 1307bottomrighty = 846
Now that we have them, let’s define the width and height of a box:
boxwidth = (bottomrightx - topleftx)/8boxheight = (bottomrighty - toplefty)/8
Explanation: when looking at a row of cells, the distance from the leftmost to the rightmost cell is 8 box widths. The same goes for columns. Hence we must divide the coordinate difference by 8.
Now we have to define a function that replaces zeros in sudoku array with the right numbers.
def fillsudoku(nr, pos):    global sudoku    indexlocx = int((pos[0] - topleftx + boxwidth/2)//boxwidth)    indexlocy = int((pos[1] - toplefty + boxheight/2)//boxwidth)    sudoku[indexlocy][indexlocx] = nr
I’m not going to explain the math but it takes a Box object and finds the “index-coordinates” of our number. If you have done everything correctly so far then it will work perfectly.
Finally, let’s fill the array with information on our screen:
for i in range(1, 10):    for pos in pag.locateAllOnScreen(str(i)+'.png'):        fillsudoku(i, pos)
If you aren’t quick enough to jump from your IDE to web browser then add the following delay method to the beginning of your code:
import timetime.sleep(5)  #this makes a 5 second delay to executing the script
Great! Now we have a method that uses screen information for filling our sudoku array. Since we have our solving algorithm then we can solve this instantly. We only need the following function to do so:
def fillcell(nr, x, y):    xcoord = topleftx + boxwidth * x    ycoord = toplefty + boxheight * y    pag.click(xcoord, ycoord)    pag.press(str(nr))
Here we have three input parameters:
nr: this is the number we want to insert;
x: column index of the cell we are going to insert nr;
y: row index of the cell we are going to insert nr;
Lines 2–3 find the approximate coordinates of the midpoint of the cell we must insert our number. Lines 4–5 click on that cell and write that number.
Oh yes! One more thing. Before solving the sudoku, please create a copy of the sudoku array. I’ll explain later.
import copysudokucopy = copy.deepcopy(sudoku)
To be on the same line, I’ll embed a Github gist below:
So far we read the sudoku from our screen and solve it. We also have two identical arrays of our sudoku before trying to solve it: sudoku and sudokucopy.
The sudokucopy was made to compare the initial and final state of our puzzle and fill the empty cells.
for x in range(9):    for y in range(9):        if sudokucopy[x][y] == 0:            fillcell(sudoku[x][y], y, x)
Now it’s ready for execution and if your configurations are correct, then it should look like this:
Of course, some improvements can be made:
It takes me approximately 20 seconds to load the data from the screen to the sudoku array. You can use some different modules (or create your own) to make it faster.Modify the sudoku solving algorithm to make the automated solving part look a bit more like a human would think. No one in real life solves the puzzles as shown in the animation above.
It takes me approximately 20 seconds to load the data from the screen to the sudoku array. You can use some different modules (or create your own) to make it faster.
Modify the sudoku solving algorithm to make the automated solving part look a bit more like a human would think. No one in real life solves the puzzles as shown in the animation above.
Feel free to modify this implementation and test it on different puzzles (even the most extreme ones).
I’d like to thank you if you read the entire article. If you have any thoughts or have modified the code into something better, let me know about it in the comments section. :)