Beating the world record in Tetris (GB) with genetics algorithm | by Duc Anh Bui | Towards Data Science

A couple of weeks ago, I found this awesome Game Boy emulator written in Python with support for AI/bots. I was very excited because now I could use my Machine Learning knowledge to create agents for games that haunted my childhood.
In this article, I’ll show how we can use a simple implementation of genetics algorithm and neural network to create an agent that can play the original Tetris on Game Boy extremely well and able to beat the world record which is set at 999999 and 523 lines according to TwinGalaxies.
Tetris, one of the best selling video game franchises, is a puzzle game where the main goal is to reach the highest possible score by clearing lines of Tetrominoes (yes, that’s what those pieces are called). The rules of the game are very simple, yet, it’s actually a NP-complete problem [1]. This means coding a rules based agent might not be the wisest approach and heuristics, such as a genetics algorithm, would be a better fit.
Genetics algorithms (GA) are a set of algorithms that are inspired by Darwin’s theory of evolution. These algorithms are used to solve optimisation problems. This means we can use GA to optimise the weights of a neural network (and they can do that pretty well too). I won’t go into what is a neural network in this article as there’re plenty of resources out there for it already, such as here.
The general idea is that we’ll create a neural network (NN) that will produce a score for each action the agent does. For every new tetromino, the agent will try all possible actions it could do and choose the action with the highest score. Using GA, we wish to optimise the network to converge to an optimal state via evolution.
In GA, we have a number of key concepts: fitness, elitism, crossover, and mutation.
Fitness is how we measure the ability of the network to correlate its output with how good the move is.
Elitism is when a percentage of top performing networks are brought over to the new generation.
Crossover is the process of creating a new network using the genes (weights) of two other networks.
Mutation involves randomly modifying the weights of a network.
The steps are as follows:
Generate a population of neural networks with randomised initial weights.For each network, play until the game is over.Calculate the fitness score of each network.Bring the top x performing networks over to the next set of the population (elitism).Randomly select 2 networks (parents) from the current population and perform crossover to produce a new network (children).Mutating the new network weights with a probability of x%.Continue until we reach the number of generations wanted.
Generate a population of neural networks with randomised initial weights.
For each network, play until the game is over.
Calculate the fitness score of each network.
Bring the top x performing networks over to the next set of the population (elitism).
Randomly select 2 networks (parents) from the current population and perform crossover to produce a new network (children).
Mutating the new network weights with a probability of x%.
Continue until we reach the number of generations wanted.
Like many other Machine Learning problems, the first step we need to do is collect data, or in this case, state data from the environment. Here’s a quick example that shows how to quickly initialise the environment.
The script itself is pretty self-explanatory. The developers have made it very easy to get a captured state of the game via tetris.game_area() . We can then create a sparse matrix where 0s are the blank tiles and 1s are the pieces:
area = np.asarray(tetris.game_area())area = (area != 47).astype(np.int16)
Using the matrix above, we can define a number of features as seen in [2]:
Aggregate HeightThis calculates a sum of all the column heights. In the example above, the sum would be 16.
Number of holesThe number of holes on the board. There are 6 holes in the board below:
Number of columns with at least one holeThat would be 5 in the example above.
BumpinessThe sum of absolute height differences between the columns. In the board below, it’s: 1 + 0 + 2 + 2 + 0 + 0 + 1 + 0 + 0 = 7
Row transitionsThe number of transitions from occupied to unoccupied tiles in each row summed. Each red dot in the picture below counts as one transition:
Column transitionsLike above but column wise.
Number of pitsA pit is defined as a column without any blocks. There are 3 pits in the board below:
Deepest wellA well is best explained via the image below. In this case, the deepest well has a height of 2.
Lines clearedThe number of lines cleared with the move.
Let’s implement these features using the matrix representation:
Now, let’s implement the algorithm:
First, we create a class for the Network
Our model is a simple network with an input layer with size 9 and an output layer with size 1. This simple network will allow us to interpret the weights easily and can be represented as follows:
I’m using PyTorch to create the network but feel free to use any other library or implement your own. Using this, we can create a Population class with crossover and mutation implemented:
For each new generation, we’ll bring over the top 20% networks.
For crossover, we sample parents with probability distribution generated from population fitnesses (the higher a Network fitness it the more likely it will be chosen). Each weight is chosen randomly with a 50/50 chance from either parent.
Afterwards, weights have 20% chance of being mutated with the addition of random Gaussian noise multiplied by a factor of 0.5.
The next step is to code some logic to play the game. First, let’s define some helper functions to move the blocks:
For each new piece, we’ll try all the possible moves with that piece. The number of turns and sideways moves we need to check is dependant on the block type. We can use these 3 functions to determine them:
Now we have all we needed to start simulating the networks. For each new piece, we pass the board information to get_score() to get the output of the network for each possible action. We choose the action with the highest output. Once the game is over, we return the fitness of the network which is the score of the game.
Due to the randomness of the game, the network won’t produce the same result for each run. We could evaluate multiple runs for each network via run_per_child to better approximate the network fitness.
As the agent gets better at the game, training time can get much longer. To speed training up, we stop the game once the agent has reached the maximum score (999999) and utilise multiprocessing. It’s simple enough to spawn multiple processes to evaluate networks parallelly like so:
And there you have it, we have implemented a genetic algorithm to create a simple agent that can play Tetris. Here’s an example run with the best model obtained after just 10 generations with a population size of 50:
Scores: [999999.0, 999999.0, 729892.0, 372490.0, 999999.0, 296117.0, 299328.0, 533870.0, 282609.0, 525193.0]Average: 603949.6Lines: [1031, 1002, 825, 415, 935, 393, 364, 622, 389, 582]Average: 655.8
There’re definitely some limitations to the agent due to its simplicity, however. As the agent only takes into account of present reward and disregards future rewards, it’s very likely to get into situations where placing a block gets lines cleared but worsen the board positioning. In addition, due to the small board size (18x10), if the agent gets multiple bad blocks in a row, it’s very likely to lose. These lead to a very high standard deviation between runs like the example above where, in the best run, the agent clear 1031 lines and 364 lines in the worst one.
And that’s it, I hope you had fun reading my approach to this project. Feel free to check out the full source code available at and make any improvements as you wish: https://github.com/uiucanh/tetris
References:
[1] Erik D. Demaine, Susan Hohenberger, David Liben-Nowell. Tetris is Hard, Even to Approximate.
[2] Renan Samuel da Silva, Rafael Stubs Parpinelli. Playing the Original Game Boy Tetris Using a Real Coded Genetic Algorithm