A simple, but strong AI bot for Tron/Lightriders on a hexagonal grid | by Gilles Vandewiele | Towards Data Science

A new academic year, which means a new batch of bright students taking their first steps in their engineering career. As last year, three of my fellow course assistants (Elias, Pieter and Ozan) and I created a platform for an AI bot competition, to be hosted during the course of Informatics (in which they learn to program in Python), given by prof. Dhoedt at Ghent University. This year, we decided to host the game of Tron (sometimes called continuous snake or Lightriders), as shown below.
Since both Riddles.io and CodinGame (which you should definitely check out if you are interested in AI bots!) host an AI competition for Tron as well, we decided to slightly adapt the game to be played on a hexagonal grid, instead of a square. This to avoid students just copy-pasting available solutions of those platforms. A screenshot of the game can be found below.
As last year, I decided to compete in the competition myself, but did not find that much spare time. Hence, my solution wasn’t a very strong one. Nevertheless, I ended up pretty high on the leaderboard (5th spot), with a minimal code base and a simple heuristic, based on path finding.
To discuss my solution, I will not need as much time/lines as last year, and therefore I decided to add some extra content. I split this blog post up in several sections:
Game and competition rulesA detailed overview of my strong heuristic (with code)A write-up by some of the top competitorsAn overview of how the competition platform was made (separate post)
Game and competition rules
A detailed overview of my strong heuristic (with code)
A write-up by some of the top competitors
An overview of how the competition platform was made (separate post)
The rules of the game are rather straight-forward. Both players start at a random position (mirrored around the center of the map) and with a random orientation (there are 6 orientations in total). Each player must then decide if they want to rotate their snake (120° or 60° CW/CCW or 0°), before making a move forward (as depicted in the figure below). Each snake leaves behind an impenetrable trail. The first snake that crashes into a wall or a trail of himself or opponent, loses the game.
The submitted agents have to follow a predefined interface:
def generate_move(board, positions, orientations):  """Generate a move.     Args:        board (np.array):    playing field        positions (list):    list of positions `(x, y)`        orientations (list): list of orientations (int in [0,5])     Returns:        move (int):          integer in [-2,2]    """  return 0
Moreover, each agent must comply to the following constraints:
It contains no import statements, with the exception of: numpy, functools, itertools, collections, operator, heapq, math, random, time
The file size must not be larger than 1 Megabyte
The execution time of generate_move may not exceed 1 second
After submitting your first agent to our platform, you will be matched up with another player every 30 minutes. After each match, rankings (TrueSkill or ELO) of both players are adjusted according to the match result, and these rankings are used for the leaderboard. For more information on the ranking systems, please refer to my next post.
The competition lasted for about 1 month. We noticed that luck had a significant impact on your final rank (due to the matchmaking algorithm). To combat this, and to end up with an as objective ranking as possible, we took the top-competitors from the leaderboard after a certain amount of time (first-year students only). These competitors were chosen to compete in the “play-offs”, where each agent played all other agents four times. For each win, an agent was assigned three points, while a draw (of which there were many) resulted in both agents being awarded a single point.
Before implementing more complex techniques, such as reinforcement learning, genetic algorithms and alpha-beta pruning, it is always good practice to first implement an agent based on a heuristic. These can be used to test the strengths of your opponents, but more importantly the strength of later versions of your agent. Unfortunately, I did not manage to implement more complex techniques this year due to a lack of time and only uploaded my baseline agent, based on a heuristic (which turned out to be very strong). I will now discuss my implementation thoroughly.
The main reasoning behind my agent is rather simple: “always pick the move which results in a maximal difference between the number of tiles you can reach first, and the number of tiles your opponent can reach first.” Obviously, there will be many situations in which each possible move results in the same score (especially in the beginning of the game). We break ties by minimizing the number of remaining legal moves after making a certain move and by minimizing the distance towards the middle (if it still can be reached). The main game loop thus looks as follows:
Now let’s break everything down, starting from (probably) the most simple part in the code: getting all legal moves (get_legal_moves). In this function, we iterate over each integer in the interval [-2, 2] which are the five possible moves. We map each integer to a tuple of offsets (get_steps) that correspond to our move and update our positions and orientations hypothetically (_update_pos_orient) according to these offsets. Then, we calculate whether we still reside in the hexagonal grid (_valid_position) and whether we did not crash into an already occupied cell (_no_crash):
The code is rather straight-forward and probably does not need a lot of elaboration. To check whether we still reside in the hexagonal grid, we have to ensure that we do not get an IndexError by checking if our x and y coordinates do not go out of the bounds of our playing fields. Moreover, since the board representation is a square with some cells that are not reachable, we need to ensure that our agent does not move onto one of these cells. By inspecting the smaller (7x7) playing field depicted on the homepage we notice that the sum of x and y of the legal cells is bounded by 3 and 9.
This generalizes to all playing field sizes and we can thus say that:
with k the size of the playing field. Moreover, to check whether we are not moving on an already occupied cell, we can just take the sum along the final axis of our playing board (remember that the final axis of this board contains a bitmap for each player) and check whether the value at the coordinate we want to move to is equal to 0. Further, the order of the offsets in _get_steps is order specifically such that we can directly index them using our newly calculated orientation, which is equal to (orientation + action) % 6. Using this new orientation, we can get the corresponding offsets and add these to our current coordinates to calculate our new coordinate.
Now that we have a list of all the moves that will not result in us immediately losing, we can calculate a score for each of these moves, based on a heuristic. The heuristic we used is based on distances to each possible tile in the game field. We want to maximize the number of tiles we can reach in less moves than our opponent while minimizing the number of tiles he or she can reach first. A nice GIF, taken from the winning solution’s write-up in the Google Tron AI Challenge is displayed below.
To calculate the minimal distance from a certain coordinate to another cell, we can use the A* path-finding algorithm or Dijkstra. But since we need to calculate a distance to all cells in our grid, we use a slightly adapted version. We keep track of a list of cells that still need to be visited (and the distances to each of these cells), called a frontier. For each cell in our frontier, we expand in all possible directions and add these new cells to our frontier (with a distance equal to our old distance + 1) if they have not yet been explored or if we discovered a path with a smaller amount of moves. Once we have calculated the distances to all cells for both players, we can easily calculate the number of tiles both players can reach first:
And that’s it! It’s that simple. In summary, we retrieve a list of all possible legal moves we can make, we then iterate over these legal moves and calculate a score for each of them. The score is based on the number of tiles we can reach first from the new position after making that move and the number of tiles our opponent can reach first. There are a few extra details, such as subtracting a penalty term from the score if we are too close to our opponent to avoid head-to-head crashes and the handling in case of ties (which are broken based on the number of legal moves remaining to fill up the board and based on the distance to the middle, since that is important in the start of the game).
As mentioned, only a limited amount of time was spent this year on writing my agent. Therefore, many possible improvements are left open. The most important ones are listed here:
An adaptation of the heuristic function could result in significant gains. According to the write-up of the Google Tron AI Challenge this resulted in the author winning the competition. In his blog post, a nice example game state is provided, where the heuristic function used for my agent would assign a negative score while we (again we are red) are actually on the winning hand:
Instead of calculating a score for each possible move, we could calculate a score for each possible sequence of moves, thus looking ahead more than 1 move. To solve this, we usually apply techniques such as alpha-beta pruning, but a few extra considerations will need to made since both players move simultaneously. One simple hack is to first retrieve all legal moves for your opponent, before iterating over all your legal moves (changing the game state) and then recursively call your opponent, passing along those moves. This ensures that both players can calculate their legal moves, using the same game state during the recursion.
Calculating the score for each possible move (or sequence of moves) using Monte Carlo simulations, instead of using a heuristic function. Here, we would simulate a large number of games starting from a certain game state where each agent would make a random, but legal, move. The fraction of games we win from these simulations could be a good indication of the strength of a certain game state. Of course, it should be noted that the maximum time to make a move is only one second. I have not tested how many simulations one could make within that second.
Evolving neural networks to play the game for you, is a very generic strategy that requires no knowledge of the game itself. It would require quite a lot of work to write a framework for this (or configure an already existing one), but once it is written (or get the hang of the existing framework), it could easily be adapted for other games. I found a lot of examples for the single-player snake game (e.g. here and here) but only a very small amount for continuous snake or tron (e.g. here).
Reading the write-ups of the other top competitors, and combining the most important insights from all to create a superbot.
Below are write-ups of some of the solutions of students that were ranked higher than my agent in the end:
The bot is a result of all its predecessors. Instead of writing new code all over again each time I made a lot of meta-functions. A lot of smaller functions can always be recycled to the next generation. These functions range from determining whether a move is legal, to calculating the number of available cells and distances to these cells. But combining all these functions can turn into something rather complex.
The code first calculates if the opponent is in the same field as him, if he isn’t he takes the most left moves with the maximal number of available cells (floodfill). If they are in the same field it’s all about having the best floodfill in the future. It starts by going to the center (6, 6). If the snakes start at the border he goes as wide as possible to try and trick the opponent, otherwise it takes the most central route to try to cut the field in equal halves. The center of the playing field is a very important position. I made a special function for this move. He looks which positions could give him advantage without giving the opponent advantage. If he doesn’t find a such position he moves towards the center, resulting in many possible options.
Once the bots are done fighting for center, I’ve defined a functions that simulates the field if my snake goes straight in every possible direction. It analyses if my bot can take advantage from there (bigger floodfill) and if distances to that spot is shorter. It analyses situations as ‘success’, ‘maybe draw’ or ‘success but longer distance’. For the moves I could make he also runs that function as if he was to opponent and doesn’t do moves that could result in advantage for the opponent. This can take a while, so it stops doing this after 0.7 seconds and it just assumes the opponent won’t have advantage. If no success moves are possible he tries to stay as close to center and the enemy as possible.
At the beginning of every turn, the agent generates several variables based on the data it receives, some of them being 2D arrays with their length and width equal to the length and width of the playing board, others being single-element integers or Booleans. These variables serve as input for eventually deciding which hexagon adjacent to the agent is the most suitable to go to next, and consequently how the agent should rotate to reach that hexagon, which is the output.
The aforementioned variables generated by the agent include:
* a 2D array where each element that corresponds to a hexagon refers to the number of adjacent hexagons that are not occupied by one of both agents or their trails
* a 2D array where each element that corresponds to a hexagon refers to the shortest possible distance from that hexagon to the edge of the playing board (expressed in hexagons)
* a Boolean that indicates whether both agents can still reach each other
After calculating these variables, the agent assigns weights to the 2D arrays based on the values of the single-element variables. The 2D arrays are multiplied by their respective weights, and subsequently added together to construct a final 2D array. Each element in this final array that corresponds to a hexagon on the board has been assigned a value that is comparatively high if the agent estimates a comparatively high chance of winning the game by moving to that hexagon, and comparatively low if it estimates a comparatively low chance. Finally, the agent will check every hexagon around its position for the hexagon with the highest corresponding value in the final array and will output the rotation necessary to reach that hexagon.
The final agent has two major versions, v8 and v10, where v10 has almost the exact same strategy as v8 except it’s much faster because it represents the board as a graph (which I’ll explain later).
Both agents have a large set of utility functions that allow to write code much faster because these can be copied between versions since their implementation stays the same (e.g. is position x playable? How many cells can the agent reach from position x? What is the distance between position x and y? What is the closest position in set s to position x?).
The rest of the code consists of algorithms to determine the best move. Most of these algorithms filter the moves that are playable and passes them on to another algorithm to break ties. The first algorithm determines if we must decide between two or more areas, and only passes the moves that lead to the larger area. Then the agent checks if we can reach the opponent, if that’s not possible, we can just fill the remaining area with a filler algorithm.
In all other cases it’s almost always optimal to go to the center of the board. If the agent is about to go to the center of the board, it first checks if the opponent can reach the center too. If he can, then the agent can possibly cut off the opponent by moving slightly to the left or slightly to the right, but only if the start positions were at the edge of the board and if cutting off results in a larger area than the area of the opponent.
If none of the above special cases occurred, the agent falls back on its general flooding algorithm, which makes the move with results in the most cells that it can reach first. If the opponent can reach a cell at the same time as the agent, it counts as half a cell.
If the flooding results in equal scores for several moves, it passes those moves to the multi ray casting algorithm, which is a rather old algorithm (v6) that casts rays from the position of both players and determines which move results in the largest area.
The biggest difference between v8 and v10 lies in the filler algorithm. While v8 only casts single rays from the players position to determine the largest possible area, v10 composes a graph from the grid. This is done by mapping every position in the grid to a number from 0 till 126. Because ray casting could still be useful, I had to define a direction for the nodes in the graph, so I defined a 127x5 grid where get_next_nodes()[node][direction % 6] would give me the node the agent would reach if it went in that direction. Then a rather complex convert function would turn the grid into an adjacency list. Instead of ray casting, the agent could now determine articulation points in the graph. The agent can now fill the area by making the move that results in the least articulation points in the next two moves. This results in a perfect fill of any area.
IMPORTANT: Maxim published the code on his GitHub. Be sure to check out the winning solution of the competition. You can even simulate games between mine and his agent!
All code for the agent discussed within this blog post is available on GitHub. I did my best to clean and optimize the code as much as possible, but it is definitely not yet perfect. Therefore, submitting a pull request or opening issues is more than welcome!
Moreover, as always, if anything in this blog post was unclear or requires further clarification, feel free to leave a comment, or contact me! If you have any cool suggestions for a game for next year, please feel free to leave a comment as well!
Wow, you reached the end of this blog post! Kudos to you. See you next year for the third AI competition ;),Gilles