Create AI for Your Own Board Game From Scratch — Minimax — Part 2 | by Haryo Akbarianto Wibowo | Towards Data Science

Hi, How’s life? Welcome to the second part of the series of articles about project on making EvoPawness (Temporary Name) board game. Here, we will implement on how to add AI to to the game. In this article, we will focus on implementing some classic algorithms. It’s minimax and alpha beta pruning minimax. We will recap what we’ve done in the previous part and what I’ve done in the repository about the code of the board game (GitHub).
Before we dive on how to implement it, I will tell you that this series of articles will tell step by step on creating the board game. So, I will write about how I design and code the game, the GUI, and the AI as detail as possible. I want to share about my experience on doing it. So, I will do it gradually and tell my success and failed attempt on creating the AI.
We haven’t covered the GUI yet. Later, I will create the article about how to make and design the GUI. For now, we will skip the GUI and do the AI part first.
This article is targeted for the one who is interested in AI and designing a game. If you are not one of them, of course you can still see this article. I hope that my writing skill will increase by publishing this article and the content benefits you 😄.
RepositoryPrevious part recapEvaluation and Utility Function EngineeringMinimax Algorithm and ImplementationA-B Pruning Minimax Algorithm and ImplementationConclusionAfterwords
Repository
Previous part recap
Evaluation and Utility Function Engineering
Minimax Algorithm and Implementation
A-B Pruning Minimax Algorithm and Implementation
Conclusion
Afterwords
Here is the repository
github.com
I’ve added several changes to the repository
Balance the game. I’ve played several times with friends and the previous rule of the game is really unbalance.
Implement the GUI with PyQt5 library
Change the rule to make the game deterministic for the sake of this article. It will be changed on the next part.
Add Minimax, AB-Pruning Minimax, and random agent
Reformat the folder’s structure
If you want to play it in the Graphical User Interface (GUI) version, install PyQt5 with pip install PyQt5 command in your Command Line Interface (CLI).
In the previous part, we’ve defined some components to be used by AI Algorithm. We’ve stated Game’s Rule, State Representation, Result Function, Actions, Result Functions, and Terminal Test. We also have implemented them.
In this article, let’s make use the function that we’ve implemented before to make an agent that can choose the action by seeing opponent’s possible action.
Let’s recap, Utility Function Defines the final numeric value for a game when it’s in the terminal state for a player. The numeric value formula is defined by us. Evaluation Function Defines an estimate of the expected utility numeric value from a given state for a player. This function is called when the game hasn’t ended. The receive input of state and player and output a numeric score that represent the outcome and desirability of the state by the player. Negative score means that the player is in a disadvantage position.
There aren’t any rule to define the Evaluation and Utility Function. So, we need to think on how to formulate it. The function should differentiate whether the player is in an advantage situation or not.
This part is really important as this function will define the smartness of our agent.
Here’s the formula that I’ve came up with:
In the utility function, we will define the utility as high as you want and should be higher than evaluation function that we will define after this one.
We’ve stated in the previous part that our terminal state has 2 conditions:
When one of the king is deadWhen all of player’s pawns are dead
When one of the king is dead
When all of player’s pawns are dead
Here, if the terminal state is reached, it will return -120 (negative means disadvantage), 120 otherwise.
In the evaluation function, We will formulate it by considering the pawn’s hp, atk, step, and dead status. Every attributes is weighted by:
Weight of HP = 0.3Weight of ATK = 0.1Weight of Step = 0.1Weight of dead’s pawn = 10Weight of King’s HP = 1
Weight of HP = 0.3
Weight of ATK = 0.1
Weight of Step = 0.1
Weight of dead’s pawn = 10
Weight of King’s HP = 1
We give 10 weight to the dead’s pawn because it’s very bad on losing a single pawn in this game. So, we hope by give it a high weight, the AI will avoid dead of its pawn. King is really an important piece here. So, we will give a higher point to it.
We will define the utility and evaluation function in the AIElements.evaluation_function() function which calls evaluation function in class State.
Minimax is a decision rule which simulate the decision of a player to find the optimal move for the player. It assumes that the opponent will play with optimal choice too. There are two actors in the Minimax. It’s maximizer and minimizer. The maximizer will search the highest score as possible and the minimizer will search the lowest score as possible. Each players will see the opponent’s move best move and get the best one for him/her. Usually, the searching is represented in tree structure data.
Here’s the example how the Minimax algorithm
Look at Image 1, Max will output B,C,and D actions. First, the minimizer will see the possible action resulted in B . Get the lowest score min(3,12,8) = 3. Then do that to C and D. After that, the Maximizer will pick the highest score. Which is max(3,2,2) = 3 .
At our board game, the algorithm will not expand the state/node until terminal state. It will have memory issue or takes very long time to process. We should define a cut-off to our search algorithm by defining the max_depth that limit the node that should be expanded. It will call evaluation function that will calculate the heuristic of the condition.
Let’s implement it into our EvoPawness (Temporary Name):
We will define a class for Minimax algorithm that receive max_depth and player_color as its input. The player_color is a parameter to points out who is the maximizer here.
Let’s see the Minimax class:
As you can see, the Minimax Agent will do Depth First Search with recursive search. It will output the score (best_value) and the best action action key name. It will take the largest score if it’s the maximizer’s turn and take the smallest score if it’s the minimizer’s turn. It will search recursively until terminal state or maximum depth is reached.
We shuffle the list to make the agent choose action with the same score randomly.
If you are wondering the type of the action_target, it’s str type. Remember that our game action is a dict type which save the information of the action and the action key name. For example:
‘p1a2’: {‘action’: ‘activate’, ‘pawn_atk’: 1, ‘pawn_hp’: 5, ‘pawn_index’: 2, ‘pawn_step’: 1, ‘pawn_x’: 4, ‘pawn_y’: 1, ‘player_index’: 1},
We will play with the MinimaxAgent with depth = 4
Go to line 52 in controller/game_controller.py and change it into
self.ai_agent = MinimaxAgent(max_depth=4, player_color=0)
Run the game with your CLI
python main_gui.py
The AI will prioritize reaching the rune first. Then, activating its pawn. After that attacking our pawn first. If possible, the AI will evolve its pawn into higher type. Of course there’s some flaw such as let our pawn attacking their king. Maybe there is a better evaluation function that can make the AI protect the king.
If you’ve tried the AI, you will see that the consumed time on choosing the action is very long. Let’s see:
As we can see above, as the game progressed, the expanded_node is also increasing because of the number of possible actions. It tends to rise the time of the algorithm as the expanded node increasing. We can see the time on turn 24 is 21 second. Who wants to wait that long?
This is the weakness of this algorithm as it will search all of possible state until the desired cut-off or terminal state. Fortunately, it’s possible that we can effectively cut the expanded node. We will be using a technique called pruning to compute the correct Minimax decision without looking at every node. It’s called alpha-beta pruning.
Alpha Beta Pruning is an optimization technique for Minimax algorithm. This will cut the some nodes that should not be expanded because there is a better move already found. When applied to the Minimax algorithm, it will returns the same action as Minimax would, but it will be more faster. It’s called Alpha Beta Pruning because it needs 2 new parameters for the parameters called Alpha and Beta.
Alpha : the best value that maximizer can guarantee in the current state or before in the maximizer turn
Beta : the best value that minimizer can guarantee in the current state or before it in the minimizer turn
The general principle is : given a state s somewhere in the tree. If player has a choice of making action m which is found before s has been found, then s will never be reached.
Let’s take Image 1 as the example:
Remember the searching algorithm is using DFS (Depth First Search)!
Let’s dive in step by step:
Initialize Alpha to -inf and Beta to Inf. Alpha will be updated if its in the Maximizer turn and Beta is update if its Minimizer turn. They will be passed to the child of the state/node.Expand A (the initial state) possible action. It will produce B,C,D state. start from the first index (B)It’s minimizer B turn. Expand B possible action, go into the child and check its score. Update Beta as minimum as possible. We will find Beta = 3 = min(3,12,8,inf). Alpha is still -inf.It’s assumed that the depth is 2, return the minimum score (which is 3) to the A. Now it’s back to Maximizer turn A, update the Alpha = 3 = max(-inf,3). Then we go to C state/node, the second child:Now go to state C, the minimizer turn C with the passed Alpha parameters 3. In C, we find that the first child’s score that the C found is 2. Update the Beta = 2 = min(inf,2). Now, let’s think. Will the maximizer choose this action? The maximizer has already found better move, which is scored at 3. Of course not, if it’s optimal agent. So, don’t bother check another possible action in the C. This is the pruning process. Return the best score (it’s 2). To make it simpler if beta ≤ alpha then prune it.Maximizer Turn A: calculate max(alpha, 2) = alpha. Check the last child C.Minimizer Turn C : Calculate min(inf,14) , update beta to 14 and check if beta ≤ alpha (14 ≤ 3) and it’s not pruned. Calculate the next child. beta ≤ alpha (5 ≤ 3) and it’s not pruned. At last, check beta ≤ alpha (2 ≤ 3) it should be pruned. Since there are not any child after this node. No node that can be pruned. It can be pruned if the child that returned 2 is found at first. After this, return 2 to A.Maximizer Turn A, calculate max(alpha,2) = alpha. Then it’s done. return 3 as best score that has been found in the tree.
Initialize Alpha to -inf and Beta to Inf. Alpha will be updated if its in the Maximizer turn and Beta is update if its Minimizer turn. They will be passed to the child of the state/node.
Expand A (the initial state) possible action. It will produce B,C,D state. start from the first index (B)
It’s minimizer B turn. Expand B possible action, go into the child and check its score. Update Beta as minimum as possible. We will find Beta = 3 = min(3,12,8,inf). Alpha is still -inf.
It’s assumed that the depth is 2, return the minimum score (which is 3) to the A. Now it’s back to Maximizer turn A, update the Alpha = 3 = max(-inf,3). Then we go to C state/node, the second child:
Now go to state C, the minimizer turn C with the passed Alpha parameters 3. In C, we find that the first child’s score that the C found is 2. Update the Beta = 2 = min(inf,2). Now, let’s think. Will the maximizer choose this action? The maximizer has already found better move, which is scored at 3. Of course not, if it’s optimal agent. So, don’t bother check another possible action in the C. This is the pruning process. Return the best score (it’s 2). To make it simpler if beta ≤ alpha then prune it.
Maximizer Turn A: calculate max(alpha, 2) = alpha. Check the last child C.
Minimizer Turn C : Calculate min(inf,14) , update beta to 14 and check if beta ≤ alpha (14 ≤ 3) and it’s not pruned. Calculate the next child. beta ≤ alpha (5 ≤ 3) and it’s not pruned. At last, check beta ≤ alpha (2 ≤ 3) it should be pruned. Since there are not any child after this node. No node that can be pruned. It can be pruned if the child that returned 2 is found at first. After this, return 2 to A.
Maximizer Turn A, calculate max(alpha,2) = alpha. Then it’s done. return 3 as best score that has been found in the tree.
Implement it into the EvoPawness (Temporary Name).
This is the class for Minimax Alpha Beta Pruning:
(NOTE: the self.node_expanded is for DEBUG purpose)
We add alpha and beta as the parameter with default -inf and inf. Then we add these line:
For Maximizer
alpha = max(alpha, best_value)                                       if beta <= alpha:                                           break
And
For Minimizer
beta = min(beta, best_value)                                       if beta <= alpha:                                           break
That’s all, let’s test it 😃!
We will play with the MinimaxABAgent with depth = 4
Go to line 52 in controller/game_controller.py and change it into
self.ai_agent = MinimaxABAgent(max_depth=4, player_color=0)
The behavior is the same as the minimax , but faster. Let’s see the running time until turn 24.
We can see, the time spent is faster without using pruning technique. Until the end, I found that the worst run time on searching the best action is 17s with 2218 expanded node (Turn 66). It’s faster because of the pruning that remove unnecessary node.
Go on challenge the AI. The AI is very challenging. Sometimes it will target the king in the beginning. I lost once to the AI because of a sudden attack to the king and I didn’t prepare to defend the king 😢.
Even so, it’s winnable. The AI doesn’t defend the king. I think that it needs more depth in the tree to know it. It will cost longer run time. Well, No one would want to test his/her patience on playing with AI that takes its turn too long (you want to wait a minute or more to wait its turn?).
Minimax also does not consider the state other than the state that is found in the chosen depth. Minimax only cares about the state that is found in it. It’s impossible to search all the node to the terminal state from start as it will cause Memory Error. It’s computationally expensive (O(action^depth ) ) This is the weakness of the Minimax, even with the improvement using pruning technique.
We need some technique that consider the state other than the chosen depth and can also make the AI to choose the action faster. We will see it in the next part 😄.
We’ve created the Utility and Evaluation Function that is used by Minimax algorithm. Although the performance is good, the Minimax algorithm is so slow. To mend it, we use pruning to the algorithm. It’s called Alpha Beta Pruning. The pattern of the actions is same and it’s faster without using pruning. Even so, the Minimax Alpha Beta Pruning has its flaw. It can only consider and see the movement n turns onward that n needs to be small. If n is large, it will be computationally expensive to choose an action. We need some algorithm that consider further turns and need to be faster.
Thank you for reading my third article about Artificial Intelligence. I need some constructive feedback to make me better on writing and better knowledge about Artificial Intelligence. Go easy on me please 😆!
I just want to share my knowledge to the reader. It’s good to share knowledge as it can be helpful to someone in needs. I’m in the process of learning this field and want to share what I’ve learnt. It’s nice if someone tell me if there’s something wrong with this article.
Sorry for the bad User Experience (UX) on the GUI 😞. It also has messy and not enough documentation in the code. I hope I have the time to fix it. Maybe someone want to help me on fixing it?
Next part (Part 3) will move on into Reinforcement Learning. We will explore kinds of Reinforcement Learning one by one. Before that, I will create an article on how I design the GUI and show the structure of the class in this project. It will be Part 1.5. Hope that I can release the article on this Sunday.
We will experiment it on the Reinforcement Learning. I’m afraid that It will take a lot time to experiment it. I think, two weeks is the minimum interval that I can publish the articles. Someone want to join me experimenting it? Give me the spirit to write the next part ✌️.
If you want another article from me like this one, please clap this article 👏 👏. It will boost my spirit to write my next article. I promise to make a better article.
See you on the next article!
Part 1 : Create AI for Your Own Board Game From Scratch — Preparation — Part 1
Part 2 : Create AI for Your Own Board Game From Scratch — Minimax — Part 2
Part 3 : Create AI for your Own Board Game From Scratch — AlphaZero-Part 3
Russell, Stuart J., and Peter Norvig. Artificial Intelligence: a Modern Approach. Prentice-Hall, 2010.