Randomised fast no‐loss expert system to play tic‐tac‐toe like a human

Cognitive Computation and SystemsVolume 2, Issue 4 p. 231-241 Research ArticleOpen Access Randomised fast no-loss expert system to play tic-tac-toe like a human Aditya Jyoti Paul, Corresponding Author aditya_jyoti@srmuniv.edu.in orcid.org/0000-0002-4351-2108 Cognitive Applications Research Lab, India Department of Computer Science and Engineering, SRM Institute of Science and Technology, Kattankulathur, Tamil Nadu, 603203 IndiaSearch for more papers by this author Aditya Jyoti Paul, Corresponding Author aditya_jyoti@srmuniv.edu.in orcid.org/0000-0002-4351-2108 Cognitive Applications Research Lab, India Department of Computer Science and Engineering, SRM Institute of Science and Technology, Kattankulathur, Tamil Nadu, 603203 IndiaSearch for more papers by this author First published: 09 November 2020 https://doi.org/10.1049/ccs.2020.0018AboutSectionsPDF ToolsRequest permissionExport citationAdd to favoritesTrack citation ShareShare Give accessShare full text accessShare full-text accessPlease review our Terms and Conditions of Use and check box below to share full-text version of article.I have read and accept the Wiley Online Library Terms and Conditions of UseShareable LinkUse the link below to share a full-text version of this article with your friends and colleagues. Learn more.Copy URL Share a linkShare onEmailFacebookTwitterLinked InRedditWechat Abstract This study introduces a blazingly fast, no-loss expert system for tic-tac-toe using decision trees called T3DT, which tries to emulate human gameplay as closely as possible. It does not make use of any brute force, minimax, or evolutionary techniques, but is still always unbeatable. To make the gameplay more human-like, randomisation is prioritised and T3DT randomly chooses one of the multiple optimal moves at each step. Since it does not need to analyse the complete game tree at any point, T3DT is exceptionally faster than any brute force or minimax algorithm, this has been shown theoretically as well as empirically from clock-time analyses in this study. T3DT also does not need the data sets or the time to train an evolutionary model, making it a practical no-loss approach to play tic-tac-toe. 1 Introduction Tic-tac-toe is one of the oldest and most played childhood games, dating back as early as 1300BC in Egypt. The board comprises a 3 × 3 matrix; two players alternatively place an X or an O in an empty cell and the first player to get three X's or O's in a row, column, or diagonal wins. It is a zero-sum game in which one player's win is the other's loss. It also is a full information game that is both the players have complete information about all the moves that are possible arising from a certain board state. Over the ages, people have come up with tricks or techniques to play it without being beaten. The operations in decreasing priority order are as follows: (a) making a winning move, else (b) blocking the opponent's winning move, else (c) trying to make a fork, and lastly (d) blocking an opponent's fork. The main objective of this research work is the creation of a no-loss expert system that can play tic-tac-toe by understanding the rules of the game and maintain a high win-to-draw ratio. Here no-loss refers to the game never losing against an opponent, it would draw or preferably win the game, irrespective of whether it starts first or second. It should be able to execute the operations listed above just like a human would, so a decision-tree based algorithm (T3DT) was chosen. Humans' perception of games is fuzzy and chaotic in nature and the randomness hence produced makes them fun to be played with. T3DT tries to recreate the randomness to the maximum extent possible, thus making the algorithm a fun opponent to play with. 2 Literature review Many heuristics and mathematical algorithms have been designed to win the game. In the field of two-player zero-sum games, pioneering work was done by John von Neumann, his first proof [1] of the minimax (MM) algorithm published in 1928 created ripples through the scientific and mathematical communities and is still considered one of the founding blocks of game theory. Kjeldsen [2] iterated the history of the development of Neumann's MM from his paper [1] in 1928 to his completely different proof in his book [3] with Morgenstern in 1944. In [2], Kjeldsen wrote that by considering mixed strategies and expressing player values in the bilinear form h, Neumann [1] had shown that for two-player zero-sum games, there always exists optimal mixed strategies ξ0, η0, such that (1)Kjeldsen [2] wrote that Neuman had actually proven a generalised version of MM considering a class of functions broader than the bilinear form h. However, for simpler two-player zero-sum games such as tic-tac-toe, the essence of this equation lies in the observation that the optimal strategy for both players involves pessimistically minimising the maximum damage that can be inflicted by the opponent. MM surprised people by providing an algorithm which can never be beaten, and most of the earlier approaches for tic-tac-toe relied on some form of game tree search. Claude Shannon realised the importance of variable depth search in 1949 [4], and best-first approaches with both fixed [5] and variable depth [6] have been implemented since. In general, much of the early research on game-tree search algorithms such as alpha-beta pruning [7-9], Scout [10], NegaScout [11], SSS* [12-15], fixed and dynamic node ordering [16] and aspiration windows [17, 18] make the same choice as full-width fixed depth MM. On the other hand, algorithms such as B* [19, 20], min–max approximation [21], conspiracy search [22, 23], meta-greedy search [24], singular extensions [25], and risk assessment [26] might not always do so, searching some strategies more deeply than others. Improved hardware and algorithms supporting parallelism [27-30] have made tree generation and search even faster. Here, just a cursory description of game tree search methods has been given as a detailed account is outside the scope of this paper, but [31-33] compare and contrast these methods in much greater detail for a clearer understanding. Ignoring the symmetry of the board, there are 255,168 unique games of tic-tac-toe [34], and move-generation and analysis of all these moves at runtime are tedious and time-consuming. Humans are skilled at simplifying the search process by pruning the options at hand by having an intrinsic understanding of the game and selecting a few main candidates [35]. This task is challenging for computers and thus other approaches that do not rely on game tree search were developed and have been described below. Fogel [36] laid the groundwork for using evolution in developing neural agents to play tic-tac-toe. Chellapilla and Fogel [37] demonstrated how intelligence is developed in evolutionary neural networks, and how they can learn to play any arbitrary game without depending upon human expertise or being explicitly programmed to do so. Hochmuth [38] described how this game could be played by a genetic algorithm. Soedarmadji [39] suggested a decentralised approach comprising nine agents, one for each cell in the board and one manager to choose which move to do make based on the priority number generated by each independent agent. Yau et al. [40] compared the implementations of various adaptations and related parameters in evolving agents to play games such as tic-tac-toe. Yau et al. [41] comprehensively discussed the use of co-evolution and Pareto evolution in the development of neural agents and how success is strongly dependent on the initialisation of the co-evolution process, and related empirical data is discussed. Bhatt et al. [42] did some commendable work, trying to find out no-loss strategies for the game using a customised genetic algorithm, focusing on a high win-to-draw ratio. They succeeded in finding 72,657 such strategies, and also came to some other interesting conclusions, which verify our existing notions about the game. Mohammadi et al. [43] presented a co-evolution and interactive fitness-based genetic algorithm to build a human-competitive player. Ling et al. [44] used a double transfer function and Rajani et al. [45] applied the Hamming distance classifier, on neural networks to demonstrate the advantages of each method. Singh et al. [46] devised a deterministic mathematical model to solve tic-tac-toe as a nine-dimensional problem. Karamchandani et al. [47] discussed the techniques used in [44, 45] and presented a very basic rule-based algorithm, which always targets the centre at the start and has a set of special moves, but it can still be beaten by an opponent in some cases. Garg et al. [48, 49] formally defined the tic-tac-toe problem and formulated a winning strategy for the same using a multi-tape turing machine and automata theory. Initial approaches by Hochmuth [38] and Soedarmadji [39] were innovative but not very effective. Hochmuth's method does not guarantee any loss for all possible tic-tac-toe games and was an exercise demonstrating the effectiveness of genetic algorithms, rather than the development of a perfect no-loss algorithm. Soedarmadji's heuristic approach loses in at least three board states as described in [42]. The three examples are explained through Fig. 1, in which, optimal positions for X are marked with '+'. In the first example, X should have marked any of the edge-centres, not in the top-right corner which opens a fork for the opponent, the second example is more evident as X fails to block a two O's in a row and similarly in the third example X again fails to block O. Fig. 1Open in figure viewerPowerPoint Three examples in which Soedarmadji's solution [39], playing with X, loses the game [42] It is discussed in [40, 42] and in Section 4 of this paper itself that the optimal strategy for the first and second player are different from each other. The first player has a greater chance of victory (around 1.68:1 [42]). In [50], a non-randomised decision tree approach is discussed by Sriram et al. They demonstrated some interesting observations, such as how MM fails to move optimally in as many as 25–40 cases. However, the algorithm was unclear in some aspects, such as it was not discussed how the algorithm would make a move if it had to start the game itself, even though the strategies are different for the two players as discussed above. Also, when the opponent makes the first move, it was suggested to always mark the centre if possible else mark the top-left corner, according to the tree in Fig. 2a, ignoring the edges completely. The remaining moves were to follow the tree in Fig. 2b. Furthermore, it was unclear exactly how each of the steps in the decision tree was carried out as that would have a significant effect on the actual runtime of the algorithm. Runtime comparison against vanilla MM was done solely on the basis of asymptotic complexity and not clock time complexity. Asymptotic complexity does not give a true representation of the time taken by an algorithm to run, which is needed for benchmarking. All these shortcomings are addressed in this research work. Fig. 2Open in figure viewerPowerPoint Decision trees for the algorithm suggested in [50] (a) Decision tree for the first move, (b) Decision tree for the remaining moves Thus, it is observed that initially most of the game-playing algorithms were inspired by MM and speed-ups involving pruning the game tree by various methods. Gradually approaches based on genetic and evolutionary algorithms were proposed, and these can generalise well to larger boards, but they take time to train and needs a careful tweaking of the hyperparameters. It is also difficult to estimate how many of these no-loss strategies can actually win against a sub-optimal player, which is a necessary trait to make the algorithm more human-like, for example, the 'forgetting problem' described in [41] prevents 'expert level' players from winning or even drawing against a medium level or random players. People have through the ages, tried to intuitively come up with rules to win the game. However, surprisingly not much research has been done to either try to design a randomised rule-based solution until now that tries to maximise win-to-draw ratio for tic-tac-toe and compare such an algorithm's clock time complexity to MM and similar tree search approaches. Although most of the techniques to play tic-tac-toe are centuries old, the authors in [51, 52] reported some rules, which could be incorporated into an expert system such as T3DT. However, this book [51] steers clear from starting in an edge, which was necessary for the randomisation and hence incorporated into T3DT. In [52], Ryan Aycock went to great lengths to explain the motivation for some of the rules for effective gameplay. Hence, this research work proposes a novel no-loss expert system to play tic-tac-toe, called T3DT, and compares the performance both theoretically as well as from actual runtime to existing MM-based procedures, proving that it is faster and a practical approach to play tic-tac-toe. The rest of the paper is organised as follows. In Section 3, the optimality of some existing MM-based tree-search algorithms, which have been used for benchmarking, are briefly discussed. In Section 4, the proposed algorithm is explained in detail. In Section 5, the experimental methodology is described and some newly devised metrics for the benchmarking process are elucidated. In Section 6, the algorithm is compared to the existing algorithms and the results are compared and analysed. Finally, Section 7 gives the conclusion and discusses some avenues for further work. 3 Optimality of some existing tree-search algorithms This section gives a brief overview and discusses the optimality of three commonly used MM-based no-loss strategies, namely (a) MM (b) MM with alpha–beta pruning (ABP) (c) MM with advanced alpha–beta (ABA) Optimality refers to the algorithm arriving at the goal in the least number of moves. Here the vanilla MM and MM with ABP have their usual implementations. MM with ABA has a modified score function, which subtracts the number of moves left till victory from the returned score. The vanilla MM algorithm and ABP never lose but they may occasionally make a move that results in a slower victory. For example, the opponent starts the game and after both players alternatingly making moves O1, X1, O2, X2, and O3, the bot's X3 moves chosen by each of the algorithms are shown in Fig. 3. Here, marking X3 in cell (3,1) would result in a victory on the diagonal instantly. MM and ABP do not choose this move, they still win eventually, but take a longer path. Including the depth into the evaluation function allows ABA to pick the optimal winning move (3,1) such as T3DT leading to the fastest victory. Fig. 3Open in figure viewerPowerPoint Different moves for X3 made by the algorithms (a) MM, (b) ABP, (c) ABA on the same board state. T3DT makes the same winning move as ABA in this state 4 Proposed method In this section, the no-loss system called T3DT is detailed, which is much faster than MM-based procedures, because it logically partitions the entire game into small subparts that are easier to analyse. Henceforth the computer or artificial intelligence player will be referred to as the 'bot', and it is assumed for simplicity that the bot always plays with X and the opponent plays with O. The algorithm can be divided into two strategies (a) When the bot starts the game (Sections 4.1 and 4.2) and (b) when the opponent starts the game (Sections 4.3 and 4.4). 4.1 Algorithm when the bot starts the game Start Step 1: the bot makes the first move, with absolute randomness in a corner edge or centre. Fig. 4Open in figure viewerPowerPoint Decision tree when the bot starts the game Step 2: the opponent marks any of the remaining cells. Step 3: depending on the opponent's first move in Step 2, the bot makes its second move as shown in Fig. 4, and explained in Section 4.2, Step 4: then the opponent makes its second move. Step 5: disregarding the opponent's second move, the bot always tries to first make a winning triplet or block the opponent, if no such possibilities exist, it makes use of the strategies shown in Fig. 4, and explained in Section 4.2. Step 6: the opponent makes its move. Step 7: the bot makes its consequent moves, based on the strategies mentioned in Section 4.5, which comprise winning or blocking or making a random move. Step 8: repeat steps 6 and 7 until the game ends. Step 9: declare the result. End 4.2 First few moves when the bot starts the game When the bot starts the game, to make the game truly randomised, it chooses a corner, edge, or centre with equal probability. These three cases again have their own sub-parts, which are all described in Fig. 4. In the following Sections 4.2.1–4.2.3, the motivation for each of the choices made in this decision tree is elaborated. 4.2.1 Bot starts in a corner The opponent chooses one of the following positions, based on which the bot makes the next few moves: (a) Edge: if the opponent chooses an edge, then the bot chooses the centre as its second move. Now since the bot has made two moves along a diagonal, the opponent is forced to mark the other corner of the diagonal to block the bot from winning. Now unless it needs to win or block the opponent, the bot marks a corner in its third move, such that it creates an empty V-shaped fork, this guarantees a win. In both of the games in Fig. 5, irrespective of the position of O3, the bot always wins. Fig. 5Open in figure viewerPowerPoint Bot starts in the corner and the opponent's first move is in an edge Fig. 6Open in figure viewerPowerPoint Bot starts in the corner and the opponent's first move is in a corner Fig. 7Open in figure viewerPowerPoint Bot starts in the corner and the opponent's first move is in the centre (b) Corner: if the opponent chooses another corner, then the bot marks any of the remaining free corners. The opponent is now forced to block the bot's win and the bot marks the last free corner, thus creating a fork for itself. Once again this guarantees a win for the bot, such as in the games in Fig. 6. (c) Centre: suppose the opponent marks the centre, then the bot marks the diagonally opposite corner to the first X. Now if the opponent places O2 another corner, the bot's win is guaranteed (see resulting board on left in Fig. 7); else the game will end in a draw (see the resulting board on right in Fig. 7) for further optimal play. 4.2.2 Bot starts in an edge This is the most complicated and interesting part of the algorithm when the bot chooses an edge. It would be much simpler to just prevent the bot from starting in the edge, as suggested by multiple sources [51, 52], but to make the game randomised, this starting move has to be included. The opponent may choose any of the following cases, based on which the bot's subsequent plays are explored: (a) Corner: if the opponent chooses a corner, the bot moves in the centre and in the third move, it marks any of the remaining empty edges, unless a winning or blocking opportunity arises. This move-sequence almost always results in a draw, such as the two games in Fig. 8. Fig. 8Open in figure viewerPowerPoint Bot starts in edge and the opponent's first move is in a corner Fig. 9Open in figure viewerPowerPoint Bot starts in edge and the opponent's first move is in the nearer edge Fig. 10Open in figure viewerPowerPoint Bot starts in edge and the opponent's first move is in the opposite edge Fig. 11Open in figure viewerPowerPoint Bot starts in edge and the opponent's first move is in centre (b) Edge: it is again of two types: i. Edge nearer to the bot's first move: here we will attempt to create an L-shaped fork making sure the bot wins. If the opponent chooses any of the two edges nearest to the bot's chosen edge, the bot puts a X2 in the square adjacent to both the bot's X1 and opponent's O1. Now the opponent will be forced to block the bot and the bot will now make its third move I the centre. This sequence guarantees a win for the bot, because of the L-shaped fork that is formed, as seen in Fig. 9. ii. Edge opposite to the bot's first move: if the opponent chooses the opposite edge to the bot's first move X1, the bot places a cross in any of the corners adjacent to X1. The opponent will block the bot with O2 in the corner. Now the bot places X3 in the corner that is closest to the opponent's moves O1 and O2. This sequence results in a draw, as in Fig. 10. (c) Centre: if the opponent chooses a centre, the bot marks the corner adjacent to its first move. This always ends in a draw for optimal play, as in Fig. 11. 4.2.3 Bot starts in the centre Then the opponent chooses either of the following positions, based on which the bot makes its subsequent moves: (a) Edge: if the opponent chooses an edge, the bot chooses any of the four corners randomly. Now the opponent needs to block the bot by its move O2. The bot takes advantage of this and creates a fork which is an empty V-shape, by placing X3 in the required corner. This sequence guarantees victory for the bot, as in both the games in Fig. 12. Fig. 12Open in figure viewerPowerPoint Bot starts in the centre and the opponent's first move is in an edge Fig. 13Open in figure viewerPowerPoint Bot starts in the centre and the opponent's first move is in a corner (b) Corner: If the opponent chooses a corner, the bot marks the diagonally opposite corner. Now unless the opponent moves in a corner, the bot will always win, by forming an empty V-shaped fork, as in Fig. 13. 4.3 Algorithm when the opponent starts the game Start Step 1: bot waits for the opponent to make its first move. Fig. 14Open in figure viewerPowerPoint Decision tree when the opponent starts the game Step 2: based on Step 1, the bot makes its first move as shown in Fig. 14, and explained in Section 4.4, Step 3: then the opponent makes its second move. Step 4: the bot always tries to first make a winning triplet or block the opponent, if no such possibilities exist, then it makes its move depending on the strategies shown in Fig. 14, and explained in Section 4.4. Step 5: the opponent makes its move. Step 6: the bot makes its consequent moves, based on the strategies mentioned in Section 4.5, which comprise winning or blocking or making a random move. Step 7: repeat steps 6 and 7 until the game ends. Step 8: declare the result. End 4.4 First few moves when the opponent starts the game The opponent's first move may be any position on the board – a corner, an edge, or the centre. This leads to three strategies for the bot, which are all described in Fig. 14. 4.4.1 Opponent starts in a corner If the opponent starts in the corner, the bot moves in centre as its first move. Now the opponent's second move may be either a corner or an edge, based on which the bot makes its subsequent moves. (a) Corner: if opponent's second move is a corner, bot's second X is in any of the four empty edges, unless there is a need to block the opponent. In other words, if the opponent marks the corner diagonally opposite to the first move, then the bot marks a random edge, as seen in the left board in Fig. 15, else the bot will need to block the opponent, such as in the right board in Fig. 15. These move sequences usually result in a draw. Fig. 15Open in figure viewerPowerPoint Opponent starts in the corner and the opponent's second move is in another corner Fig. 16Open in figure viewerPowerPoint Opponent starts in the corner and the opponent's second move is in an edge (b) Edge: if opponent's second move is in an edge unless the bot needs to block the opponent, its second X2 is in the corner that blocks both the O's. In other words, X2 is in the corner which is at a minimum distance from O1 and O2. These move sequences usually result in a draw, as in Fig. 16. 4.4.2 Opponent starts in an edge When the opponent starts in an edge, the bot marks any of the two corners next to the opponent's first move O1, and unless there is a need to block the opponent, which arises only if the opponent has marked O2 in the centre, the bot puts second move X2 in the centre. These move sequences usually result in a draw, such as in Fig. 17. Fig. 17Open in figure viewerPowerPoint Opponent starts in an edge 4.4.3 Opponent starts in the centre If the opponent starts in the centre, the bot's first move is in a corner, and if there's no need to block (that is if the opponent marks the cell diagonally opposite to the bot's first move), the second bot's move is placed randomly in any of the two remaining corners. This sequence of moves usually leads to a draw, line in the boards in Fig. 18. Fig. 18Open in figure viewerPowerPoint Opponent starts in the centre 4.5 Last few moves In all of the above, irrespective of whether the bot or the opponent starts the game, after the decision tree has played its part, the rest of the bot's moves comprise the three tasks in the following priority order: (a) Make a triplet and win. (b) Block the opponent's triplet. (c) Randomly select any one of the empty squares; because now there is no longer a way to win (or lose) the game, and the moves are kept randomised to add some variety. 5 Experimental methodology and proposed comparison metrics This section discusses the implementation details, experimental setup, and the newly formulated notations for this research work. The algorithm is implemented in Java 8. All times are measured in nanoseconds, using the internal nanosecond counter called by the method nanoTime() of system class [53], which has a high degree of precision. Runtimes were calculated for the four algorithms MM, ABA, ABP, and T3DT (please refer to Sections 3 and 4 for a description of these algorithms). For these comparisons, each algorithm was made to play against itself thousands of times, on Java HotSpot(TM) 64-Bit Server VM, with the JIT compiler both enabled and disabled, on multiple computers having dissimilar specifications to simulate all the possibilities and these statistics have been tabulated. JIT compiler was disabled to prevent the runtime optimisation for this benchmarking. Always the runtimes or the first 50 games were dropped, as they might have some noise due to the cold start of the compiler. Three of these setups whose findings would be presented have their specifications listed in Table 1. Table 1. Specifications of setups used for benchmarking Specifications Setup 1 Setup 2 Setup 3 processor Intel Core™ i5–7200U Intel Core™ i5-7200U AMD A4-3330MX APU cores/threads 2/4 2/4 2/2 base frequency 2.5 GHz 2.5 GHz 2.2 GHz operating system Windows 10 Ubuntu 18.04 LTS Windows 7 RAM 12 GB DDR4 12 GB DDR4 2 GB DDR3 For each particular setup and JIT state (mixed/interpreted), matrix was created having a number of rows equal to the number of games played and nine columns. Here the matrix notation indicates the time taken by the algorithm to make the jth move in the ith game; j belongs to the finite set{x|x ∈ N, 1 ≤ x ≤ 9}, while i belongs to the finite set {x|x ∈ N, 1 ≤ x ≤ Ng}, where Ng is the number of games played (2)For comparing clock-time analyses across the different setups, this paper formulates the following new notations: (a) Time per move (TPM): it refers to the response time of each move made by the bot. TPM is defined as the average of all moves of a certain move index j, for a particular setup, and state of the JIT compiler. TPM gives one an idea of how the algorithm is performing at each move. TPM for the jth move is represented by and is given by (3) (b) Time per game (TPG): TPG is another metric, which compares each algorithm on the basis of how long all the nine moves take to execute per game, it refers to the total time taken by each algorithm, on average, to play a game from start to finish. TPG for the ith game is represented by , and is given by (4) TPG, the mean of all values, is a very good estimate of the speed of an algorithm, and on a certain setup, it is given by (5)Runtimes are calculated for each move on the three setups with JIT compiler enabled and disabled, using the four algorithms being compared. These 24 matrices are then used to calculate TPMj and TPG, which are used to analyse the performance of the algorithms. TPG values are tabulated and the effective speedup of T3DT over the competing algorithms is calculated. The speedup is chosen as the benchmarking criterion here because, keeping all other variables (such as setup and JIT state) constant, it gives a true representation of each algorithm's performance. Speedup of T3DT over a certain algorithm X, on a particular setup and JIT state, is given by (6) 6 Analysis and results In this section, the claims of improved runtime are supported by irrefutable empirical evidence. Both theoretical and practical clock-time analyses have been done for the algorithms as described below. 6.1 Theoretical analysis This section explores some of the competing approaches to solve this problem and compares and contrasts the merits and demerits of each. The simplest approach for maximising win-to-draw ratio while still maintaining O(1) time complexity would have been to generate all the possible moves and store them in a tree or directed acyclic graph and map the current bo