Package: gym-tic-tac-toe3D
gym-tic-tac-toe3DOpenAI Gym Environment for a Two-Player 3D Tic-Tac-Toe.
Github Link:https://github.com/OUStudent/gym_tic_tac_toe3DRequirementsgymNumpyMatplotlibInstallpipinstallgym-tic-tac-toe3DHow it WorksTic Tac Toe is usually played on a 3x3 grid where the objective is for one player to line up their tokens in a straight
line of three. This is an extremely easy and trivial game; however, one can extend the difficulty by stacking 3x3 layers
to create a 3x3x3 cube. Now the objective is to line up three tokens in any of the directions.Here are three example games where the action was randomly generated:How to UseThe environment is a Two-Player Game, where Blue denotes Player 1 and Red denotes Player 2. Thestateandactioncontains all 27 possible positions, 9 for the first layer, 9 for the second, and 9 for the third.The input has three possible integer values for eac position, -1 for opponent, 0 for empty, and 1 for current player.
Note that no matter whom the Player is, these values holds true. Therewardis a two value list where the first index
represents the reward for Player 1 and the second for Player 2. Therewardvalue should only be used after the game is
completed. Players are rewarded for wins and receive extra points for how fast they win; while players are penalized for
losing and how fast they lost. In addition, because Player 1 has such a great advantage over Player 2 due to playing first,
they are penalized greater if they lose to Player 2 than if Player 2 lost to Player 1; in addition, Player 2 is rewarded
greater than Player 1 if they win than if Player 1 won against Player 2. Here are the current rewards:Turns TakenPlayer 1 Win:P1 RewardP2 PenalizePlayer 2 Win:P1 PenalizeP2 Reward<= 3120-102-2040<=5118-92-1836<=7116-82-1632<=9114-72-1428Else110-52-1020For example, after a game where Player 1 has won within three turns, Player 1 is rewarded 20 points while Player 2 penalized
10 points. On the other hand, if Player 2 won within 7 turns then Player 1 is penalized 16 points while Player 2 is
rewarded 32 points.Here is an example on how to create the environment with random agents:importgymimportgym_tic_tac_toe3Dimportmatplotlib.pyplotaspltenv=gym.make("tic_tac_toe3D-v0")games=3# best of threeplayer1_reward=0player2_reward=0foriinrange(0,games):state=env.reset()done=Falseplayer=1whilenotdone:env.render(player=player)plt.pause(0.5)whileTrue:action=env.action_space.sample()# Need to check if action is available in state spaceifstate[action]==0:breakstate,reward,done,info=env.step(action,player=player)# switch playersifplayer==1:player=2else:player=1# final render after completion of game to see final moveenv.render(player=player)plt.pause(1)player1_reward+=reward[0]player2_reward+=reward[1]Here is another example between two agents namedp1andp2:importgymimportgym_tic_tac_toe3Dimportmatplotlib.pyplotaspltdefplay(p1,p2,show=False,num_games=3):env=gym.make("tic_tac_toe3D-v0")player1_reward=0player2_reward=0foriinrange(0,num_games):state=env.reset()done=Falseplayer=1whilenotdone:ifshow:env.render(player=player)plt.pause(0.5)ifplayer==1:move=p1.predict(state)# returns softmax of prob's for all 27 possible actionselse:move=p2.predict(state)# returns softmax of prob's for all 27 possible actions# get action states that are emtpyviable_moves=np.where(state==0)[0].tolist()# find the empty action with the largest probabilityaction=viable_moves[np.argmax(move[0][viable_moves])]state,reward,done,info=env.step(action,player=player)# switch playersifplayer==1:player=2else:player=1ifshow:env.render(player=player)plt.pause(1)player1_reward+=reward[0]player2_reward+=reward[1]returnplayer1_reward,player2_rewardCustom RewardsIf the default rewarding scheme described above does not suite your likings, there is an option to create custom
rewarding. The rewards available for changing are only for the turn slots described above, meaning[3,5,7,9,else]for
rewarding or penalizing players 1 and 2. Here is an example:Suppose I want the following rewarding scheme:Turns TakenPlayer 1 Win:P1 RewardP2 PenalizePlayer 2 Win:P1 PenalizeP2 Reward<= 3120-22-2040<=5118-42-1630<=7112-62-1220<=918-82-1010Else15-102-1010Then this is how to create it:importgymimportgym_tic_tac_toe3Dcust_p1_win=[20,18,12,8,5]cust_p2_win=[40,30,20,10,10]cust_p1_lose=[-20,-16,-12,-10,-10]cust_p2_lose=[-2,-4,-6,-8,-10]env=gym.make("tic_tac_toe3D-v0")env.custom_rewards(reward_p1=cust_p1_win,reward_p2=cust_p2_win,pen_p1=cust_p1_lose,pen_p2=cust_p2_lose)# do other stuff