Package: gymnasium-2048
Gymnasium 2048Gymnasium environment for theGame 2048and game-playing agents using
temporal difference learning of n-tuple networks.https://github.com/Quentin18/gymnasium-2048/assets/58831477/c630a605-d1da-412a-a284-75f5c28bab46Action Spacespaces.Discrete(4)Observation Spacespaces.Box(low=0, high=1, shape=(4, 4, 16), dtype=np.uint8)Importgymnasium.make("gymnasium_2048:gymnasium_2048/TwentyFortyEight-v0")InstallationTo installgymnasium-2048with pip, execute:pipinstallgymnasium_2048From source:gitclonehttps://github.com/Quentin18/gymnasium-2048cdgymnasium-2048/
pipinstall-e.EnvironmentAction SpaceThe action is an integer representing the direction to slide the tiles:DirectionAction0UP1RIGHT2DOWN3LEFTObservation SpaceThe observation is a 3Dndarrayencoding the board state. It is encoded into 16 channels, where each channel is a 4x4
binary image. The i-th channel marks each cell of the game position that contains the i-th tile as 1, and 0 otherwise.
Each channel represents the positions of empty cells, 2-tiles, 4-tiles, ... , and 32768-tiles, respectively.This representation is mostly used for deep convolutional neural networks (DCNN).RewardsAt each step, for each tile merge, the player gains a reward
equal to the value of the new tile.
The total reward, corresponding to the game score, is the
sum of rewards obtained throughout the game.Starting StateThe game starts with two randomly generated tiles. A 2-tile can be generated with probability 0.9 and a 4-tile with
probability 0.1.Episode EndThe episode ends if there are no legal moves, i.e., all squares are occupied and there are no two adjacent tiles sharing
the same value.Argumentssize: the size of the game board. The default value is 4.max_pow: the maximum power of 2 allowed. The default value is 16.importgymnasiumasgymgym.make("gymnasium_2048:gymnasium_2048/TwentyFortyEight-v0",size=4,max_pow=16)UsageTo use the training and evaluation scripts, install thetrainingdependencies:pipinstall.[training]Play ManuallyTo play the game manually with the four arrows of your keyboard, execute:python-mscripts.playSee the arguments with the help command:python-mscripts.play-hTrain an AgentTo train an agent using temporal difference learning of n-tuple networks, execute:python-mscripts.train\--algotdl\-n100000\--eval-freq5000\--eval-episode1000\--save-freq5000\--seed42\-omodels/tdlSee the arguments with the help command:python-mscripts.train-hPlot Training MetricsTo plot training metrics from logs, execute:python-mscripts.plot\-itrain.log\-t"Temporal Difference Learning"\-ofigures/training_tdl.pngSee the arguments with the help command:python-mscripts.plot-hHere are the training metrics of trained policies over episodes:TDL smallTDLEnjoy a Trained AgentTo see a trained agent in action, execute:python-mscripts.enjoy\--algotdl\-imodels/tdl/best_n_tuple_network_policy.zip\-n1\--seed42See the arguments with the help command:python-mscripts.enjoy-hEvaluate a Trained AgentTo evaluate the performance of a trained agent, execute:python-mscripts.evaluate\--algotdl\-imodels/tdl/best_n_tuple_network_policy.zip\-n1000\--seed42\-t"Temporal Difference Learning"\-ofigures/stats_tdl.pngSee the arguments with the help command:python-mscripts.evaluate-hHere are the performances of trained policies:TDL smallTDLRandom policy performancesTestsTo run tests, execute:pytestCitingTo cite the repository in publications:@misc{gymnasium-2048,author={Quentin Deschamps},title={Gymnasium 2048},year={2023},publisher={GitHub},journal={GitHub repository},howpublished={\url{https://github.com/Quentin18/gymnasium-2048}},}ReferencesSzubert and Jaśkowski: Temporal Difference Learning of N-Tuple Networks
for the Game 2048Guei and Wu: On Reinforcement Learning for the Game of 2048AuthorQuentin Deschamps