Package: rl2048player
Reinforcement Learning to Play 2048This project trains agents to play the game 2048 using Q, SARSA, and TD0 learning agents.InstallationThis project is pip installable. On linux it can be installed by entering,pip3 install rl2048playerinto your terminal. For other operating systems, follow the pip instructions for that system.FeaturesIn order to use this package, you must first import it usingimport rl2048player as rlNow, I will review some of the main features of this package. For most of these features, there are addditional options not explained in this review. All options for any command can be found by reviewing the documentation in the code.MasksPefore creating a learning agent, you must initialize a mask. Masks translate between the game board and the learning agent. This allows you to change the way the agent understands the game board without changing the agent itself. Currently, only one mask has been implemented. This mask breaks down the board into rows, columns, and 2x2 squares. This has the effect of  decoupling parts of the board that do not interact strongly with each other. The mask can be initialized using the codemask = rl.masks.Mask_rxcx4()AgentsOnce the mask has been initialized, you can initialize the agents. Currently, there are three agents that have been implemented: one using a Q learning algorithm, one using a SARSA learning algorithm, and one using a TD0 learning algorithm. These agents can be initialized byagent = rl.agents.QAgent(mask)
agent = rl.agents.SARSAAgent(mask)
agent = rl.agents.TD0Agent(mask)respectively. There are also options to change the agent's hyperparameters when initializing them. The default values are the hyperparameters I have found to work best.TrainingTo train the agents over x number of games and record the scores use the codescores = agent.train(x)You can then look at a graph of those scores byagent.makeGraph(scores)
plt.show()There are also options to automatically save the scores and graph to a file.Making Gif'sOnce the agent has been trained, you can create a gif of the agent playing a game and save it to a file by using the commandagent.makeGif(gif_file)Saving and LoadingYou can also save trained agent by usingagent.save(agent_file)This save function only saves the look-up table for this agent, not the whole object. In order to load the agent, you need to initialize an agent with the same mask and hyperparameters and then load the look-up table usingagent = rl.agents.AppropriateAgent(AppropriateMask)
agent.load(agent_file)ExamplesIf you don't want to have to do all of this to use this package, there are some examples precoded in the package. Currently there are two examples that can be accessed usingrl.examples.example1()
rl.examples.example2()AcknowledgmentsThe game logic for this project is based on code by georgwiese which can be found athttps://github.com/georgwiese/2048-rl. The implementation of the learning agents is based on method described in the paper Temporal difference learning of n-tuple networks for the game 2048, by M.Szubertand W. Jaskowski. All code is my original work except when noted.Future WorkIn the future, I plan to work on implementing additional masks and learning agents.