Package: pokerl
PokeRLpokerlis a poker Reinforcement Learning (RL) environment.Currently the only game supported isNo Limit Texas Hold'em.InstallationYou can download and install the latest version from PyPI:agent@pokerl:~$pipinstallpokerlUsageA game is created by creating a new instance of thepokerl.Gameclass:frompokerlimportGamegame=Game()The constructor accepts various parameters, such as the number of players and the initial credits:game=Game(num_players=4,start_credits=100)game.reset()# Init game stateAt each step, there is anactive playerand anactive state, which captures the state of the game as seen by the active player. The active player performs actions by passing a valid action value togame.step(action):frompokerl.enumsimportPokerMovesgame.step(PokerMoves.CALL)game.step(PokerMoves.FOLD)game.step(PokerMoves.ALL_IN)The type ofactionmust beintornp.integerTheGame.active_stateproperty is an object of typeGame.StateViewthat has various informations that can be used by the agent to make informed decisions. The state also has a list ofvalid_actions, which is a one-hot encoded array of valid actions:# Choose random actionvalids=game.active_state.valid_actionsaction=np.random.choice(len(valids),p=valids/np.sum(valids))game.step(action)If an invalid action is passed togame.step(e.g. if you try to call a bet but you don't have enough credit) aValueErroris raised.game.stepalso returns a 3-element tuple indicating whether the game is over, the hand is over and/or the round of bets is over:done,_,_=game.step(action)ifdone:print('game is over')After each hand, the net profit of each player is saved ingame.payoffs.For more info see theWiki page.EnvironmentsIf you are familiar with OpenAI Gym, theracerl.envsmodule has an environment that exposes an API similar to that ofgym.Env. ThePokerGameEnvsimulates an entire game and returns a non-zero reward after each hand, depending on the player's payoffs and bets. To use thePokerGameEnvcreate a new instance by passing a list of opponents:frompokerlimportGamefrompokerl.envsimportPokerGameEnvdefrandom_agent(state:Game.StateView)->int:actions=state.valid_action_indicesreturnnp.random.choice(list(actions))env=PokerGameEnv([random_agent,random_agent,random_agent],num_players=4)An opponent must be a callable object which receives the game state, as seen by the agent, and returns a valid action.The usual methodsenv.reset()andenv.step(action)can be used to reset the state and perform actions on the environment:# Evaluate one episodeR=[]state=env.reset()done=Falsewhilenotdone:action=predict(state)state,rwd,done,_=env.step(action)R.append(rwd)print('score',np.sum(R))The reward is the sum of the payoffs.NetworkThenetworkbranch is an experimental branch where it is possible to play online games. Many functionalities are also available in the main branch.To create a server, create a new instance of thepokerl.network.PokerGameServerclass:frompokerl.networkimportPokerGameServerserver=PokerGameServer(num_players=3)The constructor accepts the same parameters ofGame.
To start the server and play a game, useserver.setup(host, port)followed byserver.run():server.setup('localhost',25560)#server.setup() Listen on default host and portserver.run()A client is an instance ofpokerl.network.PokerGameClient. The contructor accepts a callable objectagent, which represent the agent policy and ideally is a subclass ofpokerl.agents.PokerAgent:frompokerl.agentsimportRandomAgentclient=PokerGameClient(agent=RandomAgent)To connect to a server, useclient.connect(host, port):client.connect('localhost',25560)client.connect()# Use default host and portOnce all the players have connected, the game will begin.At the moment, the log of the game is only accessible to the server. Clients may print their active state to follow the progress of the game.ContributorsAndrea Mecchia @sneppyGuglielmo Manneschi @nondecidibile