Package: rlcard
RLCard: A Toolkit for Reinforcement Learning in Card Games中文文档RLCard is a toolkit for Reinforcement Learning (RL) in card games. It supports multiple card environments with easy-to-use interfaces for implementing various reinforcement learning and searching algorithms. The goal of RLCard is to bridge reinforcement learning and imperfect information games. RLCard is developed byDATA Labat Rice and Texas A&M University, and community contributors.Official Website:https://www.rlcard.orgTutorial in Jupyter Notebook:https://github.com/datamllab/rlcard-tutorialPaper:https://arxiv.org/abs/1910.04376Video:YouTubeGUI:RLCard-ShowdownDou Dizhu Demo:DemoResources:Awesome-Game-AIRelated Project:DouZero ProjectZhihu:https://zhuanlan.zhihu.com/p/526723604Miscellaneous Resources: Have you heard of data-centric AI? Please check out ourdata-centric AI surveyandawesome data-centric AI resources!Community:Slack: Discuss in our#rlcard-projectslack channel.QQ Group: Join our QQ group to discuss. Password: rlcardqqgroupGroup 1: 665647450Group 2: 117349516News:We have updated the tutorials in Jupyter Notebook to help you walk through RLCard! Please checkRLCard Tutorial.All the algorithms can suppportPettingZoonow. Please checkhere. Thanks the contribtuion fromYifei Cheng.Please followDouZero, a strong Dou Dizhu AI and theICML 2021 paper. An online demo is availablehere. The algorithm is also integrated in RLCard. SeeTraining DMC on Dou Dizhu.Our package is used inPettingZoo. Please check it out!We have released RLCard-Showdown, GUI demo for RLCard. Please check outhere!Jupyter Notebook tutorial available! We add some examples in R to call Python interfaces of RLCard with reticulate. SeehereThanks for the contribution of@Clarit7for supporting different number of players in Blackjack. We call for contributions for gradually making the games more configurable. Seeherefor more details.Thanks for the contribution of@Clarit7for the Blackjack and Limit Hold'em human interface.Now RLCard supports environment local seeding and multiprocessing. Thanks for the testing scripts provided by@weepingwillowben.Human interface of NoLimit Holdem available. The action space of NoLimit Holdem has been abstracted. Thanks for the contribution of@AdrianP-.New game Gin Rummy and human GUI available. Thanks for the contribution of@billh0420.PyTorch implementation available. Thanks for the contribution of@mjudell.ContributorsThe following games are mainly developed and maintained by community contributors. Thank you!Gin Rummy:@billh0420Bridge:@billh0420Thank all the contributors!Cite this workIf you find this repo useful, you may cite:Zha, Daochen, et al. "RLCard: A Platform for Reinforcement Learning in Card Games." IJCAI. 2020.@inproceedings{zha2020rlcard,title={RLCard: A Platform for Reinforcement Learning in Card Games},author={Zha, Daochen and Lai, Kwei-Herng and Huang, Songyi and Cao, Yuanpu and Reddy, Keerthana and Vargas, Juan and Nguyen, Alex and Wei, Ruzhe and Guo, Junyu and Hu, Xia},booktitle={IJCAI},year={2020}}InstallationMake sure that you havePython 3.6+andpipinstalled. We recommend installing the stable version ofrlcardwithpip:pip3 install rlcardThe default installation will only include the card environments. To use PyTorch implementation of the training algorithms, runpip3 install rlcard[torch]If you are in China and the above command is too slow, you can use the mirror provided by Tsinghua University:pip3 install rlcard -i https://pypi.tuna.tsinghua.edu.cn/simpleAlternatively, you can clone the latest version with (if you are in China and Github is slow, you can use the mirror inGitee):git clone https://github.com/datamllab/rlcard.gitor only clone one branch to make it faster:git clone -b master --single-branch --depth=1 https://github.com/datamllab/rlcard.gitThen install withcd rlcard
pip3 install -e .
pip3 install -e .[torch]We also providecondainstallation method:conda install -c toubun rlcardConda installation only provides the card environments, you need to manually install Pytorch on your demands.ExamplesAshort exampleis as below.importrlcardfromrlcard.agentsimportRandomAgentenv=rlcard.make('blackjack')env.set_agents([RandomAgent(num_actions=env.num_actions)])print(env.num_actions)# 2print(env.num_players)# 1print(env.state_shape)# [[2]]print(env.action_shape)# [None]trajectories,payoffs=env.run()RLCard can be flexibly connected to various algorithms. See the following examples:Playing with random agentsDeep-Q learning on BlackjackTraining CFR (chance sampling) on Leduc Hold'emHaving fun with pretrained Leduc modelTraining DMC on Dou DizhuEvaluating AgentsTraining Agents on PettingZooDemoRunexamples/human/leduc_holdem_human.pyto play with the pre-trained Leduc Hold'em model. Leduc Hold'em is a simplified version of Texas Hold'em. Rules can be foundhere.>> Leduc Hold'em pre-trained model

>> Start a new game!
>> Agent 1 chooses raise

=============== Community Card ===============
┌─────────┐
│░░░░░░░░░│
│░░░░░░░░░│
│░░░░░░░░░│
│░░░░░░░░░│
│░░░░░░░░░│
│░░░░░░░░░│
│░░░░░░░░░│
└─────────┘
===============   Your Hand    ===============
┌─────────┐
│J        │
│         │
│         │
│    ♥    │
│         │
│         │
│        J│
└─────────┘
===============     Chips      ===============
Yours:   +
Agent 1: +++
=========== Actions You Can Choose ===========
0: call, 1: raise, 2: fold

>> You choose action (integer):We also provide a GUI for easy debugging. Please checkhere. Some demos:Available EnvironmentsWe provide a complexity estimation for the games on several aspects.InfoSet Number:the number of information sets;InfoSet Size:the average number of states in a single information set;Action Size:the size of the action space.Name:the name that should be passed torlcard.maketo create the game environment. We also provide the link to the documentation and the random example.GameInfoSet NumberInfoSet SizeAction SizeNameUsageBlackjack (wiki,baike)10^310^110^0blackjackdoc,exampleLeduc Hold’em (paper)10^210^210^0leduc-holdemdoc,exampleLimit Texas Hold'em (wiki,baike)10^1410^310^0limit-holdemdoc,exampleDou Dizhu (wiki,baike)10^53 ~ 10^8310^2310^4doudizhudoc,exampleMahjong (wiki,baike)10^12110^4810^2mahjongdoc,exampleNo-limit Texas Hold'em (wiki,baike)10^16210^310^4no-limit-holdemdoc,exampleUNO (wiki,baike)10^16310^1010^1unodoc,exampleGin Rummy (wiki,baike)10^52--gin-rummydoc,exampleBridge (wiki,baike)--bridgedoc,exampleSupported AlgorithmsAlgorithmexamplereferenceDeep Monte-Carlo (DMC)examples/run_dmc.py[paper]Deep Q-Learning (DQN)examples/run_rl.py[paper]Neural Fictitious Self-Play (NFSP)examples/run_rl.py[paper]Counterfactual Regret Minimization (CFR)examples/run_cfr.py[paper]Pre-trained and Rule-based ModelsWe provide amodel zooto serve as the baselines.ModelExplanationleduc-holdem-cfrPre-trained CFR (chance sampling) model on Leduc Hold'emleduc-holdem-rule-v1Rule-based model for Leduc Hold'em, v1leduc-holdem-rule-v2Rule-based model for Leduc Hold'em, v2uno-rule-v1Rule-based model for UNO, v1limit-holdem-rule-v1Rule-based model for Limit Texas Hold'em, v1doudizhu-rule-v1Rule-based model for Dou Dizhu, v1gin-rummy-novice-ruleGin Rummy novice rule modelAPI Cheat SheetHow to create an environmentYou can use the the following interface to make an environment. You may optionally specify some configurations with a dictionary.env = rlcard.make(env_id, config={}): Make an environment.env_idis a string of a environment;configis a dictionary that specifies some environment configurations, which are as follows.seed: DefaultNone. Set a environment local random seed for reproducing the results.allow_step_back: DefaultFalse.Trueif allowingstep_backfunction to traverse backward in the tree.Game specific configurations: These fields start withgame_. Currently, we only supportgame_num_playersin Blackjack, .Once the environemnt is made, we can access some information of the game.env.num_actions: The number of actions.env.num_players: The number of players.env.state_shape: The shape of the state space of the observations.env.action_shape: The shape of the action features (Dou Dizhu's action can encoded as features)What is state in RLCardState is a Python dictionary. It consists of observationstate['obs'], legal actionsstate['legal_actions'], raw observationstate['raw_obs']and raw legal actionsstate['raw_legal_actions'].Basic interfacesThe following interfaces provide a basic usage. It is easy to use but it has assumtions on the agent. The agent must followagent template.env.set_agents(agents):agentsis a list ofAgentobject. The length of the list should be equal to the number of the players in the game.env.run(is_training=False): Run a complete game and return trajectories and payoffs. The function can be used after theset_agentsis called. Ifis_trainingisTrue, it will usestepfunction in the agent to play the game. Ifis_trainingisFalse,eval_stepwill be called instead.Advanced interfacesFor advanced usage, the following interfaces allow flexible operations on the game tree. These interfaces do not make any assumtions on the agent.env.reset(): Initialize a game. Return the state and the first player ID.env.step(action, raw_action=False): Take one step in the environment.actioncan be raw action or integer;raw_actionshould beTrueif the action is raw action (string).env.step_back(): Available only whenallow_step_backisTrue. Take one step backward. This can be used for algorithms that operate on the game tree, such as CFR (chance sampling).env.is_over(): ReturnTrueif the current game is over. Otherewise, returnFalse.env.get_player_id(): Return the Player ID of the current player.env.get_state(player_id): Return the state that corresponds toplayer_id.env.get_payoffs(): In the end of the game, return a list of payoffs for all the players.env.get_perfect_information(): (Currently only support some of the games) Obtain the perfect information at the current state.Library StructureThe purposes of the main modules are listed as below:/examples: Examples of using RLCard./docs: Documentation of RLCard./tests: Testing scripts for RLCard./rlcard/agents: Reinforcement learning algorithms and human agents./rlcard/envs: Environment wrappers (state representation, action encoding etc.)/rlcard/games: Various game engines./rlcard/models: Model zoo including pre-trained models and rule models.More DocumentsFor more documentation, please refer to theDocumentsfor general introductions. API documents are available at ourwebsite.ContributingContribution to this project is greatly appreciated! Please create an issue for feedbacks/bugs. If you want to contribute codes, please refer toContributing Guide. If you have any questions, please contactDaochen Zhawithdaochen.zha@rice.edu.AcknowledgementsWe would like to thank JJ World Network Technology Co.,LTD for the generous support and all the contributions from the community contributors.