Package: cfrainbow
CFRainbowImplementations OfCounterfactual Regret MinimizationIn Its Many Shapes & ColorsCFRainbow provides implementations of the basic CFR algorithm and some of the improved variants of CFR
for computingNash Equilibriain the 2-player 0-sum case as well asCorrelated Equilibriain the general-sum case.The pacakge is loosely built for modularity and general applicability by building on the Openspiel framework by Deepmind. CFRainbow isnotbuilt
for performance and will not scale well to large game implementations. Most algorithms are implemented with basic python data structures.The package is in the early WIP phase.UsageThe easiest way to use CFRainbow is to import the desired algorithm from the package, a regret minimizer, and therunmethod with a pyspiel game object or its OpenSpiel name as input.
For example, to use the Vanilla CFR algorithm:importcfrainbowfromcfrainbowimportcfr,rmcfrainbow.run(cfr.VanillaCFR,n_iter=1000,game="kuhn_poker",regret_minimizer=rm.RegretMatcher)This will run the algorithm for the number of iterations given and compute the exploitability.You can also use the solver object directly and calliterateon it after instantiating it with the correct arguments.
For example:fromcfrainbowimportcfr,rmfromcfrainbow.utilsimportload_game,KuhnPolicyPrinter,normalize_policy_profileroot_state=load_game("kuhn_poker").new_initial_states()solver=cfr.VanillaCFR(root_state,regret_minimizer_type=rm.RegretMatcher,alternating=True,# whether to do alternating or simultaneous updates)# iterate for a given number of iterations.foriinrange(1000):solver.iterate()avg_policy=normalize_policy_profile(solver.average_policy())print(KuhnPolicyPrinter(digits=2).print_profile(avg_policy))Output (Infostate --> Action Policy):PLAYER 0:
P1: Jack  | P2:   ?   | cb   --> ['check:  1.00', 'bet:  0.00']
P1: Jack  | P2:   ?          --> ['check:  0.81', 'bet:  0.19']
P1: Queen | P2:   ?   | cb   --> ['check:  0.48', 'bet:  0.52']
P1: Queen | P2:   ?          --> ['check:  0.99', 'bet:  0.01']
P1: King  | P2:   ?   | cb   --> ['check:  0.00', 'bet:  1.00']
P1: King  | P2:   ?          --> ['check:  0.42', 'bet:  0.58']
PLAYER 1:
P1:   ?   | P2: Queen | c    --> ['check:  0.99', 'bet:  0.01']
P1:   ?   | P2: Queen | b    --> ['check:  0.66', 'bet:  0.34']
P1:   ?   | P2: King  | c    --> ['check:  0.00', 'bet:  1.00']
P1:   ?   | P2: King  | b    --> ['check:  0.00', 'bet:  1.00']
P1:   ?   | P2: Jack  | c    --> ['check:  0.67', 'bet:  0.33']
P1:   ?   | P2: Jack  | b    --> ['check:  1.00', 'bet:  0.00']Note that inalternatingupdates each iteration is a single player's policy update.
Insimultaneousupdates both players' policy updates constitute one iteration.Available AlgorithmsThe following list shows the available algorithms that have been implemented (✔️), those that are still work in progress (🔨👷‍♂️),
and those that are planned to be implemented in the future (📅):AlgorithmStatusConvergence TestsPaper Results ReproducingBest-Response CFR✔️✔️🔨👷‍Discounted CFR✔️✔️🔨👷‍Exponential CFR✔️✔️❌Internal CFR🔨👷‍♂️🔨👷‍🔨👷‍Joint-Reconstruction CFR🔨👷‍♂️🔨👷‍🔨👷‍Chance Sampling Monte Carlo CFR✔️✔️🔨👷‍External Sampling Monte Carlo CFR✔️✔️🔨👷‍Outcome Sampling Monte Carlo CFR✔️✔️🔨👷‍Predictive Plus CFR✔️✔️🔨👷‍Pure CFR✔️✔️🔨👷‍Sampling CFR✔️🔨👷‍🔨👷‍Vanilla CFR✔️✔️🔨👷‍Lazy CFR📅📅📅InstallationWeb InstallPipTo install CFRainbow from PyPi using pip, run the following command:pipinstallcfrainbowPoetryIf you're using Poetry to manage your Python packages, you can add CFRainbow to your project by adding the following to your pyproject.toml file:[tool.poetry.dependencies]cfrainbow="*"Then run:poetryinstallto install your own package. Poetry will then installcfrainbowas dependency alongside your own package.Source InstallTo install CFRainbow from master, please follow these steps:gitclonehttps://github.com/maichmueller/cfrainbow.gitcdcfrainbowand install the package with pippipinstall.or poetrypoetryinstalluse the option--no-devto ensure a non-editable installation.