Package: sgamesolver
sgamesolverStochastic game solver, short: sgamesolver, is a python package to compute
stationary or Markov-perfect equilibria of stochastic games, using the
homotopy continuation method.Some useful links:Source code is hosted ongithubAlso on github is anissue trackerfor bug reports and feature requestsDocumentation is found onreadthedocssgamesolver is free and open source, under the MIT license.
If you use the program for any published research, please citeEibelshäuser/Poensgen (2019).Installationsgamesolver is hosted on PyPI, so installation is usually as simple aspipinstallsgamesolverHead over to the docs for other options for installation.UsageSolving a stochastic game is done in three steps:define a gamepick a homotopy functionset up and run the solverimportsgamesolvergame=sgamesolver.SGame.random_game(64,2,4,seed=42)qre=sgamesolver.homotopy.QRE(game)qre.solver_setup()qre.solve()print(qre.equilibrium)A quick rundown of these steps – more details are found in the docs:1. Set up a stochastic gamegame=sgamesolver.SGame.random_game(64,2,4,seed=42)Stochastic games are represented by the classSGame.
For this quick example, we are using the methodrandom_gameto
randomize a game with 64 states, 2 players, and 4 actions per player
and state. (Setting a seed just makes the result reproducible.)Of course, you probably didn't come here to solve a random game,
but have a specific game in mind. Thedocumentationcontains instructions
and examples on how to create anSGamewhich represents the stochastic
game you want to solve. (By the way, sgamesolver can also solve normal form games,
sequential games, repeated games, and also
finite Markov decision problems – each of these is just a simple case of a stochastic game.)2. Select and set up a homotopy function for your gameqre=sgamesolver.homotopy.QRE(game)sgamesolver uses thehomotopy principleto solve stochastic games,
a general technique to solve systems of non-linear equations.
In short, the idea is as follows: Instead of solving some very hard problem directly
(in our case: finding an equilibrium), a continuous transformation
is applied to the system, yielding a related, but much simpler problem,
for which one can easily obtain a solution. This transformation is then
gradually reversed while tracking the solution, until arriving at a solution
for the original problem – here, the desired stationary equilibrium.
(You can find more background in the documentation –
although such knowledge is not necessary for using the program.)The (mathematical) function used for this transformation is calledhomotopy function. In general, there are many possibilities
to construct a suitable one. sgamesolver currently includes two:
The one we picked for this example,sgamesolver.homotopy.QRE, is based on
an extension of quantal response equilibrium to stochastic games(Eibelshäuser/Poensgen 2019b). The other,sgamesolver.homotopy.LogTracing, implements the
logarithmic tracing procedure for stochastic games(Eibelshäuser/Klockmann/Poensgen/von Schenk 2022).
Which one to pick? In our experience, the former is more robust –
while the latter has the advantage that it allows to search for multiple equilibria.
More homotopy functions are to come! In any case, please make sure to cite the paper that
introduced whichever homotopy you end up using.3. Let the homotopy solver do its jobFinally, we will set up the solver and start it:qre.solver_setup()qre.solve()Then it's time to lean back and watch for a bit:==================================================
Start homotopy continuation
Step    37: t =  3.612 ↑, s =  20.47, ds =  3.418... until ...Step   247: t = 1.147e+04 ↑, s = 9.385e+04, ds =  1000.
Step   247: Continuation successful. Total time elapsed: 0:00:15
End homotopy continuation
==================================================
An equilibrium was found via homotopy continuation.... success!
Ideally, the solver will be able to find a solution without
requiring any further interaction, as in this example. In cases
where this does not work out, check out thesection on troubleshootingin the documentation.4. AftermathWe can now display the solution:print(qre.equilibrium)which outputs equilibrium strategies and values for all 64 states:+++++++++ state00 +++++++++
                       a0    a1    a2    a3  
player0 : v=15.09, σ=[1.000 0.000 0.000 0.000]
player1 : v=15.63, σ=[0.000 0.000 1.000 0.000]

+++++++++ state01 +++++++++
                       a0    a1    a2    a3  
player0 : v=14.76, σ=[0.000 0.961 0.000 0.039]
player1 : v=15.61, σ=[0.354 0.000 0.000 0.646]

+++++++++ state02 +++++++++
                       a0    a1    a2    a3  
player0 : v=14.84, σ=[1.000 0.000 0.000 0.000]
player1 : v=15.61, σ=[0.000 0.000 1.000 0.000]

... (abridged here for brevity) ...

+++++++++ state63 +++++++++
                       a0    a1    a2    a3  
player0 : v=14.92, σ=[0.000 1.000 0.000 0.000]
player1 : v=15.75, σ=[1.000 0.000 0.000 0.000]Of course, you now also can access equilibrium strategies (and values)
asnumpyarrays and use them for further calculations or simulations.eq_strat = qre.equilibrium.strategies
eq_values = qre.equilibrium.values