  The real world is awash with multi-agent problems that require collective
action by self-interested agents, from the routing of packets across a computer
network to the management of irrigation systems. Such systems have local
incentives for individuals, whose behavior has an impact on the global outcome
for the group. Given appropriate mechanisms describing agent interaction,
groups may achieve socially beneficial outcomes, even in the face of short-term
selfish incentives. In many cases, collective action problems possess an
underlying graph structure, whose topology crucially determines the
relationship between local decisions and emergent global effects. Such
scenarios have received great attention through the lens of network games.
However, this abstraction typically collapses important dimensions, such as
geometry and time, relevant to the design of mechanisms promoting cooperation.
In parallel work, multi-agent deep reinforcement learning has shown great
promise in modelling the emergence of self-organized cooperation in complex
gridworld domains. Here we apply this paradigm in graph-structured collective
action problems. Using multi-agent deep reinforcement learning, we simulate an
agent society for a variety of plausible mechanisms, finding clear transitions
between different equilibria over time. We define analytic tools inspired by
related literatures to measure the social outcomes, and use these to draw
conclusions about the efficacy of different environmental interventions. Our
methods have implications for mechanism design in both human and artificial
agent systems.
