<figure><img src="image_1.png" /></figure>

HAL
open science

# Special Issue: Multi-Agent Dynamic Decision Making and Learning

Konstantin Avrachenkov, Vivek S Borkar, U. Jayakrishnan Nair

<figure><img src="image_2.png" /></figure>

## To cite this version:

Konstantin Avrachenkov, Vivek S Borkar, U. Jayakrishnan Nair. Special Issue: Multi-Agent Dynamic Decision Making and Learning. Dynamic Games and Applications, 13, 2023, 10.1007/s13235-023-00493-z. hal-04372613

HAL Id: hal-04372613

https://inria.hal.science/hal-04372613v1

Submitted on 4 Jan 2024

HAL is a multi-disciplinary open access archive for the deposit and dissemination of scientific research documents, whether they are published or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

<figure><img src="image_3.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Special Issue: Multi-Agent Dynamic Decision Making and Learning

Konstantin Avrachenkov<fnref n="1" /> · Vivek S. Borkar<fnref n="2" /> · U. Jayakrishnan Nair<fnref n="2" />

Accepted: 21 January 2023 / Published online: 31 January 2023

© The Author(s), under exclusive licence to Springer Science+Business Media, LLC, part of Springer Nature 2023

As a large and ever-increasing part of our economic and social interactions moves to the cyberspace, data-driven algorithmic decision making by autonomous agents is fast becoming an integral and inseparable part of our lives. These agents are competing in uncertain and volatile environments and must in turn learn aspects thereof, and of each other, in order to dynamically optimize their performance. What is more, even the humans in the loop are obliged to depend more and more on data-driven signals for their own decision making, e.g., on automated rankings and recommendations. Given the inherently distributed, strategic, dynamic nature of this ethos, learning in dynamic games, with its broad spectrum of modeling and analysis tools, is a prime candidate for providing this endeavor the theoretical underpinnings, with a balance between unification of the mathematical substructure, and retaining the distinct flavors and diversity of the competing paradigms. On modeling front, this ranges from dynamic cooperative games to mean field and evolutionary games and, for learning paradigms, from reinforcement learning to learning by imitation.

This nascent role of dynamic games has already registered its presence in many different ways and is increasingly doing so. The time is thus ripe for taking stock of where we are and where we should be headed. This is the motivation behind this special issue. The subarea is still too young to be put into a straitjacket of well defined boundaries. As an outcome, we have here a collection of fourteen articles that represent the many strands in this area, some of them straddling more than one. This includes reinforcement learning, network games, evolutionary games, distributed resource allocation, prospect theoretic considerations, information structures, etc.

This article is part of the topical collection "Multi-agent Dynamic Decision Making and Learning" edited by Konstantin Avrachenkov, Vivek S. Borkar and U. Jayakrishnan Nair.

<figure><img src="image_4.png" /></figure>

<figure><img src="image_5.png" /></figure>

<footnote>Vivek S. Borkar
borkar.vs@gmail.com</footnote>

<footnote>Konstantin Avrachenkov
konstantin.avratchenkov@inria.fr</footnote>

<footnote>U. Jayakrishnan Nair
jayakrishnan.nair@ee.iitb.ac.in</footnote>

<footnote n="1">¹ INRIA Sophia Antipolis, NEO Project-team, 2004, Route des Lucioles, B.P.93, 06902 Sophia Antipolis Cedex, France</footnote>

<footnote n="2">² Department of Electrical Engineering, Indian Institute of Technology Bombay, Powai, Mumbai 400076, India</footnote>
Specifically, the contributions are as follows. There is an excellent survey by Sylvain Sorin about various approaches to continuous time models of learning in games [10]. There are several articles dedicated to reinforcement learning, a very active area in machine learning and control, now already making inroads into dynamic games. They deal with multi-agent versions of classical reinforcement learning [13], robustness and approximation issues in stochastic games [11], mean field games [1, 14], and with learning coarse correlated equilibria in stochastic games [6]. The articles [2, 3] address robustness issues in the context of network games and resource allocation problems. We also have contributions to learning aspects of evolutionary games [4], prospect theoretic learning [8], decentralized bandits [7], learning for coordination [9], information structures [12] and opinion dynamics [5].

This field interests multiple communities such as dynamic games, control theory and machine learning. The editors hope that this special issue makes a small contribution toward building bridges between them to increase the synergistic interaction that will spur further advances in this field that we all love.

We would like to thank all authors who submitted a paper to this issue. Special thanks are also due to all the reviewers for their criticisms and suggestions, without which this issue would not have been possible.

## References

1. Anahtarci B, Kariksiz CD, Saldi N (2022) Q-learning in regularized mean-field games. Dyn Games Appl. https://doi.org/10.1007/s13235-022-00450-2

2. Brown PN, Seaton JH, Marden JR (2022) Robust networked multiagent optimization: designing agents to repair their own utility functions. Dyn Games Appl. https://doi.org/10.1007/s13235-022-00469-5

3. Ferguson BL, Marden JR (2022) Robust utility design in distributed resource allocation problems with defective agents. Dyn Games Appl. https://doi.org/10.1007/s13235-022-00470-y

4. Graham T, Kleshnina M, Filar JA (2022) Where do mistakes lead? A survey of games with incompe47 tent players. Dyn Games Appl. https://doi.org/10.1007/s13235-022-00425-3

5. Jiang H, Mazalov VV, Gao H et al (2021) Opinion dynamics control in a social network with a commu- nication structure. Dyn Games Appl. https://doi.org/10.1007/s13235-021-00406-y

6. Mao W, Başar T (2022) Provably efficient reinforcement learning in decentralized general-sum Markov games. Dyn Games Appl. https://doi.org/10.1007/s13235-021-00420-0

7. Newton CJ, Ganesh A, Reeve HWJ (2022) Asymptotic optimality for decentralised bandits. Dyn Games Appl. https://doi.org/10.1007/s13235-022-00451-1

8. Phade SR Anantharam V (2021) Learning in games with cumulative prospect theoretic preferences. Dyn Games Appl. https://doi.org/10.1007/s13235-021-00398-9

9. Ramirez S, van Brandenburg LH, Bauso D (2022) Coordinated replenishment game and learning under time dependency and uncertainty of the parameters. Dyn Games Appl. https://doi.org/10.1007/s13235-022-00441-3

10. Sorin S (2022) Continuous time learning algorithms in optimization and game theory. Dyn Games Appl. https://doi.org/10.1007/s13235-021-00423-x

11. Subramanian J (2021) Robustness and sample complexity of model-based MARL for general-sum Markov games. https://doi.org/10.1007/s13235-023-00490-2

12. Tang D, Tavafoghi H, Subramanian V et al (2022) Dynamic games among teams with delayed intra-team information sharing. Dyn Games Appl. https://doi.org/10.1007/s13235-022-00424-4

13. Trivedi P, Hemachandra N (2022) Multi-agent natural actor-critic reinforcement learning algorithms. Dyn Games Appl. https://doi.org/10.1007/s13235-022-00449-9

14. uz Zaman MA, Miehling E, Başar T (2022) Reinforcement learning for non-stationary discrete-time linear-quadratic mean-field games in multiple populations. Dyn Games Appl. https://doi.org/10.1007/s13235-022-00448-w

<figure><img src="image_6.png" /></figure>