Kati8/ICLR_2023_2025 huggingface dataset:
authors: Noboru Isobe
authors: Kenshi Abe
authors: Kaito Ariu
reviews: 
Comments: The paper studied regularized MFG and proposed a mirror descent algorithm that can converge in $O(\log(1/\epsilon))$ iterations.
Strengths: The paper is well-written and the proof is clear. The result is the first last-iterate convergence in regularized MFG to my best knowledge.
Weaknesses: My main concern is that, with the monotonicity condition and the regularization $\lambda$, the results can be straightforwardly obtained given the existing proof of exponentially fast convergence of MD in monotone games. See [1] for details. 

Specially, the relationship between $D_{\mu^\ast} (\pi^\ast, \pi^{t+1})$ and $D_{\mu^\ast} (\pi^\ast, \pi^{t})$ has already been derived in Zhang et al 2023. To obtain the exponentially fast convergence rate, you would only need to get the $1-\lambda \eta$ factor out of $D_{\mu^\ast} (\pi^\ast, \pi^{t})$. One common way to achieve this is to use the monotonicity condition. From what I understand, this paper uses the same technique that has been used in papers such as [1]. 


[1] Cen, Shicong, Yuting Wei, and Yuejie Chi. "Fast policy extragradient methods for competitive games with entropy regularization." Advances in Neural Information Processing Systems 34 (2021): 27952-27964.
Recommendation: 5
Reviewer confidence: 3
Soundness/correctness: 3
Clarity: 2
reviews: 
Comments: This paper studies the convergence performance in Monotone Mean Field Games. The major issue is that the focus of this paper are all well studied. Authors cannot identify the research gap and contribution in a comparison to existing study.
Strengths: The paper is well structured and organized.
Weaknesses: 1. Position: Last-iterate convergence and learning regularized MFGs have been extensively studied. However, the discussion of related work in this manuscript is somewhat superficial. The authors should conduct a thorough literature review and clearly position their work within the existing literature. This will help avoid giving the audience the impression that their contribution is marginal, merely by applying a proximal point iteration to MFGs.

2. Exponential convergence of mirror descent: Using mirror descent to learn regularized mean field games (MFGs) is not a novel approach. To the best of my knowledge, previous studies have established a standard convergence rate of $T^{-1/2}$. In contrast, this work claims an exponential convergence rate of $e^{-T}$, which represents a significant improvement. The authors should clearly explain the specific challenges they addressed to achieve this enhanced convergence rate that previous work did not tackle. Additionally, providing numerical demonstrations of such exponential convergence is essential, given the magnitude of this improvement.

3. Presentation: the presentation of the paper is not yet suitable for scientific publication. Please avoid excessive use of colors, shapes, and unconventional layouts. Figure 1 lacks informative content. Additionally, some language and structural transitions are too casual and do not provide sufficient discussion to support the arguments presented convincingly.

4. The numerical experiments are limited. More results regarding the algorithm performance in a comparison to baselines (OMD, FP, Multi-scale, double loop, single loop) for MFGs should be provided.
Recommendation: 5
Reviewer confidence: 4
Soundness/correctness: 3
Clarity: 1
reviews: 
Comments: This paper proposes and algorithm for mean field games (MFGs) based on proximal-point-type iterations. After proving convergence of the last iterate for an ideal algorithm, the paper considers an approximate algorithm based on mirror descent updates. For regularized MFGs, a proof of convergence is provided, with a rate of convergence. A simple illustrative example is shown at the end.
Strengths: The paper is relatively clear and addresses an interesting question (last-iterate convergence). It seems to have a good theoretical foundation.
Weaknesses: The first result holds under general assumptions but the algorithm cannot be implemented. The second result holds only for regularized MFGs, which can be quite different from the original (unregularized) MFG problem and there is not much discussion about this. As a consequence, the contributions seem relatively limited.
Recommendation: 5
Reviewer confidence: 4
Soundness/correctness: 3
Clarity: 2
reviews: 
Comments: This paper proposes algorithms for achieving fast convergence to equilibria in monotone mean field games. The first (conceptual) algorithm is based on a proximal point method type of algorithm that is based on possibly hard to implement update step. Next, the authors employ the Mirror Descent algorithm for a regularized MFG which can efficiently approximate the update of the proximal point method. This process converges exponentially fast to equilibria of the regularized game.
Strengths: - the paper studies a well motivated problem (MFG games + monotonicity).
- it contains provable theoretical analysis of their last iterate results.
- the proof structure is easy to parse.
Weaknesses: - the theoretical results feel somewhat incremental from a technical stand point. E.g. the use of the regularized variant of MD to achieve fast convergence in monotone settings feel very similar to e.g. [1]. Clearly, there are more technicalities involved to adapt the results to this precise setting but arguably these technicalities do not feel conceptually/technically novel. On the somewhat positive side, this makes parsing the proof rather straightforward.
- the experiments could be expanded. For example, prior work that studied last-iterate convergence in (non-strict) monotone settings used the experiments to study the effects of equilibrium selection [2]. Although the possibility of diverse equilibria is advertised in the paper as an important element of monotone games, not present in strictly monotone games, the implications of which equilibria get selected is not addressed. I feel that this is a missed opportunity that would make the paper significantly stronger.

[1] Julien Perolat, et al. From Poincare recurrence to convergence in imperfect information games: Finding equilibrium via regularization. In ICML, volume 139 of Proceedings of Machine Learning Research, pp. 8525–8535. PMLR, 2021.
[2] Leonardos, S., et al. Exploration-exploitation in multi-agent competition: convergence with bounded rationality. Advances in Neural Information Processing Systems, 2021.
Recommendation: 5
Reviewer confidence: 4
Soundness/correctness: 3
Clarity: 3