Title: Theoretical Guarantees for the Retention of Strict Nash Equilibria by Coevolutionary Algorithms
Venue: NeurIPS 2025
Accepted: yes
Keywords: coevolution

adversarial optimisation

Nash equilibria
Abstract: Most methods for finding a Nash equilibrium rely on procedures that operate over the entire action space, making them infeasible for settings with too many actions to be searched exhaustively. Randomised search heuristics such as coevolutionary algorithms offer benefits in such settings, however they lack many of the theoretical guarantees established for exhaustive methods such as zero-regret learning. We address this by developing a method for proving necessary and sufficient conditions for a coevolutionary algorithm to be stable, in the sense that it reliably retains a Nash equilibrium following discovery. As the method provides bounds that are adapted to both application and algorithm instance, it can be used as a practical tool for parameter configuration. We additionally show how bounds on regret may be deduced from our results and undertake corresponding empirical analysis.


Peer reviews:
comments: SUMMARY:
In this paper, the authors provide a thorough theoretical analysis of the bounds of the coevolutionary algorithms (CoEAs), which specify the necessary and sufficient conditions for a CoEA to be stable. Besides, the authors also provide bounds on regret based on the obtained results. Empirical studies also confirm the theoretical guarantees obtained in the paper.

STRENGTHS AND WEAKNESSES:
The motivation of the paper is clear and reasonable. The theoretical results in the paper could have important implications in guiding the design of CoEAs for finding the Nash equilibrium stably. The paper is well-written, and the results are well-organized.

I am not an expert in theory, so I only check some parts of the theoretical analysis. From my sense, I did not find any significant weakness, and I would like to see more comments from other reviewers.

QUESTIONS:
Will the authors release the code?

LIMITATIONS:
The authors have adequately addressed the limitations of the paper.
TITLE: Official Review
CLARITY: 3
IMPACT: 3
ORIGINALITY: 3
DATE: 2025-06-19

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors study the stability of (two-player) co-evolutionary algorithms, with a particular focus on Bitstring Games (where actions are tuples with binary elements) and games with unique NE. However, some of the results apply outside of these assumptions. The goal of the paper is to establish \\gamma-stability which, informally, states that NE solutions persist in the population for a long time with high probability, given that they were present in the population to begin with. The authors then show negative results on stability for several classes of games, and finally establish regret bounds for zero-sum games.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths: \n1. The paper provides novel contributions to the community regarding the behaviour of derivative-free approaches to solving games. In particular, the formulation and result of \\gamma-stability is novel, as is the regret analysis. \n2. For the most part, the paper is well written, with definitions clearly given, alongside informal statements to aid in understanding\n3. The authors also show that their theoretical results are satisfied in practice, in particular showing their necessary and sufficient conditions for stability, on various choices of \\gamma and mutation rate q. \n4. Whilst the majority of negative results are defined for bitstring games with unique NE, the sufficient condition for stability holds for generic games and Co-evolutionary algorithms.\n5. The results are clearly of use to practitioners in terms of setting the hyperparameters of their mutation method to encourage stability of the learned solutions. \n\nWeaknesses:\n1.  The authors have not sufficiently motivated bit-string games. Whilst it is convenient to design examples (as the authors do for their experiments) they do not discuss the domains in which bitstring games are of interest. \n2. There are a number of issues with notation. In particular:\n     (a) $\\mathcal{X}^\\lambda$ is not defined. By inferring, one realises that it is a subset of $\\mathcal{X}$ with cardinality $\\lambda$. However, without defining it as a countable subset of $\\mathcal{X}$, this is merely inference.\n    (b) The dual use of Q as a subset of $\\mathcal{Y}$ and as a subset of $[0, 1]$ in Defn 2.2 is extremely confusing. Please change this.\n    (c) The selection operator $S$ should be defined implicitly with the cardinality of the set (i.e. write it as $S_\\lambda$). Otherwise its domain varies for different choices of $\\lambda$. \n3. Related to (1), the authors have not provided justification for MJBR or PlantedBilinear as experiments. I would imagine that the reason these were chosen was that their NE is known and so can be injected into the initial population. Even saying this is fine, but it ought to be stated.\n4. Graphs are poorly displayed - fonts are almost unreadable and a legend is not provided. \n5. The proof structure in the Appendix is not clear. Please provide more detail than \"Here is McDiarmid's inequality\". In particular, it would be useful to know a-priori how and where the result will be used (in the proof of Lemma 3.3). \n___\n**Note**: The above criticisms do not have a bearing on the content on the paper, but on its exposition. What follows are more serious concerns.\n___\n5. There is no attempt to provide empirical evidence of the paper's results outside of the assumptions made, (e.g. games that are not Bitstrings). This would have strengthened the paper's contributions in presenting the formalism by which stability can be analysed. This is somewhat remedied by the extensive future work section. \n6. There is *no* related work section. As such, for someone who is not an expert in the field, it is impossible to place the work in the context of other works on co-evolutionary algorithms, or learning methods. The closest is the paragraph on line 40, which argues that theirs is the first work to study stability, rather than runtime analysis (hitting time of solution). However, without a related work section, it becomes impossible to assess the technical novelty of the work in relation to existing works. \n\n\nAdditional minor remarks:\n1. Typo on line 524: \"of pairs of...\"\n2. Line 465:  It would be clearer to say that, since $M_X(x)(x) < 1$, it follows that $M_X(x)(x^\\star) < 1$ rather than in reference to eq (10).\n3. Line 537: I think you mean changing $\\{ (x_j, y_j) \\}_{j \\in [\\lambda]}$ changes the value of $f$ by at most $2 \\lambda$, rather than changing any $(x_j, y_j)$.\n\nQUESTIONS:\n1. Line 541:  I am missing the final step of this proof, could you please explain why the union bound yields the result?\n\n2. Is there a method to estimate $\\delta$ for a given problem class empirically?\n\n3. How do the authors think the results hold for elitist population-based CoEAs (i.e. in which the top candidates are selected and mutated, whilst the remaining are removed from the population)?\n\nLIMITATIONS:\nYes (This section is exceptionally written)", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 2, "IMPACT": 4, "ORIGINALITY": 3, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper addresses the challenge of finding and retaining Nash equilibria in games with large action spaces, where exhaustive search methods become infeasible. While coevolutionary algorithms (CoEAs) offer practical randomized search heuristics, they lack established theoretical guarantees, particularly regarding stability after discovering a Nash equilibrium. The authors develop a theoretical framework to derive necessary and sufficient conditions under which a CoEA remains stable (retaining a Nash equilibrium once found). The paper quantifies the relationship between algorithm stability and mutation strength, revealing that excessive mutation prevents stability, while overly weak mutation hinders exploration. Empirical results validate the theoretical predictions. Furthermore, the authors extend their framework to provide regret bounds for CoEAs, drawing parallels to classical regret guarantees for learning algorithms in small action spaces.\n\nSTRENGTHS AND WEAKNESSES:\nStrength \n\n- The paper did a reasonably good job educating me on the concepts of CoEA, mutation and selection. There seems to be many new ideas in the analysis of this problem, though I am not familiar with the prior work along this line. \n- The setting of this problem is well-motivated for general dynamics evolved among a population. The paper also provides some experiments on problems motivated from some real-world applications. The heat map plots in Figure 2 are very well designed.\n-  The paper clearly acknowledged the limitations of their results.\n\nWeakness\n\n- I am not sure if the problems studied by this paper fits into the theme of Neurips, or its result might be interesting to a general audience at this conference. \n- Perhaps a notation table can help us quickly parse through your technical results.\n\nQUESTIONS:\n- How can we model any real world problem with the framework of CoEA? Can we find a more compelling setting for Neurips audiences such as AI safety, multi-agent interactions or computational social science?\n- How does this problem related to the mean field games and the result and analysis there?\n- How should we understand the relationship between CoEA and standard multi-agent learning problem under simple normal-form games? Are any of the insight obtained from CoEA useful there?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 2, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 4, "DATE": "2025-07-03"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper investigates the theoretical guarantees of coevolutionary algorithms (CoEAs) for identifying Nash equilibria (NE) in games. The authors introduce the concept of stability for CoEAs, defined as an algorithm’s ability to retain a discovered Nash equilibrium over a long period with high probability, both increasing exponentially with population size. They derive theoretical bounds on the mutation strength under which different classes of CoEAs are guaranteed to be stable or unstable. These theoretical results are also validated empirically. Finally, by combining the stability assumption with a runtime analysis, a bound on the expected time to find a NE for the first time, the authors demonstrate that CoEAs applied to zero-sum games with a single pure NE are ε-Hannan consistent, meaning that the time-averaged regret converges almost surely to a value below ε.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\nThe paper is generally clearly written and easy to follow. In particular, the visualizations of the derived theoretical bounds with their empirical validation are helpful.\nThe authors introduce and prove a novel theory of stability for evolutionary algorithms, which traditionally lack strong theoretical guarantees.\nThe derived mutation strength bounds could be valuable for practitioners by helping them to avoid parameter settings that cause algorithmic instability.\n\nWeaknesses:\nThe notion of stability is not, by itself, particularly insightful, as it only provides a necessary condition for practical algorithm behavior. It remains unclear how mutation strength within the “stable” region affects other algorithmic properties, and thus how one should optimally select this parameter.\nDue to these limitations, the overall significance and practical impact of the theoretical results seem rather narrow.\nDefinition 2.1 of a game is somewhat confusing and unconventional, as it includes a definition of a solution concept as a simple subset of players’ actions without further clarification. \nThere appears to be a typo in Chapter 5, where the definition of y_t should use arg⁡min instead of arg⁡max.\nAn issue (and question) I have relates to Theorem (5.1). The way I understand this, it assumes pure Nash (SINGLETONSOLUTION). But this is a very pathological assumption, making many things trivial (e.g. computing Nash even in non-zero sum games becomes polynomial, etc). Am I missing something? Why is that case interesting?\n\nQUESTIONS:\nHow exactly could practitioners make use of the derived bounds on the mutation strength for the algorithm stability besides just avoiding unstable regions?\nWhat exactly is the class of games assumed in Theorem 5.1? Are those indeed only zero-sum games with just a single pure NE?\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-03"}