Title: Why Playing Against Diverse and Challenging Opponents Speeds Up Coevolution: A Theoretical Analysis on Combinatorial Games
Venue: NeurIPS 2025
Accepted: yes
Keywords: coevolution

runtime analysis

combinatorial games
Abstract: Competitive coevolutionary algorithms (CoEAs) have a natural application to problems that are adversarial or feature strategic interaction. However, there is currently limited theoretical insight into how to avoid pathological behaviour associated with CoEAs. In this paper we use impartial combinatorial games as a challenging domain for CoEAs and provide a corresponding runtime analysis. By analysing how individuals capitalise on the mistakes of their opponents, we prove that the Univariate Marginal Distribution Algorithm finds (with high probability) an optimal strategy for a game called Reciprocal LeadingOnes within $O(n^2\log^3{n})$ game evaluations, a significant improvement over the best known bound of $O(n^5\log^2{n})$. Critical to the analysis is the introduction of a novel stabilising operator, the impact of which we study both theoretically and empirically.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe main contribution of this paper is to advance the theoretical understanding of coevolution by undertaking a more detailed runtime analysis of CoEAs on combinatorial games, which are a challenging domain for CoEAs. Through this detailed analysis, it proves an improved upper bound, as well as a new lower bound that shows the result is tight up to a polylogarithmic factor. It also shows that an EA cannot efficiently optimise Reciprocal LeadingOnes if opponents are sampled purely uniformly at random.\n\nSTRENGTHS AND WEAKNESSES:\nThe idea presented in the paper seems sound. The exposition is mostly clear. Its findings are relevant to the field of CoEA and appear to be novel.\n\nPage-by-page comments and suggestions:\n\nPage 1:\n\n4: associated to -> associated with\n\n5: provide -> provide a\n\n28-32: Split into separate sentences\n\nPage 2:\n\n43: and so we consider this as a -> so we consider this a\n\n52: Add comma after \"even in such cases\"\n\n82-83: Put AlphaStar comment in new sentence\n\nPage 3:\n\n87: Add comma after \"In this paper\"\n\n88-89: Replace semicolon with period (new sentence)\n\n89: \"a method of ranking diversity\" Briefly elaborate.\n\n119: Make \"however...\" a new sentence\n\n120: \"we only use that when |S_i| = 2 the constraint...\" grammatically awkward, reword\n\n122: difference from -> difference to\n\n134: Put quotes around 'balanced' and 'stable', since these are technical terms that are being introduced\n\nPage 4:\n\n135: Suggest clarifying explicitly what the expectation is with respect to\n\n138: Suggest adding a brief informal description of LeadingOnes here\n\n148: state -> describe\n\n153: empty the -> empty, the\n\nPage 5:\n\nMove Figure 1 to this page, if possible\n\nPage 6:\n\nFigure 3: Consider adding error bars (e.g. standard error of the mean)\n\n190: Period before \"however\" (new sentence)\n\n194-195: Broken citations\n\n208: What is remarkable is that -> It is remarkable that, \n\nPage 8:\n\nFigure 5: Font size for the axes labels is too small\n\n252: who avoids -> avoids\n\n264-265: What precisely do you mean by \"is not guaranteed to exist in general\"?\n\n265: Add comma before \"there is\"\n\nPage 9:\n\n294: Add comma after \"In addition\"\n\n300: which has here been generalised -> which, here, has been generalised\n\n307: to to -> to\n\nQUESTIONS:\nTo what extent can the findings (other than Theorem 1.2/6.1, which applies generally) be generalized or extended to games beyond RLO? Would doing so require substantially different proof techniques?\n\nDo you currently have any theoretical insight on the c parameter of the Mutate operator (lines 202-212), or only empirical results?\n\nHow well do your results generalize to the non-binary case (lines 119-121)?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-18"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper continues the line of work that analyse self-playing Univariate Marginal Distribution Algorithm in combinatorial games. A game, whose optimal strategies are characterized by a sequence of independent \"correct\" moves, is introduced as a testbed to analyse the dynamic of coevolution. The analysis provides insight into the utility of maintaining diverse opponents in finding marginal move distributions aligned with optimal strategies. This insight is demonstrated empirically.\n\nSTRENGTHS AND WEAKNESSES:\nThe studied problem is significantly more challenging than optimizing LeadingOnes since there is no measure of \"progress\" towards optimal play. Rather, the algorithm is responsible for maintaining this \"measure\" via self-play, in the form of adversarial strategies. The simplicity of the combinatorial game, thus, helps expose the effect of opponent distributions on the discovery of best strategies. As such, the study gives meaningful insight into the self-play dynamics, which is reminiscent of the bias-variance trade-off phenomenon in reinforcement learning.\n\nThe theoretical findings are summarized in two theorems. The second one is particularly useful since it applies to general black-box searchers. On the other hand, the first one is conditioned on rather stringent parameter settings and the use of the novel mutation operator. A more fine-grained analysis of the effect of the mutation's parameter $c$ on the discovery time distribution would be appreciated.\n\nThe proof details are very involved, and not described in a self-contained manner. Following the proofs is challenging as the result.\n\nSome minor things:\n\nOne of the two experiments in the paper is not presented with enough information. Specifically the one on the RLO: the number of runs is not given and the variances are not shown. Adding log-scaled variance whiskers to the points in Figure 3 should suffice.\n\nFigure 4: two more trajectories (one for each subfigure) should probably be added whose starting point matches the initial distribution $p_0$ in Algorithm 1.\n\nFigure 6: perhaps the overall runtimes can be added to the second set of images to contextualize the horizontal scaling.\n\nRasterized images are used in figures. It is advisable to use vectorized images instead.\n\nQUESTIONS:\nTheorem 1.1 summarizes the upper bound as $O(n^2\\log^3n)$. This means that $C_{pop}$ in Theorem 5.2 is $O(1)$. It is not clear how the assumption of Theorem 5.2 implies this. Is it not possible that $C_{pop}$ is $\\Theta(n)$ or $\\Theta(\\log n)$?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 4, "DATE": "2025-06-30"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe manuscript applies the Estimation-of-Distribution Algorithm Tournament UMDA to a newly introduced combinatorial game, Reciprocal LeadingOnes (RLO). It derives a better upper bound on the expected runtime for solving RLO and reports empirical results on several small-scale games that suggest a performance advantage over baseline methods.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths.\n- **Improved theoretical bound** The analysis yields a better upper bound on runtime than earlier results, marking clear analytical progress.\n\n- **Potential broader impact** Although the experiments target relatively simple games, the approach could generalise to more complex domains, mirroring the way EDA-inspired ideas have influenced large-scale systems such as AlphaStar.\n\nWeaknesses.\n- **Novelty of the algorithm** Because Tournament UMDA itself was introduced in reference [3], the contribution here appears to be its application and analysis. Please clarify which aspects-parameter settings, proof techniques, or new insights-differ from the original proposal.\n\n- **Status of RLO** The paper seems to be the first to study Reciprocal LeadingOnes. If RLO is indeed novel, please state this explicitly and outline why analysing this game is interesting or representative of a broader problem class.\n\n- **Significance of the bound** The prior bound you improve upon was obtained with the same algorithm in a very recent study [3]. Please elaborate on why this incremental improvement matters-for example, does it reveal new theoretical behaviour or enable practical speed-ups on larger instances?\n\nMinor Comments\n- **Comprehensive appendix** The detailed proof in the appendix is valuable. Including a concise, reader-friendly proof sketch in the main text (e.g., an outline of key lemmas and ideas) would make the paper easier to follow without forcing readers to consult supplementary material.\n\n- **Figure 2 readability** The legend font is too small; please enlarge it for clarity.\n\nQUESTIONS:\n- Please address the three points raised in the “Weaknesses” section above.\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-06"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors conduct runtime analysis of an existing competitive co-evolutionary algorithm (CoEA), the univariate marginal distribution algorithm (UMDA), on a specific combinatorial game (Reciprocal LeadingOne, RLO). By introducing a novel mutation operator, for suitably small games the authors show that UMDA's runtime on RLO has an upper bound $\\mathcal{O}(n\\log^2n)$, improving on the previously known tightest upper bound. The authors also show that their variant of UMDA on RLO has a runtime lower bound of $\\mathcal{O}(n^2\\log n)$, which is the first lower bound established in this setting. The authors also present a corollary showing that not just population diversity, but also coevolutionary feedback between player and opponent is fundamental to learning.\n\nSTRENGTHS AND WEAKNESSES:\n# Strengths\n\n* The paper is of utmost clarity and very well-written.\n* The theoretical results, as far as I have been able to verify them in detail, represent significant progress in the chosen setting.\n* The empirical results do illustrate practical utility in the authors' chosen mutation operator across a variety of games (Table 1), although it remains unclear whether the results are statistically significant (also see Figure 2).\n\n# Weaknesses\n\n* While I appreciate the novelty of the theoretical results, I am not convinced about the wider theoretical extendability of the authors' findings outside of the chosen UMDA/RLO setting. \n* Likewise, I am not entirely convinced whether the authors' presented empirical results (Figure 2) indeed add evidence for the practical utility of their mutation operator.\n* Figure 3 seems to be missing confidence intervals.\n\nQUESTIONS:\n* RLO is arguably a very artificial game - to what extent does your approach transfer to games of more practical interest?\n* What would be the significance of finding analogous results for EDAs?\n* Can you explain - in a practical context - what \"small\" means wrt game (as you are saying that $\\mathcal{O}(\\sqrt{n})$ games may have better asymptotic bounds)?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 2, "CLARITY": 4, "IMPACT": 3, "ORIGINALITY": 4, "DATE": "2025-07-19"}