Title: Training as Computation: A Resource-Bounded Theory of Continual Self-Play Learning
Primary area: reinforcement learning
Keywords: Self-Play Learning

Abstract: We study \emph{training as computation} in a continual self-play setting, where a single reasoning model proposes tasks, solves them, and updates itself using verifiable signals from an external executor--verifier interface. Rather than focusing on one-shot models, we analyze the \emph{process-level} dynamics of learning under explicit resource budgets: each generation step is capped by an output budget, and the executor/verifier operate within bounded working space. Within this framework we (i) formalize a general generator--executor--verifier--buffer loop for continual learning with self-proposed curricula; (ii) prove \emph{resource-bounded completeness} at the process level---the set of functions computable by the evolving loop up to time $t$ matches a corresponding $\mathrm{SPACE}[\cdot]$ class determined by the budgets; and (iii) show monotone capability growth under mild, length-aware exploration schedules and curriculum learnability constraints, without assuming non-vanishing exploration or relying on supervised traces. Conceptually, the results separate \emph{capability universality} (as properties of the training \emph{process}) from \emph{alignment and safety} (properties of objectives and verifiers). Empirically, a light-weight self-play prototype on synthetic program-execution and abduction/induction tasks corroborates the theory: the loop reliably expands its reachable task set over time and benefits from curriculum learnability, while remaining grounded by verifiable rewards. This positions continual self-play as a principled path to scalable, data-free improvement under explicit resource budgets.