Title: How LLMs Learn to Reason: A Complex Network Perspective
Primary area: interpretability and explainable AI
Keywords: Interpretability, Large Language Models, Reinforcement Learning from Verifiable Rewards (RLVR), Complex Network Models, Frustration and Phase Transitions, Policy Collapse

Abstract: Training large language models with Reinforcement Learning from Verifiable Rewards (RLVR) exhibits a set of distinctive and puzzling behaviors that remain poorly understood, including a two-stage learning curve, V-shaped response-length trajectories, and a pronounced vulnerability to catastrophic forgetting. In this work, we propose that these seemingly disparate phenomena can be explained using a single unifying theory: the model’s reasoning process maps to the self-organization of a semantic complex network whose topology remains persistently sparse, with the average degree pinned close to two. This topology imposes a fundamental mechanism for forgetting and learning: it first drives the system into a maximally frustrated state where "skill islands" form, slow-learning happens, and forgetting is induced; then it enters a sharp growth phase where the new skills are "bolted on", driven by phase-transition-like learning at the web’s frontier. Equipped with the theory, we propose \textit{Annealed-RLVR}, a principled algorithm that introduces an SFT-based "heating" step at the point of maximal frustration to resolve the competitive bottleneck and enhance the reasoning capability of the model. Experiments on a 1.5B-parameter model demonstrate that the approach outperforms standard RLVR on both in-distribution and out-of-distribution benchmarks. By recasting RLVR from black-box optimization into a predictable process of structural self-organization, our work provides a new physical intuition for engineering the emergent reasoning capabilities of future AI systems.

TL;DR: We explain RLVR training dynamics in LLMs via frustration and phase-transition mechanisms, introduce a minimal complex network model, and propose Annealed-RLVR to restore exploration.