Title: The Cost of Delegation
Primary area: reinforcement learning
Keywords: theory, alignment, multi-agent reinforcement learning

Abstract: We study multi-agent reinforcement learning and alignment through the lens of hierarchical coordination, where a principal steers many delegates with partial views and coupled effects. From first principles we show that delegation induces a quadratic game surrogate but also an irreducible cubic penalty—the cost of delegation. We characterize when structured coordination (low-rank plus sparse, trees, block-sparse) yields stable, tractable control and sharp performance bounds despite this penalty. Our work provides a theoretical foundation and new perspective for designing aligned, scalable multi-agent systems.

TL;DR: You can’t captain every oar—so you hire. Alignment issues are delegation problems.