Title: The Hot Mess of AI: How Does Misalignment Scale With Model Intelligence and Task Complexity?
Primary area: foundation or frontier models, including LLMs
Keywords: Large Language Models, Scaling Laws, Misalignment, Bias-Variance

Abstract: As AIs become more capable, we entrust them with more general and consequential tasks. The risks from failure grow more severe with increasing task scope. It is therefore important to understand the ways extremely capable AIs will fail: Will they fail by systematically pursuing goals we don't intend? Or will they fail by taking nonsensical actions that don't further any goal? We operationalize this question using a bias-variance decomposition of the errors made by models. An AI's \emph{incoherence} on a task is the fraction of its error that stems from variance rather than bias in task outcome. Across a variety of tasks and frontier models, we find that the longer models spend reasoning and taking actions, \emph{the more incoherent} they become. We further find that while more capable AIs have consistently lower error, they are not consistently more coherent. On challenging tasks, incoherence may even increase with model scale. Consequently, scale alone will not eliminate incoherence. Instead, as capable AIs pursue harder tasks, requiring more actions and thought, our results predict more incoherent behavior. This implies a future where AIs sometimes cause industrial accidents (due to unpredictable misbehavior), but are less likely to exhibit consistent pursuit of a misaligned goal.

TL;DR: We propose decomposing AI error with the bias and variance framework, and analyze scaling behavior across task complexity and model scale.