Evaluating Frontier Models for Dangerous Capabilities
Authors: name: Mary Phuong; name: Matthew Aitchison; name: Elliot Catt; name: Sarah Cogan; name: Alexandre Kaskasoli; name: Victoria Krakovna; name: David Lindner; name: Matthew Rahtz; name: Yannis Assael; name: Sarah Hodkinson; name: Heidi Howard; name: Tom Lieberum; name: Ramana Kumar; name: Maria Abi Raad; name: Albert Webson; name: Lewis Ho; name: Sharon Lin; name: Sebastian Farquhar; name: Marcus Hutter; name: Gregoire Deletang; name: Anian Ruoss; name: Seliem El-Sayed; name: Sasha Brown; name: Anca Dragan; name: Rohin Shah; name: Allan Dafoe; name: Toby Shevlane
Published: 2024-03-20 17:54:26

Abstract: To understand the risks posed by a new AI system, we must understand what it
can and cannot do. Building on prior work, we introduce a programme of new
"dangerous capability" evaluations and pilot them on Gemini 1.0 models. Our
evaluations cover four areas: (1) persuasion and deception; (2) cyber-security;
(3) self-proliferation; and (4) self-reasoning. We do not find evidence of
strong dangerous capabilities in the models we evaluated, but we flag early
warning signs. Our goal is to help advance a rigorous science of dangerous
capability evaluation, in preparation for future models.