On scalable oversight with weak LLMs judging strong LLMs
Authors: name: Zachary Kenton; name: Noah Y. Siegel; name: János Kramár; name: Jonah Brown-Cohen; name: Samuel Albanie; name: Jannis Bulian; name: Rishabh Agarwal; name: David Lindner; name: Yunhao Tang; name: Noah D. Goodman; name: Rohin Shah
Published: 2024-07-05 16:29:15

Abstract: Scalable oversight protocols aim to enable humans to accurately supervise
superhuman AI. In this paper we study debate, where two AI's compete to
convince a judge; consultancy, where a single AI tries to convince a judge that
asks questions; and compare to a baseline of direct question-answering, where
the judge just answers outright without the AI. We use large language models
(LLMs) as both AI agents and as stand-ins for human judges, taking the judge
models to be weaker than agent models. We benchmark on a diverse range of
asymmetries between judges and agents, extending previous work on a single
extractive QA task with information asymmetry, to also include mathematics,
coding, logic and multimodal reasoning asymmetries. We find that debate
outperforms consultancy across all tasks when the consultant is randomly
assigned to argue for the correct/incorrect answer. Comparing debate to direct
question answering, the results depend on the type of task: in extractive QA
tasks with information asymmetry debate outperforms direct question answering,
but in other tasks without information asymmetry the results are mixed.
Previous work assigned debaters/consultants an answer to argue for. When we
allow them to instead choose which answer to argue for, we find judges are less
frequently convinced by the wrong answer in debate than in consultancy.
Further, we find that stronger debater models increase judge accuracy, though
more modestly than in previous studies.