Spotting LLMs With Binoculars: Zero-Shot Detection of Machine-Generated
  Text
Authors: name: Abhimanyu Hans, name: Avi Schwarzschild, name: Valeriia Cherepanova, name: Hamid Kazemi, name: Aniruddha Saha, name: Micah Goldblum, name: Jonas Geiping, name: Tom Goldstein

Detecting text generated by modern large language models is thought to be
hard, as both LLMs and humans can exhibit a wide range of complex behaviors.
However, we find that a score based on contrasting two closely related language
models is highly accurate at separating human-generated and machine-generated
text. Based on this mechanism, we propose a novel LLM detector that only
requires simple calculations using a pair of pre-trained LLMs. The method,
called Binoculars, achieves state-of-the-art accuracy without any training
data. It is capable of spotting machine text from a range of modern LLMs
without any model-specific modifications. We comprehensively evaluate
Binoculars on a number of text sources and in varied situations. Over a wide
range of document types, Binoculars detects over 90% of generated samples from
ChatGPT (and other LLMs) at a false positive rate of 0.01%, despite not being
trained on any ChatGPT data.