Challenges in Trustworthy Human Evaluation of Chatbots
Authors: name: Wenting Zhao, name: Alexander M. Rush, name: Tanya Goyal

Open community-driven platforms like Chatbot Arena that collect user
preference data from site visitors have gained a reputation as one of the most
trustworthy publicly available benchmarks for LLM performance. While now
standard, it is tricky to implement effective guardrails to collect
high-quality annotations from humans. In this paper, we demonstrate that three
sources of bad annotations, both malicious and otherwise, can corrupt the
reliability of open leaderboard rankings. In particular, we show that only 10\%
of poor quality votes by apathetic (site visitors not appropriately
incentivized to give correct votes) or adversarial (bad actors seeking to
inflate the ranking of a target model) annotators can change the rankings of
models by up to 5 places on the leaderboard. Finally, we discuss open
challenges in ensuring high-quality human annotations.