A Single Character can Make or Break Your LLM Evals
Authors: name: Jingtong Su, name: Jianyu Zhang, name: Karen Ullrich, name: Léon Bottou, name: Mark Ibrahim

Common Large Language model (LLM) evaluations rely on demonstration examples
to steer models' responses to the desired style. While the number of examples
used has been studied and standardized, the choice of how to format examples is
less investigated. In evaluation protocols and real world usage, users face the
choice how to separate in-context examples: use a comma? new line? semi-colon?
hashtag? etc.? Surprisingly, we find this seemingly minor choice can
dramatically alter model response quality. Across leading model families
(Llama, Qwen, Gemma), performance on MMLU for example can vary by pm 23%
depending on the choice of delimiter. In fact, one can manipulate model
rankings to put any model in the lead by only modifying the single character
separating examples. We find LLMs' brittleness pervades topics, model families,
and doesn't improve with scale. By probing attention head scores, we find that
good-performing delimiters steer attention towards key tokens in the input.
Finally, we explore methods to improve LLMs' robustness to the choice of
delimiter. We find specifying the selected delimiter in the prompt boosts
robustness and offer practical recommendations for the best-performing
delimiters to select.