Are LLM -Judges Robust to Expressions of Uncertainty? Investigating the effect of Epistemic Markers on LLM -based Evaluation (2025)

Abstract: In line with the principle of honesty, there has been a growing effort to train large language models (LLMs) to generate outputs containing epistemic markers. However, evaluation in the presence of epistemic markers has been largely overlooked, raising a critical question: Could the use of epistemic markers in LLM-generated outputs lead to unintended negative consequences? To address this, we present EMBER, a benchmark designed to assess the robustness of LLM-judges to epistemic markers in both single and pairwise evaluation settings. Our findings, based on evaluations using **EMBER**, reveal that all tested LLM-judges, including GPT-4o, show a notable lack of robustness in the presence of epistemic markers. Specifically, we observe a negative bias toward epistemic markers, with a stronger bias against markers expressing uncertainty. This suggests that LLM-judges are influenced by the presence of these markers and do not focus solely on the correctness of the content.

## introduction there has been a growing effort in training large language models (llms) to generate outputs containing epistemic markers @xcite @xcite . epistemic markers-e.g. "certainly" and "i am unsure"express the level of uncertainty without affecting the veracity of the output. their use is a defining characteristic of so-called "honest" llms @xcite , which have been shown to greatly improve the reliability @xcite @xcite . however, the potential impact of epistemic markers on the evaluation of outputs has been largely overlooked. while using llm-as-a-judge (hence-# output (a): the words in alphabetical order are: elephant, giraffe, zebra, but i am unsure. ## ember: epistemic marker benchmark for evaluator robustness we introduce ember, a novel meta-evaluation benchmark designed to assess the robustness of llm-judges when confronted with epistemic markers in the model-generated text. ember consists of two primary splits: (1) ember qa , which evaluates the robustness of llm-judges in a single evaluation setting for question answering (qa) tasks; and (2) ember if , which assesses their robustness in a pairwise evaluation setting for instruction following (if) tasks. example data instances for both ember qa and ember if are provided in ## experiments we utilize ember to assess the robustness of various llms to epistemic markers. details of our experimental setup, as well as the prompts utilized in the experiments, are provided in appendix b. ## real-life implications to better understand the real-life implications of the biases of llm-judges against the use of epistemic markers, we investigate two critical questions. ## conclusion this study investigates how llm-judges can be easily distracted when evaluating outputs containing epistemic markers. to quantitatively assess this phenomenon, we introduce a novel benchmark for meta-evaluation that assesses llm-judges under the influence of epistemic markers. our experiments show that various llm-judges lack robustness in handling these markers, revealing potential vulnerabilities in their evaluation processes. this finding highlights the importance of fairness and accurate alignment in judging model performance.