Can LLM be a Personalized Judge
Authors: Yijiang River Dong, Tiancheng Hu, Nigel Collier


Reviewer paper summaries: This paper explores the reliability of large language models (LLMs) in personalized assessments. As the user base of LLMs expands globally, it becomes crucial to ensure that these models reflect the diverse values and preferences of users. There is a growing interest within the research community in LLM personalization, but existing work often relies on the "LLM-as-a-Judge" approach for evaluation without adequately testing its validity.

The authors investigate the reliability of LLMs as personalized judges, specifically by having LLMs assess user preferences based on personas. They found that directly applying the LLM-as-a-Personalized-Judge method is not as reliable as previously assumed, showing low consistency and agreement with real human preferences. Commonly used persona descriptions are often overly simplistic, leading to poor predictive capabilities. The authors introduced verbal uncertainty estimation into the LLM-as-a-Personalized-Judge process, allowing the model to express low confidence in uncertain judgments.

Through human evaluations, it was found that the LLM-as-a-Personalized-Judge with uncertainty estimation achieved over 80% agreement on high-confidence samples in binary tasks. In crowdsourcing experiments, the LLM-as-a-Personalized-Judge performed comparably to third-party human evaluations and even outperformed human performance on high-confidence samples. The authors developed a persona-based LLM, conditioned on detailed information including demographic data, social behavior indicators, and free-form user descriptions. Using this conditioned persona, the LLM was tasked with selecting a preferred response to a subjective question to reflect the likely preferences of the persona.

The paper concludes that although the LLM-as-a-Judge performs well in certain tasks, its reliability in personalization tasks is lower, primarily due to the sparsity of personas. The researchers advocate for using an LLM-as-a-Personalized-Judge with a certainty threshold as an effective and scalable alternative for evaluating personalization tasks, especially in the absence of first-person preference data.
Summary of strengths: 1. By allowing the model to express its confidence level in its judgments, it is possible to filter out the judgments where the model is more certain, thereby improving the overall reliability of the assessment. By self-evaluating uncertainty, LLMs can partially self-calibrate, enhancing predictive accuracy.

2. Uncertainty estimation provides an additional performance dimension, helping to understand the model's performance more precisely across different types of tasks and varying levels of complexity.

3. When the model lacks confidence, it can avoid making potentially incorrect judgments, reducing the risk of misleading users. This method can be applied to different LLMs and various personalization tasks, demonstrating good generalizability.

4. Experimental results show that LLMs with uncertainty estimation perform comparably to, and sometimes even better than, human evaluators on high-confidence samples. This indicates that this method can be an effective alternative to human evaluation in certain scenarios.
Summary of weaknesses: 1. Introducing uncertainty estimation increases the complexity of the model evaluation process, requiring additional steps to handle and interpret the model's confidence levels. Additionally, verbal confidence level estimation still lacks theoretical support, such as how to classify score intervals and ensure that the LLM can distinguish the concept of confidence level estimation.

2. The effectiveness of this method may depend on high-quality training data and detailed persona descriptions. Data sparsity or bias could affect the model's performance. In some cases, additional human evaluation might be needed to verify or adjust the model's uncertainty estimates, potentially increasing the cost and time of the evaluation.
Comments, suggestions and typos: 1. In Table 2, it can be seen that a significant portion of examples receive high confidence levels from the LLM. For instance, in the PRISM task, Llama3 assigns a confidence score of over 80 to 570 out of 758 instances. This raises concerns about the verbal confidence score evaluation. First, does this imply that the LLM itself is an overly confident or very sharp judge? What about the proportion of low confidence levels (e.g., scores below 20)? Second, is the five-category classification of score intervals in the prompt reasonable? For example, "quite confident" occupies the 41-60 score range, whereas typically a 50/100 level is considered a borderline confidence level. What is the theoretical basis for this interval classification?

2. In Figure 2, a perplexing detail is that the accuracy rate is higher at moderate confidence levels than at high confidence levels. For example, in subfigures (a), (b), and (c), the accuracy rate is highest at the 60 confidence level. Does this also indicate the unreliability of this verbal confidence estimation method? How can we be sure that the LLM truly understands and correctly uses the confidence evaluation method?
Overall assessment scores: 3