GL a PE : Gold Label-agnostic Prompt Evaluation for Large Language Models (2024)

Abstract: Despite the rapid progress of large language models (LLMs), their task performance remains sensitive to prompt design. Recent studies have explored leveraging the LLM itself as an optimizer to identify optimal prompts that maximize task accuracy. However, when evaluating prompts, such approaches heavily rely on elusive manually annotated gold labels to calculate task accuracy for each candidate prompt, which hinders its generality. To overcome the limitation, this work proposes GLaPE, a gold label-agnostic prompt evaluation method to alleviate dependence on gold labels. GLaPE is composed of two critical aspects: self-consistency evaluation of a single prompt and mutual-consistency refinement across multiple prompts. Experimental results on 8 widely-recognized reasoning tasks demonstrate that GLaPE can produce more effective prompts, achieving performance comparable to those derived from manually annotated gold labels. Analysis shows that GLaPE provides reliable evaluations aligned with accuracy, even in the absence of gold labels. Code is publicly available at **Anonymous**.

## introduction as the integration of large language models (llms) into natural language processing tasks has become imperative in recent years @xcite @xcite , the sensitivity of the performance of llms to prompts has garnered significant attention @xcite . while traditional soft prompt tuning methods @xcite @xcite demonstrate effectiveness in guiding the llm to perform desired tasks, they encounter limitations when applied to private llms, such as @xcite . this situation necessitates the exploration of effective strategies for optimizing prompts without requiring gradient updates. recent studies @xcite have unveiled a noteworthy strategy, where the llm itself acts as the optimizer to seek the prompt that maximizes task accuracy. specifically, opro @xcite provides an intriguing avenue for prompt optimization based on a gold label evaluation recipe (figure 1a ). the optimization commences with an initial prompt, then iteratively evaluates existing prompts and generates novel prompts based on prior assessments. however, a significant caveat emerges as these studies heavily rely on manually annotated gold labels. concretely, the gold label, representing the ideal output, serves as a crucial ingredient for evaluating and refining prompts. nevertheless, the acquisition of such gold labels poses a formidable obstacle @xcite , introducing complexity and hindering the widespread implementation and generality of these optimization techniques. therefore, exploring alternative methodologies becomes mandatory to address these challenges and improve the efficiency of prompt evaluation and optimization for llms. to address the limitations, this work proposes a gold label-agnostic prompt evaluation (glape) method to identify prompts that facilitate consistent and accurate answers. instead of relying on gold labels, glape evaluates prompts based on two critical aspects: self-consistency evaluation and mutual-consistency refinement. inspired by @xcite , we first consider a naive solution by utilizing self-consistency (sc) as the evaluation metric instead of accuracy, as correct answers generally exhibit higher sc than incorrect ones. ## related work prompt optimization in the domain of llms @xcite @xcite , prompt engineering plays a crucial role in guiding models to generate desired outputs across diverse tasks @xcite . consequently, optimizing prompts becomes paramount for enhancing the performance and efficiency of llms. various soft prompt tuning methods @xcite @xcite have been explored in previous research to optimize prompts for open-source llms. however, these methods encounter challenges when applied to private llms, where accessing gradients is infeasible. consequently, diverse gradient-free prompt optimization techniques @xcite @xcite have been explored. recent works @xcite have embraced an iterative process for gradient-free prompt optimization, commencing from an initial prompt and iteratively assessing existing prompts while generating new ones based on prior evaluations. nevertheless, these iterative prompt optimization methods heavily depend on gold labels for prompt evaluation. relying on gold labels not only limits the generalizability of these prompt optimization methods but may also introduce other potential issues @xcite . in our work, we propose a novel gold label-agnostic prompt evaluation method and subsequently present a unique approach to optimize prompts for llms without the constraints associated with conventional gold label reliance. prompt selection prompt selection tasks aim to identify the optimal prompt among candidates for a given task, representing an alternative approach to prompt optimization. recent studies have delved into probability-based evaluation methods, utilizing diverse metrics such as mutual information @xcite , entropy @xcite , and perplexity @xcite . in contrast to these probability-centric assessments, our proposed evaluation approach exclusively relies on the output, making it applicable to private llms where only the output is accessible. ## investigating gold label-agnostic prompt evaluation according to section 3.1, the evaluation function f in existing studies measures the goodness of the prompt candidate ρ by maximizing the task accuracy. however, in real-world tasks, obtaining gold labels poses a considerable challenge, limiting figure 2: the schematic representation of our glape method integrating self-consistency (sc) evaluation and mutual-consistency (mc) refinement. this sketch illustrates how our method assesses the prompts in figure 1 ; computation details are provided in appendix a.2. notably, we observed that prompt3, as indicated by the red marker, produces an incorrect answer with high self-consistency (70%). through the mutual-consistency refinement, our glape score experiences a decrease of 20.0, rendering it more discernible when compared to prompt1 and prompt2. the texts are favorably read in colors of background. blue: self-consistency, purple: mutual-consistency refinement, green: answer "31" (gold label), orange: answer "36", yellow: glape metric. the generalization of existing prompt optimization methods. furthermore, we ultimately expect llms to solve problems for which answers are not already known. therefore, when optimizing prompts to enhance performance, gold labels are not readily available. thus, it is imperative to find a gold labelagnostic prompt evaluation method. in this section, we will investigate the challenge of gold label-agnostic prompt evaluation and study how to design an effective approach to overcome the challenge. ## rethink on gold label-agnostic prompt optimization our amalgamation of self-consistency evaluation and mutual-consistency refinement facilitates the identification of prompts leading to correct answers. however, we also observe a diminished spearman correlation coefficient between our glape and accuracy on the aqua dataset and strategyqa dataset, as depicted in ## conclusion this work presents an innovative gold labelagnostic prompt evaluation methodology that operates in the absence of gold labels. by integrating self-consistency evaluation and mutual-consistency refinement, our evaluation demonstrates a strong correlation with accuracy. subsequently, we example 1: was iggy pop named after his father? label: yes. llm answer & explanation: no. iggy pop, whose real name is james newell osterberg jr., was not named after his father. his father's name was james newell osterberg sr., so they shared the same first name but not the same nickname. iggy pop got his stage name from his first band, the iguanas, and later shortened it to iggy. example 2: is great pyramid of giza the last wonder of its kind? label: yes. llm answer & explanation: no. the great pyramid of giza is one of the seven wonders of the ancient world. it is the only wonder from that list that still exists today. however, there have been other lists of wonders created throughout history. for example, the new7wonders foundation conducted a global poll in 2007 to determine the new seven wonders of the world. incorporate our metric into prompt optimization, yielding prompts comparable to those optimized based on accuracy metrics across various tasks.