Venue: NAACL 2025

# Few-shot Personalization of LLMs with Mis-aligned Responses

Jaehyung Kim\({}^{1}\) Yiming Yang\({}^{2}\)

\({}^{1}\)Yonsei University \({}^{2}\)Carnegie Mellon University

jaehyungk@yonsei.ac.kr

This work is done when Jaehyung was postdoc at CMU.

###### Abstract

As the diversity of users increases, the capability of providing personalized responses by large language models (LLMs) has become increasingly important. Existing approaches have only limited successes in LLM personalization, due to the absence of personalized learning or the reliance on shared personal data. This paper proposes a new approach for a few-shot personalization of LLMs with their mis-aligned responses (Fermi). Our key idea is to learn a set of personalized prompts for each user by progressively improving the prompts using LLMs, based on user profile (_e.g._, demographic information) and a few examples of previous opinions. During an iterative process of prompt improvement, we incorporate the contexts of mis-aligned responses by LLMs, which are especially crucial for the effective personalization of LLMs. In addition, we develop an effective inference method to further leverage the context of the test query and the personalized prompts. Our experimental results demonstrate that Fermi significantly improves performance across various benchmarks, compared to best-performing baselines.1

Footnote 1: The code is available at [https://github.com/bbuing9/Fermi](https://github.com/bbuing9/Fermi).

+
Footnote †: _Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers)_, pages 11943–11974

## 1 Introduction

The recent development of large language models (LLMs) has significantly accelerated progress in various NLP tasks, and yielded real-world applications used by millions of users, such as coding assistants and chatbots OpenAI (2022); Team et al. (2023); Touvron et al. (2023). As the use of LLMs by diverse users in real-world applications increases, _personalization_ of LLMs, _i.e._, steering LLMs' responses towards the unique needs or preferences of individual users becomes progressively important Glaese et al. (2022); Solaiman and Dennison (2021). However, recent studies show that LLMs' responses are often biased toward certain groups but not suited for other diverse groups of users, and such biases cannot be fixed by providing simple instructions Santurkar et al. (2023).

Recent work in steering the responses of LLMs can be roughly divided into two categories. One category is prompt engineering, which heuristically incorporates the user's information into the input prompts of LLMs Salemi et al. (2023); Hwang et al. (2023). The other category focuses on learning from other users' data Li et al. (2023); Zhao et al. (2024). Both categories have their limitations: prompt engineering for each user would be too costly as it requires exploring the extensive search space of all possible prompts to find the personalized prompt. Also, the learning-based category relies on the unrealistic assumption that personal data can be shared without violating privacy considerations.

This paper addresses those limitations by introducing a new approach, namely **F**ew-shot **P**ersonalization of LLMs with **mis**-aligned responses (Fermi). Our high-level idea is to use LLM to progressively improve its input prompts based on a few examples of previous user opinions and profiles (_e.g._, demographics) in an iterative process. In addition to the current prompts' scores measured on given few-shot user opinions Yang et al. (2024), Fermi incorporates the _mis-aligned responses_ (_i.e._, LLM's responses with those prompts, which are inconsistent with given user opinions) as additional context. The contexts of mis-aligned responses include useful learning signals to update prompts such as the types of wrong predictions with the current prompts (see the empirical evidence in Section 4). Specifically, the iterative process of Fermi consists of three steps: (1) scoring the initial or current prompts with LLM, (2) updating the memory with high-scored prompts in the form of \(<\)prompt, score, context\(>\) triplets, and (3) generating new improved prompts with LLM based on the updated mem

ory. In addition, we propose Retrieval-or-Prompt, a method to improve the inference on a given test query. Retrieval-or-Prompt selectively uses one of the personalized prompts obtained from the optimization, based on the context of the test query. An overview of Fermi is presented in Figure 1.

We demonstrate the effectiveness of Fermi for few-shot personalization of LLMs, through extensive evaluations on various tasks including question-answering (QA), classification, regression, and generation. For example, we observe that Fermi exhibited 6.8% and 4.1% average accuracy improvements on two multiple-choice QA datasets, constructed to evaluate the personalization of LLMs, compared to the previous state-of-the-art heuristic and optimization approaches, respectively. We also found that the personalized prompts produced with one LLM are also effective on other LLMs, including both API-based and open-sourced ones, which is crucial for efficient deployment in practice. In addition, our in-depth analyses reveal why Fermi is more effective than other prompting methods and what are the important features of prompts for effective personalization of LLMs. We hope our work provides useful insights for the research on LLM personalization, which becomes increasingly emerging and important for the future success of LLMs in real-world applications.

## 2 Related Works

Few-shot personalization of LLMs.Few-shot personalization of LLM is to align LLM's responses to a specific user with a limited number of user information such as user profile (_e.g._, demographic information) or opinions (_e.g._, previous responses to questions by user). To this end, one line of prior works has explored how to input given user information into LLM in a heuristical manner, _i.e._, prompt engineering; for example, Santurkar et al. (2023) designs three different templates of input prompt. Salemi et al. (2023) leverages the retrieval system Izacard et al. (2022) to use the given user opinions selectively. Hwang et al. (2023) shows that using both user profile and opinions is more effective. While PEFT-based personalization method has been explored Tan et al. (2024), it's hard to be applied to recent black-box LLMs. On the other hand, another line of prior works has proposed learning from other user's data; Li et al. (2023) selects the relevant users using collaborative filtering, then learns the soft-prompt Li and Liang (2021) from the augmented training data from these users' data. Zhao et al. (2024) proposes to train an independent transformer module via meta-learning on several users' data. However, both approaches have their limitations; prompt engineering incurs the cost of designing the prompt, and could be limited to fully utilizing the user information due to the absence of learning. The learning-based one necessitates other users' data which inevitably incurs privacy issues. Therefore, we propose to only learn from target user's information and find the optimized (_i.e._, personalized) prompt for that user.

Prompt optimization with LLM.As the prior works for prompt-tuning, relying on the gradient-based update Deng et al. (2022); Lester et al. (2021); Shin et al. (2020), become inapplicable to the recent API-based LLM due to their black-box nature, other approaches have been recently explored for gradient-free prompt optimization, such as a progressive improvement using heuristic rules or LLMs Prasad et al. (2023); Yang et al. (2024); Zhou et al. (2023). For example, Pryzant et al. (2023) receives text feedback on how to update the prompts by instructing LLM. Also, after generating initial prompts with LLMs, Zhou et al. (2023) generates a semantically similar variant of the prompts with the highest accuracies. Yang et al. (2024) iter

Figure 1: **An overview of Fermi.****Fermi iterates three steps to optimize the prompt from the given user information: (1) scoring new prompts, (2) updating the memory based on the score, and (3) generating new prompts (left). After the optimization, Fermi selectively uses the personalized prompts for the inference, via Retrieval-of-Prompt (right).**



ates evaluation and generation of prompts with two LLMs, to solve the black-box optimization such as prompt optimization; Yang et al. (2024) incorporates the past generated prompts with their scores to enable the LLM for the optimization to construct new improved prompts. However, only providing the scores on training examples is insufficient to optimize the prompt for few-shot personalization of LLMs, as the context with mis-aligned responses such as the types or patterns within recursively wrong predictions can't be captured in scores. Therefore, we propose an efficient way to incorporate such context, along with an additional method to improve the inference by considering the context of the given query.

## 3 Fermi: Few-shot Personalization via Learning from Mis-aligned Responses

In this section, we present our framework proposed for **F**ew-shot **P**ersonalization of LLMs from **mis-aligned responses (Fermi)**. We first present our problem setup in Section 3.1. Then, in Section 3.2, we present our core component that optimizes the input prompt with a given user information, by using LLM as a black-box optimizer along with the additional contexts from mis-aligned responses. Lastly, we introduce an efficient inference scheme after optimizing prompts with Fermi, by utilizing the context of a test query (Section 3.3).

### Problem description

We first describe the problem setup of our interest under a question-answering (QA) scenario. Our goal is to steer LLM for a specific user using that user's information, and hence make LLM adaptively answer a given question depending on the user. Formally, let \(q\) denote the given test question and \({\mathcal{M}}\) denote the LLM, respectively. Next, for user \(u\), we assume two types of user information: \(U_{\texttt{pro}}\) and \(U_{\texttt{opi}}\). \(U_{\texttt{pro}}\) indicates explicit profile of \(u\) such as demographics information (_e.g._, region, sex, and age) or ideology (_e.g._, political affiliation). \(U_{\texttt{opi}}\) indicates \(N\) few-shot previous opinions by \(u\), which has the form of QA pairs, _i.e._, \(U_{\texttt{opi}}=\{(q_{i},a_{i})\}_{i=1}^{N}\) where \(q_{i}\) is a previously asked question and \(a_{i}\) is an opinion (answer) by the user. Then, for given test question \(q\), our goal is to predict the answer \(a\), which would be generated by user \(u\), through LLM \({\mathcal{M}}\) using both \(U_{\texttt{pro}}\) and \(U_{\texttt{opi}}\). The heuristic design of input prompt p to incorporate such user information has been previously explored Hwang et al. (2023); Santurkar et al. (2023), _i.e._, prediction \(\widehat{a}\) is obtained by conditioning \({\mathcal{M}}\) with p constructed using \(U_{\texttt{pro}}\) and \(U_{\texttt{opt}}\):

\[\widehat{a}(\text{p})={\mathcal{M}}(q;\text{p}). \tag{1}\]

However, heuristically designed prompts are limited to fully exploit the given user information; for example, the personalization of LLM was better with few user opinions (_e.g._, 3 or 8) compared to using all opinions in \(u_{\texttt{opi}}\)Hwang et al. (2023). Therefore, we tackle this limitation by finding personalized prompts that align LLM to the user, through direct learning from given user information.

### Effective prompt optimization with LLM from contexts of mis-aligned responses

To mitigate the difficulties from the large scale and black-box nature of recent LLMs, we instead optimize input prompts to learn from user information. It is motivated by the recent work Yang et al. (2024) that uses two LLMs, \({\mathcal{M}}\) and \({\mathcal{M}}_{\texttt{opt}}\), to solve black-box optimization, where \({\mathcal{M}}_{\texttt{opt}}\) denotes another LLM used for the optimization. Specifically, our key idea is incorporating the contexts of _mis-aligned_ responses (_i.e._, QAs in \(U_{\texttt{opi}}\) that \({\mathcal{M}}\)

Figure 2: **Prompt example. Example of input prompt for \({\mathcal{M}}_{\texttt{opt}}\) to generate new prompts, composed of fixed input prompt p\({}_{\texttt{opt}}\) (including fixed few-shot demonstrations) and optimization memory \(M^{t}\) (Eq. 5) on OpinionQA dataset. A full version is in Appendix B.3.

 

incorrectly predict with current prompts) during the optimization, instead of only using scores of the prompts (_e.g._, average accuracy of the prediction by \({\mathcal{M}}\) on \(U_{\texttt{opi}}\)). As the contexts of mis-aligned responses include useful learning signals such as types or patterns of common wrong predictions, they could be effective in learning how to improve the prompts.

Fermi starts with an initial prompt set \(\text{P}^{0}=\{\text{p}^{0}\}\) and we adopt heuristically designed prompts in previous works Hwang et al. (2023); Santurkar et al. (2023) for \(\text{p}^{0}\). Specifically, we use the user's explicit profile \(U_{\texttt{pro}}\) when it is available; thereby we fully utilize the given user information. If not, we adopt vanilla prompt that instructs QA without user information (see details in Appendix B.3).

Then, Fermi iterate the following 3 steps: _(1) Score Prompts_, _(2) Update Memory_, and _(3) Generate New Prompts_. See Figure 1 for the illustration.

\(\circ\)_Step 1: Score Prompts_. We first calculate the score \(s_{k}\) of each prompt \(\text{p}_{k}\in\text{P}^{t}\), by obtaining the predictions from \({\mathcal{M}}\) under \(\text{p}_{k}\) and evaluating them using the user's previous answers:

\[s_{k}=\sum\nolimits_{(q_{i},a_{i})\sim U_{\texttt{opt}}}\operatorname*{s} \bigl{(}a_{i},\widehat{a}_{i}(\text{p}_{k})\bigr{)}/N, \tag{2}\]

where \(\widehat{a}_{i}(\text{p}_{k})={\mathcal{M}}(q_{i};\text{p}_{k})\). Here, \(\operatorname*{s}(\cdot,\cdot)\) is a specific metric to evaluate the prediction (_e.g._, accuracy). During this calculation of the score \(s_{k}\) of the prompt \(\text{p}_{k}\), we also collect pair of QA pairs \(U^{k}_{\texttt{opi}}\) that \({\mathcal{M}}\) yields _mis-aligned response_\(\widehat{a}_{i}\) to the user's answer \(a_{i}\), under \(\text{p}_{k}\):

\[U^{k}_{\texttt{opi}}=\{(q_{i},a_{i})\in U_{\texttt{opi}}|\operatorname*{s} \bigl{(}a_{i},\widehat{a}_{i}(\text{p}_{k})\bigr{)}<\tau\}, \tag{3}\]

where \(\tau\) is a threshold to judge the mis-alignment; for example, we set \(\tau=0.5\) when we use the correctness of prediction as the score \(\operatorname*{s}(\cdot,\cdot)\).

\(\circ\)_Step 2: Update Memory_. Next, we construct an optimization memory \(M^{t}\), which is used for the input of \({\mathcal{M}}_{\texttt{opt}}\) to generate new improved prompts, by providing the information of well-performing prompts through the contexts of their mis-aligned responses. To be specific, the optimization memory \(M^{t}=\{(\text{p}_{l},s_{l},c_{l})\}_{l=1}^{L}\) is constructed by selecting top-\(L\) prompts among P\({}^{t}\) and \(M^{t-1}\) (where \(M^{0}=\emptyset\)), according to their scores (Eq. 2). Here, we present the triplets in \(M^{t}\) in ascending order, _i.e._, \(s_{l}<s_{l^{{}^{\prime}}}\) when \(l<l^{{}^{\prime}}\), and provide the varied context \(c_{l}\) depending on \(l\). Specifically, for \(l=1\), we construct \(c_{l}\) by concatenating QAs and mis-aligned responses by \({\mathcal{M}}\) under \(\text{p}_{l}\) on \(U^{t}_{\texttt{opi}}\):

\[c_{l}=\textsc{Concat}\{\big{(}i,q_{i},a_{i},\widehat{a}_{i}(\text{p}_{l})\big{)} |(q_{i},a_{i})\in U^{l}_{\texttt{opi}}\}. \tag{4}\]

In Figure 2, the texts corresponding to \(c_{1}\) are highlighted in blue. For other cases (_i.e._, \(l\neq 1\)), instead of the enumeration like \(c_{1}\), we construct the context \(c_{l}\) with (i) the indices of _common_ mis-aligned QA pairs between \(\text{p}_{l}\) and \(\text{p}_{1}\), and (ii) the number of _newly_ mis-aligned QAs by \(\text{p}_{l}\) compared to \(\text{p}_{1}\) (see greentexts in Figure 2 for an example). Through the presented indices in \(c_{l}\), \({\mathcal{M}}_{\texttt{opt}}\) can directly access the mis-aligned QA pairs by referring \(c_{1}\), and one can avoid unnecessary complexity of \(c_{l}\) and cost from the long input to \({\mathcal{M}}_{\texttt{opt}}\). Additionally, the number of newly mis-aligned ones offers further insight into whether \(\text{p}_{l}\) has improved, which can't be captured by the common mis-aligned ones.

\(\circ\)_Step 3: Generate New Prompts_. With the updated memory \(M^{t}\), we generate \(K\) new improved prompts \(\text{P}^{t+1}=\{\text{p}^{\texttt{new}}_{k}\}_{k=1}^{K}\) by prompting \({\mathcal{M}}_{\texttt{opt}}\) to generate the new and high-scored prompts:

\[\text{p}^{\texttt{new}}_{k}={\mathcal{M}}_{\texttt{opt}}(M^{t};\text{p}_{ \texttt{opt}}), \tag{5}\]

where \(\text{p}_{\texttt{opt}}\) is a fixed input prompt for \({\mathcal{M}}_{\texttt{opt}}\) to generate new prompts, and we use a random sampling with temperature to generate diverse new prompts from \({\mathcal{M}}_{\texttt{opt}}\). Figure 2 presents the example of the overall input of \({\mathcal{M}}_{\texttt{opt}}\) to generate new prompts, which is constructed with \(M^{t}\) and \(\text{p}_{\texttt{opt}}\).

Then, we go back to Step 1 with \(\text{P}^{t+1}\) and iterate these 3 steps for \(T\) times. After that, we obtain the optimized (_i.e._, personalized) prompts \(\text{P}^{T}=\{\text{p}^{T}_{k}\}_{k=1}^{K}\) for the user \(u\).

### Effective inference by Retrieval-of-Prompt

After \(T\) iterations of the optimization procedure, Fermi outputs \(K\) unique personalized prompts \(\text{P}^{T}=\{\text{p}^{T}_{k}\}_{k=1}^{K}\). Therefore, for a given test question \(q\), one needs to determine which prompt to apply. Selecting the prompt with the highest score, _i.e._, \(k^{*}=\arg\max_{k}s_{k}\) (Eq. 2), would be a straightforward way. However, our intuition is that better selection is possible if we utilize the context of the test question \(q\) as additional information. To this end, we propose to select the input prompt with the highest score on the subset of \(U_{\texttt{opi}}\), which only consists of the previous questions highly relevant to \(q\). Formally, we first measure the relevance \(r\) between \(q\) and previous question \(q_{i}\):

\[R(q,U_{\texttt{opi}})=\{r(q,q_{i})|q_{i}\in U_{\texttt{opi}}\}. \tag{6}\]

For the relevance \(r\), we use the cosine similarity between the embeddings of questions, extracted 

by the sentence encoder (Reimers and Gurevych, 2019). Then, we select top-\(\tilde{N}\) questions according to the calculated relevance and construct the subset \(U^{q}_{\texttt{opi}}\) with those questions. Lastly, we choose the input prompt \(\text{p}^{*}=\text{p}^{T}_{k^{*}}\) based on the score on \(U^{q}_{\texttt{opi}}\), which were already calculated, and use the prediction \(\widehat{a}(\text{p}^{*})\) by \(\mathcal{M}\):

\[k^{*}=\arg\max_{k}s^{T}_{k}(U^{q}_{\texttt{opi}}), \tag{7}\]

where \(s^{T}_{k}(U^{q}_{\texttt{opi}})=\sum_{(a_{i},a_{i})\sim U^{q}_{\texttt{opti}}} s\big{(}a_{i},\widehat{a}_{i}(\text{p}^{T}_{k})\big{)}/\tilde{N}\). Figure 1 illustrates the overview of Fermi. We note that a full version of the prompts and examples of personalized prompts are presented in Appendix B and D, respectively.

## 4 Experiments

In this section, we design our experiments to investigate the following questions:

* How does Fermi perform compare to other personalization methods? (Tables 1 and 2)
* Is the optimized prompt with Fermi from one LLM transferable to different LLMs? (Table 3)
* What is the effect of each component in Fermi? (Table 4)
* Why optimized prompt by Fermi is more effective than other prompts? (Table 5)

### Setups

First, we describe our experimental setups. More details are presented in Appendix B.

**Datasets.** For the experiments, we first use two multiple-choice QA datasets proposed to measure the steerability of LLMs for specific users (or social groups): _OpinionQA_(Santurkar et al., 2023) and _GlobalOpinionQA_(Durmus et al., 2023). For OpinionQA, we use a subsampled split released by Hwang et al. (2023), which consists of 10.5k and 15.8k training and test QA pairs across 525 users and 15 topics, respectively. For GlobalOpinionQA, since the dataset originally included the answer distribution by multiple respondents in the same country, we converted it to have a single answer by selecting the choice with the highest probability. It results in 920 training and 1,317 test QA pairs across 46 countries. We consider each country as a specific user. Next, we use three additional tasks, _LaMPtag_ (classification), _LaMPrate_ (regression), _LaMPtitle_ (generation), from a recent benchmark proposed for personalization of LLMs (Salemi et al., 2023). LaMPtag is a 15-way classification task where an input is a movie description and a label is a movie tag, and LaMPrate is a regression task where an input is a user review and a label is an integer rating (1-5). LaMPtitle is a generation task where the input is the abstract of the paper and the goal is generating the personalized title of the paper based on the given abstract. We construct these datasets by subsampling from their original validation split, which results in 1,000 training and 1,500 test QA pairs across 50 users for each dataset. On average across four datasets, for each user, 20 training QAs as previous opinions and specific profile are given, and then 30 test QAs are used to evaluate. Following Salemi et al. (2023), we report average test accuracy (Acc), mean absolute error (MAE), and Rouge-L score for _LaMPtag_, _LaMPrate_, and _LaMPtitle_, respectively.

**Baselines.** We compare Fermi against extensive baselines as follows: (1) _Uniform_: expected performance when the prediction is made uniformly at random. (2) _Vanilla_: answers the question with LLMs without any user information. (3) _Profile_: constructing prompt using all available user profiles (Santurkar et al., 2023; Hwang et al., 2023) such as demographics or nationality. (4) _Few-shot_: retrieving relevant previous questions and opinions, then append them to the prompt (Hwang et al., 2023; Salemi et al., 2023). Following (Salemi et al., 2023), we consider BM25 (Robertson et al., 2009) and Contriever (Izacard et al., 2022) for the retriever models. The number of retrieved profiles is determined among {3, 8, all} with validation performance. (5) _All Info_: using both explicit profiles and retrieved previous QAs to construct prompt (Hwang et al., 2023). We use the retrieval with the best performance in _Few-shot_.2 (6) Optimization by PROmpting (_OPRO_; Yang et al. (2024)): optimizing input prompt using both user profiles and previous opinions using LLMs. Here, all of the previous opinions are utilized during the optimization. In the experiments, the prompt with the best training score is selected for the test.

Footnote 2: In the case of OpinionQA, we additionally consider the retrieved indices originally included by (Hwang et al., 2023).

**Implementation details.** We use three recent state-of-the-art LLMs for the prediction LLM \(\mathcal{M}\) for the experiments: ChatGPT (gpt-3.5-turbo-0613) (OpenAI, 2022), GPT-4 (gpt-4-turbo-1106) (OpenAI, 2023), Mistral-7B-Instruct-v0.2 (Jiang et al., 2023) and LLaMA2-c 

\({\cal M}\), we use a temperature of \(0.0\) when calling the API or greedy decoding for LLMaMA, to remove the effect of random sampling. For the optimization LLM \({\cal M_{opt}}\), we always use GPT-4, as the prompt optimization based on the memory (Eq. 5) requires complex reasoning capability (See Appendix A), with a temperature of 1.0. For OPRO and Fermi, we use fixed values of \(K=4\), \(L=5\), and \(T=10\). Also, with previous user opinions in \(U_{\text{opi}}\), 80% is used for optimization and 20% is used as few-shot demonstrations in p\({}_{\texttt{opt}}\). We set \(\tau\), a threshold to define the mis-aligned responses (Eq. 2), 0.2 only for \(LaMP_{\texttt{title}}\) and 1.0 for other tasks. To obtain sentence embeddings for Retrieval-of-Prompt, we use the sentence encoder with MPNet Song et al. (2020) showing the best performance.3 We use a fixed \(\hat{N}=3\) for Retrieval-of-Prompt.

Footnote 3: Following the results in [https://www.sbert.net](https://www.sbert.net)

### Main results

Table 1 summarizes the experimental results on two different multiple-choice QA datasets, under ChatGPT. First, it is observed that augmenting the user information into the input prompt is effective in improving the accuracies of LLMs, but the effectiveness could be varied. For example, retrieving relevant user opinions is more effective than using the user profile for OpinionQA (49.8% vs. 48.1%), but it's vice versa in GlobalOpinionQA (61.2% vs. 66.1%). It is due to the difference between datasets, as each user is asked multiple questions on the same topic in OpinionQA while GlobalOpinionQA asks the broader topics; this result also reveals the necessity of the learning-based prompt optimization approach. From the results of OPRO and Fermi, one can observe that the optimization-based approach is actually effective, and the proposed method significantly improves it. To be specific, Fermi exhibits 6.75% average accuracy improvement compared to the previous prompting method. Furthermore, compared to the existing optimization method, Fermi exhibits 4.05% accuracy improvement in the average. In Figure 3, we additionally present detailed results on OpinionQA, a topic-wise accuracy from four representative baselines selected based on average accuracy. Here, Fermi consistently shows better performance than other baselines across all topics, which further demonstrates the effectiveness of Fermi for the personalization of LLMs.

Next, Table 2 summarizes the experimental results on LaMP\({}_{\texttt{tag}}\), LaMP\({}_{\texttt{rate}}\), and LaMP\({}_{\texttt{title}}\). We note that these datasets do not include explicit user profiles; hence, we exclude both _Profile_ and _All Info_ for the baselines. Here, it is noteworthy that the effectiveness of OPRO is significantly degraded, as the given task becomes more challenging to solve

\begin{table}
\begin{tabular}{c|c c} \hline \hline  & \multicolumn{2}{c}{Datasets (Metric)} \\ Methods & OpQA (Acc.) & GOQA (Acc.) \\ \hline Uniform & 34.2 & 31.4 \\ Vanilla & 45.5 & 62.8 \\ Profile & 48.1 & 66.1 \\ Few-shot\({}_{\texttt{bm25}}\) & 49.8 & 59.1 \\ Few-shot\({}_{\texttt{cont}}\) & 49.3 & 61.2 \\ All Info & 48.6 & 62.3 \\ OPRO & 50.2 & 71.1 \\ \hline Fermi & **54.6** & **74.8** \\ \hline \hline \end{tabular}
\end{table}
Table 1: **Main result on multiple-choice QA datasets. Test accuracy of ChatGPT over the different methods on OpinionQA (OpQA) and GlobalOpinionQA (GOQA). The best and second best scores are highlighted in bold underline, respectively.**

\begin{table}
\begin{tabular}{c|c c c} \hline \hline \multirow{2}{*}{Methods} & \multicolumn{3}{c}{Datasets (Metric)} \\  & LaMP\({}_{\texttt{tag}}\) & LaMP\({}_{\texttt{rate}}\) & LaMP\({}_{\texttt{title}}\) \\  & (Acc.) & (MAE) & (Rouge-L) \\ \hline Uniform & 6.7 & 1.65 & - \\ Vanilla & 36.1 & 0.62 & 0.394 \\ Few-shots\({}_{\texttt{no}}\)\({}_{\texttt{no}}\)\({}_{\texttt{no}}\) & 35.9 & 0.40 & 0.411 \\ Few-shot\({}_{\texttt{cent}}\) & 36.2 & 0.36 & 0.406 \\ OPRO & 34.3 & 0.57 & 0.406 \\ \hline Fermi & **37.8** & **0.34** & **0.419** \\ \hline \hline \end{tabular}
\end{table}
Table 2: **Main result on LaMP Benchmark. Test performance of ChatGPT over the different methods on LaMP benchmarks. Test accuracy \(\big{(}\texttt{Acc.}~{}(\uparrow)\big{)}\), mean absolute error \(\big{(}\texttt{MAE}~{}(\downarrow)\big{)}\), and Rouge-L (\(\uparrow\)) score are used, respectively. The best and second best scores are highlighted in bold and underline, respectively.**

\begin{table}
\begin{tabular}{l|c c c} \hline \hline Methods & \multicolumn{2}{c}{Mistral-v0.2} & LLAMA-2 & GPT-4 \\ \hline Vanilla & 57.6 & 62.4 & 56.7 \\ Profile & 67.4 & 65.5 & 77.7 \\ Few-shot & 70.1 & 60.5 & 68.9 \\ All Info & 71.7 & 65.1 & 78.2 \\ OPRO\({}^{*}\) & 66.4 & 64.5 & 76.7 \\ \hline Fermi\({}^{*}\) & **72.4** & **68.9** & **84.8** \\ \hline \hline \end{tabular}
\end{table}
Table 3: **Transferability of Fermi. Test accuracy of two LLMs on GlobalOpinionQA. For Few-shot, we use Contirever which shows higher accuracy in Table 1. For OPRO\({}^{*}\) and Fermi\({}^{*}\), prompts optimized on ChatGPT are directly used. The best and second best scores are highlighted in bold and underline, respectively.** 

(_e.g._, the average number of answer choices: 3.96 for GlobalOpinionQA vs. 15 for LaMPtag). Nevertheless, Fermi is consistently effective and outperforms the other baselines; for example, Fermi exhibits 17.7% and 4.0% relative improvement in average, compared to the vanilla and the previous best baseline, respectively.

### Analyses with Fermi

In this section, we provide additional analyses of Fermi with the experiments on GlobalOpinionQA. More analyses are presented in Appendix A.

Transferability of the optimized prompt.Here, we provide additional experiments to verify the transferability of the learned prompt with our method. To be specific, we first save the optimized prompts under ChatGPT as LLM for evaluation (Eq. 1), which are used in Table 1. Then, we directly apply these prompts to two different types of LLMs (Mistral-7B-Instruct-v0.2, LLaMA-2-chatta-70B, and GPT-4), without additional optimization as same as applying heuristically designed prompts. From Table 3, one can observe that the transferred prompts from Fermi significantly outperform the baseline prompting methods on both LLMs; for example, it exhibits 3.6% accuracy improvement compared to the best-performing baseline on average. We remark that the prompts from OPRO are even less effective than the existing baseline, which further shows the advantages of Fermi in learning the well-generalized personalized prompt.

Ablation study.To validate the effectiveness of the proposed component of Fermi in Section 3, we perform the ablation experiments by decomposing our framework into three different components: (1) including QAs that have mis-aligned responses with the initial presentation and referring via common indices (\(Add_{\texttt{Mi}}\)), (2) noting the number of QAs with new mis-aligned responses (\(Add_{\texttt{fun}}\)), and (3) Retrieval-of-Prompt for a test query (_RoP_). As shown in Table 4, all components progressively improve the few-shot personalization of LLMs. Especially, it is observable that efficiently providing the context of mis-aligned QAs during the optimization is mostly crucial for the improvement. Next, providing the number of new mis-aligned QAs makes additional improvement, as it can provide information about the effectiveness of the given prompt, which is not captured by commonly mis-aligned QAs. Lastly, for a test query, retrieving the most relevant prompt is more effective than selecting

Figure 3: **Overall topic-wise improvement. Test accuracy of ChatGPT over four different personalization methods on OpinionQA. Detailed results are presented in Appendix C.

 

with the highest training score, as it successfully utilizes the context of the test query.

Features of good input prompts for personalization.In Table 5, we further conduct the experiments to answer the following question: _what features make good personalized prompts for LLMs?_ First, we claim that the relevance of the prompt to the test query is crucial; for example, Few-shot\({}_{\texttt{top3}}\), Few-shot\({}_{\texttt{all}}\), and Few-shot\({}_{\texttt{bott3}}\) are different prompting methods by retrieving the 3 mostly relevant, all 20, and 3 mostly irrelevant previous opinions, respectively. Here, it is observable that test accuracy largely degrades when a portion of irrelevant opinions increases. Similarly, when we retrieved the most irrelevant prompt (Fermi\({}_{\texttt{irrel}}\)), _i.e.,_ take \(\arg\min\) in Eq. 7, accuracy of Fermi is also decreased.

Second, providing the user information with the proper format for LLMs is important. As shown in Figure 4, the optimized prompt by Fermi is a detailed instruction consisting of multiple sentences that condense the lessons from the user opinions and LLM's mis-aligned responses. In contrast, the previous prompt used to incorporate previous opinions is based on the specific form, which is harder to follow by LLMs. To verify the importance of the format, we convert the enumeration of all QAs (by Few-shot\({}_{\texttt{all}}\)) into the instruction of multiple sentences (denoted by Few-shot\({}_{\texttt{format}}\)), by prompting GPT-4 using the optimized prompts by Fermi as reference. Interestingly, this format conversion shows significant improvement (\(56.3\%\to 66.4\%\)) while it is still underperforming Fermi.

Lastly, effectively distillling the given user information is important. As shown in Table 5, the prompting method with higher accuracy on previous user opinions \(U_{\texttt{op1}}\) (_i.e.,_ training accuracy) has a higher test accuracy for that user as well, except Few-shot\({}_{\texttt{all}}\) which can directly access \(U_{\texttt{op1}}\). In this aspect, Fermi shows a clear advantage compared to the previous prompting optimization method; as shown in Figure 5, Fermi more effectively optimizes the prompt and achieves higher training accuracy than OPRO. These results indicate that finding a proper way to condense and incorporate the user information to design input prompts is crucial, and Fermi achieves this by using the context of mis-aligned responses.

Overall, designing personalized prompts satisfying these three properties (relevancy to test query, proper format, and effective distillation of user information) is challenging, but Fermi effectively accomplishes this goal.

## 5 Conclusion

In this work, we propose Fermi, a simple yet effective framework for improving the few-shot personalization of LLMs. Our key idea is to optimize the input prompt by learning from the user information; we propose an efficient way to incorporate contexts of mis-aligned responses by LLMs during the optimization, and a retrieval approach to select the optimized prompt relevant to test query. The effectiveness of Fermi is demonstrated by results on various personalization tasks and LLMs.

Figure 4: **Qualitative comparison. Example prompts from All Info (middle) and Fermi (bottom) for the specific question (top) from GlobalOpinionQA. Prompt is inserted to “LNS”. More examples are in Appendix D.

 



### Limitations

One possible limitation of Fermi is its computational cost. As discussed in Appendix A, our framework necessitates a strong LLM as the optimization of prompt requires complex reasoning capability. If we substitute \(M_{\text{opt}}\) from GPT-4 to ChatGPT, it can't properly optimize the input prompt; see Figure 5 and we remark the similar observation was in the previous work Yang et al. (2024).

Nevertheless, we would like to emphasize that the proposed Fermi is not just a simple consequence of more computations and costs. Compared to OPRO Yang et al. (2024), another computationally intensive method for accuracy improvement, Fermi significantly outperforms it, _i.e._, Fermi is an even more efficient way to increase performance. For instance, as shown in Figure 6(c), Fermi achieves much higher accuracy than OPRO, (73.0% v.s. 71.1% on GlobalOpinionQA), even with half of the previous question and user's responses and 1.4% less cost per user (0.33$ v.s. 0.34$). The effectiveness of Fermi for optimizing personalized prompts is from more effective optimization by extracting the effective learning signal from mis-aligned responses. On the other hand, one can directly control the computational cost and performance, by varying the number of iterations (\(T\)) which linearly increases the cost with improved performance. Regarding this, we remark that 4 iterations of optimization with Fermi yield similar results to the 10 iterations of optimization with OPRO (see Figure 5).

We further remark that the overall cost to use our framework will decrease while preserving its effectiveness, as the cost of using LLM with strong reasoning capability is continuously reduced. Currently, more than half of the overall cost occurs to use strong LLM (_e.g._, GPT4) for generating the improved prompts, and this choice is inevitable as this task requires the complex reasoning capability for LLM. However, as shown in Table 6, we empirically observe that the recently released strong yet efficient LLM (GPT-4o-mini) can successfully optimize the prompt and yield a comparable performance with GPT-4, although it only requires 0.5% input token and 1.0% output per token price compared to GPT-4.

At the same time, as we demonstrated in the experiments, the personalized prompts from our method are well-transferrable to other LLMs that are not used during optimization (Table 3), could be continuously updated with enlarged data through the user interactions (Table 9), and also reusable to convert previous prompts to have the proper format for LLMs (Table 5). Therefore, we believe that our approach could be an even more efficient way for personalization compared to the heuristical design of the prompt, after the consumption of the cost at the initial optimization.

### Broader impact and ethical implications

We strongly believe that Fermi can provide a strong positive impact in real-world applications that require personalized responses for the given user, _e.g._, search engines or chatbots. We expect that our framework would be especially beneficial for the users belonging to under-populated social groups, since LLMs are known to follow the knowledge or opinion of the major population within pre-trained data Kandpal et al. (2023); Santurkar et al. (2023). In contrast, there also exists some potential negative impacts. Since our framework needs to provide personal information to LLMs (mostly through API), it has a potential privacy risk when the provider of LLMs does not follow the safeguard and collects the given information. In addition, as our framework didn't filter out the resulting prompts separately, it can include the prompts that have socially negative impacts, _e.g._, jailbreak of LLMs Chao et al. (2023). We believe that the incorporation of an additional filtering step could be a solution to this problem Xie et al. (2023).

## References

* Chao et al. (2023) Patrick Chao, Alexander Robey, Edgar Dobriban, Hamed Hassani, George J Pappas, and Eric Wong. 2023. Jailbreaking black box large language models in twenty queries. In R0-FoMo: Robustness of Fewshot and Zero-shot Learning in Large Foundation Models, Cited by: SS1.
* Deng et al. (2022) Ringhai Deng, Jianyu Wang, Cheng-Ping Hsieh, Yihan Wang, Han Guo, Tianmin Shu, Meng Song, Eric Xing, and Zhiting Hu. 2022. RIprompt: Optimizing discrete text prompts with reinforcement learning. In Conference on Empirical Methods in Natural Language Processing (EMNLP), Cited by: SS1.
* Durmus et al. (2023) Esin Durmus, Karina Nyugen, Thomas I Liao, Nicholas Schiefer, Amanda Askell, Anton Bakhtin, Carol Chen, Zac Hatfield-Dodds, Danny Hernandez, Nicholas Joseph, et al. 2023. Towards measuring the representation of subjective global opinions in language models. arXiv preprint arXiv:2306.16388. Cited by: SS1.
* Alfaes et al. (2019) An introduction to the theory of empirical analysis. In Proceedings of the 2019 International Conference on Artificial Intelligence, pp. 1-1. Cited by: SS1.
* M. Deng, J. Wang, C. Hsieh, Y. Wang, H. Guo, T. Shu, M. Song, E. Xing, and Z. Hu 2022. RIPPrompt: Optimizing discrete text prompts with reinforcement learning. In Conference on Empirical Methods in Natural Language Processing (EMNLP), Cited by: SS1.
* S. Das, K. Lee, W. Li, X. Zhang, and L. Liu (2020)A novel approach for multi-task learning in multi-task learning. In Proceedings of the 2020 IEEE International Conference on Knowledge Discovery and Data Mining, pp. 1-1. Cited by: SS1.
* F. Gao, G. Xu, and L. Liu (2020)An efficient policy model for multi-task learning in multi-task learning. In Proceedings of the 2020 ACM SIGKDD international conference on Knowledge discovery and data mining, pp. 1-1. Cited by: SS1.
* G. Xu, L. Liu, and L. Liu (2020)Deep learning for multi-task learning in multi-task learning. In Proceedings of the 2020 ACM SIGKDD international conference on Knowledge discovery and data mining, pp. 1-1. Cited by: SS1.



Laura Weidinger, Martin Chadwick, Phoehe Thacker, et al. 2022. Improving alignment of dialogue agents via targeted human judgements. _arXiv preprint arXiv:2209.14375_.
* Hwang et al. (2023) EunJeong Hwang, Bodhisattwa Prasad Majumder, and Niket Tandon. 2023. Aligning language models to user opinions. In _Conference on Empirical Methods in Natural Language Processing (EMNLP)_.
* Izacard et al. (2022) Gautier Izacard, Mathilde Caron, Lucas Hosseini, Sebastian Riedel, Piotr Bojanowski, Armand Joulin, and Edouard Grave. 2022. Unsupervised dense information retrieval with contrastive learning. In _Transactions on Machine Learning Research (TMLR)_.
* Jiang et al. (2023) Albert Q Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, et al. 2023. Mistral 7b. _arXiv preprint arXiv:2310.06825_.
* Kandpal et al. (2023) Nikhil Kandpal, Haikang Deng, Adam Roberts, Eric Wallace, and Colin Raffel. 2023. Large language models struggle to learn long-tail knowledge. In _Proceedings of the International Conference on Machine Learning (ICML)_.
* Lester et al. (2021) Brian Lester, Rami Al-Rfou, and Noah Constant. 2021. The power of scale for parameter-efficient prompt tuning. In _Conference on Empirical Methods in Natural Language Processing (EMNLP)_.
* Li et al. (2023) Junyi Li, Ninareh Mehrabi, Charith Peris, Palash Goyal, Kai-Wei Chang, Aram Galstyan, Richard Zemel, and Rahul Gupta. 2023. On the steerability of large language models toward data-driven persona. _arXiv preprint arXiv:2311.04978_.
* Li and Liang (2021) Xiang Lisa Li and Percy Liang. 2021. Prefix-tuning: Optimizing continuous prompts for generation. In _Annual Meeting of the Association for Computational Linguistics (ACL)_.
* OpenAI (2022) OpenAI. 2022. Introducing chatgpt. [https://openai.com/blog/chatgpt](https://openai.com/blog/chatgpt).
* OpenAI (2023) OpenAI. 2023. GPT-4 technical report. _arXiv preprint arXiv:2303.08774_.
* PewResearch (2022) PewResearch. Writing survey questions. [https://www.pewresearch.org/our-methods/u-s-surveys/writing-survey-questions](https://www.pewresearch.org/our-methods/u-s-surveys/writing-survey-questions).
* Prasad et al. (2023) Archiki Prasad, Peter Hase, Xiang Zhou, and Mohit Bansal. 2023. Grips: Gradient-free, edit-based instruction search for prompting large language models. In _Conference of the European Chapter of the Association for Computational Linguistics (EACL)_.
* Przant et al. (2023) Reid Pryzant, Dan Iter, Jerry Li, Yin Tat Lee, Chenguang Zhu, and Michael Zeng. 2023. Automatic prompt optimization with" gradient descent" and beam search. In _Conference on Empirical Methods in Natural Language Processing (EMNLP)_.
* Razdaibedina et al. (2023) Anastasia Razdaibedina, Yuning Mao, Rui Hou, Madiana Khabsa, Mike Lewis, and Amjad Almhahiri. 2023. Progressive prompts: Continual learning for language models. In _International Conference on Learning Representations (ICLR)_.
* Reimers and Gurevych (2019) Nils Reimers and Iryna Gurevych. 2019. Sentence-bert: Sentence embeddings using siamese bert-networks. In _Conference on Empirical Methods in Natural Language Processing (EMNLP)_.
* Robertson et al. (2009) Stephen Robertson, Hugo Zaragoza, et al. 2009. The probabilistic relevance framework: Bm25 and beyond. _Foundations and Trends(r) in Information Retrieval_, 3(4):333-389.
* Salemi et al. (2023) Alireza Salemi, Shesheera Mysore, Michael Bendersky, and Hamed Zamani. 2023. Lamp: When large language models meet personalization. _arXiv preprint arXiv:2304.11406_.
* Santurkar et al. (2023) Shibani Santurkar, Esin Durmus, Faisal Ladhak, Cinoo Lee, Percy Liang, and Tatsunori Hashimoto. 2023. Whose opinions do language models reflect? In _Proceedings of the International Conference on Machine Learning (ICML)_.
* Shin et al. (2020) Taylor Shin, Yasaman Razeghi, Robert L Logan IV, Eric Wallace, and Sameer Singh. 2020. Autoprompt: Eliciting knowledge from language models with automatically generated prompts. In _Conference on Empirical Methods in Natural Language Processing (EMNLP)_.
* Solaiman and Dennison (2021) Irene Solaiman and Christy Dennison. 2021. Process for adapting language models to society (palms) with values-targeted datasets. In _Advances in Neural Information Processing Systems (NeurlPS)_.
* Song et al. (2020) Kaitao Song, Xu Tan, Tao Qin, Jianfeng Lu, and Ti-Yan Liu. 2020. Mpnet: Masked and permuted pretraining for language understanding. _Advances in Neural Information Processing Systems (NeurIPS)_.
* Tan et al. (2024) Zhaoxuan Tan, Qingkai Zeng, Yijun Tian, Zheyuan Liu, Bing Yin, and Meng Jiang. 2024. Democratizing large language models via personalized parameter-efficient fine-tuning. _arXiv preprint arXiv:2402.04401_.
* Team et al. (2023) Gemini Team, Rohan Anil, Sebastian Borgeaud, Yonghui Wu, Jean-Baptiste Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M Dai, Anja Hauth, et al. 2023. Gemini: a family of highly capable multimodal models. _arXiv preprint arXiv:2312.11805_.
* Touvron et al. (2023) Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajwal Bhargava, Shruti Bhosale, et al. 2023. Llama 2: Open foundation and fine-tuned chat models. _arXiv preprint arXiv:2307.09288_.

 

Zifeng Wang, Zizhao Zhang, Chen-Yu Lee, Han Zhang, Ruoxi Sun, Xiaoqi Ren, Guolong Su, Vincent Perot, Jennifer Dy, and Tomas Pfister. 2022. Learning to prompt for continual learning. In _Conference on Computer Vision and Pattern Recognition (CVPR)_.
* Xie et al. [2023] Yueqi Xie, Jingwei Yi, Jiawei Shao, Justin Curl, Lingjuan Lyu, Qifeng Chen, Xing Xie, and Fangzhao Wu. 2023. Defending chatgpt against jailbreak attack via self-reminders. _Nature Machine Intelligence_, pages 1-11.
* Yang et al. [2024] Chengrun Yang, Xuezhi Wang, Yifeng Lu, Hanciao Liu, Quoc V Le, Denny Zhou, and Xinyun Chen. 2024. Large language models as optimizers. In _International Conference on Learning Representations (ICLR)_.
* Zhao et al. [2024] Siyan Zhao, John Dang, and Aditya Grover. 2024. Group preference optimization: Few-shot alignment of large language models. In _International Conference on Learning Representations (ICLR)_.
* Zhou et al. [2023] Yongchao Zhou, Andrei Ioan Muresanu, Ziwen Han, Keiran Paster, Silviu Pitis, Harris Chan, and Jimmy Ba. 2023. Large language models are human-level prompt engineers. In _International Conference on Learning Representations (ICLR)_.
* Zhu et al. [2022] Qi Zhu, Bing Li, Fei Mi, Xiaoyan Zhu, and Minlie Huang. 2022. Continual prompt tuning for dialog state tracking. In _Annual Meeting of the Association for Computational Linguistics (ACL)_.

 



## Appendix A More Analyses with Fermi

In this section, we provide more analyses of Fermi in addition to the analyses in Section 4.3.

GPT-4 for optimization \({\mathcal{M}_{\text{opt*}}}\)As denoted in Section 4.1, we commonly use GPT-4 for LLM \({\mathcal{M}_{\text{opt}}}\) to generate new prompts from the optimization memory (Eq. 5) for all the experiments in Section 4. To validate this design choice, we conduct the experiments by substituting GPT-4 with Chat-GPT \({\mathcal{M}_{\text{opt}}}\) in both OPRO and Fermi. Figure 5 is the optimization trajectory in terms of training accuracy (_i.e._, average accuracy of the prediction by \({\mathcal{M}}\) on previous user opinions). Here, one can observe that both OPRO and Fermi suffer in optimizing the prompt when we use ChatGPT as \({\mathcal{M}_{\text{opt}}}\), similar to the previous observation Yang et al. (2024); it reveals that generating the improved prompts from the optimization memory with previous prompts, scores, and contexts requires complex reasoning capability. Therefore, using a strong LLM with sufficient capability to optimize input prompts, such as GPT-4, is necessary. But, we remark that one can substitute GPT-4 with cheaper LLMs with a sufficient reasoning capability (_e.g._, GPT-4o-mini), as shown in Table 6

Optimization with stronger LLM for evaluation \({\mathcal{M}_{\text{}}}\)Next, to explore the compatibility of Fermi with different configurations of two LLMs during the optimization, we conduct the additional experiments by substituting evaluating LLM \({\mathcal{M}}\) to GPT-4 from ChatGPT; namely, two LLMs \({\mathcal{M}}\) and \({\mathcal{M}_{\text{opt}}}\) for evaluating and generating are GPT-4. The results on GlobalOpinionQA are presented in Table 7. It is observable that one can find further improved personalized prompts in terms of test accuracy, when using stronger LLM \({\mathcal{M}}\) for evaluating (Eq. 2). For example, compared to the use of personalized prompts optimized by ChatGPT as \({\mathcal{M}}\) (Fermi*), the optimization only using GPT-4 exhibits 1.9% additional test accuracy improvement. This result clearly shows that the proposed Fermi is compatible with different types and capacities of evaluating LLMs.

Importance of initial prompts in FermiFor the experiments, we used a fixed initial prompt template across all datasets in our experiments, that maximally incorporates the given user profiles, as it has proven effective in prior studies Hwang et al. (2023); Santurkar et al. (2023); Zhao et al. (2024), as described in Section 3.2 and Appendix B.3.

Nevertheless, to further provide insights about the impact of initial prompt templates on Fermi,

Figure 5: **Optimization trajectory under different LLMs for \({\mathcal{M}_{\text{opt*}}}\). Average training accuracies on GlobalOpinionQA across optimization iterations (\(T=10\)) under OPRO and Fermi.**

we conduct additional experiments by varying the initial prompt set \(P^{0}=\{\mathrm{p_{0}}\}\). To be specific, on GlobalOpinionQA dataset, we exclude the user profiles for the construction of the initial prompt unlike the original Fermi (in Table 1), and use the prompt of _Vanilla_ for the initialization. We denote this version as Fermi\({}_{\text{van}}\). The results (in comparison with other methods are) shown in Table 8, where Fermi (our method) consistently outperforms the baselines with both choices of prompt initialization while the gain is enlarged with better initialization when incorporating the user profile.

Continual optimization of prompts.In the previous experiments, we assumed that the fixed dataset \(U_{\texttt{opi}}\) of questions and user's opinions is given. However, in the real-world, user often interacts frequently with LLMs, which means that the dataset could be continuously updated. Therefore, the iterative process of refining prompts might incur significant computational costs, if it should be conducted from scratch at certain intervals (_e.g._, when the number of new data reaches a threshold).

To mitigate this issue, we conduct additional experiments to show that the idea of continual prompt optimization Razadiabedina et al. (2023); Wang et al. (2022); Zhu et al. (2022) could be applied to Fermi, and hence such cost could be drastically reduced. Specifically, we first conduct Fermi by using half of the previous questions and the user's responses \(U_{\texttt{opi}}\) (denoted by Fermi\({}^{\texttt{half}}\)). We remark that other parameters are kept the same such as the 10 iterations of the optimization. Then, with the entire \(U_{\texttt{opi}}\), we continuously conduct Fermi under the limited number of iterations, by initializing the prompt pool with previously optimized prompts in Fermi\({}^{\texttt{half}}\) (_i.e._, substituting the initialization in line 116). We denoted the results of this continuous optimization with 1 and 5 iterations as Fermi\({}^{\texttt{cont}}_{\text{iter}~{}1}\) and Fermi\({}^{\texttt{cont}}_{\text{iter}~{}5}\), respectively.

The results are presented in Table 9. First, it is notable that even with the reduced number of data for the optimization, Fermi still outperforms the strong baselines that are based on heuristic prompt engineering (_Profile_) or using the optimization by LLMs under full data (_ORPO_). However, one can also observe that the accuracy under full data is much better (74.8 vs. 73.0), which reveals that the data quantity is still important in Fermi. Next, it is also observed that the prompts could be successfully optimized continuously when the new data is added. Here, we denote that the previously optimized prompts in Fermi\({}^{\texttt{had1f}}\) are also re-used for the pool of Retrieval-of-Prompt, to keep the knowledge of previous iterations.4 Remarkably, even with only 1 additional iteration of optimization, the accuracy is significantly increased (73.0 \(\rightarrow\) 74.0). Also, when increasing the number of iterations to 5 (_i.e._, the same amount of computations compared to the original Fermi), the accuracy is increased and slightly outperforms the original optimization under the full data. Such improvement might be from an enlarged pool of Retrieval-of-prompt that enables better exploitation of previous knowledge.

Footnote 4: Integrating new prompts into each user’s retrieval pool adds minimal computational overhead for calculating their embeddings.

These results clearly show that the proposed framework is still effective for a more realistic scenario under the continuously updated user data.

**Different hyper-parameter with Fermi.** Here, we first conduct the additional experiments by varying \(K\) (number of new generation of prompts) and \(L\) (number of prompts in the memory), and present these results in Figure 6. Here, one can observe that using too small values of \(K\) and \(L\) significantly decreases the performance as it fails to find the effective prompt within the fixed iterations. However, after certain values including the originally used ones, Fermi successfully finds the personalized prompt and significantly outperforms the previous baselines.

Next, we conduct new experiments that vary \(N\) (number of previous questions and apply our framework, Fermi. Here, one can observe that more previous questions continuously improve the personalization performance of our method. Nevertheless, one can also verify that Fermi is indeed more sample efficient; Fermi achieves much higher accuracy than OPRO (73.0% v.s. 71.1%), even using half of the user's previous questions. This is because

\begin{table}
\begin{tabular}{c|c} \hline \hline Methods & GOQA (Acc.) \\ \hline Profile & 66.1 \\ OPRO\({}^{*}\) & 71.1 \\ Fermthal\({}^{\texttt{half}}\) & 73.0 \\ Fermi\({}^{\texttt{cont}}_{\text{iter}~{}1}\) & 74.0 \\ Fermi\({}^{\texttt{cont}}_{\text{iter}~{}5}\) & 74.9 \\ \hline Fermi & **74.8** \\ \hline \hline \end{tabular}
\end{table}
Table 9: **Continual prompt optimization. Test accuracy of ChatGPT over the different prompting methods on GlobalOpinionQA. \({}^{*}\) denotes the results with a two times larger pool for Retrieval-of-Prompt.** 

Fermi enables more efficient optimization by extracting the useful learning signal from mis-aligned responses, while OPRO just uses the average score of the current prompt as a learning signal. Consequently, our framework is more sample-efficient than the previous state-of-the-art method (OPRO), and hence one can achieve better personalization results even with fewer previous questions.

## Appendix B Experimental Details

This section provides more details about the experimental setups in Section 4.

### Datasets

First, we present more detailed descriptions of the used datasets: OpinionQA Santurkar et al. (2023), GlobalOpinionQA Durmus et al. (2023), LaMPtagLaMPrateLaMPtitleSalemi et al. (2023). Dataset statistics are presented in Table 10. Example of each dataset is presented in Figure 7.

* **OpinionQA** is a multiple-choice QA dataset originally constructed based on a public opinion survey (PewResearch), to evaluate the alignment of LM with 60 US demographic groups over various topics. As OpinionQA includes the information of each respondent, this dataset has been also used to evaluate the personalization of LLMs Hwang et al. (2023) and we also adopt it. Specifically, we use a subsampled split released by Hwang et al. (2023), which consists of 10.5k and 15.8k training and test QA pairs across 525 users and 15 topics; namely, each user has 20 training QA pairs and 30 test QA pairs for each topic, on average. Also, the average number of answer choices is 3.2. Then, we use training QA pairs as given previous opinions by user, and use test QA pairs to evaluate. In addition, for the experiments, we use all 12 types of user profiles included in the dataset: {Age, Citizenship, Region, Education, Income, Marital status, Political ideology, Political party, Race, Religion, Frequency of religious attendance, Gender}.
* **GlobalOpinionQA** is a multiple-choice QA dataset constructed from cross-national surveys to capture diverse opinions on global issues across different countries. Since the dataset originally included the answer distribution by multiple respondents in the same country, we converted it to have a single answer by selecting the choice with the highest probability, and treated each country as a specific user. To be specific, we set a threshold (0.8) and selectively use the data when its highest probability is higher than the threshold to guarantee the quality of the converted. It results in 920 training and 1,317 test QA pairs across 46 countries; namely, each user (country) has 20 training QA pairs and 28.6 test QA pairs for each topic, on average. Also, the average number of answer choices is 4.1. Then, we use training QA pairs as given previous opinions by user, and use test QA pairs to evaluate. Also, nationality becomes the only available profile. The full list of countries included in the dataset is presented in Table 11. Dataset could be downloaded from [https://huggingface.co/datasets/Anthropic/llm_global_opinions](https://huggingface.co/datasets/Anthropic/llm_global_opinions).
* **LaMPtag** is is a 15-way classification data where an input is a movie description and a label is a corresponding movie tag among 15 categories: {Sci-fi, Based on a book, Comedy, Action, Twist ending, Dystopia, Dark comedy, Classic, Psychology, Fantasy, Romance, Thought-provoking, Social commentary, Violence, True story}. Since the original dataset is proposed to consider the

Figure 6: **Effect of different hyper-paramters with Fermi. Test accuracy of ChatGPT over the baseline methods and Fermi with the different choices of hyper-parameters for generating prompts on GlobalOpinionQA.**



scenario of fine-tuning LMs and hence it consists of a large number of examples, we construct our dataset by subsampling from its validation dataset to make it suitable to evaluate LLMs with inference. It results in 1,000 training and 1,500 test QA pairs across 50 users, respectively.
* \(\mathbf{LaMP_{rate}}\) is a regression data where an input is a user review and a label is an integer rating (1-5), _i.e._, 1 is mostly negative and 5 is mostly positive. Under the same motivation with LaMPtag, we construct our dataset by subsampling from its validation dataset, which results in 1,000 training and 1,500 test QA pairs across 50 users, respectively.
* \(\mathbf{LaMP_{title}}\) is a generation data where input is an abstract of the paper and a label is a title generated by the user. Under the same motivation with LaMPtag, we construct our dataset by subsampling from its validation dataset, which results in 1,000 training and 1,500 test QA pairs across 50 users, respectively. LaMP benchmarks could be downloaded in [https://github.com/LaMP-Benchmark/LaMP](https://github.com/LaMP-Benchmark/LaMP).

### Baselines

In this section, we present the specific prompts used for the experiments in Section 4. Listing 1-4 are actually used prompts for Vanilla, Profile, Few-shot, and All Info, during the experiments on GlobalOpinionQA. Also, the prompt of OPRO used for the optimization is presented in Figure 9, which is the originally used one in Yang et al. (2024). While we're trying to adapt this prompt similar to ours in Figure 8, we observed that it degrades the performance of OPRO; for example, the average test accuracy is reduced to 70.7% from 71.1%. Therefore, we use the original prompt for all the experiments. We remark that each prompt is minimally adjusted to consider the difference between datasets. For example, as OpinionQA includes many available user profiles, we fully incorporate these with the prompt in Listing 5, following Hwang et al. (2023). Also, we present the prompt of Vanilla method on LaMPratedataset in Listing 6. In addition, we present the prompt used to convert the format of input prompt by Few-shot (Table 5) in Listing 7.

### Fermi

As denoted in Section 3.2, we need to provide an initial input prompt set P\({}^{0}=\{\text{p${}^{0}$}\}\). To this end, we use the heuristically design input prompts, which are presented in B.2. Specifically, we adopt the prompts used for _Profile_ tuned for each data, when the user profile \(U_{\texttt{pre}}\) is available (both OpinionQA and GlobalOpinionQA). Since our framework only utilizes a given few-shot previous opinions during the optimization, this way of initial prompting naturally enables us to fully utilize all the user information. When the user profile is not available, we adopt the prompts used for _VanialI_. In addition, we present a more detailed version of the prompt popt used to generate new input prompts with \(\mathcal{M}_{\texttt{opt}}\) in Figure 8. We remark that popt is minimally adjusted across dataset, to match the different task and user information of each dataset.

\begin{table}
\begin{tabular}{c c c c c c} \hline \hline Dataset & Task & Users & Types of User Profiles & \# of Previous Opinions & \# of Test Questions \\ \hline OpinionQA & Multiple Choice QA & 525 & Demographic and Ideology & 10.5k & 15.8k \\ GlobalOpinionQA & Multiple Choice QA & 46 & Nationality & 920 & 1,317 \\ LaMPtag & 15-way Movie Tagging & 50 & Not Available & 1,000 & 1,500 \\ LaMPrata & 5-scale Review Rating & 50 & Not Available & 1,000 & 1,500 \\ LaMPtitlta & Paper Title Gen. from Abstract & 50 & Not Available & 1,000 & 1,500 \\ \hline \hline \end{tabular}
\end{table}
Table 10: **Dataset statistics. More descriptions and statistics of datasets used in experiments.** 

f''' Choose the proper answer to the given question among the given answer choices. Your answers should be a single alphabet among given answer choices: Question: {question} Answer choices: {answer choice} Answer: ''' ```

Listing 1: Input prompt used for Vanilla method on GlobalOpinionQA.

## Appendix C Additional Quantitative Results

In this section, we provide additional quantitative results that can't be presented in the main draft due to the limited space. First, in Table 12, we present the average and standard deviation of topic-wise accuracy, _i.e._, the average and standard deviation are calculated across 35 users where each user receives 30 test questions in the same topic. Next, we 

present the test performance of _Few-shot_ method in Section 4, under different numbers of retrieved opinions (Table 13). Lastly, we present the test performance under a different number of considered training questions \(\tilde{N}\) (Eq. 7). As one can see in Table 14, \(\tilde{N}=3\) which is commonly used in our experiments shows consistent improvements in general, although the optimal values are different 

across the datasets.

## Appendix D More Comparison Examples between Personalized Prompts

In this section, we present more qualitative comparisons between the prompts from different methods for personalization of LLMs. To be specific, we present the specific test query from each data, and three corresponding prompts from the heuristic design, OPRO, and Fermi. Figures 12-17 are the comparison results on the datasets used in Section 4. Somewhat interestingly, one can observe that the personalized prompts by Fermi exhibit non-trivial incorporation of user information. In addition, we present examples of format-converted versions of few-shot prompting of previous user opinions (_i.e._, Few-shot\({}_{\texttt{format}}\) in Table 5) in Figures 19 and 20. Here, one can observe that the converted prompts have a similar form to the personalized prompts by Fermi which is more natural to understand and follow for LLMs, and hence it significantly improves the performance up to 10.1%, as shown in Table 5. Figure 10 is the example of personalized prompt and generated response under this on LaMPtitle.

 



\begin{table}
\begin{tabular}{c|c c c c c} \hline  & & \multicolumn{4}{c}{Datasets (Metric)} \\ \cline{3-6} \(\tilde{N}\) & OPQA & GOQA & LaMP\({}_{\text{tag}}\) & LaMP\({}_{\text{rate}}\) & LaMP\({}_{\text{title}}\) \\  & (Acc.) & (Acc.) & (Acc.) & (MAE) & (Rouge-L) \\ \hline \(\tilde{N}=1\) & 54.6 & 74.8 & 37.8 & 0.341 & 0.415 \\ \(\tilde{N}=3\) & 54.5 & 74.8 & 37.8 & 0.343 & 0.419 \\ \(\tilde{N}=5\) & 54.5 & 74.4 & 37.5 & 0.341 & 0.419 \\ \(\tilde{N}=10\) & 54.1 & 74.1 & 37.7 & 0.347 & 0.417 \\ \(\tilde{N}=20\) & 54.3 & 74.2 & 36.7 & 0.338 & 0.413 \\ \hline \end{tabular}
\end{table}
Table 14: **Different \(\tilde{N}\) for RoP. Test performance of ChatGPT under different \(\tilde{N}\) for RoP (Eq. 7).**


* [14] A. Krizhevsky, S. Bengio, and R. Fergus. Visualizing the image of natural images. In _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_, pages 235-238, 2016.
* [15] M. L. LeCun, J. Cortes, and C. Vinyals. The Visual Structure of Convolutional Neural Networks. _IEEE Transactions on Image Processing_, 7(9):1269-1272, 2016.
* [16] D. Lee, Y. Li, and H. Zhang. Deep residual learning for image recognition. In _Advances in neural information processing systems_, volume 1, pages 1-10, 2016.
* [17] G. Hinton, I. Goodfellow, and Z. Shlens. Imagenet classification with deep convolutional networks. In _NIPS conference on computer vision_, pages 1398-1399, 2016.
* [18] P. Kingma and Ba. Adam: a method for stochastic optimization. _arXiv preprint arXiv:1412.6152_, 2014.
* [19] T. Lin, X. Liu, W. Dong, and Q. Weinberger. Learning generative adversarial nets for large scale data. In _CVPR 2015 International Conference on Machine Learning_, pages 1398-1405, 2015.

[MISSING_PAGE_POST]

 

* [12] M. J. C. K. R. A. S. G. and D. E. T. W. H., _Theoretical Methods for the Study of Computer Science_**47**(3): 509-68, 2000.
* [13] M. J. C. K. R. A. S. G. and D. E. T. W. H., _Computational Complexity in Computing Systems_**11**(1): 1-10, 2000.

[MISSING_PAGE_POST]

 



**Question:** Generate a title for the following abstract of a paper: Research in verification and validation (V&V) for concurrent programs can be guided by practitioner information. A survey was therefore run to gain state-of-practice information in this context. The survey presented in this paper collected state-of-practice information on V&V technology in concurrency from 35 respondents. The results of the survey can help refine existing V&V technology by providing a better understanding of the context of V&V technology usage. Responses to questions regarding the motivation for selecting V&V technologies can help refine a systematic approach to V&V technology selection.

**Ground-truth Answer:** A state-of-practice questionnaire on verification and validation for concurrent programs

**Personalized response by Few-shot\({}_{\texttt{Bm25}}\):** State-of-Practice Information on V&V Technology in Concurrency: A Survey

**Optimized prompt by Fermi:** To adeptly craft personalized titles for academic paper abstracts, it is crucial to delve into the quintessence of the text, grasping the primary research focus and outcomes presented. The objective is to distill this essence into a title that is at once compelling and indicative of the paper's content. Strive to maintain a balance between specificity--a title that zeroes in on the core findings or innovative methodologies--and generality, ensuring the title conveys the broader implications and scope of the research. Such a title should be informative, engaging, and reflective of the paper's value to its field ... By emulating these stylistic nuances and concentrating on the research's unique elements, the title is tailored to resonate with the individual's mode of expression. This method aims to augment the compatibility between the generated title and the individual's own answering pattern, thereby improving the likelihood of achieving a high matching score. As a result, each generated title should not only meet academic standards of descriptiveness but also feel authentic to the individual's characteristic way of encapsulating research in a succinct, yet thorough, manner.

**Personalized response by Fermi:** Understanding the State-of-Practice in Verification and Validation for Concurrent Programs: Insights from a Survey

* [10] A. Datta, M. C. K. Snoek, and J. E. Marsden, "A new approach to the design of a single alphabet: A review on the design of a single alphabet: A survey on the design of a 

* [10] A. M. Krizhevsky, S. Goyal, and R. L. Krause, "A fast-learning approach to learning a class of neural networks: the case of deep reinforcement learning," in _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)_, pp. 2359-2367, 2014.
* [11] J. P. Kingma and Ba Ba, "Adam: A method for stochastic optimization," in _Advances in Neural Information Processing Systems (NIPS'08)_, volume 1, pages 1349-1350, 2014.
* [12] D. Kingma and Ba, "Adam: A method for stochastic optimization," in _Advances in neural information processing systems (NIPS'14)_, volume 1, pages 1349-1357, 2014.
* [13] C. Li, Y. Bengio, and H. W. Teh, "Deep residual learning with applications to image recognition," _arXiv preprint arXiv:1412.6110_, 2014.
* [14] B. Liu, X. Chen, Z. Zhang, and Q. Sun, "Deep residual learning with deep convolutional neural network," in _IEEE International Conference on Acoustics and Speech (ICASSP)_, pp. 1-10, 2014.
* [15] T. Lin, "Learning phrase representations from tiny images," in _Proceedings of the IEEE conference on computer vision and pattern recognition_, pp. 1-10, 2014.
* [16] I. Goodfellow, I. Sutskever, and J. Dean, "Generative adversarial nets for large data sets," in _Neural machine translation, speech recognition_, vol. 3, no. 3, p. 103002, 2014.
* [17] V. LeCun, E. Vanhoucke, and O. Zeil, "Improving deep neural networks via deep belief propagation," _Journal of Machine Learning Research_, vol. 3, no. 3, p. 103002, 2014.
* [18] N. Mirza, F. Chollet, A. Courville, and J. Schmidhuber, "Batch normalization by jointly connected layers," in _Advances in neural information processing systems_, vol. 3, no. 3, p. 103002, 2014.
* [19] A. Radford, "An introduction to artificial intelligence," _Machine Learning_, vol. 3, no. 3, p. 103003, 2014.
* [20] A. Radford, "The perceptron model," in _Proceedings of the IEEE international conference on computer vision and pattern recognition_, pp. 1-10, 2014.

[MISSING_PAGE_POST]

 

* [1] A. Datta, M. G. C. K. Wilson and J. E. Marsden, "The evolution of the solar wind in a solar system: The role of the solar wind in the solar system", _Journal of Geophysical Research_**20**(4), no. 3, pp. 57-68.

[MISSING_PAGE_POST]

 



[MISSING_PAGE_POST]

 

* [1]
* Question: Which tag does this movie relate to among the following tags?Royal Tenenbaum and his wife Eteline had three children and then they separated. All three children are extraordinary -- all geniuses. Virtually all memory of the brilliance of the young Tenenhum s was subsequently erased by two decades of betrayal, failure, and disaster. Most of this was generally considered to be their father's fault. "The Royal Tenenbaunay" is the story of the family's sudden, unexpected reunion one recentwriter. Answer choices: A. sci-fi B. based on a book C. comedy D. action E. twist ending F. dystopia G. dark comedy H. classic I. psychology J. fantasy K. romance L. thought-provoking M. social commentary N. violence O. true Story Answer: [2] Question: Which tag does this movie relate to among the following tags?Sam Flynn, the tech-savy and daring son of Kevin Flynn, investigates his father's disappearance and is pulled into The Grid. With the help of a mysterious program named Quorra, Sam quests to stop evil dictator Liu from crossing into the real world. Answer choices: A. sci-fi B. based on a book C. comedy D. action E. twist ending F. dystopia G. dark comedy H. classic I. psychology J. fantasy K. romance L. thought-provoking M. social commentary N. violence O. true story Answer: A [3] Question: Which tag does this movie relate to among the following tags?Brian Cohen is an average young Jewish man, but through a series of ridiculous events, he gains a reputation as the Messiah. When he's not dogging his followers or being scolded by his thrill mother, the hapless Brian has to contend with the pompous Pontius Pilate and acronym-obsessed members of a separatst movement Rife with Monty Python's signature absurdity, the tale finds Brian's life paralleling Biblical tree, albeitwith many more laughs. Answer choices: A. sci-fi B. based on a book C. comedy D. action E. twist ending F. dystopia G. dark comedy H. classic I.psychology J. fantasy K. romance L. thought-provoking M. social commentary N. violence O. true story Answer: H [4] Question: Which tag does this movie relate to among the following tags?In 1979 Ohio, several younsters are making a zombie movie with a Super-8 camera. In the midst of filming, the friends witness a horrifying train derailment and are lucky to escape with their lives. They soon discover that the catastrophe was no accident, as a series of unexplained events and disappearances soon follows. Deputy Jackson Lamb, the father of one of the kids, searches for the terrylving truth behind the crash. Answer choices: A. sci-fi B. based on a book C. comedy D. action E. twist ending F. dystopia G. dark comedy H. classic I.psychology J. fantasy K. romance L. thought-provoking M. social commentary N. violence O. true story Answer: A Based on the above previous questions and answers, choose the proper answer to the given question among the given answer choices. Select the genre from the list provided that most closely captures the essence of the film's theme. Indicate your preference clearly by selecting the alphabet that best represents your chosen genre. When tasked with assigning a movie tag from the presented choices, it is essential to immerse oneself in the provided synopsis, allowing for a comprehensive understanding of the underlying themes and narrative direction. Delve into the intricacies of the movie's storyline, identifying the genre and the primary elements that define the movie's character. After a meticulous consideration of the synopsis, chosest the letter corresponding to the movie tag that must closely embodies the films central tenets and overarching message. This selection should be the result of a reflective process, ensuring that the tag not only aligns with the storyline's genre but also resonates with its unique emotional impact and story/tealing approach. Strive for an insightful and accurate representation of the movie's core to enhance the relevance and preciseness of your tagged classification. Figure 15: **Comparison of prompts on LaMP\({}_{tag}\). Example of question from LaMP\({}_{tag}\) (1st row), and the prompts used to answer this question with Few-shot\({}_{cont}\) (2nd row), OPRO (3rd row), and Fermi (4th row).** 

* [1]**S.-G. Galle, M. C. R. Norman, and J. A. Wheeler, "The physics of the universe: The history of evolution, cosmology and cosmology," _Physics Reports_**206**(3), 579-844, 2000.

[MISSING_PAGE_POST]

 

* [1].
* Question: What is the score of the following review on a scale of 1 to 5? I noticed that several of the sets for the people who are seen more than once are repeated which is too bad. I wish there had been either other material in their place (either from them or other people) or else just delete the multiple jokes (thereby making it shorter in length). The video quality is also not very good. Answer: 2
* [2]. Question: What is the score of the following review on a scale of 1 to 5? loved the inspiration for the TV show. Springfield plays it differently but I liked it. Answer: 5
* [3]. Question: What is the score of the following review on a scale of 1 to 5? I borrowed this from my library first and had to buy my own copy of it and the sequel--it was that good! Answer: 5
* Based on the above previous questions and answers, answer to the given question. An outstanding synthesis of profound knowledge, showcasing an exceptional mastery of the subject matter with exceptional depth and clarity. Each insightful offers precise, well-articulated information that resonates perfectly with the posed question, delivering a comprehensive overview that controlly meets the inquiry's demands butalso introduces an expert-level analysis that enhances the understanding with significant context and nuance. Use the following guide to rate the sentiment in reviews on a scale of 1 to 5. If the review is unambiguously positive without any hint of dissatisfaction, assign a rating of 5. If the review is mostly favorable with minor negative remarks or even just neutral language, provide a rating of 4. For reviews that are mixed, with an evenly balanced view reflecting both positive and negative sentiments, allocate a rating of 3. Reviews that convey a principally negative impression but may include some positive observations should be scored with a 2. Lastly, reviews that are outrightnegative, showing no signs of positivity, should be granted a rating of 1. Respond only with the numerical score that epitomizes the overall sentiment of the review, with no need for additional comments or justification.

Figure 17: **Comparison of prompts on LaMP\({}_{\texttt{rate}}\). Example of question from LaMP\({}_{\texttt{rate}}\) (1st row), and the prompts used to answer this question with Few-shot\({}_{\texttt{cont}}\) (2nd row), OPRO (3rd row), and Fermi (4th row).**



* [1].
* Question: What is the score of the following review on a scale of 1 to 5? I heard the song on a TV commercial and thought it was one of the songs from the group Cars. But now I have learned to like a new singer. Gary has a lot of interesting songs. I hope to purchase more of his music. Answer:

Figure 18: **Comparison of prompts on LaMP\({}_{\texttt{rate}}\). Example of question from LaMP\({}_{\texttt{rate}}\) (1st row), and the prompts used to answer this question with Few-shot\({}_{\texttt{cont}}\) (2nd row), OPRO (3rd row), and Fermi (4th row).**



* <INS> Your answer should be a single alphabet among given answer choices: Question: In which of the following things do you believe, ifyou believe in any? Heaven Answer choices: A. Yes B. No C. Don't know D. No answer E. Other missing; Multiple answers Mail (EVS) Answer:

[1]. Question: In which of the following things do you believe, ifyou believe in any? Hell Answer choices: A. Yes B. No C. Don't know D. No answer E. Other missing; Multiple answers Mail (EVS) Answer: A

[2]. Question: In which of the following things do you believe, ifyou believe in any? Life after death Answer choices: A. Yes B. No C. Don't know D. No answer E. Other missing; Multiple answers Mail (EVS) Answer: A

[3]

[20]. Question: Have you been the victim of a crime during the past year? Respondent Answer choices: A. Yes B. No C. Don't know D. No answer E. Missing; Not available Answer: B

Based on the above previous questions and answers, choose the proper answer to the given question among the given answer choices. When responding to the following questions, please select the answer option denoted by a single letter that best aligns with the prevailing beliefs and social norms of Libyan society. Given Libya's strong Islamic values and cultural norms, your chosen response should reflect the commonly held views within the country. In instances where the provided options do not align with your personal beliefs, or you are uncertain, opt for the choice that represents a neutral stance or a preference to abstain from expressing an opinion. Provide your response concisely as a single letter, indicative of the consensus within Libyan society, which is influenced by their Islamic faith and cultural traditions.

Figure 19: **Example of format-converted prompts. Example of question from GlobalOpinionQA (1st row), and the prompts used to answer this question with Few-shot\({}_{\tt all}\) (2nd row) and Format-converted prompts (Few-shot\({}_{\tt format}\)) by prompting GPT-4 to convert the format using the personalized prompts by Fermi as reference (3rd row).**



* [1] A. D. Kessler, M. J. S. Murray, and R. E. Stanley, _Theoretical Computer Science_**2006**, 379-485 (2006)
* [2] C. L. Bender, P. W. Anderson, and T. G. Wilson, _A practical approach to the design of a new class of systems_, Journal of Computational Physics **199**, 103-118 (2006)

[MISSING_PAGE_POST]