Venue: NAACL 2025

# Generative Prompt Internalization

Haebin Shin\({}^{1,2*}\)  Lei Ji\({}^{2}\)   Yeyun Gong\({}^{2}\)   Sungdong Kim\({}^{1}\)   Eunbi Choi Minjoon Seo\({}^{1}\)

###### Abstract

Prompts used in recent large language model based applications are often fixed and lengthy, leading to significant computational overhead. To address this challenge, we propose Generative Prompt Internalization (GenPI), a lightweight method that employs a joint training approach. GenPI not only replicates the behavior of models with prompt inputs but also generates the content of the prompt along with reasons for why the model's behavior should change accordingly. We demonstrate that our approach effectively internalizes complex prompts across various agent-based application scenarios. For effective training without interactions with the dedicated environments, we introduce a data synthesis technique that autonomously collects conversational datasets by swapping the roles of the agent and environment. This method is especially useful in scenarios where only a predefined prompt is available without a corresponding training dataset. By internalizing complex prompts, Generative Prompt Internalization enables high-performance and efficient inference without the need for explicit prompts.1.

Footnote 1: Code are available at [https://github.com/kaistai/GenPI](https://github.com/kaistai/GenPI)

+
Footnote †: _Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers)_, pages 7338–7363

+
Footnote †: _Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers)_, pages 7338–7363

## 1 Introduction

In the real world, for inference in large language model (LLM) based applications (e.g. ChatGPT), fixed and complex prompts are often used repeatedly. Although advanced prompts can improve performance, their dependence on lengthy prompts raises concerns regarding computational efficiency for service providers and cost-effectiveness for users. Longer prompts can considerably increase computational overhead, especially during multi-turn inference, making it a potential bottleneck in practical applications. Therefore, finding effective strategies to optimize these prompts while maintaining performance is crucial for the practical deployment of LLM-based applications.

To address this issue, existing methods can be categorized into two approaches. One practical approach only for efficient prompt computation is compressing prompts into external embeddings Mu et al. (2023); Chevalier et al. (2023); Ge et al. (2024) or compressed text Jiang et al. (2023b); Li et al. (2023); Jiang et al. (2024); Pan et al. (2024). However, these methods still require additional tokens, limiting their ability to fully internalize prompts and effectively modify the model's behavior as intended. Alternatively, fine-tuning Kim and Rush (2016); Zou et al. (2024) or distillation Askell et al. (2021); Snell et al. (2022); Choi et al. (2023); Li et al. (2024), have been explored as canonical approaches for internalizing prompts. These methods adjust the language model's behavior to follow the prompt's intentions without requiring a prompt during inference.

However, these internalization methods are limited in that the model cannot reference the prompt's content during training. Instead, they rely on indirect training based on the model's output Kim and Rush (2016) or distribution Askell et al. (2021); Snell et al. (2022); Choi et al. (2023); Li et al. (2024) of the original model when the prompt is provided. This leads to significant performance degradation in information-rich tasks or specific requirements, such as agent's action requirements or schema grounding Choi et al. (2023).

To overcome the limitations, we propose Generative Prompt Internalization (GenPI), a method that is trained to generate the target prompt, rather than merely using it as input. GenPI employs joint loss training, combining two approaches: 1) mimicking the teacher's output to guide behavior, similar to distillation approaches Askell et al. (2021); Snell et al. (2022); Choi et al. (2023); Li et al. (2024), and 2) generating the content of the prompt while inferring why the output should change based on that prompt. GenPI employs a lightweight internalization process for each prompt, requiring only 0.5%

additional parameters on 1,000 samples. Focusing on scenario where only prompts are available without a corresponding training dataset for internalization Liu et al. (2023); Choi et al. (2023), we also introduce a data synthesis technique to generate a multi-turn pseudo conversational dataset. By simply swapping the roles of user and assistant within the given context, the model simulates both sides of conversation, enabling it to autonomously collect conversational datasets.

We evaluate our method in agent application scenarios with lengthy prompts using AgentBench Liu et al. (2023). GenPI maintains strong performance even without the need for prompt input, while outperforming other distillation or compression-based baselines. GenPI achieves 100% performance retention on OS interaction agent task, while maintaining at least 82% performance on web-based agent tasks with over 1,000 tokens. Furthermore, GenPI demonstrates 39% efficiency improvement in handling environments with lengthy prompts, outperforming other compression-based methods.

## 2 Related Works

**Prompt Internalization** is a methodology designed to embed prompt information within a language model, enabling the model to perform various tasks without requiring explicit prompt input. For instance, Askell et al. (2021) internalize persona-related prompts to facilitate helpful, honest, and harmless alignment, while Snell et al. (2022) incorporates both prompts and scratch pads into the internalization process to enhance performance on more complex tasks. Li et al. (2024) applies prompt internalization to internalize demonstrations for in-context learning scenarios. While these methods focus on internalizing broad and general coarse-grained information, Choi et al. (2023); Zou et al. (2024) introduce a more fine-grained approach, targeting more specific and predetermined prompts. Choi et al. (2023) focuses on short chat histories or task-specific instructions, and Zou et al. (2024) relies on a tailored training dataset to retrieve similar examples based on the given user input. Following these previous works, our goal is to enable the model to internalize the specific predetermined prompts. However, existing methods still limit the model's ability to directly learn the content of the prompt, as they rely on training a student model using outputs from a teacher model. We address this limitation by generating prompts that allow the model to learn their content directly, enabling it to handle more realistic and information-rich prompts.

**Prompt Compression** is one of the practical approach to reducing the computational overhead caused by lengthy prompts. In case of the API-based large language models (LLMs) services, systematic caching solutions allow frequently used prompts to be stored between API calls.23 For users of API-based LLMs, text-based prompt compression methods are proposed, where key segments of long prompts are selected on a token-by-token basis to generate a compressed version of the original prompt Jiang et al. (2023b); Li et al. (2023); Jiang et al. (2024); Pan et al. (2024). Although this approach reduces prompt length, it often still results in relatively long token sequences, as essential tokens must be retained. On the other hand, embedding-based prompt compression methods generate cached token embeddings for prompts, which can be utilized as a more efficient representation in LLMs Mu et al. (2023); Chevalier et al. (2023); Ge et al. (2024). These methods offer the advantage of using fewer token embeddings compared to text-based compression but often require modifications to the model architecture, making it challenging to leverage the compressed vectors across different models.

Footnote 2: [https://www.anthropic.com/news/prompt-caching](https://www.anthropic.com/news/prompt-caching)

Footnote 3: [https://ai.google.dev/gemini-api/docs/caching](https://ai.google.dev/gemini-api/docs/caching)

## 3 Problem Definition

Following Choi et al. (2023), we assume a scenario where an application-specific prompt \(p\) is predetermined. Our goal is to guide the model to behave as if the prompt is given, even in its absence. Similar to the previous distillation approaches Askell et al. (2021); Snell et al. (2022); Choi et al. (2023); Li et al. (2024), we define the teacher model \(T\) and the student model \(S\) based on whether the prompt \(p\) is provided to the same model \(\theta\).

The teacher model \(T\) is defined as a function that takes the prompt \(p\) and the input \(x_{i}\) at turn \(i\), generating the teacher's output: \(y^{T}_{i}=T(x_{i},p)=f_{T}(x_{i},p;\theta)\). The student model \(S\) is a function that takes the input \(x_{i}\) at turn \(i\), generating the student's output: \(y^{S}_{i}=S(x_{i})=f_{S}(x_{i};\theta)\). We approximate the student model's behavior to match that of the teacher model, conditioned on the prompt \(p\) over multiple turns, as shown in Equation 1,

\[P(y^{S}_{i}|x_{i})\approx P(y^{T}_{i}|x_{i},p).\] (1) 



## 4 Generative Prompt Internalization

In this section, we demonstrate our novel method, Generative Prompt International (GenPI), which internalizes the given prompt by generating its own contents. Basically, GenPI involves training the student model to mimic the behavior of the teacher model given the prompt \(p\). We finetune the student model on the multi-turn outputs generated by the teacher model (\(y_{i}^{T},i\in\{0,1,...,N\}\)), inspired by sequence-level knowledge distillation approaches Kim and Rush (2016); Touvron et al. (2023b) which follow the hard label distribution from the teacher model.

\[\mathcal{L}_{SFT}=-\sum_{i=0}^{N}\log P(y_{i}^{T}|x_{<i},y_{<i}^{T},x_{i}) \tag{2}\]

However, the model is still unable to learn the content of the prompt directly; it only learns indirectly through the teacher's output. We introduce an additional loss function, Prompt Generation loss (PG), where the loss is calculated directly on the prompt. This loss involves training the student model to understand why its behavior should align with the teacher's behavior based on the prompt content. As illustrated in Figure 1, for a given input \(x\), the model is trained to generate a prompt \(p\) along with a reason \(r\) for why the output should be changed. In this process, the student's output \(y_{i}^{S}\) is considered the initial state ("AS-IS"), while the teacher's output \(y_{i}^{T}\) represents the desired state ("TO-BE"). This process is formalized in Equation 3:

\[\mathcal{L}_{PG}=-\log P(p,r|y_{0}^{S},y_{0}^{T},x). \tag{3}\]

We utilize a hyperparameter \(\lambda\) to combine the losses into a joint loss function, resulting in the final joint loss function:

\[\mathcal{L}=(1-\lambda)~{}\mathcal{L}_{PG}+\lambda~{}\mathcal{L}_{SFT} \tag{4}\]

Our goal is to enable lightweight training and inference for each prompt, while also adapting to changes in the prompt effectively. To achieve this, we employ QLoRA Dettmers et al. (2023) to learn prompt-specific adaptors, allowing us to tailor our approach to each individual prompt.

## 5 Components for Prompt Internalization

Following Choi et al. (2023), we assume a realistic scenario where the predetermined prompt has never been encountered before and is not included in any training dataset for prompt internalization. Consequently, we first generate the components as a pseudo training dataset from the prompt \(p\).

Figure 1: Overview of Generative Prompt Internalization. SFT loss learns the teacher model’s behavior based on the user input. Prompt Generation loss internalizes the prompt by generating both the content of the prompt and the reason for why the model’s behavior should be modified. This process is guided by comparing the student model’s output (“AS-IS”) with the teacher model’s output (“TO-BE”). SFT loss and Prompt Generation loss are combined into a joint loss to train the prompt-specific adaptor.



Pseudo User Input.Similar to typical query generators Lewis et al. (2021); Choi et al. (2023); Oh et al. (2024), we simply utilize a large language model to generate pseudo user input \(x\) by prompting to generate a probably questionable user input from the given prompt \(p\). In this paper, we generate only 1,000 pseudo user inputs. Implementation details and examples are provided in Appendix A.1 and Appendix A.2.

Pseudo Conversational Outputs.In many task-specific applications, such as agent applications, the model interacts with the real environment across multiple turns Liu et al. (2023); Zeng et al. (2023); Chen et al. (2024). This means that collecting multi-turn conversation outputs requires interacting with the actual environment at every turn, which is a non-trivial problem. As our primary objective is to gather the teacher model's behavioral patterns, rather than to optimize the task performance, we introduce a simple method: _Self Role-Playing Conversation_. This method simply involves reversing the role between agent and environment. As illustrated in Figure 2, we provide the model \(\theta\) with a role-reversed context to simulate the task environment. By swapping the agent and environment roles in the prompt, a single model \(\theta\) can effectively embody both environment and agent personas, enabling the collection of self-conversational outputs \(y\). We report the examples and additional quality evaluation in Appendix A.3. Implementation details and examples are provided in Appendix A.1 and Appendix A.2.

Reason.To collect the supervision for the reason \(r\), we utilize a large language model by prompting to generate from the prompt \(p\), user input \(x\), and outputs \(y^{S}\), \(y^{T}\). As illustrated in Figure 1, the reason explains why the student's output ("AS-IS") should be changed to teacher's output ("TO-BE"). Implementation details and examples are provided in Appendix A.1 and Appendix A.2.

## 6 Experiments Setup

### Dataset

We evaluate our method on the three agent benchmark tasks: OS-Interaction Liu et al. (2023), Web Browsing Deng et al. (2023), Web Shopping Yao et al. (2022). As mentioned in Section 5, we assume our scenario where an application-specific prompt \(p\) is predetermined before the model is deployed. For that, we utilize the AgentBench Liu et al. (2023) settings, which organize a prompt with multi-turn evaluation samples for each task. Each task's prompt consists of general task description, agent's action space description, and shot-examples for agent behavior. Detailed information for each task's prompt is described in Appendix B.

OS Interaction Liu et al. (2023).This task involves interacting with an Ubuntu Docker container using bash commands and committing answers, with a 474 token4 prompt and 144 evaluation samples. The agent's performance is measured by the Success Rate.

Footnote 4: All tokens are calculated by the LLaMA tokenizer Dubey et al. (2024).

Web Browsing Deng et al. (2023).This task formulates element selection as a multi-choice QA problem. We follow the AgentBench Liu et al. (2023) setting, where the agent operates within an HTML action space (e.g., click, type, select options) using a 1,424 token prompt and 100 evaluation samples. The agent is evaluated using a Success Rate metric, which is based on two criteria: the correctness of the selected element and whether the predicted operation matches the ground truth value for each step.

Figure 2: Self Role-Playing conversation. Collecting pseudo conversational output by switching the role in the prompt.

 

Webb Shopping (Yao et al., 2022).In this task, the agent navigates scraped _amazon.com_ pages to identify the correct product using click or search actions. The task consists of a 1,285 token prompt and 200 evaluation samples. The task is completed when the agent clicks "buy now" or exceeds the turn limit, and performance is evaluated using a reward function that assesses the similarity between the expected and actual product attributes as following the Liu et al. (2023). Please refer to Appendix B for more details on the metric.

### Baselines

We explore various baselines to internalize the prompt, ranging from distillation approaches Snell et al. (2022); Choi et al. (2023) to compression approaches Pan et al. (2024); Ge et al. (2024).

Full Fine-tuning.Since previous typical prompt internalization methods are based on much smaller model Choi et al. (2023) or are not publicly available Snell et al. (2022), we adopt recent knowledge distillation approaches Lin et al. (2020); Gu et al. (2024); Ko et al. (2024) for autoregressive large language models in our prompt internalization setup. (1) The basic distillation approach Hinton et al. (2015) employs the Kullback-Leibler Divergence loss (KLD) between the logit distributions of the student and teacher model. (2) Sequence-level Knowledge Distillation (SeqKD) Kim and Rush (2016) enforces the student model to generate the teacher model's outputs on a fixed dataset. (3) As a strong baseline, we also employ a joint loss (SeqKD+KLD), inspired by recent knowledge distillation works Song et al. (2020); Gu et al. (2024), which incorporates the language modeling loss during distillation. This approach can be interpreted as a hybrid distillation loss that combines the benefits of both soft labels and hard labels from the teacher model.

Prompt Prepending.One straightforward approach to consider is prepending the prompt during finetuning. If the prompt is consistently prepended during training, we expect the model to indirectly contextualize the prompt. However, since the model needs to predict without the prompt during inference, there is a potential mismatch between the training and inference distributions. To address this concern, we compare two baseline approaches: (1) always prepending the prompt during training (100% probability), and (2) stochastically prepending the prompt during training (50% probability). This baseline utilizes QLoRA Dettmers et al. (2023) with the same settings as our method. This approach can be viewed as an extension of the method that relies exclusively on SFT loss for training.

Text-based Prompt Compression.LLMLingua-2Pan et al. (2024) is a prompt-agnostic method for generating compressed texts. LLMlingua-2 explicitly compresses tokens using a smaller model, such as XLM-RoBERTa-large Conneau et al. (2020). By performing with the compressed prompt, this baseline is expected to achieve efficient inference while maintaining comparable performance.

Embedding-based Prompt Compression.We utilize ICAE Ge et al. (2024) to compress the prompt into cached prompt embeddings. Following Ge et al. (2024), we compress the prompt into 128 tokens and prepend compressed tokens to the user input at each inference time. Since the official checkpoint of ICAE exhibits significantly low performance on AgentBench Liu et al. (2023), we finetune the baseline specifically for AgentBench Liu et al. (2023). Additional details regarding this baseline are described in Appendix C.3.

Upper Bound.We utilize the teacher model as an upper bound that inputs the full prompt, consistent with previous studies Snell et al. (2022); Choi et al. (2023). In our preliminary experiments (see Appendix C.2), we observe that the fine-tuned model performed worse than the original model in AgentBench Liu et al. (2023). Consequently, we consider the original model with the full prompt as the teacher model following the setup of Liu et al. (2023). Since ICAE Ge et al. (2024) is based on the Mistral-7B Jiang et al. (2023) model, we establish a separate upper bound specifically for ICAE Ge et al. (2024) to ensure a fair comparison. For a detailed explanation of the upper bound, please refer to the Appendix C.2.

### Implementation Details

To internalize the agent-based prompt into the language model, we utilize the LLaMA-3-8B-Instruct Dubey et al. (2024) as the target model. Following our scenario in Section 5, we fine-tune the model using the 1,000 pairs of pseudo dataset as our train dataset, and this is applied equally to all baselines. We utilize QLoRA Dettmers et al. (2023) with rank \(r=16\) only requires 0.5% of pa 

rameters. Additional training details are described in Appendix C.4.

## 7 Results

### Performance

Compression baselinesLLMLingua-2 Pan et al. (2024) exhibits a significant performance drop as the compression rate increases. Although the semantics of the prompt can be inferred, the loss of format crucial for agent tasks leads to substantial performance drops. In the Web Shopping scenario, LLMLingua-2 Pan et al. (2024) achieves the best performance at a compression rate of 90%. However, compression rates exceeding 30% led to a failure in all tasks. In the case of ICAE Ge et al. (2024), despite fine-tuning, it struggles to handle the agent application prompts. Many embedding-based compression approaches Mu et al. (2023); Chevalier et al. (2023), including ICAE Ge et al. (2024), are primarily optimized for general texts such as articles, and thus exhibit limitations in compressing task-specific information, as required by AgentBench Liu et al. (2023).

Distillation/Finetune BaselinesWhen trained with only the basic distillation loss (KLD), the model fails in AgentBench Liu et al. (2023), suggesting that 1,000 training examples may not be enough to train the entire model. While SeqKD helps promising results in the Web Browsing task, its performance is still lacking in other tasks. Among prompt prepending baselines, stochastically prepending prompts demonstrates some performance gains but ultimately fails to overcome the distribution mismatch at inference time, resulting in poorer performance compared to our approach.

Generative Prompt InternalizationGenPI achieves superior performance in the OS interaction task, reaching 100% of the upper bound. It also consistently demonstrates high performance of over 82% across Web Browsing and Web Shopping tasks, both of which have longer prompts that exceed 1,000 tokens in length. The incorporation of PG loss into SFT loss shows a notable performance improvement of approximately 25% in web shopping tasks. Furthermore, when compared to Prompt Prepending which is based on SFT loss and QLoRA Dettmers et al. (2023) adaptor, our approach exhibits higher performance by leveraging PG loss, rather than relying solely on prepending prompts to the input.

\begin{table}
\begin{tabular}{l r r r r r r r} \hline \hline \multirow{2}{*}{Methods} & \multicolumn{1}{c}{\multirow{2}{*}{w/o Prompt}} & \multicolumn{2}{c}{OS Interaction} & \multicolumn{2}{c}{Web Browsing} & \multicolumn{2}{c}{Web Shopping} \\ \cline{3-6}  & & SR & Norm. & SR & Norm. & Rewards & Norm. \\ \hline \hline Upper Bound & ✗ & 17.36 & 100.00 & 17 & 100.00 & 54.16 & 100.00 \\ \hline Text-based Compression & & & & & & & \\ LLMLingua-2 (x0.9) & ✗ & 4.16 & 23.96 & 9 & 52.94 & **50.78** & **93.76** \\ LLMLingua-2 (x0.7) & ✗ & 3.47 & 19.99 & 1 & 5.88 & 43.69 & 80.67 \\ LLMLingua-2 (x0.3) & ✗ & 0 & 0.00 & 0 & 0.00 & 0 & 0.00 \\ \hline Embedding-based Compression & & & & & & & \\ Upper Bound* & ✗ & 10.41 & 100.00 & 13 & 100.00 & 11 & 100.00 \\ ICAE & ✗ & 4.16 & 39.96 & 0 & 0.00 & 1.14 & 10.36 \\ \hline Full Finetune & & & & & & & \\ KLD & ✓ & 0 & 0.00 & 0 & 0.00 & 0 & 0.00 \\ SeqKD & ✓ & 3.47 & 19.99 & **15** & **88.24** & 38.79 & 71.62 \\ SeqKD + KLD & ✓ & 4.86 & 28.00 & **15** & **88.24** & 40.26 & 74.34 \\ \hline Prompt Prepending & & & & & & & \\
100\% probability & ✓ & 0 & 0.00 & 0 & 0.00 & 0 & 0 \\
50\% probability & ✓ & 13.19 & 75.98 & 12 & 70.59 & 31.09 & 57.40 \\ \hline Ours & & & & & & & \\ only SFT loss & ✓ & 15.97 & 91.99 & 12 & 70.59 & 35.64 & 65.81 \\ Generative Prompt Interalization & ✓ & **17.36** & **100.00** & 14 & 82.35 & 44.46 & 82.09 \\ \hline \hline \end{tabular}
\end{table}
Table 1: Performance evaluation results for AgentBench Liu et al. (2023) dataset. _SR_ denotes the Success Rate, and _Norm_. denotes the normalized score using Upper Bound following Choi et al. (2023). Since ICAE Ge et al. (2024) is based on Mistral 7B Jiang et al. (2023), we report the normalized score using Mistral-7B-instruct-v0.2 as the upper bound score (Upper Bound*). The best results are in bold, while second-best ones are underlined.

 



### Analysis

To understand the factors behind the performance improvement of GenPI, we perform three types of ablation studies. First, we compare the impact of PG loss and SFT loss on task performance. Second, we investigate the influence of the reason and prompt in PG loss by removing each component during generation. Finally, instead of omitting components, we explore the effect of reordering the input and output elements in PG loss while keeping the overall information level constant.

#### 7.2.1 Impact of the Loss

As shown in Table 2, the removal of SFT loss causes the model to suffer significantly in task performance. This suggests that task performance is fundamentally dependent on SFT loss. However, incorporating PG loss consistently improves performance across all tasks. Our findings indicate that while SFT loss focuses on task behavior, PG loss plays a complementary role in enhancing task performance by concentrating on prompt internalization.

#### 7.2.2 Impact of the Reason/Prompt

In Table 2, the prompt \(p\) has a greater impact on performance than the reason \(r\) across all tasks. This is interpreted as being due to the fact that the prompt is longer in length than the reason and generally contains more information, thereby exerting a more significant influence on the Prompt Generation loss. This highlights our primary contribution, directly generating the prompt itself, allowing the model to internalize and utilize prompt information more effectively.

#### 7.2.3 Impact of Input/Output Ordering

Despite ensuring consistency in the overall amount of information in Table 3, we observe a decline in overall performance when certain components are removed during the generation process similar to Table 2. Except for the OS task which has relatively short prompt length than Web Browsing or Web Shopping, \(P(r|p,x,y)\) consistently outperformes \(P(p|x,y,r)\). This result is attributed to the fact that providing the model with a prompt, input, and output, followed by generating a reason, aligns more closely with the typical dialogue flow of an LLM, compared to inferring the prompt from an input, output, and reason. Since the prompt generally contains more information than the reason, inferring a prompt based on the reason is likely to be a more challenging task for the model.

Similarly, when comparing _w/o reason_ in Table 2 (equivalent to \(P(p|x,y)\)) and \(P(p|x,y,r)\), we see that while incorporating the reason as input slightly improved scores in the OS task, the performance drops in the web browsing and web shopping tasks. The prompt in the OS task is relatively shorter, making it easier to infer prompts based on the reason. However, as the length of the prompt increases, it becomes more challenging for the model, leading to a negative impact on training. When comparing with _w/o prompt_ in Table 2 (equivalent to \(P(r|x,y)\)) and \(P(r|p,x,y)\), we observe that adding prompt information leads to an improvement in all tasks.

## 8 Efficiency

To compare the efficiency between Generative Prompt Internalization and other baselines that require fixed prompts, we sample an example with more than 5 turns from a Web Shopping task containing over 1200 tokens.5 We then compare the MACs (Multiply-Accumulate Operations), FLOPs (Floating Point Operations), and latency to LLaMA-3-8B-Instruct Dubey et al. (2024) based baselines. All metrics were measured by DeepSpeed Profile6.

Footnote 5: Efficiency is measured on a single NVIDIA A6000 GPU and AMD EPYC 7513 CPU, featuring 32 physical cores.

\begin{table}
\begin{tabular}{c c c c} \hline \hline  & OS & WB & WS \\ \hline \hline Ours & 17.36 & 14 & 44.46 \\ \hline \multicolumn{4}{l}{Prompt Generation loss ablation} \\ w/o reason & 14.58 & 14 & 38.41 \\ w/o prompt & 13.19 & 13 & 32.34 \\ \hline \multicolumn{4}{l}{Joint loss ablation} \\ w/o \(\mathcal{L}_{PG}\) & 15.97 & 12 & 35.64 \\ w/o \(\mathcal{L}_{SFT}\) & 0 & 0 & 0 \\ \hline \end{tabular}
\end{table}
Table 2: Results of Ablation Studies for Prompt Generation loss and joint loss. _WB_ refers to the Web Browser task, and _WS_ refers to the Web Shopping task.

\begin{table}
\begin{tabular}{c c c c} \hline \hline Ordering & OS & WB & WS \\ \hline \hline \(P(p,r|x,y)\) (Ours) & 17.36 & 14 & 44.46 \\ \(P(r|p,x,y)\) & 14.58 & 14 & 36.26 \\ \(P(p|x,y,r)\) & 15.27 & 13 & 35.51 \\ \hline \end{tabular}
\end{table}
Table 3: Results of Input/Output Ordering Analysis.



As shown in Figure 3, the overhead for baselines that require prompts accumulates and increases with each turn. While LLMLingua-2 Pan et al. (2024) exhibits slightly better performance due to the reduced number of prompt tokens, it still incurs a larger overhead compared to ours. Furthermore, when considering compression overhead, the LLMLingua-2 Pan et al. (2024) which is based on XLM-RoBERTa-large Conneau et al. (2020), shows relatively low MACs and FLOPs than inferring with the prompt. However, it requires high compression latency proportional to the prompt length, resulting in extra latency in the first turn.

To simulate a more realistic scenario used in the real-world chat application, we conduct an additional experiment inspired by recent KV caching Pope et al. (2023) works across multi-turn dialogues Kwon et al. (2023); Zheng et al. (2023); Gao et al. (2024). We assume that the previous turn's attention values are pre-cached to reduce the latency. The prompt is also considered as a previous turn's conversation and is pre-cached. In this setting, the overhead between each turns is independent, and we report the conversation-level total overhead over 5 turns.

In Figure 4, the prompt caching time executed before the first turn incurs higher computational overhead due to the use of a larger model compared to the compression model. From the second turn onward, although caching previous conversation histories eliminates redundant computations, the computational overhead still increases proportionally to the length of the prompt. Even without considering in caching time, GenPI demonstrates a \(39\%\) improvement in MACs and FLOPs, along with a 17% improvement in latency. Consequently, despite caching, baselines inevitably require more computational overhead compared to Generative Prompt Internalization.

## 9 Discussion

In this section, we discuss the strengths and limitations of two prompt internalization strategies: generalized prompt-agnostic compression and prompt-specific internalization approach.

Prompt compression approaches, such as LLMLingua-2 Pan et al. (2024) and ICAE Ge et al. (2024), effectively compress short instruc

Figure 4: Comparison of computational overhead in LLaMA-based baselines applying KV caching Pope et al. (2023) across the multi-turn conversation. Even if the previous contents are cached, a long context still creates extra overhead. Best viewed in color.

Figure 3: Comparison of computational overhead in LLaMA-based baselines as the conversation turn progresses. All generations within a turn are reported with KV caching Pope et al. (2023) applied. Best viewed in color.

 

tions without requiring additional training for new prompts (Mu et al., 2023; Chevalier et al., 2023; Ge et al., 2024). However, as shown in Table 1, these methods struggle with long, information-rich application prompts, where key details are often lost in the compression process.

Prompt internalization approach, such as GenPI, overcome this limitation by explicitly internalizing prompts through targeted training. While GenPI requires training for each new prompt, this overhead is minimal in practical applications where only a limited set of prompts is needed. Once trained, switching between adapters incurs negligible computational cost, making the approach both flexible and scalable. Each adapter requires only 1,000 data samples for training. Even when generating the dataset with a 70B LLM, entire process can be completed in under an hour using four A6000 GPUs with vLLM (Kwon et al., 2023), while training takes approximately three hours on a single A6000 GPU.

## 10 Conclusion

In this paper, we propose a novel prompt internalization method, Generative Prompt Internalization (GenPI), which generates both the contents of the prompt and the reasoning behind changes to the model's output while mimicking the behavior of the teacher model. To address the challenging scenario where only predetermined prompts are available without additional training data, we introduce Self Role-Playing Conversation, a method that generates a pseudo-conversational training dataset from the given prompt. Our approach demonstrates that even in scenarios with lengthy agent prompts, GenPI maintains high performance without relying on the prompt itself. Moreover, GenPI improves efficiency by fully internalizing the prompt without requiring any additional tokens.

### Limitations

We assume the use of long and fixed prompts from realistic application scenarios. However, collecting official prompts from real-world applications (such as ChatGPT) poses significant challenges, leading us to rely on prompts from academic agent-based applications. This introduces a limitation, as only a representative prompt for each task are considered, and our approach handles a relatively small set of prompts. In future work, we plan to further explore context internalization in various domains, including long chat histories, in-context learning, and retrieval-augmented generation (RAG), as well as multimodal applications involving video and image data.

## Acknowledgements

This project was supported by Microsoft Research Asia. Additionally, this research was supported by the MSIT (Ministry of Science, ICT), Korea, under the Global Research Support Program in the Digital Field program (RS-2024-00436680) supervised by the IITP (Institute for Information & Communications Technology Planning & Evaluation). And, this work also supported by Institute for Information & communications Technology Promotion (IITP) grant funded by the Korea government (MSIT) (RS-2024-00398115, Research on the reliability and coherence of outcomes produced by Generative AI).

## References

* Askell et al. [2021] Amanda Askell, Yuntao Bai, Anna Chen, Dawn Drain, Deep Ganguli, Tom Henighan, Andy Jones, Nicholas Joseph, Benjamin Mann, Nova Dassarma, Nelson Elhage, Zac Hatfield-Dodds, Danny Hernandez, John Kernion, Kamal Ndousse, Catherine Olsson, Dario Amodei, Tom B. Brown, Jack Clark, Sam McCandlish, Christopher Olah, and Jared Kaplan. 2021. A general language assistant as a laboratory for alignment. _arXiv_.
* Chen et al. [2024] Zehui Chen, Kuikun Liu, Qiuchen Wang, Jiangning Liu, Wenwei Zhang, Dahua Lin, Kai Chen, and Feng Zhao. 2024. Agent-flan: Designing data and methods of effective agent tuning for large language models. _arXiv_.
* Chevalier et al. [2023] Alexis Chevalier, Alexander Wettig, Anirudh Ajith, and Danqi Chen. 2023. Adapting language models to compress contexts. In _EMNLP_.
* Choi et al. [2023] Eunbi Choi, Yongrae Jo, Joel Jang,oonwoon Jang, and Minjoon Seo. 2023. Fixed input parameterization for efficient prompting. In _Findings of ACL_.
* Conneau et al. [2020] Alexis Conneau, Kartiky Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzman, Edouard Grave, Myle Ott, Luke Zettlemoyer, and Veselin Stoyanov. 2020. Unsupervised cross-lingual representation learning at scale. In _ACL_.
* Deng et al. [2023] Xiang Deng, Yu Gu, Boyuan Zheng, Shijie Chen, Samuel Stevens, Boshi Wang, Huan Sun, and Yu Su. 2023. Mind2web: Towards a generalist agent for the web. In _NeurIPS_ 

Tim Dettmers, Artidoro Pagnoni, Ari Holtzman, and Luke Zettlemoyer. 2023. Qlora: Efficient finetuning of quantized llms. In _NeurIPS_.
* Dubey et al. (2021) Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, Anirudh Goyal, Anthony Hartshorn, Aobo Yang, Archi Mitra, Archie Sravankumar, Artem Korenev, Arthur Hinnsvark, Arun Rao, Aston Zhang, Aurelien Rodriguez, Austen Gregerson, Ava Spataru, Baptiste Roziere, Bethamy Biron, Binh Tang, Bobbie Chern, Charlotte Caucheteux, Chaya Nayak, Chloe Bi, Chris Marra, Chris McConnell, Christian Keller, Christophe Touret, Chunyang Wu, Corinne Wong, Cristian Canton Ferrer, Cyrus Nikolaitis, Damiem Allonsius, Daniel Song, Danielle Pintz, Danny Livshits, David Esioub, Dhruv Choudhary, Dhruv Mahajan, Diego Garcia-Olano, Diego Perino, Dieuwke Hupkes, Egor Lakomkin, Ehab AlbBadawy, Elina Lobanova, Emily Dinan, Eric Michael Smith, Filip Radenovic, Frank Zhang, Gabriel Synnaeve, Gabrieile Lee, Georgia Lewis Anderson, Graeme Nahi, Gregoire Misalon, Guan Pang, Guillem Cucurell, Hailey Nguyen, Hannah Korevaar, Hu Xu, Hugo Touvron, Iliyan Zarov, Imanol Arrieta Ibarra, Isabel Kluomann, Ishan Misra, Ivan Evtimov, Jade Copet, Jaewon Lee, Jan Geffert, Jana Vranes, Jason Park, Jay Mahadeokar, Jeet Shah, Jelmer van der Linde, Jennifer Billock, Jenny Hong, Jenya Lee, Jeremy Fu, Jianfeng Chi, Jianyu Huang, Jiawen Liu, Jie Wang, Jiecao Yu, Joanna Bitton, Joe Spisak, Jongsoo Park, Joseph Rocca, Joshua Johnstun, Joshua Saxe, Junting Jia, Kalyan Vasuden Alwala, Kartikeva Upasani, Kate Plawiak, Ke Li, Kenneth Heafield, Kevin Stone, Khalid El-Arini, Krithika Iyer, Kshiftiz Malik, Kuenley Chiu, Kunal Bhalla, Lauren Rantala-Years, Laurens van der Maaten, Lawrence Chen, Liang Tan, Liz Jenkins, Louis Martin, Lovish Madan, Lubko Malca, Lukas Blecher, Lukas Landzaat, Luke de Oliveira, Madeline Muzzi, Mahesh Pasupuleti, Mannat Singh, Manohar Paluri, Marcin Kardas, Mathew Oldham, Mathieu Rita, Maya Pavlova, Melanie Kambadur, Mike Lewis, Min Si, Mitesh Kumar Singh, Mona Hassan, Naman Goyal, Narjes Torabi, Nikolay Bashlykov, Nikolay Bogoychev, Niladr Chatterji, Olivier Duchenne, Onur Celebi, Patrick Mirassy, Pengchuan Zhang, Pengwei Li, Petar Vasic, Peter Weng, Prijwal Bhargava, Pratik Dubal, Praveen Krishnan, Punit Singh Korura, Puxin Xu, Qing He, Qingxiao Dong, Ragavan Srinivasan, Raj Ganapathy, Ramon Calderer, Ricardo Siliveira Cabral, Robert Stojnic, Roberta Railaanu, Rohit Girdhar, Rohit Patel, Roman Sauvestre, Ronnie Polidoro, Roshan Sumbaly, Ross Taylor, Ruan Silva, Rui Hou, Rui Wang, Saghar Hosseini, Sahana Chennbaspappa, Sanjay Singh, Sean Bell, Seohyurn Sonia Kim, Sergey Edunov, Shaoliang Nie, Shararan Narang, Sharthaparthy, Sheng Shen, Sheng Wan, Shruit Bhosale, Shunn Zhang, Simon Vandhende, Soumya Batra, Spencer Whitman, Sten Sootla, Stephane Collot, Suchin Gururangan, Sydney Borodinsky, Tamar Herman, Tara Fowler, Tarek Sheasha, Thomas Georgiou, Thomas Scialam, Tobias Speckbacher, Todor Mihalvo, Tong Xiao, Ujjwal Karn, Vedanuj Goswami, Vibhor Gupta, Vitghens Ramanathan, Viktor Kerlex, Vincent Gonquet, Virginie Do, Visth Vogeth, Vladaru Petrovic, Weiwei Chu, Wenhan Xiong, Wenjun Fu, Whitney Meers, Xavier Martinet, Xiaodong Wang, Xiaqqing Ellen Tan, Xinfeng Xie, Xuchao Jia, Xuewei Wang, Yaelle Goldschlag, Yashesh Gaur, Yasmine Babaei, Yi Wen, Yiwen Song, Yuchen Zhang, Yue Li, Yuning Mao, Zacharie Delpierre Coudert, Zheng Yan, Zhengying Chen, Zoe Papakiros, Aadiurg Singh, Aaron Grattafi, Abha Jain, Adam Kelsey, Adam Shajnfeld, Aditya Gangidi, Adolfo Victoria, Ahiva Goldstand, Ajay Menon, Ajay Sharma, Alex Boesenberg, Alex Vaughan, Alexei Baevski, Allie Feinstein, Amanda Kallet, amit Sangani, Anam Yunus, Andreiu Lupu, Andres Alvarado, Andrew Caples, Andrew Gu, Andrew Ho, Andrew Poulton, Andrew Ryan, Ankit Ramachandran, Annise Franco, Aparajita Sarraf, Arkabandhu Chowdhury, Ashley Gabriel, Ashwin Bharampe, Assaf Eisenman, Azadeh Yazdan, Beau James, Ben Maurer, Benjamin Leonhardi, Bernie Huang, Beth Loyd, Beto De Paola, Bhargavi Paranjape, Bing Liu, Bo Wu, Boyu Ni, Braden Knock, Bram Wasti, Brandon Spence, Brani Stojkovic, Brian Gamido, Britt Montalvo, Carl Parker, Carly Burton, Catalina Meijia, Changhan Wang, Changkyu Kim, Chao Zhou, Chester Hu, Ching-Hsiang Chu, Chris Cai, Chris Tingtal, Christoph Feichtenhofer, Damon Civin, Dana Beaty, Daniel Kreymer, Daniel Li, Danny Wyatt, David Adkins, David Xu, Davide Testugengine, Delia David, Devi Parikh, Diana Liskovich, Didem Foss, Dingkang Wang, Duc Le, Dustin Holland, Edward Dowling, Eissa Jamil, Elaine Montgomery, Eleonora Presani, Emily Hahn, Emily Wood, Erik Brinkman, Esteban Arcuate, Evan Dunbar, Evan Smothers, Fei Sun, Felix Kreuk, Feng Tian, Firat Ozgenel, Francesco Cageioni, Francisco Guzman, Frank Kanayer, Frank Seide, Gabriela Medina Fiorez, Gabriella Schwarz, Gada Badeter, Georgia Swee, Gil Halpern, Govind Thattai, Grant Herman, Grigory Sizov, Guangyi, Zhang, Guna Lakshminarayanan, Hamid Shojanazeri, Han Zou, Hannah Wang, Hanwen Zha, Haroun Habeeb, Harrison Rudolph, Helen Suk, Henry Asepgren, Hunter Goldman, Ibrahim Damali, Igor Molbyog, Igor Turanov, Irina-Elena Veliche, thei Mai, Jake Weissman, James Geboski, James Kohli, Jahpet Asher, Jean-Baptiste Gaya, Jeff Marcus, Jeff Tang, Jennifer Chan, Jenny Zhen, Jeremy Reizenstein, Jeremy Teboul, Jessica Zhong, Jian Jin, Jingyi Yang, Joe Cummings, Jon Cavvill, Jon Shepard, Jonathan McPhie, Jonathan Torres, Josh Ginsburg, Juinjie Wang, Kai Wu, Kam Hou U, Karan Saxena, Karthik Prasad, Karltaky Chanddelwal, Katayoun Zand, Kathy Matosich, Kaushik Veeraraghavan, Kelly Michelena, Keqian Li, Kun Huang, Kunal Chawla, Kwalsh Lalhotia, Kyle Huang, Lailin Chen, Lakshya Garg, Lavender A, Leadro Silva, Lee Bell, Lei Zhang, Liangpeng Guo, Licheng Yu, Liron Moshkovitch, Luca Wehrstedt, Madian Khabsa, Manav Avaliani, Mnish Bhatt, Maria Tsimpoulkist, Martynas Markus, Matan Hasson, Matthew Lennie, Matthias Reso, Maxim Groshev, Maxim Naumov, Maya Lathi, Meghan Kenelly, Michael L 

Seltzer, Michal Valko, Michelle Restrepo, Mihir Patel, Mik Vyatskov, Mikayel Samvelyan, Mike Clark, Mike Macey, Mike Wang, Miquet Jubert Hermoso, Mo Matani, Mohammad Rastegari, Munish Bansal, Nandhini Santhanam, Natascha Parks, Natasha White, Navyata Baua, Nayan Singhal, Nick Egebo, Nicolas Usunier, Nikolay Pavlovich Laptev, Ning Dong, Ning Zhang, Norman Cheng, Oleg Chernoguz, Olivia Hart, Omkar Sapekar, Ozlem Kalinli, Parkin Kent, Partb Parekh, Paul Saab, Pavan Balaji, Pedro Rittner, Philip Bontrager, Pierre Roux, Piotr Dollar, Polina Zvyagina, Prashant Ratanchadani, Pritish Yuvraj, Qian Liang, Rachad Alao, Rachel Rodriguez, Rafi Ayub, Ragbotham Murthy, Raghu Nayani, Rahul Mitra, Raymond Li, Rebekkah Hogan, Robin Batley, Rocky Wang, Rohan Maheswar, Russ Howse, Rudt Rinott, Siayi Bosnohd, Smayak Datta, Sara Chugh, Sara Hunt, Sargum Dhillon, Sasha Sidorov, Satadur Pan, Saurabh Verma, Seiji Yamamoto, Sharadh Ramaswamy, Shaun Lindsay, Shawn Feng, Shenghao Lin, Shengzin Cindy Zhao, Shiva Shankar, Shuqiang Zhang, Sinong Wang, shena Agarwal, Soji Saijuyghee, Soumith Chintala, Stephanie Max, Stephen Chen, Steve Kehoe, Steve Statterfield, Sudarshan Govindaprasad, Sumit Gupta, Sungmin Cho, Sunny Virk, Suraj Subramanian, Sy Choudhury, Sydney Goldman, Tal Remez, Tamar Glaser, Tamara Best, Thilo Kohler, Thomas Robinson, Tianhe Li, Tianjun Zhang, Tim Matthews, Timothy Chou, Tezook Shaked, Varun Vontimitta, Victoria Ajayi, Victoria Montanez, Yijian Mohan, Vinay Satish Kumar, Vishal Mangla, Victor Albiero, Vlad Ionescu, Vlad Poenaru, Vlad Tiberiu Mihailescu, Vladimir Ivanov, Wei Li, Wenchen Wang, Wenwen Jiang, Wes Bouaziz, Will Constable, Xiaocheng Tang, Xiaofang Wang, Xiaojian Wu, Xiaolan Wang, Xide Xie, Xiilun Wu, Xinbo Gao, Yanjun Chen, Ye Hu, Fe Jia, Ye Qi, Yenda Li, Yilin Zhang, Ying Zhang, Yossi Adi, Youngjin Nam, Yu, Wang, Yuchen Hao, Trudu, Yuli Gan, Yuzi He, Zach Rait, Zachary DeVito, Zef Rosbnrick, Zhaoduo Wen, Zhenyu Yang, and Zhiwei Zhao. 2024. The llama 3 herd of models. _arXiv_.
* Gao et al. (2024) Bin Gao, Zhuomin He, Puru Sharma, Qingxuan Kang, Djordje Jevdjic, Junbo Deng, Xingkun Yang, Zhou Yu, and Pengfei Zuo. 2024. Cost-efficient large language model serving for multi-turn conversations with cachedattention. In _USENIX ATC_.
* Ge et al. (2024) Tao Ge, Jing Hu, Lei Wang, Xun Wang, Si-Qing Chen, and Furu Wei. 2024. In-context autoencoder for context compression in a large language model. In _ICLR_.
* Gu et al. (2024) Yukian Gu, Li Dong, Furu Wei, and Minlie Huang. 2024. Mini1lm: Knowledge distillation of large language models. In _ICLR_.
* Hinton et al. (2015) Geoffrey E. Hinton, Oriol Vinyals, and Jeffrey Dean. 2015. Distilling the knowledge in a neural network. _arXiv_.
* Jiang et al. (2023) Albert Q. Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guil laume Lample, Lucile Soulnier, Lello Renard Lavaud, Marie-Anne Lachaux, Pierre Stock, Teven Le Scaco, Thibaut Lavril, Thomas Wang, Timothee Lacroix, and William El Sayed. 2023a. Mistral 7b. _arXiv_.
* Jiang et al. (2024) Huiqiang Jiang, Qianhui Wu, , Xufang Luo, Dongsheng Li, Chin-Yew Lin, Yuqing Yang, and Lili Qiu. 2024. LongLLMLingua: Accelerating and enhancing LLMs in long context scenarios via prompt compression. In _ACL_.
* Jiang et al. (2023) Huiqiang Jiang, Qianhui Wu, Chin-Yew Lin, Yuqing Yang, and Lili Qiu. 2023b. LLVMEUR: Compressing prompts for accelerated inference of large language models. In _EMNLP_.
* Kim et al. (2023)Seungone Kim, Se June Joo, Doyoung Kim, Joel Jang, Seonghyeon Ye, Jamin Shin, and Minjoon Seo. 2023. The cot collection: Improving zero-shot and few-shot learning of language models via chain-of-thought fine-tuning. In _EMNLP_.
* Kim and Rush (2016) Yoon Kim and Alexander M. Rush. 2016. Sequence-level knowledge distillation. In _EMNLP_.
* Ko et al. (2024) Jongwoo Ko, Sungnyun Kim, Tianyi Chen, and SeYoung Yun. 2024. Distillm: Towards streamlined distillation for large language models. In _ICML_.
* Kwon et al. (2023) Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph E. Gonzalez, Hao Zhang, and Ion Stoica. 2023. Efficient memory management for large language model serving with pagedattention. In _SIGOPS_.
* Lewis et al. (2021) Patrick Lewis, Yuxiang Wu, Linqing Liu, Pasquale Minervini, Heinrich Kuttler, Aleksandra Pittus, Pontus Steentorp, and Sebastian Riedel. 2021. PAQ: 65 million probably-asked questions and what you can do with them. _TACL_.
* Li et al. (2024) Yichuan Li, Xiyao Ma, Sixing Lu, Kyumin Lee, Xiaohu Liu, and Cheleni Guo. 2024. Mend: Meta demonstration distillation for efficient and effective in-context learning. In _ICLR_.
* Li et al. (2023) Yucheng Li, Bo Dong, Frank Guerin, and Chenghua Lin. 2023. Compressing context to enhance inference efficiency of large language models. In _EMNLP_.
* Lin et al. (2020)Alexander Lin, Jeremy Wohlwend, Howard Chen, and Tao Lei. 2020. Autoregressive knowledge distillation through imitation learning. In _EMNLP_.
* Liu et al. (2023)Xiao Liu, Hao Yu, Hanchen Zhang, Yifan Xu, Xuanyu Lei, Hanyu Lai, Yu Gu, Hangliang Ding, Kaiwen Men, Kejuan Yang, Shudan Zhang, Xiang Deng, Aohan Zeng, Zhengxiao Du, Chenhui Zhang, Sheng Shen, Tianjun Zhang, Yu Su, Huan Sun, Minlie Huang, Yuxiao Dong, and Jie Tang. 2023. Agentbench: Evaluating llms as agents. _arXiv_.
* Mu et al. (2023) Jesse Mu, Xiang Lisa Li, and Noah Goodman. 2023. Learning to compress prompts with gist tokens. In _NeurIPS_ 

Hanseok Oh, Haebin Shin, Miyoung Ko, Hyunji Lee, and Minjoon Seo. 2024. Ktrl+f: Knowledge-augmented in-document search. In _NAACL_.
* Pan et al. (2024) Zhuoshi Pan, Qianhui Wu, Huiqiang Jiang, Menglin Xia, Xufang Luo, Jue Zhang, Qingwei Lin, Victor Ruhle, Yuqing Yang, Chin-Yew Lin, H Vicky Zhao, Lili Qiu, and Dongmei Zhang. 2024. LLMLingua-2: Data distillation for efficient and faithful task-agnostic prompt compression. In _Findings of ACL_.
* Pope et al. (2023) Reiner Pope, Sholto Douglas, Aakanksha Chowdherry, Jacob Devlin, James Bradbury, Anselm Levskaya, Jonathan Heek, Keran Xiao, Shirani Agrawal, and Jeff Dean. 2023. Efficiently scaling transformer inference. In _ MLSys_.
* Snell et al. (2022) Charlie Snell, Dan Klein, and Ruiqi Zhong. 2022. Learning by distilling context. _arXiv_.
* Song et al. (2020) Kaitao Song, Hao Sun, Xu Tan, Tao Qin, Jianfeng Lu, Hongzhi Liu, and Tie-Yan Liu. 2020. Lightpaff: A two-stage distillation framework for pre-training and fine-tuning. _arXiv_.
* Tourron et al. (2023a) Hugo Tourron, Thibaut Lavril, Gautier Lacard, Xavier Martinet, Marie-Anne Lachaux, Timothee Lacroix, Baptiste Roziere, Naman Goyal, Eric Hambro, Faisal Azhar, Aurelien Rodriguez, Armand Joulin, Edouard Grave, and Guillaume Lampe. 2023a. Llama: Open and efficient foundation language models. _arXiv_.
* Touvron et al. (2023b) Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, Dan Bikel, Lukas Blecher, Cristian Canton Ferrer, Moya Chen, Guillem Cucurull, David Esibou, Jude Fernandes, Jeremy Fu, Wenyu Fu, Brian Fuller, Cynthia Gao, Vedanuj Goswami, Naman Goyal, Anthony Hartshorn, Saghar Hosseini, Rui Hou, Hakan Inan, Marcin Kardas, Viktor Kerkez, Madian Khabsa, Isabel Kloundann, Artem Korenev, Punit Singh Koura, Marie-Anne Lachaux, Thibaut Lavril, Jenya Lee, Diana Liskovich, Yinghai Lu, Yuning Mao, Xavier Martinet, Todor Mihaloylov, Pushkar Mishra, Igor Molbyog, Yixin Nie, Andrew Poulton, Jeremy Reizenstein, Rashi Runga, Kalyan Saladi, Alan Schelten, Ruan Silva, Eric Michael Smith, Ranjan Subramanian, Xiaojing Ellen Tan, Binh Tang, Ross Taylor, Adina Williams, Jian Xiang Kuan, Puxin Xu, Zheng Yan, Iiiayan Zarov, Yuchen Zhang, Angela Fan, Melanie Kambadur, Sharan Narang, Aurelien Rodriguez, Robert Stojnic, Sergey Edunov, and Thomas Scialom. 2023b. Llama 2: Open foundation and fine-tuned chat models.
* Wei et al. (2022) Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed Chi, Quoc Le, and Denny Zhou. 2022. Chain-of-thought prompting elicits reasoning in large language models. In _NeurIPS_.
* Yao et al. (2022) Shunya Yao, Howard Chen, John Yang, and Karthik Narasimhan. 2022. Webshop: Towards scalable real-world web interaction with grounded language agents. In _NeurIPS_.
* Yao et al. (2023) Shunya Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. 2023. React: Synergizing reasoning and acting in language models. In _ICLR_.
* Zeng et al. (2023)Aohan Zeng, Mingdao Liu, Rui Lu, Bowen Wang, Xiao Liu, Yukiao Dong, and Jie Tang. 2023. Agentuning: Enabling generalized agent abilities for l1ms. _arXiv_.
* Zheng et al. (2023)Linaminin Zheng, Liangsheng Yin, Zhiqiang Xie, Jeff Huang, Chuyue Sun, Cody Hao Yu, Shiyi Cao, Christos Kozyrakis, Ion Stoica, Joseph E. Gonzalez, Clark Barrett, and Ying Sheng. 2023. Efficiently programming large language models using sglang. _arXiv_.
* Zou et al. (2024) Jiaru Zou, Mengyu Zhou, Tao Li, Shi Han, and Dongmei Zhang. 2024. Promptntern: Saving inference costs by internalizing recurrent prompt during large language model fine-tuning. _arXiv_.

 

Details of Components Generation

### Implementation details

Pseudo User Input.We utilize LLaMA-3-70B-Instruct Dubey et al. (2024) as a psuedo user input generator. Each samples are generated by nucleus sampling with threshold 0.9 and temperature 1.0. To ensure the quality of pseudo user input, we randomly select 5 demonstrations from the validation samples of each task. System prompt and user prompt for Pseudo Input generation are as below.

```
Thedocumentbelowarethecodeofconductforaspecificagentperforminggivenproblem.##ThisisthegivenDocument:(context)
```

Basedonthegivendocument,generateseveralproblemsfors{taskname}task.Generateonlyproblemswithnumberedlist.This{num_gen_once}generatedProblems.[N-shotdemonstrations]

Pseudo Conversational Outputs.We provide an additional system prompt for the environment persona model in the Self Role-Playing Conversation (Figure 2). In the N-shot examples within the given prompt \(p\), we swap the roles of the agent and environment and collected the teacher's conversational behavior patterns with a temperature of 0. In this paper, we use LLaMA-3-8B-Instruct Dubey et al. (2024) as the target model \(\theta\).

Reason.As same as the pseudo input generator, we utilize LLaMA-3-70B-Instruct Dubey et al. (2024) with nucleus sampling threshold 0.9 and temperature 0.7. We prompt the model to generate a reason of approximately 5 sentences to mitigate excessive hallucination. This is based on the statistics from the CoT Collection Kim et al. (2023), which includes 1.84M rationales across 1,060 tasks, with an average length of about 3 sentences. System prompt and user prompt for reason generation are as below.

```
Youareacommentatorcomparightenswersoftwomodelstoquestion.Iwillgiveyoufourcomponents:context,input,badoutput.good output.BadOutputisresponseprovidedwithoutanycontext,onlybasedonthemipuriscu.GoodOutputisresponseprovidedwithbothcontextandinput.Explainingreasonfromchangeinresponse,referringtothecontext,insaboutfivesentences.##Context:(context) """Input:(input)
##BadOuput:(student_output)
##GoodOutpu:(tacher_output)
##Reson:
```

###ComponentsExamples

Figure 8, 9, 10 present examples from the conversational dataset, including a pseudo user input and pseudo conversational outputs generated by Self Role-Playing Conversation. In Figure 12, 13, 14, the examples of reason describe the differences in format and actions required from the prompt.

### Quality of Self Role-Playing Conversation

As shown in Table 4, we perform quantitative evaluations using statistics and three distinct error types.

How similar is it to the gold turn?The primary objective of the pseudo-corservational outputs is to emulate multi-turn dialogue capabilities based on the content of the prompt. To this end, we consider the shot examples provided in the prompt as gold dialogue, and compare the similarity in the number of turns. Across all three tasks, the generated multi-turn dialogues successfully mimic the structure of the gold turns. Even in the Web Browsing task, which consists of N-shot single-turn examples, the number of turns remains comparable to the gold examples.

How many samples exhibit abnormally long conversations?We evaluate whether the conversation terminates due to reaching the maximum turn limit. We set the maximum turn limit for each task (10 turns for OS, 2 turns for Web Browsing, and 5 turns for Web Shopping), based on the shot examples and expected number of turns to find the answer following Liu et al. (2023). Among the 1,000 generated dialogue, only fewer than 2% samples exceed the maximum limit. We truncate this samples under the maximum limit.

How many samples are abnormally terminated?During the Self Role-Playing Conversation, we instruct the environment persona model to generate a special termination token when the agent generates the final output for the given user input. We consider samples to be abnormally terminated if there are no termination tokens. In Table 4, all 

samples successfully stop the conversation with environment persona model's termination token.

_How many samples are missing the final action?_ We evaluate whether the final action, which serves as the scoring target, is correctly generated. In this evaluation setup, we restrict the evaluated action space that can be inferred solely from the prompt. For instance, we evaluate whether OS tasks ended with 'finish' or 'answer' actions, Web Browsing tasks with 'click', 'type', 'select', 'None', and Web Shopping tasks with 'search' or 'click' actions. Note that the specific values for click or type actions are excluded on this quantitative evaluation. This information cannot be inferred from the prompt, as it pertains to the agent's capability to respond based on the user input. In the OS task, approximately 5% of the samples omit the final action. As shown in Figure 11, while most samples in this group contain the correct bas commands, they lack the 'answer' or 'finish' keyword required for proper termination. Similarly, for the Web Browsing task, many samples in this group do not adhere to the predefined action space, generating undefined actions (e.g. COMPARE, ENTER USERNAME AND PASSWORD in Figure 11) in response to user input.

## Appendix B Details for Evaluation Dataset

We follow the prompt configurations defined in AgentBench7(Liu et al., 2023) for three agent-based tasks: OS Interaction Liu et al. (2023), Web Browsing Deng et al. (2023), and Web Shopping Yao et al. (2022). Each prompt comprises a task description, a detailed explanation of the agent's expected actions, the formatting guidelines that the agent should adhere to, and a set of N-shot examples illustrating interactions between the agent and the environment. As suggested by Liu et al. (2023), all agent outputs are formatted using the Chain-of-Thought (CoT) style Wei et al. (2022), which has become the standard approach for this type of evaluation in conjunction with action-based responses Yao et al. (2023). We utilize these multi-turn prompts as the chat history context for our agent interactions.

Footnote 7: We select three tasks because several tasks in AgentBench Liu et al. (2023) either lack sufficiently long prompts (DB task), exhibit significantly low performance on open-source LLMs Game-grounded tasks, or face reproducibility issues due to Freebase server limitations (KG task).

OS Interaction Liu et al. (2023).The prompt for the OS Interaction task includes a brief task description and the formulation of an interaction trajectory using a 1-shot example. An example of this prompt is illustrated in Figure 5. Agents are evaluated based on Success Rate (SR), determined by comparing the final output against the expected solution.

Web Browsing Deng et al. (2023).The prompt for the Web Browsing task contains 3-shot CoT-style examples. An illustration of this prompt can be found in Figure 6. In alignment with Liu et al. (2023), we report Step Success Rate, which indicates the independent accuracy of each action step taken by the agent.

Web Shopping Yao et al. (2022).The prompt for the Web Shopping task consists of the task description along with a 1-shot CoT-style example. During each turn, the agent interacts with the HTML text observation by exploring and making decisions accordingly. An example of this configuration is depicted in Figure 7. Consistent with Liu et al. (2023), we evaluate the agent's performance using a reward metric, which quantifies the similarity between the expected attributes of a product and the attributes of the purchased item, mapping this similarity to a value between 0 and 1.

\[{\rm Reward}=\frac{|G_{at}\cap Y_{at}|+|G_{op}\cap Y_{op}|+\mathbb{I}[y_{pr} \leq g_{pr}]}{|G_{at}|+|G_{op}|+1}\cdot r_{type}\]

\[r_{type}=\left\{\begin{array}{ll}0,&~{}\text{if TM}=0,\\ 0.1,&~{}\text{if TM}<0.1,\\ 0.5,&~{}\text{if TM}>0.2~{}\text{and}~{}c=1,\\ 1,&~{}\text{otherwise}\end{array}\right.\]

\(G\) and \(Y\) denote the goal and the chosen product, respectively, while \(at\) and \(op\) represent attributes

\begin{table}
\begin{tabular}{l c c c} \hline \hline  & OS & WB & WS \\ \hline \hline Gold Turns (in Prompt) & 3 & 1 & 4 \\ Avg. Turns & 2.71 & 1.03 & 3.21 \\ \hline Max Turns Limit & 1.8\% & 1.6\% & 1.7\% \\ Abnormal Termination & 0\% & 0\% & 0\% \\ Abnormal Final Action & 5.2\% & 3.9\% & 0\% \\ \hline \hline \end{tabular}
\end{table}
Table 4: Overall Qualities and Statistics about Pseudo Conversational Outputs. Note that WB denotes Web Browsing task Deng et al. (2023), and WS denotes Web Shopping task Yao et al. (2022).

 

and options. Note that \(r_{type}\) compares the product category sequences (\(c=1\) if matched) listed on the Amazon website. Additionally, TM represents a text match comparison between titles, focusing on the proun, noun, and propn tags. For further details on these metrics, please refer to Yao et al. (2022).

## Appendix C Details for Baselines

### Criteria of selecting the Base Model

We assume a multi-turn conversational application scenario, where an instruct-tuned model is deployed with a fixed, predetermined prompt. Due to this setup, we select LLaMA-3-8B-Instruct Dubey et al. (2024) model as the base model instead of using LLaMA-3-8B.

### Criteria of selecting Upper Bound

In case of Choi et al. (2023), they finetune a relatively small model (approximately 200M parameters) on gold training dataset, due to concerns that the language model may not adequately perform on unseen tasks, thereby limiting its effectiveness as an upper-bound model. In contrast, we examine a scenario where only a prompt is provided, without a corresponding training dataset for prompt internalization. Moreover, we specifically focus on a billion-scale language model applied to the AgentBench Liu et al. (2023), which lacks a dedicated gold training dataset.

To establish a more robust upper bound under these conditions, we conduct preliminary experiments comparing the performance between a model fine-tuned on a pseudo training dataset and the original model. The model that demonstrates superior performance is then selected as the upper bound. As shown in Table 5, the original model outperforms the fine-tuned model when prompts are provided. This suggests that the limited size of the pseudo training dataset (only 1,000 examples) may be insufficient to optimize the LLM for agent-specific tasks, or that there exists a distributional mismatch between the pseudo training dataset and the actual gold test set. Since optimizing agent performance through fine-tuning is beyond the scope of this paper, we adopt the original model, which demonstrated higher performance, as the upper bound, following the approach of Snell et al. (2022); Liu et al. (2023).

### Criteria of selecting Embedding-based Compression Baseline

Recent embedding-based prompt compression approaches Mu et al. (2023); Chevalier et al. (2023); Ge et al. (2024) have been developed as a general language models that first cache the given prompt first and then infer user input using compressed prompt embeddings. Gisting Mu et al. (2023) and AutoCompressor is based on LLaMA-7B Touvron et al. (2023) and LLaMA-2-7B Touvron et al. (2023), respectively. These models perform significantly poorly on AgentBench Liu et al. (2023) with scores close to zero. Therefore, we select the baseline ICAE Ge et al. (2024), which is based on Mistral-7B-Instruct Jiang et al. (2023).

### Implementation Details

We utilize a single NVIDIA A6000 GPU to train our method and baselines. For Generative Prompt Internalization, we apply QLoRA Dettmers et al. (2023) with rank \(r=16\), 32 batch size, 1e-4 learning rate, 10 epochs. The joint loss hyperparameter \(\lambda=0.7\) (as defined in Equation 4) is selected through hyperparameter tuning over [0.3, 0.5, 0.7]. For full finetining baselines, we use 4 NVIDIA A100 80G GPUs with 64 batch size, 1e-5 learning rate, 10 epochs, following the setting from Gu et al. (2024). Prompt Prepending, a variation of our approach, uses the same hyperparameters as GenPI. For ICAE Ge et al. (2024), we follow the original settings with a LoRA rank of \(r=512\) and 128 compression tokens.

## Appendix D Comparison with Chain-of-Thought

To discuss of the role of reason \(r\) in PG loss, we compare the differences between chain-of-thought (CoT) rationales Wei et al. (2022) and reason \(r\), focusing on their mechanisms, and impact on inference.

Rationale vs. Behavioral Change.CoT Wei et al. (2022) generates explicit stepwise rationales

\begin{table}
\begin{tabular}{l r r r} \hline \hline  & OS & WB & WS \\ \hline \hline original & 17.36 & 17 & 54.16 \\ fine-tuned & 16.6 & 17 & 51.12 \\ \hline \end{tabular}
\end{table}
Table 5: Preliminary experiments to select the Upper Bound model. All models are based on LLaMA-3-8B-Instruct Dubey et al. (2024) and are evaluated on their performance when provided with the agent prompt.

 

to explain the final answer for solving a task-intensive problem (e.g., an explicit reasoning process for solving a math problem). In contrast, reason \(r\) aims to compare the differences between the outputs of the teacher and student models, and identify the key changes in the model's behavior. That is, while indirectly describing the prompt \(p\), PG loss is designed to induce behavioral changes in the model rather than explicitly providing explanations for the correct answer.

Inference Efficiency.Since GenPI does not generate rationales at inference time, it differs from CoT (Wei et al., 2022) in inference efficiency. By internalizing the prompt, PG loss ensures higher efficiency (in Section 8) while maintaining the original performance.

 

* <USER>:
* You are an assistant that will act like a person, I'will play the role of linux(ubunt) operating system. Your goal is to implement the operations required by me or answer to the question proposed by me. For each of your turn, you should first think what you should do, and then take exact one of the three actions: "bash", "finish" or 'answer'.
* 1. If youthlink yous should execute some bash code, take bash action, and yous should print like this: Think: put your thought here. Act: bash
* ...bash
* @ put your bash code here
* 2. If youthlink yous have finished the task, take finish section, and yous should print like this: Think: put your thoughts there. Act: finish
* 3. If youthlink yous have got the answer to the question, take answer section, and yous should print like this: Think: put your thoughts there. Act: answer(Yours answer to the question should be put in this pair of parentheses) If the output is too long, I will truncate it. The truncated output is not complete. Yow have to deal with the truncating problem by yourself. Attention, your bash code should not contain any input operation. Once again, yous should take only exact one of the three actions in each turn. Now, my problems is: tell me how many files aren the directory ''etc"? <AGENT>: Think: To count the files in /etc, I need to print all the files in it. Act: bash
* '''bash ls, / etc <<USER>: The output of the OS: cpi ron.hourly fuse.cantiproute2 llvm networkd-dispatcher protocols selinux tmpfiles.d[truncated because the output is too long] <AGENT>: Think: The output has been truncated because it is too long, so I needed to count files by script directly. Act: bash
* '''bash !s_1 /etc | we -1 ..
* <USER>: The output of the OS: 220 <AGENT>: Think: Now I get the answer, it is 220. Act: answer(220) <ASER>: Now, I will start a new problem in a new OS. My problem is: {user_input}

Figure 5: Agent Prompt for OS Interaction. Following the task setup from AgentBench (Liu et al., 2023), we describe all content, including the system prompt and demonstrations, as a multi-turn strategy using ¡USER¿ and ¡AGENT¿¿



*<USER>:
*<html><div><div><div><ntextbomepage/><buttonid=0bookasreversion.toggleeppen><span>Bookafreservation</ipan></button><buttonbookabasteverstation.logstepeppen><bitbucketing<</ultra><di><di><objectidentif><optionrestervationstrue>Dine in</opinion><option><option><option><option><option<views>>>Delivery<options><option><option><question><views>>Wineries>Winerings<option><optional>>Eversighting</option><selected><div>id=2><>Celebratingandsupportingleading supremumshakingup the industry.</p><span>Explorenow</span><dir></dir></dir></dir></dir></dir></html> """> Basedonthe HTMLwcbpageabove.trytocompleteoftlowingtask; Task:Checkforpickpickertawantalisabledin Boston,NYonMarch18.5mwithjustoneset Previousactions; NoneWhatshouldbehennextaction?Pleaseselectfromthefollowingchoices(Ifthecorrectsctioniscontinthepageabove,pleaseelectA.Noneoftheabove'): A.Noneoftoftheloveablexentown.toggleeppen><span>Booka C.<selectid=idtype<option><operation><wideppen><peduanghedigwormshakingup D.<divididx><p>Celebatingandsupportingleadingwomanshakingup

<AGNNF>:

Though:Inedtoesbecti pickupresturantfirst. Answer:C. ActionSELECTValue:Pickup

<USER>:
*<html><div><mainim><sectionsubmapst><div><ultslist><litabheadinglevel3searchmd><//i><tlid0tbheadinglevel3searchmd><//i><tlid0tbheadinglevel3searchmd>>|eventmd>>|eventmd>>|eventmd>|eventmd>|eventmd>|eventmd>|eventmd>|eventmd>|eventmd>|eventmd>|eventmd>|eventmd>|eventmd>|eventmd>|eventmd>|eventmd>|eventmd>|eventmd>|eventmd>|eventmd>|eventmd>|eventmd>|eventmd>|eventmd>|eventmd>|eventmd>|eventmd>|eventmd>|eventmd>Eventmd>|eventmd>|eventmd>|eventmd>|eventmd>|eventmd>|eventmd>|eventmd>|eventmd>|eventmd>|eventmd>|eventmd>|eventmd>|eventmd>|eventmd>|eventmd>|eventmd>|eventmd>eventmd>|eventmd>|eventmd>eventmd>|eventmd>eventmd>|eventmd>eventmd>|eventmd>eventmd>|eventmd>eventmd>|eventmd>eventmd>|eventmd>eventmd>|eventmd>eventmd>|eventmd>eventmd>|eventmd>eventmd>eventmd>|eventmd>eventmd>eventmd>|eventmd>eventmd>eventmd>|eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>eventmd>md>eventmd>eventmd>eventmd>eventmd>md>eventmd>eventmd>eventmd>md>eventmd>eventmd>eventmd>md>eventmd>eventmd>eventmd>md>eventmd>eventmd>md>eventmd>eventmd>md>eventmd>eventmd>md>eventmd>eventmd>md>eventmd>eventmd>md>eventmd>eventmd>md>eventmd>eventmd>md>eventmd>eventmd>md>eventmd>eventmd>md>eventmd>eventmd>md>eventmd>eventmd>md>eventmd>md>eventmd>eventmd>md>eventmd>md>eventmd>eventmd>md>eventmd>md>eventmd>md>eventmd>md>eventmd>md>eventmd>md>eventmd>md>eventmd>md>eventmd>md>eventmd>md>eventmd>md>eventmd>md>eventmd>md>eventmd>md>eventmd>md>eventmd>md>eventmd>md>eventmd>md>eventmd>md>eventmd>md>eventmd>md>eventmd>md>eventmd>md>eventmd>md>eventmd>md>eventmd>md>eventmd>md>eventmd>md>eventmd>md>eventmd>md>eventmd>md>eventmd>md>eventmd>md>eventmd>md>eventmd>md>eventmd>md>eventmd>md>eventmd>md>eventmd>md>eventmd>md>eventmd>md>eventmd>md>eventmd>md>eventmd>md>eventmd>md>eventmd>md>eventmd>md>eventmd>md>eventmd>md>eventmd>md>eventmd>mdeventmd>mdeventmd>mdeventmd>mdeventmd>mdeventmd>eventmd>mdeventmd>eventmd>mdeventmd>eventmd>mdeventmd>eventmd>mdeventmd>eventmd>mdeventmdeventmd> 

<MSER>:
* [SEP] !need a long lasting 6.76 f1 or bottle of l'eam d'issey, and price lower than 100.00 dollars [SEP] Back to Search [SEP] !Prev [SEP] size [SEP] 2.5 f1 or [SEP] 6.76 f1 or (pack of l) [SEP] L'eam D'issey By Issey Miyake for MenEur De Teilite Spray, 6.7 F1 Or Bottic [SEP] Price; 564.98 [SEP] Rating; N.A. [SEP] Description [SEP] Features [SEP] Reviews [SEP] Bay Now

[MISSING_PAGE_POST]

* [SEP] Day
* [SEP]
 

* [123] M. C. B. K., and J. A. L. Szymanski, "Theoretical evaluation of the OSS method for the OSS method in a general pseudo user", _Journal of Computational Chemistry_**46**(5):709-718, 2000.

[MISSING_PAGE_POST]

 

<USER>: ...
*html><div><div><divid=0><h2>HotelSearch</h2><form><inputid=1type=textplaceholder="City,Airport,orHotel"/><buttonid=2>Search</button></form></div></div></html> ...

Based on the HTML webpage above, try to complete the following task: Task: Search for hotels in New York City Previous actions: [link] Hotels->CLICK Whatshould be the next action? Pleaseselect from thefollowing choices (If the correctaction is notinthe pageabove,pleasselectA.'Noneoftheabove'): A. Noneoftheabove B.<inputid=1type=textplaceholder="City,Airport,orHotel"/> C.<buttonid=2>Search</button> <AGENT>: Thoughst: The search formisalready displayed,sothenextactionistonenterhesearchcriteria. Answer: B Action: TYPE Value: New York City

<USER>: <html><div><selectedid=0><option>1guest</option><option>2guests</option><option>3guests </option></select><buttonid=1>SearchHotels</button></div></html> ...

Based on the HTML webpage above,trytocompletethefollowingtask: Task: SearchforhotelisInLes Vegasfor2guests Previousactions: [link] Hotels->CLICK What should be thenextaction?Pleaseselectfromthefollowingchoices(Ifthecorrectactionisnotinthepageabove,pleaseselectA.'Noneofttheabove'): A.Noneoftheabove B.<selectid=0> C.<buttonid=1>SearchHotels</button> <AGENT>: Thought:Ineedtoselectthemumberofguests. Answer: B. Action:SELECT Value:2guests

Figure 9: Examples of a pseudo dataset for the Web Browsing task. Following the prompt, pseudo user input is generated as a multiple-choice problem based on the webpage. This example involves a hotel searching scenario, where the agent selects an answer and predicts its actions along with the corresponding values.

 

* [ENDFIGURE]

Figure 10: Example of Pseudo Conversational Dataset on Web Shopping task. Upon receiving a pseudo user input, the environment persona model generates a webpage that includes clickable elements, simulating changes in the webpage based on the agent’s click actions.

 

* [10] M. C. R. Jones, J. S. H. Lee, and D. A. Lowe, "A new approach to the classification of a class of objects in a class of object recognition systems," _IEEE Transactions on Pattern Analysis and Machine Intelligence_**53**(2):47-68, 2009.
* [11] L. K. Johnson, "The automated classification of objects using convolutional neural networks," _Journal of Computer Vision and Pattern Recognition_**40**(1):1-10, 1994.
* [12] P. B. Anderson, "An introduction to machine learning for object detection," _IEEE transactions on pattern analysis and machine intelligence_**53**(1):1-10, 1994.
* [13] G. E. Hinton, "Learning from tiny images by reducing noise," _arXiv preprint arXiv:1407.6090_ (2014).
* [14] T. Jaynes, "A fast algorithm for detecting objects with large numbers of labels," _IEEE transactions on pattern analysis and machine intelligence_**53**(1):1-10, 1994.

[MISSING_PAGE_POST]

 



## Input: Showmethefirst5linesofthe/etc/profilefile. ------------------------------------------------
###BadOutput: l'dbehappytohelpyowwiththat.However,l'malargelanguagemodel,Idon'thavedirectaccesstoyoursystemorfiles.Butlcamprovideyowoithsomegeneralinformationaboutthe/etc/profilefile. The/etc/profilefileisasystem-wideconfigurationfileinLinuxandotherUnix-likeoperatingsystems. Itiseexecutedbytheshellwhenauserlogsin,anditsetsvariousenvironmentvariablesandshell settings. Ifyouwanttonesethefirst5linesofthe/etc/profilefileonyoursstem,youcannseacommandlikethis: `head-m5/etc/profile' Thiswilldisplaythefirst5linesofthefile.However,pleasenotethatethecontentsofthe/etc/profile filecanvarydependingontheLinuxdistributionandversionyoungareusing. Ifyou'reusingacloud-basedshelloravirtualmachine,youmaynothaveaccesstothe/etc/profilefile. Inthatascape,youcantryotfindasimilarfileorconfigurationfilethatsetsenvironmentvariablesandshellsettingsforyourspecificsystem. ------------------------------------------------
###GoodOutput: Think:Toshowthefirst5linesofthe/etc/profilefile,Ineedtocusetheheadcommand. Act:bash ""bash head-m5/etc/profile """ Reason: Thereasonforthdifferenceinseponsisthatthebadoutputlackscontext,specificallytheunderstandingthatthemodeliscatingsasmassistantwithinainx(Ubuntu)operatingsystem.Withoutthiscontexlt,thebadoutputprovidesaggenericandvagueresponse,explainingtheconceptofthe/etc/profilefileandsuggestingaxcommandtowiveitscontens,butnotactuallyexceptingthecommand.Incontrast,thegoodoutputtakesintoaccountthecontextoftheworsation,recognizingthatitisansistantwithintheLinuxsystem,andthereforecansecutethecommanddirectly.Thisallowsthegoodoutputtopovideamorespecificandrelevantresponse,usingthe'headcommandtoldisplaythefirst5linesofthe/etc/profilefile. ```

Figure 12: Examples of reason in OS Interaction task. Refer to Agent Prompt in Figure 5



###Input:

 <html><div><h2id=0>FindYourDreamHome</h2><p>Getstartedbyselectingalocationbelow</p><ul><li><aid=1>NewYork</a></li><li><aiid=2>California</a></li><li><<aiid=3>Florida</a></li></ui></di></hlm>

BasedontheHMLwebpageabove,trytocompletethefollowingtask: Task:FindhomessforsaleinCalifornia. Previousactions:[link]RealEstate->CLICK Whatshouldothenextaction?Pleaseselectfromthefollowingchoices(Ifthecorrectactionisnotinthepageabove,pleaseselectA.'Noneofttheabove'): A.Noneoftheabove B.<h2id=0>FindYourDreamHome</h2> C.<aid=1>NewYork</a> D.<aid=2>California</a>

####BadOutput: ThecorrectanswerisD.<aid=2>California</a> ThetaskisttofindhomessforsaleinCalifornia,andtheoptiontoselectCaliforniaisprovidedinthedHMLcodas<aid=2>California</a>.

####GoodOutput: ThoughethatestsofthandnessforsaleinCalifornia,soIshouldclickonthesCalifornialink. Answer:D. Action:CLICK

####Reason: Thereasonfortchangeinseponsefromethebadoutputtothegoodoutputisthatthegoodoutputtakesintoaccountthecontextofthestaskandthepreviousactions.Thebadoutputsimplyselectstheopionthatmatchesthetaskdescription,withoutconsideringthecontext.Inthiscase,thegoodoutputcorrectlyinfersthattheuserneedstolickontheCalifornialinktofindhomessforsaleinCalifornia,asindicatedbythepreviousactionofclickingonthe"RealEstate"link.Thegoodoutputprovidesamorethoughtfandomtext-awareresponse,whereasthebadoutputismorespecriticalandlacksunderstandingofthetask'scontext.

Figure 13: Examples of reason in Web Browsing task. Refer to Agent Prompt in Figure 6.





##Input: Observation: WebShop [SEP] Instruction: [SEP] i'm looking for a 40 inch 4k smart tv with built-in wi-fi, and price lower than 500.00 dollars [SEP] Search Available Actions: {'has_search_bar'; True, 'clickables': ['search']}

###BadOutput: I'd be happy to help you with that! I've searched for 40-inch 4Ksmart TVs with built-in Wi-Fi and a price lowerthan S500. Here are some options:
1. TCL 40S325 40" 4K UHD Smart TV - S349.99 * This TV has a 40-inch 4K UHD display, built-in Wi-Fi, and Roku TV built-in forseamless streaming.
2. VIZIO D4D-ID 40" 4K UHD Smart TV - S399.99 * This TV features a 40-inch 4K UHD display, built-in Wi-Fi, and VIZIO's SmartCast platform for easy streaming.
3. Hisense H40B7 40" 4K UHD Smart TV - S449.99 * This TV has a 40-inch 4K UHD display, built-in Wi-Fi, and Hisense's Vidaas smart TV platform for streaming and more. All of theseoptions are under S500 and meet your requirements. Would you like meto providemore information about any of theseoptions or would you like to explore other options?

####Good Output: Thought: I think I should use the search barto look for the product I need. Action: search[40 inch 4k smart t vwi-fi < 500.00]

####Reason: The main reason for the difference in response is that the bad output does not take into account the context of the web shopping scenario, whereas the good output does. The bad output appears to be a general response to the input, providing a list of options that meet the specified criteria, but it doesn't acknowledge the fact that the user is currently on a webshopping platform and needs to interact with it. In contrast, the good output recognizes the context and responds accordingly. It thinks about using the search barto look for the product, which is a relevant action in the webshoping scenario. The action *search[40 inch 4k smart tv wi-fi < 500.00]'is a specific and context-appropriate response that takes into account the available actions provided, which includes the search bar.

Figure 14: Examples of reason in Web Shopping task. Refer to Agent Prompt in Figure 7.