CoGenesis: A Framework Collaborating Large and Small Language Models for Secure Context-Aware Instruction Following (2024)

Abstract: With the advancement of language models (LMs), their exposure to private data is increasingly inevitable, and their deployment (especially for smaller ones) on personal devices, such as PCs and smartphones, has become a prevailing trend. In contexts laden with user information, enabling models to both safeguard user privacy and execute commands efficiently emerges as an essential research imperative. In this paper, we propose CoGenesis, a collaborative generation framework integrating large (hosted on cloud infrastructure) and small models (deployed on local devices) to address privacy concerns logically. Initially, we design a pipeline to create personalized writing instruction datasets enriched with extensive context details as the testbed of this research issue. Subsequently, we introduce two variants of CoGenesis based on sketch and logits respectively. Our experimental findings, based on our synthesized dataset and two additional open-source datasets, indicate that: 1) Large-scale models perform well when provided with user context but struggle in the absence of such context. 2) While specialized smaller models fine-tuned on the synthetic dataset show promise, they still lag behind their larger counterparts. 3) Our CoGenesis framework, utilizing mixed-scale models, showcases competitive performance, providing a feasible solution to privacy issues.

heading: 1 Introduction
text: Large Language Models (LLMs)1 have demonstrated significant potential in advancing artificial intelligence, exhibiting exceptional ability in instruction following and achieving superior performance in various tasks such as writing, coding,
* Corresponding author 1This paper defines large LMs (LLMs) as both closed and open-source models, designed for universal application and advanced performance, and intended for cloud deployment. Conversely, small LMs (SLMs) refer to models tailored for specific tasks and deployed on local devices.
and other text-based activities (Achiam et al., 2023; Bubeck et al., 2023; Touvron et al., 2023a,b). LLMbased AI assistants play a crucial role in executing instructions, aiding in writing tasks, and accelerating work processes, thereby fostering content innovation (Zhang et al., 2023a; Haase and Hanel, 2023). LLMs often require extensive context information for generating more personalized and effective content, owing to their in-context learning abilities (Brown et al., 2020). Retrieval Augmented Generation (RAG) (Gao et al., 2023) has proven beneficial in incorporating additional context to enhance the informativeness and personalization of LLM outputs. However, current instructionfollowing tasks often fail to consider rich user context in their design (Wang et al., 2023c; Xu et al., 2023b; Ding et al., 2023). Incorporating personal
4295
experiences and activity logs could significantly augment the effectiveness of these instructions, called context-aware instruction following.
Despite their advancements, the most sophisticated LLMs, including GPT-4, Claude, and Gemini, are primarily commercialized and deployed on cloud services. This API-based deployment ensures the privacy of the LLMs but potentially compromises user privacy (Xiao et al., 2023; Zhang et al., 2023c). Although more powerful LLMs are being open-sourced, like Llama-2 (Touvron et al., 2023b), Qwen (Bai et al., 2023), and Mistral (Jiang et al., 2024), their stable deployment on local devices with limited resource remains challenging. Recent advancements in smaller LMs (SLMs) equipped with billions of parameters now enable their deployment on consumer desktops and smartphones, achieving satisfactory performance (Bai et al., 2023; Zhang et al., 2024; Singer et al., 2024; Hu et al., 2024). The balance between performance and privacy in LLMs and SLMs raises three critical questions: (1) How effectively can LLMs operate without stringent user privacy contexts? (2) To what extent can specialized models, boasting billions of parameters, excel in context-aware instruction following? (3) Is it possible to navigate the trade-off between performance and privacy through collaborations between large and small models?
As indicated in Figure 1, considering the following scenario in ➍: smaller, personalized LMs are deployed on user devices with limited resources. These SLMs can access private data and activity logs on the devices while processing instructions. In contrast, the more advanced general LLMs operate on cloud services and receive only general instructions. In this setup, the LLMs provide high-level knowledge like deeper planning, superior outlines, and even “dark knowledge”. Meanwhile, the SLMs utilize the context information and knowledge provided by the LLMs to collaboratively generate personalized content. Current privacy protection methods for API-based services are limited (Cummings et al., 2023); they are either capable of handling only simple classification tasks with text sanitization (Kan et al., 2023; Chen et al., 2023b) or require encryption or noise addition (Zhou et al., 2022; Wu et al., 2023), which still pose a risk of data leakage. In contrast, the collaborative generation approach involving SLMs and LLMs can logically prevent privacy breaches without the need to upload context information.
Overall, we highlight our contributions as follows: (1) We introduce the context-aware instruction-following task that incorporates extensive user privacy context information. To support this, we design a four-step data construction process and synthesize a modest amount of instructional data for experimental validation. (2) We investigate the performance of LLMs and SLMs on this task. Our findings indicate that SLMs, when provided with context, can outperform LLMs lacking context but lag behind the performance of LLMs equipped with context. (3) For contextaware instruction generation, we present the CoGenesis framework. CoGenesis comprises sketchbased and logit-based variants to facilitate collaboration between large and small language models. This approach not only safeguards context privacy but also ensures performance gains.

heading: 2 Context-aware Instruction Following

heading: 2.1 Task Definition
text: Current instruction formats either consist of standalone instructions or instructions accompanied by additional inputs (Wang et al., 2023c). While these instructions typically cover generic tasks such as writing, searching, and coding, the inputs often contain specific task information, such as tables and coding problems. We classify the contextaware instruction-following task within the domain of controllable conditional text generation (Zhang et al., 2023b), enriching standard instructions with additional personal context. The task T , focused on generating a response r relevant to user-specific data marked by privacy and personal style, is directed by instructions t and contextual information C. It underscores the significance of contextual integration for enhancing output personalization and relevance, formally expressed as:
T (t, C) : r = g(t, C; θ) (1)
This generative model g with parameters θ aims to adaptively respond to user instructions within the nuanced context of individual data attributes.

heading: 2.2 Data Construction
text: As illustrated on the left side of Figure 2, we delineate a novel four-step pipeline for crafting contextaware instructions aimed at generating personalized and creative text with AI assistants for diverse user groups. Our methodology begins with the
creation of detailed user group portraits, capturing demographics, professional backgrounds, and interests to identify specific AI application scenarios. Individual user profiles are then elaborated, incorporating unique writing styles, fictional personal details, and smart device usage to construct nuanced characters for AI writing tasks. These profiles inform the design of writing tasks that resonate with each character’s lifestyle and digital interactions, ensuring task realism and relevance. Finally, we generate personalized texts that reflect the characters’ professional and personal narratives with stylistic accuracy, demonstrating our approach’s efficacy in producing coherent, context-specific content for AI-facilitated text generation.
An example is illustrated on the right side of Figure 2, where the user instruction comprises a general section and a personal section. The latter, in conjunction with the provided profile, experience, activities, and previous articles, constitutes the privacy-sensitive context information.

heading: 3 Collaborative Generation Framework

heading: 3.1 Overview of CoGenesis
text: We present the CoGenesis framework that capitalizes on the strengths of two differently scaled models: a LLM with parameters θl and a SLM with parameters θs. This framework is centered around the fusion strategy, denoted as f(·), which intelligently combines the outputs from both models. Specifically, θl generates replies solely based on the general instruction t, while θs considers both user instruction t and additional personal context C for its output generation. The fusion strategy
f(·) aims to synergistically blend the outputs of θl(r|t) and θs(r|t, C). Intuitively, the combined performance is expected to not only surpass that of the individual models but also closely match the performance of θl had it processed both r and C.
In our collaborative framework, sketches (or outlines) of content and next token logits from LLMs are considered forms of high-level knowledge. The two approaches to the function f(·) are identified as sketch-based and logit-based, respectively. The sketch-based approach is model-agnostic, whereas the logit-based method requires LLMs and SLMs to share the same tokenizer in our present configuration. In the following sections, we will detail these two implementations of f(·) sequentially.

heading: 3.2 Sketch-based CoGenesis
text: Recognizing the strengths of LLMs in planning and SLMs in crafting contextualized responses, we introduce a "sketch-then-fill" approach to synergize their capabilities for personalized content generation. As depicted on the left side of Figure 3, this approach consists of two crucial steps:
Step1: Sketch Generation by LLMs. Given the substantial cost and complexity, especially with API-dependent LLMs, we simplify the process by directly prompting LLMs with a general instruction t. The sketch rsketch of content is derived through text decoding from the LLMs using a sampling strategy, succinctly represented as:
rsketch = DecodingLLM(t; θl) (2)
Step2: Content Personalization by SLMs. After acquiring the sketch rsketch, the SLM utilizes it,
along with the initial instruction t and personal context C, to tailor personalized content r. This phase focuses on fine-tuning the SLM’s parameters θs to optimize content relevance and personalization:
r̂ = argmax r
P (r|t, C, rsketch; θs) (3)
In this formula, r denotes the final, customized content, and P is the likelihood of generating r given instruction t, context C, and sketch rsketch with parameters θs. This approach delineates the use of LLMs for foundational text sketching based on user prompts, without necessitating parameter adjustments, and SLMs for further content refinement. This ensures the final output aligns with user specifications and their interaction history, highlighting the distinct yet complementary roles of LLMs and SLMs in personalized content creation.

heading: 3.3 Logit-based CoGenesis
text: The logits produced in the final layer of language models encapsulate a wealth of information, reflecting the models’ internal dark knowledge. Previous efforts, such as contrastive decoding (Li et al., 2022) and emulator tuning (Mitchell et al., 2023), have explored the synergistic use of logits from both LLMs and SLMs to diminish hallucinations (Sennrich et al., 2023), augment reasoning capabilities (O’Brien and Lewis, 2023), and streamline the fine-tuning process of LLMs (Liu et al., 2024). Motivated by these works, our logit-based strategy involves integrating the logits of LLMs and SLMs under different inputs, ensuring collaborative determination of the subsequent token. A notable aspect of our method is the differential context exposure for the models: SLMs access the
full privacy context, while LLMs are provided with only broad instructions, as shown in Figure 3.
Defining the response sequence up to the kth token as r<k, and denoting the kth token probabilities over vocabulary generated by LLMs and SLMs as plk and p s k respectively, we leverage a lightweight combined model, denoted as CombModel, with parameters θc, to derive fusion weights w for final combined probabilities pck. The computation of the logit-based method proceeds as follows:
psk = θs(r<k, t, C), plk = θl(r<k, t) (4)
w = CombModel(plk, p s k) (5)
pck = w · psk + (1− w) · plk (6)

heading: 4 Experiments

heading: 4.1 Datasets
text: Synthetic Dataset. Following the construction process outlined in § 2.2, we synthesize a contextaware instruction-following dataset with GPT-4. Due to cost considerations, we construct a dataset representing thousands of fictitious users, based on hundreds of group portraits. After conducting quality and format filtering, we obtained a total of 1,500 users for training and validation. Additionally, we selected approximately 200 users from diverse group portraits to serve as the test set.
Open-source Datasets. In addition to our synthesized context-aware instruction datasets, we also utilize publicly accessible, personalized context writing datasets, although they are limited to specific tasks in domains such as email and academic
papers. Specifically, we employ the processed Avocado Research Email and Citation Network Papers datasets in LaMP (Salemi et al., 2023). Furthermore, we refine these datasets to facilitate the generation of email bodies and paper abstracts, considering previous emails and papers from the same users as contextual information.
Further details about the synthesized and processed datasets can be found in Appendix A.

heading: 4.2 Baselines
text: Settings. We primarily evaluate four configurations in our experiments: 1) LLM with context. We engage LLMs with additional context to facilitate personalized generation. It’s an upper bound that may compromise context privacy. 2) LLM w/o context. Given the importance of privacy in contextual data, it is advisable to limit requests to cloud-based LLMs to general instructions only. This setting serves as a lower bound that preserves user privacy. 3) SLM with context. This setting establishes the baseline for privacy-protected, ondevice personalized generation. 4) SLM + LLM with context. This is our proposed collaborative generation between large- and small-scale models. Within this framework, we evaluate sketch-based and logit-based methods.
Models. Our selection of LLMs encompasses both commercial API-based and open-source models. For the commercial segment, we concentrate on GPT-3.5-turbo and GPT-4-turbo. In the realm of open-source, we opt for the largest models within the Llama-2 (Touvron et al., 2023b), Qwen (72B in versions 1 and 1.5 2) (Bai et al., 2023), and Mistral (Jiang et al., 2024) series. Regarding SLMs, we prioritize the most recently released models with 1∼2 billions of parameters. This includes TinyLlama (Zhang et al., 2024), Qwen (1.8B in versions 1 and 1.5), StableLM 3, and H2O-Danube (Singer et al., 2024). For both LLMs and SLMs, our focus is on the chat versions, employing the default template for each model for consistency.
Additionally, zero-shot LLMs are used in both with context and w/o context settings. For SLM with context, we include both zero-shot and fine-tuned models, whereas only fine-tuned SLMs are utilized in SLM + LLM with context. Further information on prompts for LLMs and fine-tuning details for SLMs is presented in Appendix B.
2https://github.com/QwenLM/Qwen1.5 3https://huggingface.co/stabilityai

heading: 4.3 Evaluation Metrics
text: For instruction-following evaluation, LLM-based evaluators like GPT-4 have shown high consistency and effectiveness (Chang et al., 2023), particularly in the generation of personalized content, outperforming human evaluators in consistency (Wang et al., 2023b). Therefore, we employ GPT-4-turbo as a judge to assess generated content from multiple perspectives. This evaluation encompasses several criteria: personalization, alignment with the user profile, helpfulness, relevance, depth, creativity, and the level of detail, collectively contributing to the overall score (Ovl.). To further analyze the performance, we introduce two distinct prompts: one incorporating user context (Ovl.(w)) and the other excluding it (Ovl.(w/o)), enabling a comparative assessment of LLMs in both personalized content generation and broad instruction adherence. Additionally, the personalized scores of the responses are assessed independently, indicated as Per.

heading: 4.4 Main Results
text: As illustrated in Table 1, we analyze the experimental results considering the following aspects:
Results on LLMs. In the with context setting, GPT-4-turbo outperformed all other models across all metrics and datasets, followed by Qwen-Chat(v1.5) with the second-best performance. Similar outcomes were observed in the w/o context setting. However, a comparison between the two settings reveals that all LLMs exhibit diminished performance in terms of personalization and overall scores in the absence of context information, underscoring the value of context. Interestingly, while context greatly influences the scores in with context setting, it has a negligible effect on scores in w/o context setting, reflecting advanced capability of LLMs in adhering to general instructions. Moreover, when comparing our synthesized context-aware dataset and the email dataset to the paper abstract dataset, the latter demonstrates limited personalization factors resulting in high performance in w/o context setting.
Results on SLMs. Zero-shot SLMs exhibit highly varied performances, influenced by pre-training and supervised fine-tuning factors. Notably, StableLM-Zephyr achieves the highest performance, with H2O-Danube-Chat and Qwen-Chat(v1.5) closely competing for second place. Interestingly, with the advantage of context, zero-shot SLMs can outperform LLMs in scenarios
lacking context. However, LLMs consistently outperform SLMs in overall scores without context, underscoring their superior capabilities in general instruction generation. After fine-tuning, SLMs surpass the performance of many LLMs with context, demonstrating the benefits of specialization. Yet, SLMs do not reach the performance levels of the most powerful LLMs, such as GPT-4-turbo and Qwen-Chat(v1.5). These results highlight the promise of collaboration between specialized SLMs and LLMs for achieving better personalized scores, deeper writing, and enhanced instruction generalization.
Results on Mixed-Scale Models Collaborations. In our exploration of mixed-scale model collaboration, we utilize the sketch-based method for all SLMs and the logit-based method exclusively within the Llama and Qwen model families. This comparison reveals that collaborations between mixed-scale models achieve results comparable to those of LLMs alone, while also safe-
guarding privacy. Collaborative efforts generally enhance the overall scores of SLMs, both in evaluations with and without user context. Nevertheless, incorporating sketch-based collaboration, in particular, might slightly detract from personalization scores due to the reliance on LLMs that inference without context. Between the logit-based and sketch-based approaches, the former proves more efficacious, contingent upon the SLMs and LLMs utilizing a common tokenizer. Overall, this collaborative strategy between mixed-scale models offers a promising avenue for balancing efficiency and privacy considerations, though it still necessitates further refinements to optimize performance.

heading: 4.5 Ablation Study

heading: 4.5.1 Sketch-based CoGenesis
text: Sketch vs. Full Content. Figure 4 contrasts the efficacy of employing merely the sketch versus the entire content provided by LLMs. The findings indicate that incorporating full content generally
detracts from model performance, in contrast to utilizing no content or only the sketch. This discrepancy can be attributed to the potential overload of redundant information. As explored in (Weston and Sukhbaatar, 2023), an excess of content does not invariably enhance performance and risks the inclusion of extraneous details.
Generalization Capabilities of Sketch. For small models fine-tuned with sketches generated by GPT-4, we explore their generalization potential by utilizing sketches from a variety of LLMs during testing. Specifically, we focus on StableLM-Zephyr for detailed ablation analysis. Figure 5 demonstrates that employing sketches from alternative LLMs marginally affects the overall and personalized scores negatively but enhances the overall score in scenarios lacking context, relative to GPT-4. This suggests that sketches generated by different models vary and exhibit limited generalization capabilities.

heading: 4.5.2 Logit-based CoGenesis
text: Logits Fusing Strategy. We implement a learnable model designed for merging logits from contextfree LLMs and context-inclusive SLMs in § 3.3. Additionally, we explore straightforward max and mean pooling strategies, acknowledged as robust baselines in (Ormazabal et al., 2023). According to Table 2, both max and mean pooling methods enhance the performance of SLMs, with max pooling
proving to be superior. Nonetheless, given the distinct input conditions for LLMs and SLMs, a learnable fusion model becomes essential in contextaware environments, outperforming simple pooling techniques significantly.
Logits of LLMs and SLMs. Within the fusion model, we integrate logits from LLMs and SLMs using self-adjusting weights. Figure 6 showcases the output generated by both Qwen-72B and Qwen-1.8B, employing color-coded weights to delineate their respective contributions: red signifies SLMs, blue denotes LLMs, and white represents an evenly balanced weight of 0.5. It is observed that LLMs predominantly influence the sketch of the generated content, whereas SLMs play a more significant role across the majority of tokens, underscoring the importance of collaboration.

heading: 4.6 Human Evaluation
text: Evaluations by LLMs have not yet been demonstrated to be infallible; consequently, we conducted human evaluations. To streamline the complexity of this assessment, annotators were instructed
contributions, red indicates LLM contributions, with darker shades representing higher weights.
to compare outcomes under various settings to those of the finetuned SLM, recording results as win/tie/lose, mirroring the methodology utilized in lmsys/chatbot-arena-leaderboard 4. Additionally, we employed traditional word overlap metrics such as BLEU and ROUGE-L, calculated using the evaluate library5.
As shown in Table 3, the results of human evaluation are consistent with GPT-4 evaluations, where CoGenesis performs better than finetuned SLMs and LLMs without context separately, and performs closely to LLMs with context.

heading: 5 Discussion
text: Although our experiments have been limited to models of the same family using identical tokenizers, these methods could potentially be expanded through a tokenizer alignment strategy (Fu et al., 2023; Wan et al., 2024). This principle aligns with other logits-based decoding techniques such as speculative decoding, contrastive decoding. By aligning the tokens and probabilities of models with different tokenizers, it is feasible to facilitate knowledge transfer across various LLMs and SLMs.
Recent studies have demonstrated the potential to reconstruct prompts based on the distribution of next token tokens (Morris et al., 2023, 2024). However, the accuracy of extraction, particularly the exact match scores, remains discouragingly low. Furthermore, since only the top-k logits for each token are utilized in our experimental, the cost of reconstruction is prohibitively high. Therefore, logits-based collaboration remains sufficiently
4https://huggingface.co/spaces/lmsys/ chatbot-arena-leaderboard
5https://github.com/huggingface/evaluate

heading: Settings Ovl.(w/) Per. Ovl.(w/o)
text: secure, and can be further enhanced with the implementation of encryption and noise addition algorithms. As shown in Table 4, we have investigated the logit-based CoGenesis approach, which enhances privacy by uploading only the previous few tokens instead of the entire response. This approach, inspired by the principle that a good start leads to effective completion (Jain et al., 2024; Wang and Zhou, 2024), suggests that initial guidance from LLMs on the first few tokens can direct SLMs to independently generate the remainder of the response. Performance data indicate improvements when transferring just 8, 16, or 32 tokens to cloud LLMs, compared to a fine-tuned SLM alone, with the entire response typically exceeding 500 tokens. Increasing the number of transferred tokens boosts the LLM + SLM performance, allowing us to balance enhanced performance against reduced privacy risks.

heading: 6 Related Works
text: The advent of LLMs has revolutionized the field of instruction following, with models being trained
on diverse and complex instruction sets, enabling them to perform a wide array of tasks from creative writing (Franceschelli and Musolesi, 2023) to coding (Qian et al., 2023) and debugging (Jimenez et al., 2023). The push towards collecting highquality instruction data (Wang et al., 2023c; Ding et al., 2023; Xu et al., 2023a) has allowed for the development of both proprietary and open-source medium-scale language models adept at following instructions. However, the reliance on cloudbased proprietary models like ChatGPT and GPT4 for instruction execution raises significant privacy concerns due to the potential risks associated with uploading sensitive data (Achiam et al., 2023; Team et al., 2023; Liu et al., 2023). To mitigate these risks, various privacy-preserving techniques have been employed, albeit with limitations in completely securing user data (Cummings et al., 2023; Kan et al., 2023; Chen et al., 2023b; Wu et al., 2023). Furthermore, there is a notable gap in instruction datasets, particularly in the inclusion of contextual information (Salemi et al., 2023; Wang et al., 2023b), highlighting an area for further exploration in context-aware instruction formulation to enhance privacy and personalization.
In parallel, the exploration of mixed-scale model collaboration emerges as a promising avenue to address the scalability, efficiency (Xia et al., 2024), and privacy (Yao et al., 2023) challenges inherent to LLMs. While larger models benefit from increased capabilities, their high inference costs and privacy concerns contrast with the lower costs and greater accessibility of smaller models (Bai et al., 2023; Gunasekar et al., 2023; Zhang et al., 2024; Grangier et al., 2024; Singer et al., 2024). Research in this domain is bifurcated into collaborative training and inference strategies, including offsite-tuning (Xiao et al., 2023) and speculative decoding (Leviathan et al., 2023), aiming to leverage the strengths of both large and small models (Mitchell et al., 2023; Liu et al., 2024). This paper specifically investigates collaborative inference methods, including sketch-based and logit-based approaches, to enhance the efficiency and privacy of LLMs, suggesting a promising direction in utilizing mixed-scale models for instruction following tasks.

heading: 7 Conclusion
text: This paper investigates context-aware instruction following, enriching prompts with detailed user privacy information. We outline a pipeline for
creating context-aware instructions. To mitigate privacy issues, we introduce CoGenesis, a collaborative framework between SLMs and LLMs utilizing sketches and logits. Our results highlight the advantages of mixed-scale model collaboration, suggesting fruitful directions for future research.

heading: Limitations
text: This study explores context-aware instruction following, introducing strategies for collaboration between large and small models to address privacy concerns. We developed a synthetic dataset for context-aware instruction following to empirically test our approaches. Our findings suggest that this model collaboration can significantly mitigate privacy risks associated with using public API-based LLMs. However, our dataset is limited in size and was specifically crafted for preliminary validation. Future work will focus on expanding this dataset to enhance its quality, realism, and diversity. Additionally, our proposed methods, particularly the logits-based approach, are currently restricted to models sharing the same tokenizer. Further research on tokenizer alignment is necessary to broaden the applicability of our strategies.

heading: Ethics Statement
text: The advent of LLMs has underscored the urgent need to address privacy and security concerns within the realm of artificial intelligence. This paper concentrates on the privacy challenges posed by context-aware instruction-following applications of LLMs, proposing methods to mitigate these concerns without compromising the models’ effectiveness. We emphasize the ethical imperative of protecting user data, adopting a strategy that involves generating synthetic datasets using GPT-4 for training and testing. This approach ensures that our research does not compromise real-world user privacy by preventing any potential data leakage. In essence, our work not only seeks to advance the technological capabilities of LLMs but also to uphold the highest standards of ethical responsibility by safeguarding user privacy through innovative and secure data handling practices.

heading: Acknowledgements
text: This work is supported by the National Key R&D Program of China (No. 2022ZD0119101). We extend our gratitude to the anonymous reviewers for their insightful feedback.

heading: A Dataset Details

heading: A.1 Context-Aware Instructions
text: The four steps for constructing context-aware instructions are as follows.
Group Portraits. We begin by constructing highly diverse user group portraits from the real world, encompassing a wide range of groups such as college students, programmers, and various other professions. For each group, we define their age range, identify their professional field or occupation, and enumerate typical activities and hobbies to capture the group’s unique interests. Additionally, we delineate specific scenarios in which these groups might utilize an AI assistant for personalized and creative text generation in both their professional and personal lives. Examples of such use cases include drafting business emails, writing creative blogs, composing academic papers, and crafting extended tweets.
User Profile. Building upon our diverse user group portraits, we next develop individual user profiles with rich detail. Emphasizing consistency and realism, our process involves four steps: 1) Personal Writing Style: Tailoring language use and expression unique to each character. 2) Private Information: Creating 5-10 fictional details for each profile, including life events and technology interactions. 3) Smart Device Usage: Generating 5-10 fictional activity logs per profile, covering messages, purchases, schedules, and more. Our aim is to shape distinct, multi-dimensional characters for a variety of AI writing applications.
Writing Task Instructions. In this phase, we craft text creation tasks tailored to our user characters, ensuring alignment with their professions, hobbies, and lifestyles. These tasks are intricately linked to their mobile phone activity logs and personal details, weaving the characters’ experiences, social media activities, and AI assistant interactions into the narratives. For each character, we develop K tasks that are both realistic and contextually relevant.
Personalized Generations. Here, we produce personalized texts adhering to specific guidelines. Our focus is on crafting authentic and coherent narratives that vividly reflect each character’s professional and personal life. By adapting to the user’s unique writing style, we aim to create personalized and stylistically distinctive content. To ensure coherence and relevance across various tasks, content for each task is generated sequentially.
Following these four steps, we utilize GPT-4 to create instructions with user context and responses for subsequent experiments.

heading: A.2 LaMP Dataset Processing
text: To minimize the personalization gap between the target content and user profiles in the Avocado Research Email and Citation Network Papers datasets in LaMP 6, we have implemented a two-step processing approach. Initially, we retrieve content that most closely aligns with the user profile using intfloat/e5-mistral-7b-instruct embeddings (Wang et al., 2023a). Subsequently, we encode the target content and the entire profile using style embeddings (Wegmann et al., 2022; Patel et al., 2023) , selecting only the most personalized samples. Furthermore, the content lengths and profiles of samples in LaMP vary significantly, ranging from 10 to 100,000 tokens 7, exceeding the context length capabilities of contemporary models. Consequently, we have selected samples with a minimum length of 128 characters for paper abstracts and 64 for email bodies, and a maximum length of 1024 characters for both. Owing to the unavailability of a test dataset in LaMP, we repurposed the dev dataset as our test set. Additionally, we divided the filtered training data into actual training and validation datasets, using a 9:1 split ratio.

heading: A.3 Dataset Statistics
text: The statistical results of the final processed dataset are presented in Table 1.
Our synthetic instruction datasets include a variety of instruction types, such as preparing a speech, designing a plan, and more. All of these tasks require models to utilize personal context information, including previous activities and schedules. To illustrate, we display the top 10 most common root verbs and top 10 direct noun objects in our constructed datasets as shown in Figure 6
6https://lamp-benchmark.github.io/download 7https://github.com/openai/tiktoken/

heading: Verb Percent (%) Verb Percent (%)

heading: B Model Details
text: For all open-source Large Language Models, we utilize vLLM 8 for efficient inference, setting the temperature to 0.7, top-p to 0.9, and the maximum number of new tokens to 1024.
Regarding the specialization of Small Language Models (SLMs), we apply a range of learning rates {5e-6, 8e-6, 1e-5, 2e-5, 5e-5} across different models and datasets. Furthermore, we implement an early stopping strategy to identify the optimal model based on validation performance as the specialized models. We fine-tune each model using a batch size of 8, max sequence length of 4096, across four A6000 48GB GPUs.
For the combined model, we employ a threelayer neural network featuring ReLU activation, with sigmoid activation applied to determine the final weights. At each generation step, only the top 10 logits from both LLMs and SLMs are utilized. The intermediate hidden layers are configured with sizes of 512 and 16, respectively. The model is trained using a learning rate of 2e-3 and a batch size of 2. The combined model is trained on the training dataset, with both LLMs and SLMs assigned the same target response. Additionally, an early stopping strategy based on the validation set performance is employed to select the optimal combined model. For all the aforementioned experiments, we calculate the mean scores using three distinct random seeds.

heading: C Evaluation Details
text: Owing to the costs associated with evaluation, we assess only a portion of the test samples. To account for GPT-4’s evaluation stability, we plot a curve illustrating the relationship between scores and the number of evaluated samples. As depicted
8https://github.com/vllm-project/vllm
in Figure 7, the results stabilize once the number of samples reaches 100. Therefore, we randomly select 100 samples from the test set, of which only 80 samples are utilized in the widely recognized benchmark, MT-Bench 9.

heading: D Prompt Details
text: Prompts designed for querying Large Language Models (LLMs) both with and without context are outlined in Table 7. Prompts intended for extracting outlines are illustrated in Table 8. Prompts used for GPT-4 based evaluation are depicted in Table 9.

heading: E Related Works
text: E.1 Instruction Following and Privacy
Large language models (LLMs), after being trained on high-quality instruction data and calibrated to align with human intentions, have acquired the capability to execute instructions across a range of activities such as creative writing (Franceschelli and Musolesi, 2023), coding (Qian et al., 2023), debugging (Jimenez et al., 2023), and various other textbased tasks (Bubeck et al., 2023). Contemporary research in instruction following prioritizes the acquisition of high-quality data (Wang et al., 2023c), which encompasses instructions of varied complexity (Xu et al., 2023a) and diversity (Ding et al., 2023; Cui et al., 2023), as well as ensuring a minimal dataset size for effective generalization (Chen et al., 2023a; Wei et al., 2023). Thanks to these technological advancements along with alignment algorithms (Rafailov et al., 2023), medium-scale language models with around ten billion parameters have been made open-source and perform adeptly at following instructions (Tunstall et al., 2023; Jiang et al., 2023; Zhang et al., 2024).
9https://huggingface.co/spaces/lmsys/mt-bench
Nevertheless, premier chat models like ChatGPT, GPT-4, Claude, and Gemini remain proprietary, largely due to commercial considerations, necessitating that our instruction data be uploaded to the cloud (Achiam et al., 2023; Team et al., 2023). While these models serve as potent AI assistants in daily professional and personal endeavors, they also pose significant privacy risks (Liu et al., 2023). To address privacy concerns, LLMs have employed various techniques (Cummings et al., 2023), including text sanitization (Kan et al., 2023), differential privacy (Chen et al., 2023b; Wu et al., 2023), and hidden representations (Zhou et al., 2022, 2023). However, these methods still involve uploading potentially sensitive data to the cloud, which inherently cannot eliminate the risk of privacy breaches.
Moreover, current instruction datasets predominantly cover general domains, with insufficient focus on contextual information modeling. The most closely related works involve personalized response generation, evolving from traditional benchmarks (Salemi et al., 2023; Wang et al., 2023b). The integration of extensive context information into general open-domain instructions remains an area of ongoing exploration. This paper aims to delve into context-aware instruction formulation as a means to advance research on privacy considerations within the realm of instruction following.

heading: E.2 Mixed-Scale Models Collaboration
text: The “scaling law” in language modeling posits that models with a greater number of parameters exhibit enhanced capabilities (Kaplan et al., 2020). However, these more robust models also encounter challenges related to higher inference costs, efficiency (Xia et al., 2024), and privacy concerns (Yao et al., 2023). Conversely, smaller models, ranging from 1 to 2 billion parameters, are gaining popularity due to their increasingly impressive performance (Bai et al., 2023; Gunasekar et al., 2023; Zhang et al., 2024; Grangier et al., 2024; Singer et al., 2024). These specialized models are coupled with lower inference costs and the feasibility of deployment on consumer-grade desktops and smartphones (MLC team, 2023). Collaborations between mixed-scale models represent a promising research avenue. The body of current research in this area primarily falls into two categories: training and inference. For collaborative training, Offsite-tuning has been introduced as a method to protect both user data and the privacy of large models (Xiao et al., 2023; Zhang et al., 2023c). This approach
involves using an emulator derived from LLMs, fine-tuning it on specific downstream data, and subsequently integrating the learned parameters back into the LLMs. On the inference front, techniques like speculative decoding (Leviathan et al., 2023; Xia et al., 2024) and contrastive decoding (Li et al., 2022; O’Brien and Lewis, 2023) aim to enhance and expedite LLMs’ inference processes by leveraging smaller draft or expert models. Additionally, emulator tuning (Mitchell et al., 2023) and proxy tuning (Liu et al., 2024) have been devised to economize on fine-tuning large models; however, they can also be considered forms of collaborative decoding during inference. This paper focuses on examining collaboration during inference, specifically investigating sketch-based and logit-based methods.

heading: SYSTEM PROMPT FOR REQUEST WITH CONTEXT IN PERSONALIZED EMAILS AND PAPERS

heading: SYSTEM PROMPT FOR REQUEST WITH CONTEXT IN CONTEXT-AWARE

heading: SYSTEM PROMPT FOR REQUEST WITHOUT CONTEXT

heading: PROMPT FOR EXTRACTING OUTLINE OF PAPER.

heading: PROMPT FOR EXTRACTING OUTLINE OF EMAIL.

heading: PROMPT FOR EXTRACTING OUTLINE OF CONTEXT-AWARE.

heading: Future Work
text: Future work will focus on expanding this dataset to enhance its quality, realism, and diversity. Additionally, our proposed methods, particularly the logits-based approach, are currently restricted to models sharing the same tokenizer. Further research on tokenizer alignment is necessary to broaden the applicability of our strategies