How Far Can In-Context Alignment Go? Exploring the State of In-Context Alignment (2024)

Abstract: Recent studies have demonstrated that In-Context Learning (ICL), through the use of specific demonstrations, can align Large Language Models (LLMs) with human preferences known as In-Context Alignment (ICA), indicating that models can comprehend human instructions without requiring parameter adjustments. However, the exploration of the mechanism and applicability of ICA remains limited. In this paper, we begin by dividing the context text used in ICA into three categories: format, system prompt, and example. Through ablation experiments, we investigate the effectiveness of each part in enabling ICA to function effectively. We then examine how variants in these parts impact the model’s alignment performance. Our findings indicate that the example part is crucial for enhancing the model’s alignment capabilities, with changes in examples significantly affecting alignment performance. We also conduct a comprehensive evaluation of ICA’s zero-shot capabilities in various alignment tasks. The results indicate that compared to parameter fine-tuning methods, ICA demonstrates superior performance in knowledge-based tasks and tool-use tasks. However, it still exhibits certain limitations in areas such as multi-turn dialogues and instruction following. Source codes and scripts are available at https://github.com/li-aolong/how-far-can-ica-go.

## introduction pre-trained large language models (llms) typically require an instruction fine-tuning phase during which model parameters are adjusted to align the model with human preferences, enabling it to follow human instructions and function as an interactive assistant @xcite @xcite . although this phase is much less costly than pretraining, it still involves complex and challenging recent studies have proposed that by crafting specific demonstrations, base models can achieve alignment with human preferences through in-context learning (icl). this process is known as in-context alignment (ica) @xcite . ica enables models to understand instructions without the need for parameter fine-tuning. this cost-effective nature positions ica as a potentially viable alternative to the fine-tuning-based alignment methods. however, the exploration of ica in these studies has been limited to exploring the alignment capabilities of llms as open-domain assistants in single-turn dialogue. detailed analysis lacks regarding the underlying mechanisms and broader applicability of ica, leading to our main research question: how far can ica go? this raises several questions: 1. does ica rely on specific parts or the entirety of the context to align with human preferences? 2. considering the significance of various contextual parts, would substituting them with alternative variants affect ica performance to varying extents? 3. with a preferable setting identified by exploring the previous question, what other tasks can ica effectively perform beyond single-turn dialogue? these questions are related to each other and we propose to address them sequentially. to begin exploring the first question, we initiate a pilot ica experiment with a pre-trained model, as illustrated in figure 1 . the purpose is to investigate the alignment effect of format and system prompt in the simplest scenario. we apply the chatml foot_0 format, commonly used for fine-tuning models, to llama2-7b-base model @xcite . to our surprise, the model can understand user queries and generate meaningful responses, a capability that direct prompting methods could not achieve. by probing the effects of the format and system prompt, we further explore the roles that these parts, along with the examples, play in ica. in this paper, we initially explore how contextual content affects ica by dividing it into three parts: format, system prompt, and example (section 3). building on this, we address the second question by designing variants for each part to investigate their impact on ica performance (section 4). finally, for the third question, we explore the effectiveness of ica in other instruction tasks (section 5). additionally, we train two sft models and compare them with the original chat model. our contributions can be summarized as follows: ## how different format, system and example variants affects ica? in this section, we explore the impact of different variants of format, system, and example on ica when all parts are present. ## conclusion in this work, we explore which parts of the context influence ica and how it can be affected. we divide the entire prompt into three parts: format, system, and example. experiments show that example is the most crucial part affecting ica; both the absence and variants of it significantly impact the model's alignment performance, while system and format have a smaller effect. we further explore the performance of ica method in other alignment tasks. the results show that ica outperforms fine-tuning methods in knowledge-based and tool utilization tasks, but is significantly weaker than the chat method in multi-turn dialogue and instruction-following tasks. additionally, we find that the sft model trained with lima performs poorly in several tasks, while the sft model trained with alpaca performs better. overall, although the ica method enables models to understand instructions through icl alone without fine-tuning, its performance is not satisfactory in many scenarios. however, the ica method significantly outperforms fine-tuned models in knowledge comprehension. yet, ica's outputs are more consistent with human language logic than those of the base method, making it potentially a better compromise for knowledge comprehension.