LLM can Achieve Self-Regulation via Hyperparameter Aware Generation (2024)

Abstract: In the realm of Large Language Models (LLMs), users commonly employ diverse decoding strategies and adjust hyperparameters to control the generated text. However, a critical question emerges: Are LLMs conscious of the existence of these decoding strategies and capable of regulating themselves? The current decoding generation process often relies on empirical and heuristic manual adjustments to hyperparameters based on types of tasks and demands. However, this process is typically cumbersome, and the decoding hyperparameters may not always be optimal for each sample. To address the aforementioned challenges, we propose a novel text generation paradigm termed Hyperparameter Aware Generation (HAG). By leveraging hyperparameter-aware instruction tuning, the LLM autonomously determines the optimal decoding strategy and configs based on the input samples, enabling self-regulation. Our approach eliminates the need for extensive manual tuning, offering a more autonomous, self-regulate model behavior. Experimental results spanning six datasets across reasoning, creativity, translation, and mathematics tasks demonstrate that hyperparameter-aware instruction tuning empowers the LLMs to self-regulate the decoding strategy and hyperparameter. HAG extends the current paradigm in the text generation process, highlighting the feasibility of endowing the LLMs with self-regulate decoding strategies.

## introduction in recent years, the rapid development of large language models (llms) has unveiled a spectrum of capabilities previously unimagined in the realm of natural language processing @xcite . these models, equipped with vast knowledge and contextual understanding, have revolutionized how we interact with machine intelligence. users commonly employ diverse decoding strategies and manually adjust hyperparameters for various scenarios, such as setting a temperature of 0.8 for code generation @xcite and a temperature of 0 for model evaluator @xcite . however, a critical inquiry emerges: do llms possess the intrinsic capability to realize the existence of decoding strategies and regulate themselves? to regulate the model output, the current approaches can be classified into three main categories: (1) instruction regulation: manipulating the model's behavior through delicately designed prompts @xcite or utilizing in-context learning with demonstrations @xcite to regulate llms in generating desired outputs. while these methods have shown promise in enhancing the generation of llms, instruction regulation and feedback regulation primarily focus on modifying the model's inputs without endowing the model with the capability to alter its hyperparameter settings. in contrast, existing hyperparameter regulation methods predominantly involve manual adjustments, often relying on heuristic and experiential tuning based on task requirements. however, this process can be burdensome and lacks the assurance that the decoding hyperparameters are relatively optimal for each given input. therefore, we are considering whether llm can autonomously self-regulate decoding hyperparameters to different contextual demands like the human body adjusting physiologically based on the external environment. the human body possesses a comparable self-regulation mechanism to dynamically adjust physiological parameters for optimal performance in various internal activities @xcite . consider the human body's response to physical exertion-during exercise, the heart rate and blood pressure increase to ensure an adequate supply of oxygen and nutrients to the active muscle tissues. similarly, during social interactions or casual conversations, humans experience the release of hormones that facilitate emotional expression and foster social connections, enabling individuals to navigate a spectrum of social nuances effectively. addressing this gap, we propose a novel paradigm: hyperparameter aware generation (hag). this approach significantly diverges from existing methodologies by enabling llms to autonomously determine and adjust decoding hyperparameters in response to specific input through leveraging hyperparameter-aware instruction tuning. the model generates suitable hyperparameters in the first stage based on the user's input questions. subsequently, these model-derived hyperparameters are used to adjust the model's decoding strategies and hyperparameters, followed by generating results under these new settings in the second stage. our approach eliminates the need for extensive manual tuning, offering a more autonomous self-regulation model behavior. we conduct experiments on six datasets across reasoning, creativity, translation, and mathemat-ics tasks. we summarize the main findings from our experiments and try to provide preliminary answers to our proposed research questions: (1) do llms realize the existence of decoding strategies? a: our model demonstrates proficiency in generating hyperparameters within a normal and effective range, implying the model's capacity to perceive the presence of decoding hyperparameters and provide rational configurations for hyperparameters. (2) can llms regulate decoding hyperparameter? a: our proposed generative framework hag endows the model with the capability to generate decoding hyperparameters in the first stage, subsequently modifying these hyperparameters for self-regulation during the second stage of generation under new hyperparameter settings. (3) hag surpasses alternative parameter settings such as random and default in most scenarios, demonstrating that hyperparameter-aware instruction tuning empowers the llms to self-regulate the decoding strategy. our approach extends the current llms paradigm in the text generation process, breaking free from the confines of static hyperparameter settings. by endowing llms with the ability to selfregulate, we pave the way for more autonomous and self-regulation model behavior. our main contributions are as follows: ## related work in the realm of regulating language models (llms) for text generation, existing research can be broadly categorized into three types. our research work belongs to category (3), but it involves automatic regulation of the llm's hyperparameters, distinct from ( 1 ) and ( 2 ) which pertain to regulating the output generation through manipulation of the llm's input. in contrast to the existing works in category (3), which involve manual regulation or search strategies for task-specific hyperparameter design, our self-regulation approach not only mitigates the high cost associated with manual hyperparameter selection but also allows the model to provide tailored hyperparameter settings for each distinct input. ## conclusion in this study, our primary focus was on assessing the model's ability for self-regulation, particularly in the decoding hyperparameter domain. departing from previous research that relied on manual settings or search-based approaches, we aimed for the emergence of self-regulation capabilities in the model, allowing it to adjust its own hyperparameter config based on changing tasks or inputs. by drawing inspiration from the self-regulation mechanisms observed in the human body, we introduced a two-stage paradigm called hyperparameter aware generation (hag). this framework enables llms to regulate their decoding hyperparameters autonomously in response to varying tasks and contexts. the comprehensive experiments conducted across scenarios like reasoning, creativity, translation, and mathematics underscored the model's capacity for hyperparameter-aware generation and self-regulation. these results not only demonstrate the feasibility and effectiveness of our approach but also push the boundaries of llm flexibility, opening new horizons for ai-human interactions.