Self-controller: Controlling LLMs with Multi-round Step-by-step Self-awareness

## Abstract

Abstract The applications of large language models (LLMs) have been widely spread across all domains.
However, the basic abilities such as the controllability of LLMs are still limited.
To address this, we propose ” Self-controller ”, a novel agentic framework bringing self-awareness into LLMs’ reasoning logic.
The core idea of this work is to maintain states based on the LLM’s response, letting the LLM become self-aware of current status and think step by step in a multi-round chain-of-thought paradigm.
Our experiment on the state of textual length has shown the controllability and effectiveness of the Self-controller. We further implement a binary search algorithm to accelerate the generation process based on the linearity and monotonicity of the textual length state. Another advantage of the Self-controller comes with DeepSeek’s Context Caching technology, which significantly saves computational token consumption when a cluster of conversations shares the same prefix of context. Theoretically, we prove that in this scenario the extra time complexity is O ⁢ ( c ⁢ log ⁡ n ) 𝑂 𝑐 𝑛 O(c\log n) italic_O ( italic_c roman_log italic_n ) . Results of the back-of-the-envelope estimation suggest that the token consumption of our method is no more than twice as much as that of the trivial single-round generation. Furthermore, our ablation study on word constraints demonstrates the Self-controller’s consistent controllability across all foundation models.

## 1 Introduction

Humans are constantly troubled with self-control issues. Naturally, we ponder the question: do LLMs learning from humans inherit the same characteristics?
The willpower of self-control was first discussed in psychology.
The study called ”the marshmallow test” (Mischel, ), was originally developed in the 1960s. The test’s goal is to observe how children make decisions between waiting 15 minutes longer for two marshmallows they eagerly wanted or settling for just one right away.
The experimental results imply that without accurate reward assessment, the actions of human beings might converge to a local optimum.

The self-awareness mechanism is embedded in the mind. In real practice, to reach a specified goal such as finishing an article with a fixed page size, humans often verify their current progress repetitively.
As a result, self-awareness promotes self-control. Adam and
Eve ”realizes they are naked” in the garden and feels ashamed, starting to wear clothes ever after (Bratcher, ). In the meantime, the advancement of LLMs shows new emergent abilities, demonstrating great potential for mimicking such primary intelligent behaviors. However, building self-awareness upon LLMs to ensure controllability is still underexplored.

Length control is a classical task that reflects controllability in LLMs.
Traditional length control approaches highly rely on trivial instructions, post-hoc verification, and supervised fine-tuning (SFT). These approaches either lack controllability or are heavily resource-consuming.
To address this, we propose a novel framework called ”Self-controller”. The framework comprises a state reflector and a multi-round dialogue session with LLMs. The reflector engages with LLMs in each round and provides state information on precise statistics of the textual length, as shown in Figure 1.
The empirical results on multiple datasets show that our framework can significantly enhance controllability in LLMs, without significant performance degradation. Based on the linearity and monotonicity of the textual length state, we propose a binary search optimization to achieve efficiency. Combined with DeepSeek’s Context Caching technology, we prove that the theoretical extra time complexity is $O(c\log n)$.

Figure: Figure 1: A simplified demonstration of the Self-controller summarizing for ”Oedipus Rex”
Refer to caption: x1.png

To sum up, this paper’s main contributions are as follows:
(1) A novel agentic method ”Self-controller” is proposed for length control, containing a multi-round paradigm;
(2) We propose the conception of states realizing self-awareness, which applies to any variables as long as it follows reductionism;
(3) A binary search algorithm is proposed to speed up the multi-round generation process, reaching an $O(c\log n)$ extra complexity.

## 2 Related Works

#### Large Language Models

In recent years, LLMs’ abilities continually grow. Raffel et al. () revealed the broad application potential of large-scale pre-trained models in natural language processing (NLP) tasks through fine-tuning techniques. Brown () demonstrated that even with limited sample sizes, LLMs can still exhibit excellent performance through autoregressive mechanisms. Chowdhery et al. () further explored the relationship between model size and performance, confirming the advantage of larger models in capturing linguistic complexity. With the advent of models such as ChatGPT and GPT-4 (Liu et al., ; Achiam et al., ), LLMs have drawn even more widespread attention within the field, which can generate text that approaches or even rivals human quality.

#### Self-awareness

There are attempts to assess self-awareness in LLMs. Li et al. () propose a unified taxonomy for awareness in LLMs, including capability, mission, emotion, culture, and perspective. Wang et al. () introduces the knowledge quadrant for multimodal LLMs, knowing knowns and unknowns. To enhance trustworthiness and self-awareness in LLMs, various approaches have been proposed, such as Chain-of-thought (Wei et al., ; Kojima et al., ), Self-consistency (Wang et al., ), ReAct (Yao et al., ), Tree-of-thought (ToT) (Muralidharan & Thomas, ) and Think-Solve-Verify (Liu et al., ). LLMs’ hallucination problems have been alleviated with these fine-grained stepwise generation frameworks, while the accurate output manipulation remains underexplored.

#### Controllability

The trivial method to improve controllability is to include extra zero-shot instructions (such as ”Output in a sonnet style.”) to the prompt (Kojima et al., ).
Post-hoc verification utilizes additional check programs evaluating LLMs’ responses. Chain-of-Verification (CoVe) designs a planning module to raise a set of verification questions and self-corrects them to get the final response afterward (Dhuliawala et al., ). Zhang & Gao () applies a similar idea for news claim verification along with the help of the web search. Recently, generating longer text has drawn attention. Bai et al. () proposes AgentWrite, leveraging ”divide-and-conquer” agents to generate 10,000+ words, while the length control for each paragraph remains trivial.
On the other hand, supervised fine-tuning has been the most prevailing way of ensuring controllability. Juseon-Do et al. () realizes sentence compression with a prompt and improves its performance through instruction-based fine-tuning. Yuan et al. () adds length constraints to the prompt and designs a contrastive proxy task to fine-tune via Reinforcement Learning from Human Feedback (RLHF). Jie et al. () uses reinforcement learning and beam sampling for length control, on GPT-2 models with less than 0.8 billion parameters. Anonymous () embeds length constraints into positional encodings to execute fine-tuning achieving precise length control. The major problems with SFT methods are that they are resource-consuming and lack generalizability.

## 3 Methodology

### 3.1 Overview of the Self-controller

Figure: Figure 2: Demonstration of the Self-controller workflow
Refer to caption: x2.png

The abstract workflow of the Self-controller, shown in Figure 2, contains a state reflector submodule. Given a specified task, the Self-controller will maintain relevant state variables in the state reflector.
In length control, the state variable is the textual length, making transitions from 0 to the requested word constraint $L_{\text{request}}$. The state reflector parses the new LLM response at each round and updates state variables accordingly. The state is then passed as natural words to the LLM generating the next-step response. The reflection procedure will be recycled indefinitely until the framework gets satisfactory results.
In practice, JSON output is necessary for ensuring the stability of the Self-controller, by excluding irrelevant information in the output, especially for weaker foundation models.

### 3.2 Binary Search on the Textual Length State

Based on the linearity and monotonicity of the textual length state, we design the binary-search algorithm for the length control, demonstrated in Algorithm 1. The major difference between the trivial and the binary-search state reflector is the initiative of the state reflector. The trivial state reflector only provides textual length states for the LLM to be self-awareness, including the requesting textual length $L_{\text{request}}$ and the current number of words $\operatorname{len}(S_{\text{output}})$. In binary-search state reflector, the reflector not only provides basic statistics but also actively asks the LLM to output $\frac{1}{2}[L_{\text{request}}-\operatorname{len}(S_{\text{output}})]$ words in the next round, thus executing a binary search on the linear space of $[0,L_{\text{request}}]$. In practice, we set the algorithm to go back to the trivial mode when the surplus textual length is no longer valid for constructing a complete sentence.

Figure: Algorithm 1 Binary Search for Length Control

### 3.3 Context Caching in Multi-round Sessions

Compared with trivial single-round prompts, the cost of the Self-controller is proportional to the total rounds in the muli-round dialogue. An optimization to this is to build contextual caches, thanks to DeepSeek’s Context Caching technique, which is made possible by the MLA architecture in DeepSeek-V2 (DeepSeek-AI et al., ). Context caching builds textual blocks on disks and saves token consumption by an order of magnitude. When duplicate inputs (only prefixes) are detected, the repeated parts will be retrieved from disks, thus bypassing redundant recomputation.

The theoretical token consumption of 3 different methods analysis is shown in Figure 3. The single-round generation is the trivial prompting method. The multi-round generation is the plain implementation of the Self-controller. The binary search multi-round generation is an optimized version with both Context Caching and binary search on top of the plain Self-controller.

Figure: Figure 3: The consumption of 3 different methods: single-round, multi-round, and binary search
Refer to caption: x3.png

Assume the input length is $L_{\text{input}}$, and the requested textual length constraint is $L_{\text{request}}$. The cost of output tokens is $k$ times as much as input tokens. For simplicity, we ignore the difference in token consumption for each method due to prompting variance, and the difference between the textual length and the number of tokens.
For the trivial single-round generation, the overall cost is:

$$ $\text{cost}_{\text{single-round}}=L_{\text{input}}+k\cdot L_{\text{request}}$ (1) $$

For the multi-round generation, we generate one sentence at a time. The generated textual length in each round is $L_{\text{sentence}}$ on average. We denote the number of total rounds as $n=\cfrac{L_{\text{request}}}{L_{\text{sentence}}}$. The overall cost is:

$$ $\displaystyle\text{cost}_{\text{multi-round}}$ $\displaystyle=n\cdot L_{\text{input}}+k\cdot\cfrac{1}{2}\cdot n\cdot(L_{\text{ sentence}}+L_{\text{request}})$ (2) $\displaystyle=n\cdot\left[L_{\text{input}}+\cfrac{k}{2}\cdot(L_{\text{sentence }}+L_{\text{request}})\right]$ (3) $$

For the binary search multi-round generation, the expected generated textual length is proportional to the rest of $L_{\text{request}}$ at each round. $c$ is the cost scaling coefficient due to context caching, meaning the cost of cache-hitting tokens is $c$ times as much as cache-free input tokens ($c<1$). The number of total rounds is $\log n=\log\cfrac{L_{\text{request}}}{L_{\text{sentence}}}$. The overall cost is:

$$ $\displaystyle\text{cost}_{\text{binary-search}}$ $\displaystyle=\left[1+c(\log n-1)\right]\cdot L_{\text{input}}+k\cdot\left[L_{ \text{request}}+c\cdot L_{\text{request}}\cdot\sum_{i=1}^{\log n-1}(1-\cfrac{1 }{2^{i}})\right]$ (4) $\displaystyle=\left[1+c(\log n-1)\right]\cdot L_{\text{input}}+k\cdot L_{\text {request}}\cdot\left[(1+c\left(\log n-1-\sum_{i=1}^{\log n-1}\cfrac{1}{2^{i}} \right)\right]$ (5) $\displaystyle=\left[1+c(\log n-1)\right]\cdot(L_{\text{input}}+k\cdot L_{\text {request}})-k\cdot L_{\text{request}}\cdot c\cdot\sum_{i=1}^{\log n-1}\cfrac{1 }{2^{i}}$ (6) $\displaystyle=\left[1+c(\log n-1)\right]\cdot\text{cost}_{\text{single-round}} -k\cdot L_{\text{request}}\cdot c\cdot\sum_{i=1}^{\log n-1}\cfrac{1}{2^{i}}$ (7) $\displaystyle<\left(1+c\log n\right)\cdot\text{cost}_{\text{single-round}}$ (8) $$

The result shows that the extra time complexity is $O(c\log n)$ compared with the trivial method.
Let’s have a back-of-the-envelope estimation. In practice, long output length such as Bai et al. () can generate $L_{\text{request}}\approx 10,000$ words. On average, the textual length of natural sentences has $L_{\text{sentence}}<20$. For DeepSeek, $c=0.1$. Therefore, we have

$$ $\displaystyle c\log n=c\log\cfrac{L_{\text{request}}}{L_{\text{sentence}}} \approx 0.1\times\log\cfrac{10,000}{20}\approx 0.1\times 8.96<1$ (9) $$

which indicates

$$ $\text{cost}_{\text{binary-search}}<\left(1+c\log n\right)\cdot\text{cost}_{ \text{single-round}}<2\cdot\text{cost}_{\text{single-round}}$ (10) $$

This proves the efficiency of the binary search multi-round approach.

## 4 Experiments

### 4.1 Length Control

In this experiment, 3 datasets are selected from HuggingFace: webis/tldr-17 (short as ”tldr”), argilla/news-summary (short as ”news”), and ccdv/arxiv-summarization (short as ”arxiv”). For each dataset, we sample 128 instances with textual lengths ranging in $[800,1200]$ words.
For foundation models, we choose the following models: GLM series (GLM-4-air, GLM-4-flash), GPT series(GPT-4o, GPT-4o-mini, GPT-3.5-turbo), and Deepseek-V2. The word constraint $L_{\text{request}}$ is 250 words.
Results are within Table 1.

**Table 1: Length control results across different datasets and LLMs (word constraint = 250). Grey and green color represent better performance between single-round and multi-round separately for each model. Box indicates the best result in each column.$|\Delta|$ is the absolute value of the difference between the average output textual length (AVG) and $L_{\text{request}}$. STD is the standard deviation of output textual lengths. And $\text{ratio}=\text{STD}/\text{AVG}$**
|  | Dataset | tldr | news | arxiv |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Model |  | $|\Delta|$ | STD | ratio | $|\Delta|$ | STD | ratio | $|\Delta|$ | STD | ratio |
| GLM-4-flash | single-round | 122.43 | 22.15 | 17.4% | 104.30 | 34.20 | 23.5% | 92.45 | 28.70 | 18.2% |
| multi-round | 58.41 | 95.05 | 49.6% | 62.44 | 94.26 | 50.3% | 46.27 | 78.95 | 38.8% |  |
| GLM-4-air | single-round | 86.61 | 31.62 | 19.4% | 73.77 | 43.83 | 24.9% | 54.06 | 31.31 | 16.0% |
| multi-round | 80.58 | 39.18 | 23.1% | 59.16 | 62.79 | 32.9% | 69.33 | 34.22 | 18.9% |  |
| GPT-3.5 | single-round | 127.09 | 18.23 | 14.8% | 100.09 | 17.69 | 11.8% | 106.73 | 18.35 | 12.8% |
| multi-round | 37.21 | 45.82 | 21.5% | 1.63 | 17.61 | 7.0% | 8.91 | 33.56 | 13.9% |  |
| GPT-4o-mini | single-round | 38.70 | 29.64 | 14.0% | 23.21 | 11.11 | 4.9% | 26.27 | 11.45 11.45 11.45 | 5.1 % percent 5.1 5.1 % |
| multi-round | 6.27 | 23.68 | 9.2% | 10.13 | 7.67 7.67 7.67 | 2.9 % percent 2.9 2.9 % | 8.32 | 20.92 | 8.1% |  |
| GPT-4o | single-round | 16.32 | 12.56 | 5.4% | 0.19 0.19 0.19 | 23.79 | 9.5% | 2.52 2.52 2.52 | 12.71 | 5.1% |
| multi-round | 6.42 | 6.17 6.17 6.17 | 2.4 % percent 2.4 2.4 % | 6.85 | 23.56 | 9.2% | 5.39 | 16.26 | 6.4% |  |
| DeepSeek-chat | single-round | 24.38 | 31.30 | 13.9% | 10.61 | 35.72 | 13.7% | 11.28 | 32.29 | 12.4% |
| multi-round | 2.66 2.66 2.66 | 8.52 | 3.4% | 4.95 | 24.09 | 9.4% | 6.41 | 20.58 | 8.0% |  |

To ensure the textual generation quality, we evaluate the former results on BERTSCORE proposed by Zhang et al. (). We introduce a hypothesis here that the ground-truth reference is the generated text by GPT-4o in single-round respectively. Therefore, the BERTSCOREs of GPT-4o in single-round equals 1 on all datasets. The evaluation results are illustrated in Figure 4, which shows no significant textual quality loss on most models. The degradation and low performance in Table 1 on GLM-4-flash may indicate the inner inability of weaker models. The gap between GPT-4o and all other models may be caused by fine-grained writing styles.

Figure: Figure 4: Average BERTSCORE evaluation across different datasets and LLMs (word constraint = 250)
Refer to caption: x4.png

### 4.2 Binary Search on Length Control

To study the effectiveness of the binary-search method, we select 128 samples with longer textual lengths from the tldr and the arxiv dataset, ranging in $[2000,2500]$ words. The news dataset doesn’t contain enough satisfiable samples thus being excluded in this section.
The word constraint is set to 500 words. The results are shown in Table 2. For most cases, the binary-search method achieves better results evidentially, proving our optimization’s effectiveness. One thing worth mentioning is that DeepSeek-chat maintains strong consistency on all settings both in Table 1 and Table 2.

**Table 2: Length control results across different datasets and LLMs (word constraint = 500)**
|  | Dataset | tldr | arxiv |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- |
| Model |  | $|\Delta|$ | STD | ratio | $|\Delta|$ | STD | ratio |
| GLM-4-flash | single-round | 171.71 | 73.24 | 22.3% | 112.13 | 43.57 | 11.2% |
| binary-search | 22.17 | 92.72 | 19.4% | 31.92 | 99.80 | 21.3% |  |
| GLM-4-air | single-round | 152.63 | 66.99 | 19.3% | 90.55 | 55.50 | 13.6% |
| binary-search | 30.75 | 100.52 | 21.4% | 27.43 | 57.09 | 12.1% |  |
| GPT-3.5-turbo | single-round | 330.47 | 60.41 | 35.6% | 203.71 | 49.46 | 16.7% |
| binary-search | 43.46 | 14.82 | 3.2% | 38.88 | 22.19 22.19 22.19 | 4.8 % percent 4.8 4.8 % |  |
| GPT-4o-mini | single-round | 60.46 | 48.25 | 11.0% | 13.79 | 38.72 | 7.5% |
| binary-search | 16.20 | 7.82 | 1.5% | 12.29 | 114.96 | 23.6% |  |
| GPT-4o | single-round | 52.17 | 49.41 | 11.0% | 22.92 | 40.85 | 8.6% |
| binary-search | 11.47 11.47 11.47 | 6.68 6.68 6.68 | 1.3 % percent 1.3 1.3 % | 3.31 3.31 3.31 | 61.47 | 12.2% |  |
| DeepSeek-chat | single-round | 155.24 | 137.80 | 40.0% | 38.69 | 82.88 | 15.4% |
| binary-search | 28.13 | 40.41 | 8.6% | 11.01 | 28.47 | 5.8% |  |

### 4.3 Ablation Study

#### Efficiency

We examine the efficiency between the multi-round generation and binary search multi-round generation. We choose 64 samples from the tldr dataset ranging in $[2000,2500]$ words, along with the representative word constraints as $[200,400,600,800,1000]$.
Results are illustrated in Figure 5. For weaker LLMs (GPT-3.5-turbo and GLM-4-flash), the efficiency is insignificant in terms of dialogue rounds. For stronger LLMs with moderate instruction following ability, the number of rounds for multi-round generation grows rapidly with the growth of word constraints, while the binary-search version grows slowly at a near log scale. This empirical finding aligns with the theoretical results.

#### Word Constraint

As observed in former experiments, the overall performance of the Self-controller varies according to different word constraint parameters.
In this part, we carefully investigate the influence of word constraints.
we set the word constraint within a dynamic range of $[50,1300]$.
All other settings are equivalent to the ablation study on efficiency.
The results are shown on scatter plots in Figure 6.
This clearly shows that GPT-series are gaining immanent controllability with OpenAI’s pretraining advancement on foundation models, which counteracts the Self-controller’s controlling effect on small constraints ($L_{\text{request}}<500$). However, the results imply that the Self-controller takes effect on all word constraints and all models, demonstrating its substantial generalizability.

Figure: Figure 5: Efficiency study for the binary search multi-round generation method on the tldr dataset
Refer to caption: x5.png

Figure: Figure 6: Ablation study on word constraint
Refer to caption: x6.png

## 5 Discussion

#### Exploring Automatic State Management

Although the states are not confined to any specific task, the self-controller’s prompts still require revision for each new scenario to manage novel states effectively. For future work, we can develop real-time state management based on user intentions, utilizing a society of agents. Inspired by the binary-search optimization, more heuristic reflections may accelerate the convergence of state transitions.

#### Enhancing Prompt Engineering Techniques

Improving output quality is a key area for further development. An additional refinement process for the self-controller’s final output can enhance textual quality and detail expression. For shorter constraints $L_{\text{request}}<500$, advanced foundation models like GPT-4o already perform well. However, longer outputs may benefit from planning techniques such as the divide-and-conquer approach proposed by Bai et al. (). In the future, exploring the integration of planning within the multi-round paradigm of the self-controller could provide global controllability on a book-scale.

#### New Paradigm of Thoughts

Towards the dawn of reasoning, ReAct and ToT represent two paths for future directions on CoT. The major difference between these two paths is the way in which the message is passed. ReAct can reason within a single session, while ToT requires assembly in different sessions. Recently, Diagram-of-thought (DoT) (Zhang et al., ) proposes a similar idea to our Self-controller, executing reasoning in multi-round sessions. With the development of near-infinite input and output length of LLMs, the multi-round sessions can be more informative than a cluster of generated responses in the future.

## 6 Conclusion

In this paper, we introduced the Self-controller, a novel framework designed to enhance the controllability of LLMs by incorporating self-awareness into their reasoning processes. By maintaining state variables and enabling multi-round dialogue sessions, the Self-controller allows LLMs to refine their outputs iteratively, achieving more precise control over tasks such as length control. Our experiments demonstrated that the Self-controller significantly improves the controllability of LLMs across various datasets and models, without compromising the quality of the generated text. Implementing binary search optimization further enhances the efficiency of the multi-round generation process, reducing the computational overhead. The results also suggest that the Self-controller is adaptable to different foundation models and various word constraints, showcasing its generalizability and robustness. Additionally, we explored the potential of the Self-controller in other tasks, such as keyword extraction, indicating its broader applicability beyond length control. Future work will focus on automating state management and exploring more advanced prompt engineering techniques to enhance output quality. Integrating planning techniques within the multi-round paradigm could also provide global controllability for more complex tasks. In conclusion, the Self-controller represents a significant step towards building more controllable and self-aware LLMs, paving the way for more reliable and versatile applications in natural language processing.

## Peer Reviews

confidence: 4
contribution: 2
initial_rating: 3
presentation: 2
questions: More detailed feedback is listed below. I also include grammatical and stylistic errors but only for the first two sections, and this list is not comprehensive, so I strongly encourage the authors to carefully re-read the manuscript and correct all similar errors. Research questions/comments about the content for all sections are included below.

Line 29: "Humans are constantly troubled with self-control issues."
  How so? Example? Citation?

General: (minor) Use `` for left double quotation marks rather than ".

Line 33: "The experimental results imply that
without accurate reward assessment, the actions of human beings might converge to a local optimum"
  How so? Do all children end up picking one option if the delay is not set appropriately?

Line 37: Adam and Eve, being non-historical figures, don't really serve as good supporting evidence for the claim that self-awareness is important in human intelligence.

Line 37: "realizes" -> "realize" (grammatical number agreement)

Line 40: "building self-awareness upon LLMs"
  It isn't totally clear what the precise definition of self-awareness and self-control is.

Line 49: "DeepSeek’s Context Caching"
  Needs citation.

Line 50: "extra time complexity is O(c log n)"
  What is c? What is n?

Line 101: "the accurate output manipulation" -> "accurate output manipulation" or "the accurate manipulation of output"

Line 108: "the web search" -> "web search"

Line 145: "initiative of the state reflector"
  What does "initiative" mean here?

Section 3.3 and Figure 3: What does "consumption" mean here? What is the cost measured in, here? Number of prompts? Number of forward passes/FLOPs? Reading further, I think the cost is in terms of the numbed of input and output tokens (as would be charged in the use of an API for an LLM). But would there be any difference in cost if the LLM is running locally (i.e., in terms of number of forward passes)? I think further clarification is required, as well as a comparison with the cost in the non-API setting.

Why only test on text lengths between 800 and 1200? Why not try much shorter and longer lengths? Similarly, why not try shorter lengths in the binary search experiments?

Are there any experiments validating the use of BERTSCORE in this task? For example, human evaluation of a subsample of outputs?

Table 2: Why not compare binary-search with multi-round?

Line 423: "dawn of reasoning"
  This is highly vague and abstract. I assume the authors are referring to the time at which prompting methods were developed that significantly enhanced LLM performance on reasoning-intensive tasks. I strongly suggest re-wording and avoiding the use of "dawn."

Line 427: "near-infinite"
  Though context sizes have increased, they are still quite a ways off from infinite. If what was intended was that the context size is sufficiently large for most inputs, then I suggest rewording this to be more descriptive, and avoiding comparisons to infinity.
soundness: 2
strengths: - The authors present a new prompting approach that guides LLMs to more accurately meet the length constraint in a length-constrained generation task.
- The proposed method is relatively simple and could theoretically be extended to other tasks that would benefit from the model's awareness of task state information.
summary: The authors present Self-controller, a multi-round prompting approach to more efficiently guide LLMs to generate text with length constraints. The authors claim their framework can improve the self-awareness and self-control of LLMs.

The authors apply Self-controller on the task of length-constrained text generation, but not to any other tasks, and so it is not known whether their approach would generalize to other tasks and actually improve the self-reflection of LLMs more generally. How does high performance on this task imply self-awareness? Wouldn't multi-round prompting without binary search alone be sufficient to demonstrate "self-awareness"?

The prompting approach is described as a binary search, but this analogy isn't very clear: What is the item that the search algorithm is aiming to find?
weaknesses: - The claim that the proposed approach facilitates self-reflection is overly broad. The authors only demonstrate an improvement on the task of length-constrained generation, and it is not known to what extent their results generalize to other tasks that benefit from self-reflection.
- The cost model is assumed to be that of an LLM service via an API, where the cost of each prompt is given by the number of input tokens plus output tokens (possibly with each input token being more/less expensive than an output token). But this is not explicitly stated, and there is no comparison to the cost in a local setting, where the model's cost is more determined by the number of forward passes.
- The prompting approach is described as a binary search, but this analogy is unclear/imprecise.

---

confidence: 3
contribution: 2
initial_rating: 5
presentation: 3
questions: Questions:

1.	Can authors elaborate on the novelty of their method as compared to the existing length control baselines? Is the state reflector module the main novelty?

2.	Can authors elaborate a little bit more on the hypothesis mentioned in row 276? What is the purpose of this hypothesis?

3.	Why is it that the weaker models such as GLM-4-flash and GLM-4-air do worse in multi-round than in the single round? (See Figure 4)
soundness: 3
strengths: Strengths:

1.	The use of a binary search algorithm combined with the state reflector for length control shows an effective approach for optimizing response generation in LLMs.

2.	Context caching method reduces the cost for multi-round sessions, increasing the real-world applicability.
summary: In this paper, the authors propose a novel framework called Self-controller for length control in LLMs. The main idea in the paper is is to maintain states based on the LLM’s response, letting the LLM become self-aware of current status and think step by step in a multi-round chain-of-thought paradigm. More specifically, the framework comprises a state reflector and a multi-round dialogue session with LLMs. The reflector engages with LLMs in each round and provides state information on precise statistics of the textual length. Based on the linearity and monotonicity of the textual length state, the authors propose a binary search optimization to achieve efficiency. The authors also obtain theoretical guarantees for the extra time complexity.
weaknesses: Weaknesses:

1.	It seems like the weaker models such as GLM4-flash and GLM-4-air do not benefit much from the proposed multi-round strategy.

2.	The methodology is novel, however, I believe the contribution is not enough for an ICLR acceptance. Please see the question below.

---

confidence: 3
contribution: 2
initial_rating: 3
presentation: 2
questions: The double quotation marks looks strange in the paper, the author may change the double quotation marks.
soundness: 3
strengths: 1. The Self-controller introduces a novel idea by applying self-awareness mechanisms to control LLM behavior in a multi-round step-by-step manner. 
2. The paper offers a detailed explanation of the methodology and providing empirical evidence of the system’s effectiveness across different models and datasets.
3. The paper is generally well-written and structured, with clear diagrams and explanations.
summary: The paper presents Self-controller, a novel framework for improving the controllability of large language models by incorporating self-awareness into their reasoning processes. The framework maintains state variables and iteratively refines outputs through multi-round dialogue sessions, allowing LLMs to adjust their responses dynamically. It also leverages a binary search algorithm for textual length control, enhancing efficiency in generating content of a specific length. Furthermore, the DeepSeek’s Context Caching technology is integrated to optimize token consumption, reducing computational overhead. The framework demonstrates effectiveness in controlling text length across various datasets and models without significant degradation in text quality.
weaknesses: 1. This paper lacks of the comparison with other baselines. The authors need to add more baselines (as described in related work) to compare, this could better show the effectiveness of the proposed method.
2. This paper lacks of the qualitative examples demonstrations, which could make the reader better understand the task and method. I suggest the authors put some examples to the appendix at least.

---

confidence: 3
contribution: 2
initial_rating: 3
presentation: 2
questions: N/A
soundness: 2
strengths: This method can effectively control the output length of LLMs.
summary: This paper proposes 'self-controller,' a new framework that brings self-awareness into LLMs' reasoning logic. Specifically, the authors use multiple rounds of conversations to make the output length of LLMs more controllable. In addition, a binary search algorithm is implemented along with DeepSeek's context caching technology to reduce the complexity of the proposed method. Experimental results show that the proposed methods can better control the output length without significant textual quality loss.
weaknesses: 1. I feel this paper is over-claiming. Controllability of LLMs is a very broad research topic, but the study in this paper mainly focuses on controlling the output length of LLMs, which is a very small part of this research area. This narrows down the applications of the proposed methods to other areas, such as sentiment control, keyword control, etc.

2. There are basically no baselines except for the trial single-round baseline. Again, more controllability aspects of LLMs should be studied to demonstrate the effectiveness of the proposed methods.