Paper: Unveiling and Manipulating Prompt Influence in Large Language Models (ICLR-2024)

Reviewer comment (Theory):
The TDD method is based on multiple assumptions listed at the bottom of page 4. Some assumptions are strong without any theoretical and experimental proof, especially the first and second assumptions.

Author response:
[**W1**] The TDD method is based on multiple assumptions listed at the bottom of page 4. Some assumptions are strong without any theoretical and experimental proof, especially the first and second assumptions.

[**Q1**] We are sorry for the insufficient clarity in articulating the theoretical and empirical foundations for the assumptions presented on page 4. In response, we reassess the assumptions underlying TDD from both theoretical and empirical perspectives.

**Theoretical Justification**

Geva et al. (2021; 2022) have theoretically demonstrated that in GPT2, token representations at each layer can be projected as evolving distributions over the vocabulary. Considering the consistent decoder-only transformer architecture in contemporary autoregressive language models (LLMs) such as GPTJ and LLaMA2, this theoretical analysis is extendable to these models as well since their mathematical derivations are exactly the same. This supports our first assumption/ finding: the concept of “token representations at each layer as evolving distributions” is applicable across various LLMs.
However, their empirical study only focuses on GPT2. To validate the generalization empirically, we conducted experiments across various LLMs and datasets.

**Empirical Justification**

Detailed descriptions of the experiments and their settings are provided in Appendix A. 

1) Finding 1: The theory of “token representations at each layer as evolving distributions over the vocabulary” can be generalized across various LLMs.
Given an input text, we first visualize the token distributions at each layer projected by the LM head for various LLMs including GPT2, GPTJ, BLOOM, Pythia and LLaMA2 in Figure 2 in Appendix A. Using LLaMA2 in Figure 2e as an example, for the initial input “ Some”, intermediate layers produce outputs such as “Someone” and “Somewhere”. With the input “Some boys discover”, outputs like “Some boys discover themselves” emerge. In the upper layers, the prediction logit incrementally rises, converging to the final prediction. These visualizations underscore that token representations from any layer can be depicted as meaningful token distributions over the vocabulary. Furthermore, this notation can be applied to all input tokens, instead of the only last token for prediction.

To quantitatively assess the dynamics of token distributions across layers, we treat the token distribution at the final layer as the “ground-truth”. We then compute the KL-divergence between intermediate layer token distributions and this ground-truth to study their convergence patterns. We employ datasets from BLiMP to estimate the KL-divergence for each token. Figure 2f depicts the mean divergence across layers for all input tokens in all datasets. It can be observed that the learning process reflects the evolving token distributions over the vocabulary, which converge in a roughly monotonic manner to the distribution of the final layer.

Both qualitative and quantitative analyses corroborate our first finding regarding the generalizability of token representation theory across various LLMs.

2) Finding 2: Projected token distributions from the LM head hold the potential to elucidate causal relationships between prompts and LLM outputs.
Our first finding confirms that the LM head projects complex token representations at each layer as distributions over the vocabulary. Advancing this concept, we observe that these distributions are both interpretable and integral to the model's generative process. Each dimension of the token distribution represents a specific token in the vocabulary, with the value of each logit indicating the likelihood of the subsequent token based on current and preceding tokens. This capability of the LLM to track prediction or distribution changes with the progressive introduction of tokens enables us to infer which tokens influence the prediction of a target token, offering explanatory insights. Thus, our second finding is that projected token distributions from the LM head hold the potential to elucidate causal relationships between prompts and LLM outputs.

The experiments detailed in Section 4 further validate our assumptions and findings. These experimental results show that utilizing the token distributions projected by the LM head yields the most faithful explanations. This substantiates our assertion that the LM head possesses the potential to clarify causal relationships between prompts and LLM outputs.

Detailed analysis and experiments are provided in Section 3.2 and Appendix A in our revised paper.

Reviewer comment (Evaluation):
For controllable text generation, this paper uses simple meaningless space tokens as alternative tokens in toxic language mitigation. In sentiment direction, this paper uses simple key tokens “positive” or “negative” as alternative tokens. Although this can prove that the method in this paper has potential application value in controllable text generation tasks, this method is too simple for practical applications. And whether such a way will impact the reasonableness of the evaluation of the results.

Author response:
[**W2**] For controllable text generation, this paper uses simple meaningless space tokens as alternative tokens in toxic language mitigation. In sentiment direction, this paper uses simple key tokens “positive” or “negative” as alternative tokens. Although this can prove that the method in this paper has potential application value in controllable text generation tasks, this method is too simple for practical applications. And whether such a way will impact the reasonableness of the evaluation of the results.

[**A2**] Thanks for highlighting your concerns. We would like to address your concerns by clarifying three key points:

1) Our scenario is **zero-shot** controlled text generation, distinct from general controlled text generation. The zero-shot setting is complex yet common in real-life applications. In scenarios with limited data and GPU availability, our method offers a practical solution by treating LLMs as black-box models without requiring training samples or extensive computational resources. TDD is particularly effective in zero-shot contexts, and it significantly outperforms other baselines, demonstrating its practicality and effectiveness.

2) To ensure reasonableness, we have added comprehensive causal analyses. These include experiments like swapping target and alternative tokens, randomly assigning importance to tokens, and applying different explanation methods while maintaining consistency in other operations. These experimental results, detailed in Appendix K, confirm that TDD's enhanced control over text generation primarily stems from its ability to accurately identify token saliency, which is in line with our primary contribution of this paper: taking the token distribution as a new medium to estimate token saliency.

3) The controlled text generation experiments are designed to evaluate TDD's ability to identify key words in input prompts that influence LLM outputs. The main contributions of this paper are threefold: firstly, introducing token distributions as a novel medium for assessing token saliency; secondly, proposing TDD as a technique to leverage token distribution dynamics for understanding prompt influence on LLM generations; thirdly, demonstrating two practical applications of TDD - zero-shot toxicity suppression and sentiment steering. Our findings in Section 5 confirm TDD's efficacy in pinpointing pivotal tokens in prompts for manipulating LLM outputs in zero-shot scenarios.

Consequently, we believe that the contribution of TDD is significant, addressing a challenging yet common application scenario. We deeply value your feedback and, in response, are committed to further refining TDD for a wider range of applications and more complex scenarios in our future work.

Reviewer comment (Evaluation):
In controllable text generation tasks, only automatic evaluation is used, manual evaluation is lacking, and the indicators used in automatic evaluation are relatively simple. It is difficult to fully reflect the quality of the generated text, especially when there is no ground truth for the task of this paper.

Author response:
[**W3**] In controllable text generation tasks, only automatic evaluation is used, manual evaluation is lacking, and the indicators used in automatic evaluation are relatively simple. It is difficult to fully reflect the quality of the generated text, especially when there is no ground truth for the task of this paper.

[**A3**] We apologize for excluding manual evaluations. In the revised version of our paper, we have incorporated manual evaluation metrics and results, as detailed below.

In alignment with prior studies (Liu et al., 2021; Yang et al., 2023), we randomly selected 300 prompts and their corresponding generations for evaluation. Three annotators assessed each generation based on two criteria: the text quality of the generated sentences and the presence of the target attribute. These aspects were rated on a scale from 1 to 5, with higher scores indicating better performance.

Table 12 in Appendix L summarizes the results of human evaluations. While achieving similar quality scores of the generated texts by all methods, TDD achieves the highest scores in terms of attribute control, underscoring its superior performance in comparison to baseline methods.

This revision is provided in Section 5.3 and Appendix L.

Reviewer comment (Experiments):
LLMs can already complete controllable text generation tasks well, such as by giving LLMs toxic prefixes and designed prompts, making LLMs generate non-toxic text. This paper lacks a comparison with them, which may reduce the application value of this work.

Author response:
[**W4**] LLMs can already complete controllable text generation tasks well, such as by giving LLMs toxic prefixes and designed prompts, making LLMs generate non-toxic text. This paper lacks a comparison with them, which may reduce the application value of this work.

[**A4**] Thank you for highlighting this oversight. We apologize for not previously providing detailed justifications for our baseline selections. In response, we have expanded our comparison to include a broader range of baselines.

1) While prefix-tuning is a common approach for controlled text generation, it necessitates training samples to train the prefix tokens. This requirement does not align with our focus on zero-shot controlled text generation. Hence, we opted not to include baselines derived from prefix-tuning methods in our analysis.

2) Designed prompts for controlled text generation such as prompting for style transfer can be utilized in the zero-shot scenario. We employ two methods: 1) Style Prompting (SP), which integrates specific style requirements, such as toxicity and sentiment, directly into the input prompts; 2) augmented Style Prompting (ASP), which serves as a formidable baseline by prompting the model with various examples of sentence rewriting techniques, thereby facilitating a range of sentence rewriting tasks. Detailed information regarding these two prompting strategies can be found in Section 5.1 and Appendix M. 


The efficacy of these methods in zero-shot toxic language suppression and sentiment manipulation is presented in Tables 3 and 4 in our revised paper. These results reveal that SP and ASP are less effective compared to other state-of-the-art baselines like WORDFILTER and FFNControl. This indicates models like GPT-2 struggle with comprehending these prompts, although they may be more effective for larger LLMs such as GPT-4.

This revision is provided in Sections 5.1 and 5.2.

Reviewer comment (Novelty):
The attribution of the prompt for the next token prediction is a simple problem that has been explored in many text classification tasks. The SOTA method such as integrated gradients (IG) should be also considered as a baseline. Furthermore, I think the attribution of the prompt for the sequence of generated text is a more practical task than the current one.

Author response:
[**W5**] The attribution of the prompt for the next token prediction is a simple problem that has been explored in many text classification tasks. The SOTA method such as integrated gradients (IG) should be also considered as a baseline. Furthermore, I think the attribution of the prompt for the sequence of generated text is a more practical task than the current one.

[**A5**] Thanks for your valuable feedback. In response, we have incorporated additional SOTA baselines, including Gradient * Input (GI) and Gradient Norms (GN), into our analysis. The experimental results, summarized in Table 2, reveal that both GI and GN are outperformed by their contrastive versions Con-GI and Con-GN. Importantly, our proposed TDD method continues to significantly surpass all these baselines in performance.

To the best of our knowledge, we are the first to explore token distribution as a medium to estimate token saliency. TDD can be applied to various scenarios including 1) a single target token and a single alternative token, 2) only target tokens, 3) multiple target tokens and multiple alternative tokens, and 4) controlled attributes in generated sequences. More details can be found in Appendix B.2 in our revised paper.

Therefore, while the application of TDD to entire sequences has not been extensively discussed or explored in this paper, we have provided a thorough analysis and conducted detailed experiments at the token level across various scenarios. We are grateful for your insightful feedback and plan to investigate the application of TDD to sequence-level explanations in our future research endeavors.

Reviewer comment (Experiments):
If it is in controllable text generation under multi-attribute control, what is the scalability and performance of this method?

Author response:
[**Q1**] If it is in controllable text generation under multi-attribute control, what is the scalability and performance of this method?

[**A6**] Thank you for raising this important question. Our current research is focused on zero-shot controlled text generation, specifically targeting single-attribute control. Multi-attribute control, indeed, presents a more complex challenge, primarily due to the increased dimensionality and potential conflicts between attributes. While our current study does not extend to this area, we recognize its significance in the broader context of text generation. In future work, we aim to explore strategies such as hierarchical attribute control or advanced learning algorithms to manage the intricacies of multi-attribute control. These approaches could potentially enhance scalability and improve performance in handling multiple attributes simultaneously, which we believe is a valuable direction for the evolution of text generation technologies.

Reviewer comment (Evaluation):
In the task of controllable text generation, the data used in this paper does not have ground truth, and the existing evaluation results cannot fully reflect the quality of the generated text. How does the method in this paper perform under manual evaluation?

Author response:
[**Q2**] In the task of controllable text generation, the data used in this paper does not have ground truth, and the existing evaluation results cannot fully reflect the quality of the generated text. How does the method in this paper perform under manual evaluation?

[**A7**] Please refer to our response [A3] to weakness 3 [W3].

Reviewer comment (Experiments):
The TDD methods heavily depend on the alternative word $w_a$. The datasets chosen in this paper (those BLiMP datasets) provide sentence pairs with exactly one-word differences. Other datasets may not have such well-defined alternative words. This greatly limits the potential applicability of the proposed TDD methods.

Author response:
Thank you for highlighting your concerns. We apologize for any lack of clarity regarding the scope of TDD in our paper. Actually, **TDD's effectiveness is not strictly dependent on alternative tokens. TDD is versatile and can be applied in various scenarios, including those with 1) a single target token and a single alternative token, 2) only target tokens, 3) multiple target tokens and multiple alternative tokens, and 4) controlled attributes in generated sequences.** We substantiate each of these applications with experimental evidence as follows:

**Scope of TDD**

1) Single Target and Single Alternative Token: Section 4 details our main experiments, which validate TDD's effectiveness in scenarios involving a single target token and a single alternative token.

2) Target Token Only: TDD is applicable even in the absence of alternative tokens. By assigning a zero probability to alternative tokens in Equations 2 and 4 in the paper, we can negate the necessity of alternative tokens. In the added experiments, we estimate each token’s importance by using only target tokens, and the alternative tokens are not provided. The results, summarized in Table 5, show TDD's significant outperformance of strong baselines—by about 3%—in scenarios without alternative tokens. 

3) Multiple Target and Alternative Tokens: TDD accommodates scenarios involving multiple target and alternative tokens by aggregating their probabilities in Equations 2 and 4. The effectiveness of TDD in these settings was assessed through experiments using AG’s News and SST2 datasets, with details provided in the Appendix B.2.3 of our revised paper. These results in Table 6 demonstrate TDD’s effectiveness for multiple target and alternative tokens.

4) Generated Sequence Attribute Control: The experiments described in Section 5, focusing on toxic language suppression and sentiment steering, further confirm TDD's applicability in controlling attributes of generated sequences.

More details are available in Section 3.6 and Appendix B.2 in our revised manuscript.

**Defined Target and Alternative Tokens in Section 5**

For zero-shot toxic language suppression, we utilized a predefined list of toxic words from WORDFILTER as target tokens, treating all other tokens as alternatives. In this context, considering all other tokens as alternatives is equivalent to having no alternatives.
For zero-shot positive sentiment steering, we identified negative words from SenticNet as target tokens, using positive words as alternative tokens. This approach allows TDD to identify key words that could lead to negative outcomes and suppress them. To generate negative responses, we simply reverse the roles of the target and alternative tokens, while keeping other settings constant.

Reviewer comment (Reproducibility):
A related point: it is unclear to me how the $w_a$ in the Section 5 experiments are identified.

Author response:
Thank you for highlighting your concerns. We apologize for any lack of clarity regarding the scope of TDD in our paper. Actually, **TDD's effectiveness is not strictly dependent on alternative tokens. TDD is versatile and can be applied in various scenarios, including those with 1) a single target token and a single alternative token, 2) only target tokens, 3) multiple target tokens and multiple alternative tokens, and 4) controlled attributes in generated sequences.** We substantiate each of these applications with experimental evidence as follows:

**Scope of TDD**

1) Single Target and Single Alternative Token: Section 4 details our main experiments, which validate TDD's effectiveness in scenarios involving a single target token and a single alternative token.

2) Target Token Only: TDD is applicable even in the absence of alternative tokens. By assigning a zero probability to alternative tokens in Equations 2 and 4 in the paper, we can negate the necessity of alternative tokens. In the added experiments, we estimate each token’s importance by using only target tokens, and the alternative tokens are not provided. The results, summarized in Table 5, show TDD's significant outperformance of strong baselines—by about 3%—in scenarios without alternative tokens. 

3) Multiple Target and Alternative Tokens: TDD accommodates scenarios involving multiple target and alternative tokens by aggregating their probabilities in Equations 2 and 4. The effectiveness of TDD in these settings was assessed through experiments using AG’s News and SST2 datasets, with details provided in the Appendix B.2.3 of our revised paper. These results in Table 6 demonstrate TDD’s effectiveness for multiple target and alternative tokens.

4) Generated Sequence Attribute Control: The experiments described in Section 5, focusing on toxic language suppression and sentiment steering, further confirm TDD's applicability in controlling attributes of generated sequences.

More details are available in Section 3.6 and Appendix B.2 in our revised manuscript.

**Defined Target and Alternative Tokens in Section 5**

For zero-shot toxic language suppression, we utilized a predefined list of toxic words from WORDFILTER as target tokens, treating all other tokens as alternatives. In this context, considering all other tokens as alternatives is equivalent to having no alternatives.
For zero-shot positive sentiment steering, we identified negative words from SenticNet as target tokens, using positive words as alternative tokens. This approach allows TDD to identify key words that could lead to negative outcomes and suppress them. To generate negative responses, we simply reverse the roles of the target and alternative tokens, while keeping other settings constant.

Reviewer comment (Evaluation):
The analyses presented in this paper are not really causal analyses, despite claims that the LM head “elucidate causal relationships between prompts and LLM outputs” (page 4). No controlled variable is specified, and no treatment effect is measured. This paper could be much stronger if some sort of causal scores are computed, for example, how large is the treatment effect of changing $w$ to $w_a$ causes the toxicity of the generated texts. Currently, the numbers shown in Tables 3 and 4 look similar to the treatment effects, but without more descriptions about how the variables are controlled, we can’t really say the numbers are treatment effects.

Author response:
We appreciate your insightful feedback. In response, we have expanded our causal analysis by conducting three distinct experiments and analyses to assess treatment effects: 1) Swapping target and alternative tokens; 2) Randomly assigning importance to each input token; 3) Employing varied explanation methods while other operations remain the same. These strategies are designed to confirm that the superior generation-control performance of TDD primarily stems from its ability to effectively identify token saliency for explanations, rather than from other extraneous factors, such as the mere substitution of an input token with a space token.

Table 10 in Appendix K presents results for toxic analyses utilizing these three strategies.  Table 11 shows the results of sentiment steering using the three strategies. The token swapping (TDD-CTA), random saliency allocation (TDD-RA) and other explanation methods including Con-GI and Con-GN are significantly outperformed by TDD. These results verify that the superior generation-control performance of TDD primarily stems from its ability to effectively identify token saliency for explanations, rather than from other extraneous factors.

The revisions and details are available in Appendix K in our revised paper.

Reviewer comment (Experiments):
In section 5, which variant of TDD is used? Are TDD-forward used there?

Author response:
We apologize for the lack of clarity regarding our choice of TDD variants for the controlled text generation. In our application experiments in section 5, we specifically chose the TDD-bidirectional variant for both toxicity suppression and sentiment steering. This decision was based on its superior performance observed in our main experiments, which are thoroughly discussed in Section 4. We have added such clarification in our revised manuscript.

Reviewer comment (Writing):
The difference between the TDD variants is not well discussed. While the applications are very interesting, the authors could have used the space to elaborate on the differences between the introduced variants.

Author response:
We appreciate your feedback and have incorporated detailed discussions about the TDD variants and its use in our steering experiments.

1. In our steering experiments, we specifically chose the TDD-bidirectional variant, as detailed in the revised manuscript. This decision was based on its superior performance observed in our main experiments, which are thoroughly discussed in Section 4.

**2. Detailed Variant Discussion.** 

a. Calculation Direction: The TDD-forward evaluates token importance sequentially from the first to the last input token. Conversely, TDD-backward assesses saliency in reverse, from the last input token to the first. TDD-bidirectional, drawing inspiration from bidirectional neural networks, combines saliency estimates from both TDD-forward and TDD-backward.

b. Information Perspective: TDD-backward is less influenced by linguistic conventions, offering a more targeted approach. The TDD-forward overlooks the inherent linguistic influence of individual tokens. Consider the sentence completion by the LLM: “ This design is quite novel and fantastic. I really ___”, our aim is to explain why the LLM generates “like” instead of “hate”. Linguistic conventions render it improbable to predict tokens such as “like” or “hate” following “is” or “novel”, indicating potential inaccuracies in token contribution assessments. TDD-backward can mitigate this issue. Specifically, in TDD-backward, the words “I really ___” are fed to the LLM in the first two iterations. Subsequently, other tokens, such as “fantastic”, are progressively introduced This initial phase sharpens the model's focus, enhancing its accuracy in predicting words such as “like” or “hate” as the LLM consistently generates predictions following the phrase "… I really __" in each iteration.

**3. Application Scenarios** TDD-forward is preferable in time-sensitive or computationally constrained scenarios, as it requires only one forward propagation due to the auto-regressive structure of LLMs. TDD-backward and TDD-bidirectional are better suited for contexts where time is not a constraint, and there is a higher demand for explanation fidelity. They demand iterations equal to the input length for saliency estimation. The revised paper’s Table 2 showcases the higher faithfulness of explanations by TDD-backward and TDD-bidirectional, while Table 9 confirms the computational efficiency of TDD-forward. 

More details are provided in Appendix B.1 and G in our revised paper.

Reviewer comment (Presentation):
Captions Table 1 and Table 3 are too sparse.

Author response:
Thanks for pointing this out. We have revised the captions for Tables 1, 3 and 4 accordingly in our modified manuscript.

Reviewer comment (Novelty):
While not being a contrastive XAI method, important related work on explainability for autoregressive LMs missing: ATMAN: Understanding Transformer Predictions Through Memory Efficient Attention Manipulation. Björn Deiseroth, Mayukh Deb, Samuel Weinbach, Manuel Brack, Patrick Schramowski, Kristian Kersting. In Proceedings of NeurIPS 2023

Author response:
Thank you for your insightful feedback. We apologize for the oversight regarding the omitted reference and have addressed this in our revised manuscript. To clarify, the primary distinctions between our TDD method and the AtMan approach are as follows:

1. Saliency Estimation Medium: AtMan evaluates input saliency by iteratively manipulating the attentions of each input token and observing changes in cross-entropy. In contrast, TDD utilizes the token distribution as the medium for calculating token saliency.

2. Inference Process: AtMan introduces noise at the beginning of forward propagation, whereas TDD performs inference directly at the end of propagation, after the forward propagation process is complete.

3. Parameter Dependence: TDD has the advantage of not requiring any hyperparameters. On the other hand, AtMan necessitates setting hyperparameters, such as the suppression factor and the lower bound of cosine similarity, which can vary across different datasets.

The revision can be found in Section 2.1 in our revised paper.

Reviewer comment (Experiments):
One simple way to control LLMs' generations is via prompting for style transfer, e.g., ask the model to transform the outputs into "less toxic" content, or "positive/negative" sentiment. How would this baseline compare to the proposed TDD?

Author response:
We appreciate the valuable suggestion offered. In response, we have incorporated two robust baselines from the field of text style transfer aimed at regulating the toxicity and sentiment in LLM outputs.

We compare TDD with two baselines from this field. Style Prompting (SP) (Reif et al., 2022) integrates specific style instructions like toxicity into prompts, while Augmented Style Prompting (ASP) (Reif et al., 2022) enhances this approach by introducing varied rewriting examples for a broader application. Detailed information regarding these two prompting strategies can be found in Appendix H and M.

In our revised manuscript, Tables 3 and 4 detail the experimental results. SP records a toxicity of 0.47, ASP attains 0.41, whereas TDD achieves a notably lower score of 0.20, significantly outperforming SP and ASP. In terms of negative sentiment steering, TDD surpasses both SP and ASP by a substantial margin of about 0.30. For positive sentiment steering, TDD exceeds SP and ASP by more than 0.20, underscoring its considerable advantage over style transfer methods.

Reviewer comment (Experiments):
The models used in experiments are relatively smaller models (maximum size 7B), how would the proposed approach work on larger models (e.g., llama-2 13B, 70B)? What is the computation cost (efficiency, memory) for running TDD over larger models?

Author response:
Thank you for your insightful feedback. In response, we have expanded our research to include experiments on explanation faithfulness using larger models such as LLaMA2-13B and OPT-30B to ensure its scalability. Additionally, we have included a comparison of the computational costs for both our methods and the baseline approaches. The key findings are summarized below.

**Scalability**. We conduct experiments with LLaMA2-13B and OPT-30B (Zhang et al., 2022) to assess the effectiveness of TDD in explaining larger models. The summarized results in Table 8 in the revised manuscript reveal that TDD-forward outperforms the baselines by margins of 3.15% in AOPC and 4.4% in Suff using LLaMA2-13B. Both TDD-backward and TDD-bidirectional demonstrate superior performance, exceeding the baselines by more than 4% in AOPC and over 5% in Suff. For OPT-30B, TDD surpasses competitive baselines by a significant margin of 4% to 6%. These results prove TDD's scalability and effectiveness in larger models.

**Computation Cost**. For the computational cost analysis, we evaluate the average memory usage and processing time required by our method for processing a single input sample. Consistency is maintained across all experiments, which are conducted on an NVIDIA RTX A5000 GPU. For models larger than 6 billion parameters, their 4-bit versions are utilized. Detailed memory and time metrics are presented in Table 9 in the paper. Regarding memory consumption, Rollout and the three TDD variants (TDD-forward, TDD-backward, and TDD-bidirectional) are the most efficient. In terms of processing time, TDD-forward and Rollout emerge as the fastest, whereas TDD-backward and TDD-bidirectional exhibit slightly longer processing time.

More details are provided in Section 4.6 and Appendix F, G in our revised manuscript.

Reviewer comment (Experiments):
Could the authors add a simple baseline by prompting the LLMs for style transfer on the outputs?

Author response:
We appreciate the valuable suggestion offered. In response, we have incorporated two robust baselines from the field of text style transfer aimed at regulating the toxicity and sentiment in LLM outputs.

We compare TDD with two baselines from this field. Style Prompting (SP) (Reif et al., 2022) integrates specific style instructions like toxicity into prompts, while Augmented Style Prompting (ASP) (Reif et al., 2022) enhances this approach by introducing varied rewriting examples for a broader application. Detailed information regarding these two prompting strategies can be found in Appendix H and M.

In our revised manuscript, Tables 3 and 4 detail the experimental results. SP records a toxicity of 0.47, ASP attains 0.41, whereas TDD achieves a notably lower score of 0.20, significantly outperforming SP and ASP. In terms of negative sentiment steering, TDD surpasses both SP and ASP by a substantial margin of about 0.30. For positive sentiment steering, TDD exceeds SP and ASP by more than 0.20, underscoring its considerable advantage over style transfer methods.

Reviewer comment (Experiments):
What is the computation cost (efficiency, memory) for running TDD over larger models?

Author response:
Thank you for your insightful feedback. In response, we have expanded our research to include experiments on explanation faithfulness using larger models such as LLaMA2-13B and OPT-30B to ensure its scalability. Additionally, we have included a comparison of the computational costs for both our methods and the baseline approaches. The key findings are summarized below.

**Scalability**. We conduct experiments with LLaMA2-13B and OPT-30B (Zhang et al., 2022) to assess the effectiveness of TDD in explaining larger models. The summarized results in Table 8 in the revised manuscript reveal that TDD-forward outperforms the baselines by margins of 3.15% in AOPC and 4.4% in Suff using LLaMA2-13B. Both TDD-backward and TDD-bidirectional demonstrate superior performance, exceeding the baselines by more than 4% in AOPC and over 5% in Suff. For OPT-30B, TDD surpasses competitive baselines by a significant margin of 4% to 6%. These results prove TDD's scalability and effectiveness in larger models.

**Computation Cost**. For the computational cost analysis, we evaluate the average memory usage and processing time required by our method for processing a single input sample. Consistency is maintained across all experiments, which are conducted on an NVIDIA RTX A5000 GPU. For models larger than 6 billion parameters, their 4-bit versions are utilized. Detailed memory and time metrics are presented in Table 9 in the paper. Regarding memory consumption, Rollout and the three TDD variants (TDD-forward, TDD-backward, and TDD-bidirectional) are the most efficient. In terms of processing time, TDD-forward and Rollout emerge as the fastest, whereas TDD-backward and TDD-bidirectional exhibit slightly longer processing time.

More details are provided in Section 4.6 and Appendix F, G in our revised manuscript.