Paper: Word Importance Explains How Prompts Affect Language Model Outputs (ICLR-2024)

Reviewer comment (Experiments):
The rationale for selecting a specific model, such as the FlagEmbedding model "BAAI/bge-large-en," is not adequately explained. It is crucial to provide a clear justification for choosing this particular model over others, highlighting its relevant features, performance, or suitability for the research objectives. By providing a comprehensive rationale, readers can better understand the motivations behind the model selection and its implications for the study.

Author response:
Justification for Model Selection: We understand your concern regarding the rationale for selecting the FlagEmbedding model "BAAI/bge-large-en". We further investigated the choice of embedding models and decided to pick “all-MiniLM-L6-v2” for the embedding model in our experiments. We agree that a wider study on the impact of embedding models could be valuable research but was not part of the scope of this study.

Reviewer comment (Reproducibility):
The explanation of the Scoring and Impact Calculation method lacks clarity. It is essential to provide a detailed and step-by-step description of how the scoring and impact calculation process works. This should include the specific metrics used, the mathematical formulas or algorithms employed, and any relevant considerations or assumptions. A clear and explicit explanation of this methodology will ensure that readers can comprehend and replicate the calculations performed.

Author response:
Clarity in Scoring and Impact Calculation Method: We agree that our initial explanation of the scoring and impact calculation method was not sufficiently clear. We have now revised this section to include a step-by-step description of the process, complete with the specific metrics used, mathematical formulas, and any assumptions made. This should enhance the readability and reproducibility of our methods.

Reviewer comment (Evaluation):
The dataset used in the study is generated by GPT4. Merely relying on a dataset generated by GPT4 may not sufficiently capture the range of subjective opinions on explainability. Including a user study or evaluation process would provide valuable insights into the perceptions and interpretations of explainability, enhancing the robustness and validity of the research findings.

Author response:
Dataset and Model Diversity: Your point about the limitations of solely using a dataset generated by GPT4 is well-taken. In response, we have supplemented our dataset with human-generated data and expanded our experiments to include an open source model. This addition enhances the robustness and validity of our findings.

Reviewer comment (Presentation):
The algorithm chart provided in the paper is blurry and difficult to read. It is essential to ensure that all visual elements, such as charts or diagrams, are of sufficient quality and clarity to convey the intended information effectively.

Author response:
Improvement in Visual Elements: We acknowledge that the algorithm chart in our initial submission was not of sufficient quality. We have replaced this chart with a high-resolution version and ensured that all other visual elements in the paper are clear and effectively convey the intended information.

Reviewer comment (Reproducibility):
Could you give more details or an example of how the score is calculated in the Scoring and Impact Calculation part of the proposed method?

Author response:
Clarity in Scoring and Impact Calculation Method: We agree that our initial explanation of the scoring and impact calculation method was not sufficiently clear. We have now revised this section to include a step-by-step description of the process, complete with the specific metrics used, mathematical formulas, and any assumptions made. This should enhance the readability and reproducibility of our methods.

Reviewer comment (Evaluation):
The depth of the experiments conducted in the study is extremely limited as only three metrics which cover Flesch Ease, word count, and topic similarity (cosine embedding) have been explored.

Author response:
Depth of Experiments and Model Variation: You rightly pointed out the limited scope of our experiments and the exclusive use of GPT-3.5 Turbo. In response, we have expanded our experimentation to include Llama2-13B and tested both models on our artificial data but also using questions from SQuAD 2. This ensures a more diverse and representative analysis, addressing the concern of bias towards OpenAI products.

Reviewer comment (Experiments):
There is no ablation or in-depth exploration. This form of limitation needs to be addressed for inclusion to ICLR.

Author response:
Exploration of Linguistic Scoring Features and Embedding Models: In this study, we did not explore whether certain linguistics scoring features are better than others as depending on the context and application one would be preferable over others. That said, we expanded our experiments to include an ablation study of embedding models.

Reviewer comment (Experiments):
There are several obvious limitations of the proposed methodology involving masking each word in the prompt. The method seems to be not practical for prompts that are considerably long, which is realistically common in most interdisciplinary fields. This should be discussed thoroughly in the paper.

Author response:
Methodological Limitations: We acknowledge the limitations in our initial methodology involving masking each word in the prompt. We have further explained the rationale for keeping non-essential words (e.g., stopwords) from the iteration process. However, to reduce computational expenses, users can automatically exclude all stopwords from the analysis. We also briefly discuss the potential of using our approach in a hierarchical way by first masking larger parts of the prompt.

Reviewer comment (Experiments):
For given words that are obviously non-important (e.g., stopwords), it would be computationally expensive and impractical to still iterate and compute the importance of these words in the prompt. The proposed methodology seems to have no workaround for optimization and compression.

Author response:
Methodological Limitations: We acknowledge the limitations in our initial methodology involving masking each word in the prompt. We have further explained the rationale for keeping non-essential words (e.g., stopwords) from the iteration process. However, to reduce computational expenses, users can automatically exclude all stopwords from the analysis. We also briefly discuss the potential of using our approach in a hierarchical way by first masking larger parts of the prompt.

Reviewer comment (Novelty):
The paper is basically prompt engineering as it optimizes the quality of generation based on some measure of word importance. The author should explicitly mention this as it directly aligns with the task covered by the paper. It would also help other researchers discover similarities with works on optimizing prompts / explainable prompts in general.

Author response:
Clarification on Prompt Engineering: We appreciate your observation regarding prompt engineering. We have revised our manuscript to explicitly mention and discuss how our study aligns with the field of explainability but can also contribute to the field of prompt optimization. Authors are working on another study with a focus on prompt optimization but due to the limitation on number of pages, the authors decided to solely focus on the word importance technique in this paper.

Reviewer comment (Theory):
How does using embeddings capture topics? The method only captures semantic relatedness as it only uses cosine similarity. Also, why the FlagEmbedding model? What’s the justification for using this specifically?

Author response:
Exploration of Linguistic Scoring Features and Embedding Models: In this study, we did not explore whether certain linguistics scoring features are better than others as depending on the context and application one would be preferable over others. That said, we expanded our experiments to include an ablation study of embedding models.

Reviewer comment (Evaluation):
The paper introduces “word count” as a measure of deviation. It is unclear to me how this is motivated, i.e., how a change in word count related to an LLM generation reflects the importance of a word that has been removed in its input.

Author response:
Methodology of "Word Count" as a Measure: Your critique regarding the use of "word count" as a measure of deviation was well-received. As we have highlighted in the paper, the word importance method is metric agnostic but we have revised the paper to provide a clearer rationale for its inclusion, elaborating on why adding this text score adds value to our analysis.

Reviewer comment (Evaluation):
To measure word importance, the paper uses absolute values of Flesch reading-ease and topic similarity. However, both metrics are directional in that an increase or decrease after perturbing the input is informative. Absolute values of such deviations should therefore not be used.

Author response:
Use of Absolute Value for Flesch Reading-Ease and Topic Similarity Metrics: It is worth highlighting that we believe both values of the score can be used. In the case of the example provided in Figure 2 of the paper, we use relative scores to compare across metrics. However, in general, we recommend using absolute values because it provides practical information, such as information on the impact a word has on the output.

Reviewer comment (Presentation):
The presentation can be improved. For example, there is a Figure in page 4 with a very low resolution and no caption. Page 5 states “refer to the appendix” without explicitly stating which section/paragraph is meant.

Author response:
Improvements in Presentation: We have addressed the specific issues you highlighted in our presentation. The schematic illustration of the algorithm before Section 4 has been replaced with a high-resolution version and is now accompanied by a descriptive caption. Additionally, references to the appendix have been made explicit, with clear indications of the relevant sections/paragraphs.

Reviewer comment (Experiments):
What was the motivation for using an LLM-generated dataset as opposed to one consisting of human-written texts?

Author response:
Dataset and Bias Concerns: We acknowledge your concern about the potential bias in our experiments due to the dataset being generated with an LLM. In response, we have now included a widely used dataset comprising human-written samples to ensure a more balanced and representative analysis. This addition aims to strengthen the validity of our results when applied to human-generated data.

Reviewer comment (Novelty):
The contribution of the paper is limited, similar topics have been investigated before while this paper didn’t pose any more valuable conclusions.

Author response:
Dear Reviewer,

Thank you for your detailed and constructive comments regarding our manuscript titled "Word Importance Explains How Prompts Affect Language Model Outputs". We have carefully considered your feedback and made revisions to address the concerns outlined in your review. Below, we detail how we have addressed each specific point you raised:
- Contribution and Novelty: We understand your concern regarding the perceived limited contribution of our study. In response, we have expanded our discussion to better highlight the novel aspects of our research and its implications in the field of explainable AI. We have included additional references to prior work to clarify how our study provides new insights and advances the current state of knowledge.

Reviewer comment (Presentation):
All the tables are also hasty and careless.

Author response:
Dear Reviewer,

Thank you for your detailed and constructive comments regarding our manuscript titled "Word Importance Explains How Prompts Affect Language Model Outputs". We have carefully considered your feedback and made revisions to address the concerns outlined in your review. Below, we detail how we have addressed each specific point you raised:
- Improvement in Presentation: We have taken your feedback regarding the presentation of our paper seriously:
Tables: We have revised all tables to ensure they are concise, clear, and informative.

Reviewer comment (Writing):
The term LLM lacks its full name in the abstract part.

Author response:
Dear Reviewer,

Thank you for your detailed and constructive comments regarding our manuscript titled "Word Importance Explains How Prompts Affect Language Model Outputs". We have carefully considered your feedback and made revisions to address the concerns outlined in your review. Below, we detail how we have addressed each specific point you raised:
- Improvement in Presentation: We have taken your feedback regarding the presentation of our paper seriously:
Terminology Clarification: We have added the full name and explanation of all acronyms, including LLM, in the abstract and relevant sections of the paper.