CELL your Model: Contrastive Explanations for Large Language Models

## Abstract

Abstract The advent of black-box deep neural network classification models has sparked the need to explain their decisions. However, in the case of generative AI, such as large language models (LLMs), there is no class prediction to explain. Rather, one can ask why an LLM output a particular response to a given prompt. In this paper, we answer this question by proposing, to the best of our knowledge, the first contrastive explanation methods requiring simply black-box/query access. Our explanations suggest that an LLM outputs a reply to a given prompt because if the prompt was slightly modified, the LLM would have given a different response that is either less preferable or contradicts the original response. The key insight is that contrastive explanations simply require a scoring function that has meaning to the user and not necessarily a specific real valued quantity (viz. class label). We offer two algorithms for finding contrastive explanations: i) A myopic algorithm, which although effective in creating contrasts, requires many model calls and ii) A budgeted algorithm, our main algorithmic contribution, which intelligently creates contrasts adhering to a query budget, necessary for longer contexts. We show the efficacy of these methods on diverse natural language tasks such as open-text generation, automated red teaming, and explaining conversational degradation.

## 1 Introduction

Generative artificial intelligence (AI) has rapidly transformed society and will continue to do so for the foreseeable future, albeit in ways we do not yet know. Thusfar, it has impacted how people conduct their jobs (e.g., code generation for software engineers [], text summarization for lawyers [] and doctors []) as well as how people conduct their daily activities (e.g., rewriting emails, seeking advice, or designing vacation itineraries). As AI has advanced over the last two decades, so did the need for explaining how the AI was making decisions (e.g., why was a customer denied a bank loan or why was an image classified as a pedestrian crossing a street). Such explanations are even the topic of regulations in the USA with the AI Bill of Rights [] and in Europe with the GDPR [] and the recent EU AI Act [].

Much has been done in explainable AI typically regarding classification and regression (see surveys Guidotti et al. [] and Yang et al. []) mostly focusing on black-box models, e.g., deep neural networks. Explanation methods vary and include attribution methods such as LIME [], SHAP [], and saliency [], and contrastive explanations such as CEM [] and CAT [].

Figure: Figure 1: Contrastive explanations for natural language generation by meta-llama/ llama-2-13b-chat. Colors match what is changed between input and contrastive prompts. These explanations suggest that the input prompt generated the input response because if the highlighted changes were made, the new contrastive prompt would generate a different response which contradicts the input response. Prompts taken from the Moral Integrity Corpus []
Refer to caption: x1.png

The focus of this paper is contrastive explanations for Large Language Models (LLMs). In the typical classification setting, contrastive explanations dictate that a classifier predicted label $y$ on sample $x$ because if $x$ was slightly modified to be $x_{c}$, the classifier would have predicted label $y_{c}$ instead. In the case of LLMs, there is no classifier and the output is a sequence of words. While explainability is a well-studied area for classifiers, explanations for LLMs are still limited. A recent method, MExGen [], derives LIME and SHAP methods for LLMs based on mask infilling while TextGenSHAP [] speeds up SHAP for LLMs using “speculative decoding”, but these are attribution methods which highlight words in the prompt to maintain the response (not change it) also requiring real valued representations of the response. Such attribution methods are complimentary to our proposal of a contrastive method; they are more restrictive as they can explain only individual tokens or need more information when explaining entire responses.

To the author’s knowledge, this paper offers the first contrastive explanation methods for LLMs. Consider the examples in Figure 1. Given an input prompt that is fed to an LLM, we ask why the LLM output a particular response. Our methods create perturbations of the input prompt, called contrastive prompts, which when fed to the LLM result in a contrastive responses that differ from the input response in some user defined manner, e.g., a contrastive response that contradicts the input response. In the top example, the contrastive explanation dictates: the LLM responded with ways to avoid financial ruin such as diversifying investments because if the prompt had asked about *other* people instead of *rich* people, it would have responded with financial advice for the average person.

*The key insight here is that contrastive explanations simply require a scoring function that has meaning to the user and not necessarily a specific real valued quantity (viz. class label).* Moreover, given that input prompts may have large contexts (viz. in Retrieval Augmented Generation (RAG)), we also propose an approach that can efficiently find contrasts with a limited number of calls to the black-box model, something that is not considered in previous works.

Contributions. We propose the first methods to generate contrastive explanations for LLMs: a myopic algorithm that is effective for small prompts and a budgeted algorithm that scales for large contexts. We demonstrate quantitatively that these algorithms are effective and practical. Finally, we offer two new use cases for contrastive explanations: red teaming and conversational AI, showcasing the efficacy of our methods in varied applications.

## 2 Related Work

Danilevsky et al. [] considered explainability for natural language processing, primarily for classification tasks where local explanations were provided, among which our focus is on post-hoc methods that explain a fixed model’s prediction. One large group of explainability methods are feature based where the explanation outputs some form of feature importance (i.e., ranking, relevance, etc.) of the words in text [, , , , , ]. Other types of local post-hoc explanations include exemplar based [, , ] that output similar instances to the input.

Among local methods, our focus is on contrastive methods [, , , ]. Contrastive explanations are complementary to attribution and exemplar style explanations [] discussed above as they provide ways to realistically manipulate the input in a minimal manner so as to change the output. In our setup, we want to modify the input prompt so that an LLM produces an output with a different user specified quality or characteristic (viz. fairness, preference, etc.). The latter distinguishes our work from prior contrastive explanation works which are mainly for the classification setting.

Another contrastive method POLYJUICE [] is a human-in-the-loop method requiring supervision about the type of modification to be performed to the text such as negation, word replacement, etc. A contrastive latent space method [] does not generate contrastive text, but rather highlights (multiple) words in the input text that are most likely to alter a classification prediction, and is furthermore not a black-box method. Similarly, Yin and Neubig [] highlight words that influence a model predicting a target output instead of a *foil*; this work is related to saliency and uses gradient-based scoring. A few works use LLMs to generate contrastive explanations [, , ] but focus on classification.

## 3 Formulation

We here formulate the contrastive explanation problem for LLMs. Denote by $x_{0}$ an input prompt and $\mathcal{X}$ the space of prompts, i.e., strings. Let $LLM(x)$ be the response of an LLM to prompt $x$. Define $g(x_{0},y_{0},x_{c},y_{c})$ as a scoring function that inputs a prompt $x_{0}$, the initial response $y_{0}=LLM(x_{0})$, a perturbed version of $x_{0}$ denoted as $x_{c}$, and the response to $x_{c}$ denoted as $y_{c}=LLM(x_{c})$. Also denote $f(x_{0},x_{c})$ as a measure of similarity between two prompts $x_{0}$ and $x_{c}$. We formulate the contrastive explanation problem for LLMs as

$$ minimize $\displaystyle\quad f(x_{0},x)$ (1) subject to $\displaystyle\quad g\left(x_{0},x,LLM(x_{0}),LLM(x)\right)\geq\delta$ $\displaystyle\quad x\in\mathcal{X}$ $$

Assuming bounded $\mathcal{X}$, Problem (1) is a combinatorial optimization problem over all possible prompts in $\mathcal{X}$. Note that this generalizes contrastive explanations [] or adversarial attacks [, ] where typically $LLM(\cdot)$ is replaced by a classifier and the constraint is such that the predicted class of $x_{0}$ changes. Contrastive explanation methods further constrain the contrastive explanation, i.e., the solution to (1), to lie on a manifold that maintains it to be a realistic example. In the case of language generation, such constraints will be enforced by infilling masks, i.e., replacing missing word(s).

Similarity: Experiments in this paper measure prompt similarity $f(x_{0},x_{c})$ as the number of mask and infill operations applied to a string $x_{0}$ to obtain string $x_{c}$. Other functions could be considered based on commonly used text similarity metrics such as BLEU or ROUGE. Our choice is selected to focus on minimizing the number of LLM queries made by the algorithms described below.

### 3.1 Scoring Functions

The framework defined by problem (1) requires users to provide a scoring function for their particular usecase. We here formalize the scoring functions used throughout this paper, but note that these are particular to the tasks considered here. As they are task-dependent, scoring functions below can depend on any subset of the inputs $x_{0},y_{0},x_{c},\mbox{and }y_{c}$, as defined above. It is also important to note that these user-defined scoring functions need not be symmetric and can incorporate direction. For example, a preference score defined below incorporates direction (whether preference increases or decreases) but can also be defined by the absolute value of the score instead.

Preference: This scoring function outputs a score defining which of two responses is preferable for a given prompt. Specifically, we use the stanfordnlp/SteamSHP-flan-t5-xl LLM available on HuggingFace [] which is trained to predict how helpful each response is for the prompt. We normalize these scores to act as probabilities, and our preference score is the difference between the two probabilities. Such a scoring function can be used for explaining natural language generation.

Contradiction: This scoring function inputs two responses $y_{1}$ and $y_{2}$ obtained from an LLM. A Natural Language Inference (NLI) model is used to score the likelihood that $y_{1}$ contradicts $y_{1}$, denoted as $p_{1}$, and the likelihood that $y_{2}$ contradicts $y_{1}$, denoted as $p_{2}$. We define the contradiction score as the difference $p_{2}-p_{1}$. Note that $p_{1}$ should be small for a good NLI model but is still computed here to give a reference point. Experiments in this paper use the NLI model cross-encoder/nli-roberta-base available on HuggingFace. Such a scoring function can be used for explaining natural language generation as well as red teaming.

BLEU_SUMM: The BLEU score, between 0 and 1, measures the similarity of two text samples (closer to 1 is more similar). Given two prompts $x_{1}$ and $x_{2}$ and their corresponding responses $y_{1}$ and $y_{2}$, we measure the BLEU score between prompts $x_{1}$ and $x_{2}$, denoted as $a$, and the BLEU score between responses $y_{1}$ and $y_{2}$, denoted as $b$. The BLEU_SUMM score is output as $w_{1}\cdot a+w_{2}\cdot(1-b)$, meaning a higher score is given for having similar prompts and dissimilar responses. Our experiments use $w_{1}=0.2,w_{2}=0.8$ to give more importance to dissimilarity of responses. Such a scoring function can be used for text summarization where contrastive explanations seek minor perturbations to prompts that result in large changes to the summaries.

Conversational maxims: Using the definition of conversational maxims from Miehling et al. [], we define a class of metrics spanning six categories (quantity, quality, relevance, manner, benevolence, transparency) to evaluate conversational turns. Each metric takes as input a *context* (history of turns) and a *response* (the most recent turn) and generates a score on a particular submaxim dimenion using an LLM-based labeling procedure (see Appendix D for details). These metrics can thus be viewed as LLM-as-a-judge metrics. We present examples of evaluating conversational turns with respect to helpfulness, harmlessness, harm, and informativeness (see Figure 8 in the Appendix).

Figure: Figure 2: Illustration of the CELL and CELL-budget algorithms. Both algorithms can be summarized as an iterative process that repeats a) Select substrings of the prompt to search, b) Generate perturbed prompts (mask & infill), c) Generate responses for each perturbed prompt (via the LLM), d) Score each perturbed prompt/response. The main difference between the budgeted method and the myopic method is in the *selection* block – the budgeted method augments the search process with a prompt seed generation step (see Algorithm 1 for details). The budgeted method’s search is an iterative loop subject to an inner loop budget before repeating the prompt seed generation step, whereas the myopic method’s search simply enumerates over substrings.
Refer to caption: x3.png

Figure: Algorithm 1 CELL-budget

## 4 Methods

In this section, we describe two variants of our
contrastive explanation method for large language models (CELL) for searching the space of contrastive examples. In practice, this is done by splitting a prompt into $n$ substrings and searching over the space of all possible masked and infilled subsets of these $n$ substrings. The first algorithm, CELL, is a myopic search over potential substrings to replace. The second algorithm, CELL-budget, involves an adaptive search constrained by a budget on the number of calls to the LLM being explained. Such calls can become expensive due to long documents (e.g., as with text summarization tasks). A key novelty over previous contrastive explanations (for classifiers), such as Chemmengath et al. [], Dhurandhar et al. [], and Madaan et al. [], is the insight to use scoring functions that relate the input prompt to responses generated by modified prompts; this is the essence of defining contrastive explanations for a generator, such as an LLM, versus a classifier.

Both of our methods require the following inputs: an LLM to be explained, a scoring function $g(\cdot,\cdot,\cdot,\cdot)$ as defined above, and an infiller $I(\cdot)$ that receives an input a string with a <mask> token and outputs a string with the <mask> token replaced by new text. Various options exist for the infiller model; these include BERT-based models that replace <mask> with a single word, and BART or T5-based models that replace <mask> with potentially multiple words (allowing for addition or deletion of words in addition to simple substitution). Figure 2 illustrates the general logic common to both methods. Specifically, at each iteration, a list of perturbed prompts are selected and passed to the infiller to generate new perturbed prompts. These prompts are then passed through the LLM to generate corresponding responses. A task-dependent score is computed based on the input prompts and response and the perturbed prompts and responses (or any subset of these prompts and responses). Lastly, the score is used to determine which perturbed prompts to continue searching from until a sufficiently modified contrastive prompt is found.

CELL and CELL-budget split prompts into split_k consecutive words, where split_k is a parameter. Setting split_k=1 splits prompts into individual words. Setting split_k=2 splits prompts into consecutive pairs of words, and so forth. Hence higher split_k results in a smaller search space.

### 4.1 CELL

Our myopic search, CELL, uses the following strategy: An input prompt is first split into $n$ substrings (according to split_k); the contrastive example will be a perturbed prompt that masks and replaces a subset of these $n$ substrings. In the first iteration, each of the $n$ substrings is masked and infilled, the $n$ perturbed prompts are passed through the LLM to generate $n$ responses, and these responses are used to compute $n$ scores. If a response results in a sufficiently large score, the corresponding perturbed prompt and response is deemed the contrastive example; otherwise, the perturbed prompt resulting in the largest score is used as the initial prompt and the same steps are followed on the $n-1$ remaining original substrings. These steps are repeated until either a contrastive example is found or all substrings have been perturbed without finding a contrastive example. Pseudocode for CELL can be found in Algorithm 2 in the Appendix.

### 4.2 CELL-budget

When the search is over a prohibitively large number of substrings, as is typical in text summarization for example, one might be conscientious of how many times the LLM is called. The next algorithm, called CELL-budget, our main algorithmic contribution, explores new perturbations from the input prompt while also exploiting perturbations already made. This algorithm, detailed in Algorithm 1, is inspired by Dhurandhar et al. [] which adaptively samples a continuous search space subject to a budget; their task is to find a trust region that satisfies local explainability properties whereas our task is to find a region that satisfies a score criterion.

Each iteration is broken down into three main blocks: 1) Compute the number of seeds, i.e., prompts, to perturb, 2) Generate seeds, and 3) Search around these seeds (inner loop). Note that each iteration of the inner loop samples a particular number of prompts around those seeds in order to use the total budget. Function NUM_SEEDS could take various forms; one such form (Algorithm 3 in the Appendix) is inspired by optimal sampling from continuous distributions.

Our method deviates from Dhurandhar et al. [] because it is a search over a discrete space. CELL-budget employs a prompt seed-driven approach where some seeds are generated from the initial prompt and others from previously perturbed prompts. This allows the search to explore new perturbations of the initial prompt while also taking advantage of favorable perturbations that were already made (the balance is controlled by the hyperparameter $\alpha$). The search around prompt seeds in the inner loop first samples a fixed number of perturbations around each seed using function SAMPLE_SEEDS (Algorithm 5 in the Appendix) and checks if a contrastive example was found. The next iteration of this inner loop reduces the number of seeds sampled from the current list of perturbed prompts and increases the number of samples taken around each seed. The decrease/increase in seeds/samples focuses more heavily on perturbations more likely to lead to contrastive examples. Function BEST_SUBSET outputs prompt seeds as ordered by $g(\cdot,\cdot,\cdot,\cdot)$.

## 5 Experiments

To the author’s knowledge, explaining LLMs through contrastive explanations is a novel direction for LLM explainability. LLMs have been used to generate contrasts, but that remains a very different task. Without a known comparison, we show how CELL(-budget) performs against a baseline that prompts the LLM being explained for a contrast. Additionally, we demonstrate CELL(-budget) across several performance measures such as the number of model calls made. All experiments were conducted with 1 A100_80gb GPU and up to 64 GB memory.

Datasets and Models: We consider two datasets for the following experiments: the Moral Integrity Corpus (MIC) [] (using 500 prompts) and the Extreme Summarization (XSum) dataset [] ( using 250 documents). Three LLMs are used: meta-llama/Llama2-13b-chat, meta-llama/Llama2-70b-chat-q, and faebook/bart-large-xsum, all available on HuggingFace. Infilling is done using T5-large. All corresponding standard error tables are in the Appendix.

### 5.1 Preference Comparisons

We investigate the quality of the contrast (i.e., the response to the perturbed prompt) compared to one generated by prompting the LLM being explained. A baseline contrast generator is defined by 1) prompting the LLM to generate response $y$ to prompt $x$, and 2) prompting it again to generate a less preferable response to prompt $x$. The template prompt used to generate the less preferable response is: ‘‘Answer the following prompt in one sentence, less than 20 words, and to the point: Give a less preferable response than {} to the prompt: {}.’’ The two {}’s contain the input response and prompt, respectively. This template was finalized after several variations and manual inspections. It is crucial to recall that no baselines exist in previous literature; hence we pursue this baseline as a natural comparison.

Results are shown in Table 1 applying Llama LLMs to prompts from the MIC dataset. Each entry dictates which response is preferred for a given input prompt as measured by the scoring function preference defined in Section 3.1. Table 1 (left) compares responses of the corresponding Llama LLMs to the contrastive responses output by CELL(-budget) and the Table 1 (right) compares the Baseline to CELL(-budget). Each row is for a different value of CELL parameter split_k.

We observe that CELL and CELL-budget produce similar results across different values of split_k, likely due to short prompt lengths in MIC. Importantly, positive numbers mean that the initial Llama responses and Baseline responses were found to be preferable to CELL responses. Corresponding experiments using the contradiction scoring function can be found in the Appendix.

**Table 1: Average preference scores comparing Llama, CELL, and baseline responses. Positive numbers for Llama vs CELL represent a higher preference for responses from Llama than CELL (similarly for Baseline vs CELL). Higher #s (i.e. lower preference for CELL) indicate CELL is better. Budget denotes CELL-budget which shows similar trends to CELL. The positive numbers overall signifies that the initial Llama responses and Baseline responses were found to be preferable to CELL(-budget) responses, which is the desired effect of the algorithms.**
| Scoring function: Preference |  |  |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
| split_k | Llama vs CELL(-budget) | Baseline vs CELL(-budget) |  |  |  |  |  |  |
| Llama2-13b | Llama2-70b | Llama2-13b | Llama2-70b |  |  |  |  |  |
| CELL | Budget | CELL | Budget | CELL | Budget | CELL | Budget |  |
| 1 | 0.33 | 0.32 | 0.33 | 0.33 | 0.17 | 0.16 | 0.17 | 0.17 |
| 2 | 0.34 | 0.35 | 0.34 | 0.35 | 0.19 | 0.22 | 0.21 | 0.23 |
| 3 | 0.34 | 0.35 | 0.33 | 0.35 | 0.21 | 0.25 | 0.22 | 0.26 |

**Table 2: Average edit distances, flip rates, and # model calls comparing CELL vs CELL-budget explaining Llama models. Smaller edit rates, larger flip rates, and smaller # model calls are better.**
| Scoring function: Preference |  |  |  |  |  |  |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| split_k | Average Edit Distance | Average Flip Rate | Average # Model Calls |  |  |  |  |  |  |  |  |  |
| Llama2-13b | Llama2-70b | Llama2-13b | Llama2-70b | Llama2-13b | Llama2-70b |  |  |  |  |  |  |  |
| CELL | Budget | CELL | Budget | CELL | Budget | CELL | Budget | CELL | Budget | CELL | Budget |  |
| 1 | 0.12 | 0.15 | 0.12 | 0.16 | 0.92 | 0.88 | 0.89 | 0.89 | 25.3 | 13.5 | 27.6 | 19.5 |
| 2 | 0.16 | 0.21 | 0.16 | 0.21 | 0.94 | 0.97 | 0.93 | 0.95 | 13.2 | 13.5 | 13.4 | 14.1 |
| 3 | 0.23 | 0.28 | 0.23 | 0.28 | 0.92 | 0.93 | 0.91 | 0.94 | 9.2 | 12.7 | 9.5 | 12.5 |

### 5.2 Contrastive Explanation Properties

We evaluate CELL(-budget) perturbed prompts across 3 properties considered in works on contrastive explanations for classifiers [, ]: flip rate, edit rate, and content preservation. The flip rate measures the percentage of times CELL finds a contrastive explanation. Edit distances compute a word-level Levenstein distance between the input prompt and the contrastive prompt. This is the minimum number of changes (additions, deletions, etc.) to get from one prompt to the other, and we normalize by the number of words in the input prompt. Content preservation quantifies how much content is preserved in the contrastive prompt from the input prompt. Following previous works above, we compute the cosine similarity between prompt embeddings obtained from a bert-base-uncased model.

Figure: Figure 3: Average # model calls for CELL-budget applied to text summarization. Shaded regions denote standard error.
Refer to caption: extracted/5930116/figures/textsumm_modelcalls.png

Table 2 shows edit distances and flip rates using the MIC data with the preference scoring function. Edit distances are comparable to previous literature for explaining classifiers [, ]. Flip rates are lower than in those works where the flip rate is typically $\geq$ 0.95, but this reflects the difficulty in explaining LLMs versus classifiers for which many of the methods are not black-box and have access to gradients for selecting important words. Content preservation was found to be $\geq 0.99$ across all models and scoring functions which is significantly higher than most results seen in Chemmengath et al. [] and Ross et al. [], likely due to the better and more flexible infilling models used here.

Table 2 also shows the average number of model calls made both by algorithms. We observe here that CELL-budget is not always more efficient; CELL requires less model calls and hence is faster than CELL-budget when split_k=3 because of the reduced search space, but CELL-budget is more efficient than CELL when split_k=1 due to the larger search space. Interestingly, the number of model calls is similar for both Llama LLMs. Putting these trends together with those from Table 1 suggests the slightly better quality can sometimes be obtained with the higher split_k=3 which is also more efficient here using CELL rather than CELL-budget. Corresponding experiments using the contradiction scoring function can be found in the Appendix.

Lastly, we consider the number of model calls by CELL(-budget) on longer documents from the Extreme Summarization (XSum) dataset [] in a text summarization task, illustrated in Figure 3. These explanations use the BLEU_SUMM scoring function defined in Section 3.1. Note that other recent explainability works for LLMs [, ], albeit attribution methods, do not report on such efficiency statistics in practice; users typically most value explanation quality. The figure illustrates how the number of model calls vary across different length documents as well as different values of parameter split_k. We only consider CELL-budget because it is specifically designed for use cases with long prompts as in this task. Interestingly, the number of model calls plateaus, illustrating the effectiveness of the budgeted strategy. We also observe that setting split_k $>1$ shows improvement on large texts (due to a reduced search space), but statistically, we do not see reason to go beyond split_k $=2$ here.

Figure: Figure 4: Red teaming examples on business conduct guidelines. Colors match between what is changed between input and contrastive prompts. The top example finds conflicting responses about being allowed to perform consulting services according to whether the services are for a competitor or not. The phrase ‘‘is a competitor" is modified to ‘‘is not part" and elicits a response that contradicts the input response. In the bottom example, changing the combined phrases ‘‘If you are a non-ABC company" to ‘‘If you have a company" insinuates that the prompter is an ABC employee and the new response pertains to compliance dealing with antiboycott requirements which does not answer the question. These are considered successful red teaming examples because the contrastive responses are either incorrect (top) or erratic (bottom).
Refer to caption: x4.png

## 6 Use Cases

We next demonstrate CELL-budget on two use cases, automated red teaming and explaining conversational degradation. Both use budgets of 100 queries. These use cases demonstrate two practical uses of contrastive explanations for LLMs.

Figure: Figure 5: Example explanations for conversational degradation. Colors match between what is changed between input and contrastive prompts (restricted to the assistant). The top example presents a helpful assistant response (input), which, after modifying the assistant’s turn in the prompt (made less relevant), results in a significantly less helpful response (contrastive). The bottom example contains an initial assistant response that recognizes the emergency of the stated situation. Modifying the previous assistant response results in a less urgent response (contrastive), illustrating that the cause of the original urgency was the statement that the roommate was harming themself.
Refer to caption: x6.png

### 6.1 Automated red teaming

With the deployment of LLMs has come the need for checks and balances so that the LLMs do not output incorrect/derogatory/etc. responses. This has led to the use of *red teaming* in which one searches for prompts that lead to such responses. The search has typically been done manually, with humans set up to prompt LLMs and try to *break* them [, ].

Recent works have focused on automated red teaming, using LLMs to generate prompts that lead to improper responses. Perez et al. [] use zero and few-shot generation, as well as finetuning an LLM via RL, to generate harmful prompts. Casper et al. [] add diversity penalties to the RL finetuning and Hong et al. [] further add a novelty penalty. These directions all focus on replacing humans and generating prompts from scratch.

We propose another method for automated red teaming whereby prompts from a test set are perturbed. We use CELL-budget to produce contrastive explanations for a chatbot with the specific goal of perturbing a prompt to elicit a response that contradicts the response of the original prompt (using the contradiction scoring function). To the author’s knowledge, this a novel use of NLI.

Many companies have their own set of publicly available Business Conduct Guidelines (e.g., IBM, Siemens, Boeing, Exxon Mobile, etc.) and some are known to have internal chatbots to answer questions concerning business practices. We conduct red teaming for a chatbot finetuned on mistralai/Mistral-7B-Instruct-v0.2 to a dataset created based on a public company’s Business Conduct Guidelines. Examples are shown in Figure 4 where we refer to the company as ABC (see caption for details and the Appendix for more examples). CELL-budget was used with split_k=3. The key idea here is that minor perturbations to a prompt is still a likely prompt but a response that contradicts the initial response might be an incorrect response.

### 6.2 Explaining conversational degradation

LLMs are approaching levels of sophistication that enable them to maintain coherent conversations with human users. However, current models still suffer from various issues, especially in long-form dialogue, leading to conversational degradation [, ]. Improving a model’s conversational capability requires fine-grained identification of their weak points and subsequently obtaining training data to fill these gaps.

In this section, we illustrate how our contrastive explanations method can be used to help understand why models generate specific responses subject to the conversational context. We define various metrics, adopted from the definitions of conversational degradation from Grice [], Higashinaka et al. [], Galetzka et al. [], Miehling et al. []. Details of the labeling procedure can be found in Appendix D. We explain each label by leveraging CELL-budget to minimally modify the previous assistant turn(s) until the threshold is passed. Fig. 5 illustrates two example conversations, one for helpfulness and the other for harm (see caption for details and the Appendix for more examples). Beyond explanations, the generated contrastive examples produced by our method provide useful data for improving the model’s conversational ability.

## 7 Conclusion

To the author’s knowledge, this paper proposes the first contrastive explanations for large language models. Novel insight into what a contrast should mean regarding LLMs led us to propose two algorithms for generating contrastive explanations: a myopic method that is effective for explaining responses to small prompts and a novel search strategy that takes into account a model query budget.

Our two novel use cases of contrastive explanations explicitly provide actionable explanations. In terms of red teaming, such explanations can be used to debug a chatbot. The top example in Figure 4 could lead a team to investigate the training data for examples where an employee was allowed to consult. In terms of conversational degradation, such explanations could be used to generate training data to improve conversational agents. One might want to generate data where one dimension is modified and the others remain fixed. By explaining the top example in Figure 5 over other submaxims, it could potentially be used as an example of not being helpful while maintaining other submaxims. This also suggests future algorithmic work, where we would like to adapt CELL so that the search explicitly moves in such directions.

## Appendix A Pseudocodes

This section contains several algorithms described in the paper. Algorithm 2 is the pseudocode for CELL. Algorithms 3, 4, and 5 are the helper functions to CELL-budget. One other assumed function for SAMPLE_CENTERS is a function SAMPLE that outputs $m$ random entries from any set $\mathcal{S}$.

Figure: Algorithm 2 CELL

Figure: Algorithm 3 NUM_SEEDS

Figure: Algorithm 4 GENERATE_SEEDS

Figure: Algorithm 5 SAMPLE_SEEDS

## Appendix B Additional quantitative experiments

Additional experimental results are given in this section. Each experiment in the main paper that generated explanations with a preference metric were also conducted with a contradiction metric using NLI model nli-roberta-base. Table 3 corresponds to the results in the Preference Comparisons subsection albeit with the contradiction metric. Table 4 corresponds to the results in the Efficiency subsection, again with the contradiction metric, and finally Table 5 corresponds to the results in the Contrastive Explanation Properties subsection. Similar patterns and trends are seen across all experiments between the preference and contradiction metrics. Standard errors for all experiments in the paper are given by Tables 6 and 7.

## Appendix C Additional qualitative examples

Two additional examples on natural language generation from the MIC data can be found in Fig. 6. Two additional red teaming examples can be found in Fig. 7. Two additional examples on conversational degradation can be found in Fig. 8.

**Table 3: Average preference scores comparing Llama, CELL responses, and a baseline contrastive response. Positive numbers for Llama vs CELL represent a higher preference for responses from Llama than CELL (similarly for Baseline vs CELL). Higher #s (i.e. lower preference for CELL) indicate CELL is better. K refers to the split_k parameter. Bdgt denotes CELL-budget. Contrastive explanations here were generated using a contradiction metric.**
| Metric: Contradiction |  |  |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
|  | Llama vs CELL | Baseline vs CELL |  |  |  |  |  |  |
|  | Llama2-13b | Llama2-70b | Llama2-13b | Llama2-70b |  |  |  |  |
| K | CELL | Bdgt | CELL | Bdgt | CELL | Bdgt | CELL | Bdgt |
| 1 | 0.21 | 0.21 | 0.22 | 0.21 | 0.12 | 0.11 | 0.1 | 0.1 |
| 2 | 0.22 | 0.24 | 0.23 | 0.23 | 0.14 | 0.14 | 0.14 | 0.15 |
| 3 | 0.25 | 0.24 | 0.24 | 0.24 | 0.17 | 0.17 | 0.16 | 0.16 |

**Table 4: Average # model calls and time comparing CELL vs CELL-budget explaining Llama models. K refers to split_k. Smaller #s are better for all metrics. Contrastive explanations here were generated using a contradiction metric.**
| Metric: Contradiction |  |  |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
|  | Average # Model Calls | Average Time (s) |  |  |  |  |  |  |
|  | Llama2-13b | Llama2-70b | Llama2-13b | Llama2-70b |  |  |  |  |
| K | CELL | Bdgt | CELL | Bdgt | CELL | Bdgt | CELL | Bdgt |
| 1 | 38.9 | 28.1 | 45.3 | 31.2 | 175.4 | 143.4 | 267.4 | 178.0 |
| 2 | 20.0 | 22.1 | 21.1 | 23.8 | 104.5 | 130.3 | 136.9 | 160.5 |
| 3 | 13.7 | 18.4 | 14.3 | 20.7 | 99.6 | 81.2 | 102.5 | 136.3 |

**Table 5: Average edit distances and flip rates comparing CELL vs CELL-budget while explaining Llama models. K refers to the split_k parameter. Smaller edit rates and larger flip rates are better. Contrastive explanations here were generated using a contradiction metric.**
| Metric: Contradiction |  |  |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
|  | Average Edit Distance | Average Flip Rate |  |  |  |  |  |  |
|  | Llama2-13b | Llama2-70b | Llama2-13b | Llama2-70b |  |  |  |  |
| K | CELL | Bdgt | CELL | Bdgt | CELL | Bdgt | CELL | Bdgt |
| 1 | 0.15 | 0.17 | 0.16 | 0.18 | 0.74 | 0.67 | 0.67 | 0.6 |
| 2 | 0.23 | 0.23 | 0.24 | 0.24 | 0.74 | 0.79 | 0.75 | 0.77 |
| 3 | 0.31 | 0.31 | 0.34 | 0.33 | 0.7 | 0.8 | 0.67 | 0.75 |

**Table 6: Standard errors of average preference scores comparing Llama, CELL responses, and a baseline contrastive response, both using preference and contradiction as the metric. Results generated from 500 prompts taken from the Moral Integrity Corpus (test split). K refers to the split_k parameter which controls how many consecutive words are masked together. Bdgt denotes CELL-budget.**
| Metric: Preference |  |  |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
|  | Llama vs CELL | Baseline vs CELL |  |  |  |  |  |  |
|  | Llama2-13b | Llama2-70b | Llama2-13b | Llama2-70b |  |  |  |  |
| K | CELL | Bdgt | CELL | Bdgt | CELL | Bdgt | CELL | Bdgt |
| 1 | 0.01 | 0.01 | 0.01 | 0.01 | 0.02 | 0.02 | 0.02 | 0.02 |
| 2 | 0.01 | 0.01 | 0.01 | 0.01 | 0.02 | 0.02 | 0.02 | 0.02 |
| 3 | 0.01 | 0.01 | 0.01 | 0.01 | 0.02 | 0.02 | 0.02 | 0.02 |
| Metric: Contradiction |  |  |  |  |  |  |  |  |
|  | Llama vs CELL | Baseline vs CELL |  |  |  |  |  |  |
|  | Llama2-13b | Llama2-70b | Llama2-13b | Llama2-70b |  |  |  |  |
| K | CELL | Bdgt | CELL | Bdgt | CELL | Bdgt | CELL | Bdgt |
| 1 | 0.01 | 0.01 | 0.01 | 0.01 | 0.02 | 0.02 | 0.03 | 0.02 |
| 2 | 0.01 | 0.01 | 0.01 | 0.01 | 0.02 | 0.02 | 0.02 | 0.02 |
| 3 | 0.01 | 0.01 | 0.01 | 0.01 | 0.02 | 0.02 | 0.02 | 0.02 |

**Table 7: Standard errors of average # model calls and average time (top two tables) and edit distances and flip rates (bottom two tables) comparing CELL vs CELL-budget on prompts from the Moral Integrity Corpus, both using preference and contradiction metrics. K refers to the split_k parameter which controls how many consecutive words are masked together.**
| Metric: Preference |  |  |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
|  | Std Error # Model Calls | Std Error Time (s) |  |  |  |  |  |  |
|  | Llama2-13b | Llama2-70b | Llama2-13b | Llama2-70b |  |  |  |  |
| K | CELL | Bdgt | CELL | Bdgt | CELL | Bdgt | CELL | Bdgt |
| 1 | 0.93 | 0.4 | 1.24 | 0.62 | 5.6 | 2.57 | 9.43 | 2.81 |
| 2 | 0.48 | 0.4 | 0.49 | 0.43 | 2.83 | 2.35 | 2.78 | 2.71 |
| 3 | 0.32 | 0.4 | 0.4 | 0.37 | 3.33 | 2.65 | 2.55 | 2.34 |
| Metric: Contradiction |  |  |  |  |  |  |  |  |
|  | Std Error # Model Calls | Std Error Time (s) |  |  |  |  |  |  |
|  | Llama2-13b | Llama2-70b | Llama2-13b | Llama2-70b |  |  |  |  |
| K | CELL | Bdgt | CELL | Bdgt | CELL | Bdgt | CELL | Bdgt |
| 1 | 1.63 | 0.72 | 2.09 | 0.78 | 8.46 | 4.39 | 14.68 | 4.88 |
| 2 | 0.87 | 0.72 | 0.87 | 0.73 | 5.45 | 4.6 | 6.36 | 5.36 |
| 3 | 0.64 | 0.61 | 0.59 | 0.66 | 5.48 | 3.01 | 4.87 | 4.74 |
| Metric: Preference |  |  |  |  |  |  |  |  |
|  | Std Error Edit Distance | Std Error Flip Rate |  |  |  |  |  |  |
|  | Llama2-13b | Llama2-70b | Llama2-13b | Llama2-70b |  |  |  |  |
| K | CELL | Bdgt | CELL | Bdgt | CELL | Bdgt | CELL | Bdgt |
| 1 | 0.0 | 0.01 | 0.0 | 0.0 | 0.01 | 0.01 | 0.01 | 0.01 |
| 2 | 0.01 | 0.01 | 0.01 | 0.01 | 0.01 | 0.01 | 0.01 | 0.01 |
| 3 | 0.01 | 0.01 | 0.01 | 0.01 | 0.01 | 0.01 | 0.01 | 0.01 |
| Metric: Contradiction |  |  |  |  |  |  |  |  |
|  | Std Error Edit Distance | Std Error Flip Rate |  |  |  |  |  |  |
|  | Llama2-13b | Llama2-70b | Llama2-13b | Llama2-70b |  |  |  |  |
| K | CELL | Bdgt | CELL | Bdgt | CELL | Bdgt | CELL | Bdgt |
| 1 | 0.01 | 0.01 | 0.01 | 0.01 | 0.02 | 0.02 | 0.02 | 0.02 |
| 2 | 0.01 | 0.01 | 0.01 | 0.01 | 0.01 | 0.02 | 0.02 | 0.02 |
| 3 | 0.01 | 0.01 | 0.01 | 0.01 | 0.02 | 0.02 | 0.02 | 0.02 |

Figure: Figure 6: Additional contrastive explanations for natural language generation by llama-2-13b-chat. Colors match between what is changed between input prompt and contrastive prompt. These explanations explain that the Input Prompt generated the input response because if the modifications were made to the input Prompt, the new contrastive prompt would generate the contrastive response instead, which contradicts the input response (as per NLI model cross-encoder/nli-roberta-base). Prompts taken from the Moral Integrity Corpus [].
Refer to caption: x8.png

Figure: Figure 7: Additional red teaming examples on business conduct guidelines. Colors match between what is changed between input prompt and contrastive prompt. The top example shows that changing ‘‘Should you register" to ‘‘Would you allow" elicits a misleading response because one would want the chatbot to disclose that registering the device is an option as dictated by the initial response. In the bottom example, while the contrastive response suggests that one could use the analysis if it was obtained in a legitimate way, it seems very suspicious that one could ever receive an analysis of a competitor’s pricing in a legitimate way. The input response is better suited even to answer the contrastive prompt in this case.
Refer to caption: x10.png

Figure: Figure 8: Additional examples of explanations for conversational degradation. The top example presents a simple assistant response (in input response) but after modifying the assistant’s first of two turns to make the assistant seem more *excited* and part of the event, the resulting response is more informative. The bottom example illustrates an example where changing a single word can alter the response to decrease helpfulness (since a different question is answered). This is often possible on simple question/answer type scenarios.
Refer to caption: x12.png

## Appendix D Evaluating Conversational Degradation

Detecting conversational degradation requires monitoring subtle changes in a conversation’s flow (i.e., sentiment and meaning across multiple turns). This requirement largely precludes the use of standard (prompt-response) score functions. As a result, we employ a synthetic labeling pipeline that uses a separate LLM to generate a score for a given turn, based on a natural language description.

To label turns, we create scoring rubrics that reflect the submaxims of Miehling et al. []. These scoring rubrics are constructed by first describing (in natural language) the requirement of each submaxim then including in-context examples to aid the model with the labeling task. The turn to evaluate is then appended to the prompt. We use mistralai/Mixtral-8x7B-Instruct-v0.1 to generate the labels. Additionally, we query the model multiple times, and average the resulting scores, to obtain a more robust label. Some sample prompts for helpfulness, harm, and informativeness are presented in Figs. 9, 10, and 11, respectively.

Figure: Figure 9: Sample prompt for evaluating *helpfulness* (model: mixtral-8x7b- instruct-v01).
Refer to caption: x14.png

Figure: Figure 10: Sample prompt for evaluating *harm* (model: mixtral-8x7b-instruct-v01).
Refer to caption: x15.png

Figure: Figure 11: Sample prompt for evaluating *informativeness* (model: mixtral-8x7b- instruct-v01).
Refer to caption: x16.png

## Peer Reviews

confidence: 4
contribution: 2
initial_rating: 3
presentation: 2
questions: See the questions highlighted in the "Weaknesses" section.
soundness: 2
strengths: 1. The paper provides a clear formulation for finding the contrastive explanation. Such a clear formulation allows many search algorithms to be applied rather than simply relying on heuristics. I think the formulation also has the potential to be leveraged in other setups.
2. The method part of the paper is well-written. Also, it's commendable that the paper takes the search budget into account by proposing CELL-budget. From my perspective, designing algorithms to jointly optimize performance and cost is a very important direction for LLM-in-the-loop search algorithms as these algorithms are inherently much more expansive than traditional search algorithms, and high costs may limit the usage of the algorithm.
summary: This paper proposes a contrastive explanation method for large language models that require simply black-box/query access.The method requires a scoring function and formulates finding the query/prompt for contrast as a search problem. Concretely, the paper offers two algorithms for finding contrastive explanations, (1) a myopic algorithm, and (2) a budgeted algorithm which creates contrasts adhering to a query budget. The paper further demonstrates the potential use cases of the contrastive explanation method, including automated red teaming and explaining conversational degradation.
weaknesses: 1. One major weakness of this paper is that the setup and takeaways for the Experiments section is not very clear. The major question I have is how to define the success of finding contrastive explanations. In Section 5.1, the results use the scoring function preferences, which characterize the pairwise preference between the given explanation and the baseline explanation generated by LLM through direct prompting. However, given the pairwise comparison model (i.e., stanfordnlp/SteamSHP-flan-t5-xl) is not specifically trained to provide preference for such explanation, **I am wondering whether the scores output by the model have high correlation with human judgment**. As a follow-up question, **what's the criterion to evaluate the contrastive explanation if we ask humans to evaluate it**. I think defining what is a good explanation is essential and non-trivial. With this concern, even though Section 5.2 analyze the properties of contrastive explanation algorithms (i.e., flip rate, edit rate, and content preservation), I don't exactly know how to interpret the results or what are the actionable takeaways since the definition of contrastive explanations for LLMs is not exactly the same with the contrastive explanations for classifiers.
2. It's commendable that in Section 6, the authors provide concrete use cases for the proposed contrastive explanations in the context of LLMs. However, especially for the first use case, automated red teaming, I don't know **what's the advantage for using the contrastive explanation compared to using other red teaming**. Providing results on comparing using contrastive explanation and other red teaming approaches could strengthen the argument. For the second use case, the paper mentions "beyond explanations, the generated contrastive examples produced by our method provide useful data for improving the model’s conversational ability", **I am wondering if you can elaborate on this or is there any proof-of-concept result**.

---

confidence: 5
contribution: 2
initial_rating: 3
presentation: 2
questions: What is the rationale for considering four scoring functions?
soundness: 2
strengths: The paper is easy to follow. The proposed methods are simple and straightforward for generating contrastive explanations for LLMs.
summary: This paper focuses on contrastive explanations for Large Language Models (LLMs). Two methods have been proposed for generating contrastive explanations: 1) a myopic algorithm called CELL that searches over potential substrings in the input prompt to replace, and 2) a budgeted algorithm called CELL-budget, which conducts an adaptive search constrained by a budget on the number of LLM calls. Both methods are evaluated using three models (Llama2-13b-chat, Llama2-70b-chat-q, and bart-large-xsum) on two datasets: the Moral Integrity Corpus and Extreme Summarization.
weaknesses: -	The novelty and contribution are not prominent. The authors claim that “this paper offers the first contrastive explanation methods for LLMs.” However, [1] has already proposed counterfactual explanations for LLMs, which are conceptually similar to contrastive explanations. This prior work is notably absent from the related literature review. While the authors emphasize that many previous studies focus on using LLMs to generate contrastive explanations for classification tasks, I do not see significant technical differences between classification and generation tasks in this context.
-	The clarity of the paper could be improved. For instance, in the upper part of Figure 1, the contrastive response does not contradict the original response. However, in the lower part, there seems to be a typo in the prompt, as “kill and save” presents a contradiction. Additionally, the original and contrastive responses are generated from different input prompts, making it unclear why it is reasonable to compare the preferences for the two responses. 
-	The evaluation is limited to Llama 2 (13b, 70b) and bart-large-xsum. It would be beneficial to include evaluations of more recent LLMs, such as Llama 3, GPT-4, Gemini, and Claude. 
-	The results are not compelling. For instance, CELL and CELL-budget produce similar outcomes across different values of split_k, and CELL-budget is not consistently more efficient. 
-	There is a lack of evaluation metrics for the explanations, such as faithfulness and plausibility [2, 3]. 
-	The rationale for considering four scoring functions is unclear. Do they operate similarly, or do they provide different insights in various settings? 
-	The analysis and discussion of the two use cases lack depth and insight. It may be reasonable to reduce the size of Figures 4 and 5 to save space, allowing for a more detailed discussion about the use cases. 

[1] Andreas Madsen, Sarath Chandar, and Siva Reddy. 2024. Are self-explanations from Large Language Models faithful?. In Findings of the Association for Computational Linguistics: ACL 2024, pages 295–337, Bangkok, Thailand. Association for Computational Linguistics.

[2] Lyu, Qing, Marianna Apidianaki, and Chris Callison-Burch. "Towards faithful model explanation in nlp: A survey." Computational Linguistics (2024): 1-67.

[3] Zhao, Haiyan, et al. "Explainability for large language models: A survey." ACM Transactions on Intelligent Systems and Technology 15.2 (2024): 1-38.

---

confidence: 4
contribution: 2
initial_rating: 5
presentation: 3
questions: see the Weaknesses section
soundness: 2
strengths: 1. It provides a practical framework for explaining LLM outputs through contrastive examples, offering a way to understand why models generate specific responses by showing how small input changes lead to different outputs. 
2. The proposed CELL-budget algorithm demonstrates improved efficiency by intelligently managing model queries, making it valuable for working with longer texts while maintaining performance quality. 
3. The framework is highly flexible, supporting multiple scoring functions (preference, contradiction, BLEU) and working effectively across different LLM architectures and tasks. 
4. The work has practical applications in critical areas like automated red teaming for identifying problematic model behaviors and explaining conversational degradation, making it valuable for both debugging and improving LLM systems.
summary: This paper introduces the first methods for generating contrastive explanations for LLMs, addressing the challenge of explaining why an LLM generates a particular response to a given prompt. The authors propose two algorithms: CELL, an algorithm effective for small prompts, and CELL-budget, an intelligent algorithm that creates contrasts while adhering to a query budget for longer contexts. The framework explains LLM outputs by showing how slight modifications to the input prompt would result in different responses that either contradict or are less preferable than the original response. The methodology employs scoring functions to evaluate contrast quality, uses mask-and-infill operations to generate prompt perturbations, and validates results using multiple metrics including edit distance, flip rate, and model call efficiency. The paper demonstrates practical applications through natural language generation tasks, automated red teaming for finding problematic model behaviors, and explaining conversational degradation. Results show competitive performance compared to baseline methods, efficient model calls especially for CELL-budget on longer texts, and high content preservation across all models and scoring functions.
weaknesses: 1. The framework for generating contrastive explanations lacks clear definitional boundaries. For instance, in Figure 2's example ("My car is making a weird noise when I accelerate. Can you help me diagnose the problem?"), there's insufficient clarity regarding the relative importance hierarchy among phrases like "car," "weird noise," "accelerate," etc.
2. A potential typographical error appears in Line 190, where "y1 contradicts y1" likely should read "y1 contradicts y2."
3. The examples presented in Figure 4 appear to misalign with standard red teaming definitions and practices. A more rigorous adherence to red teaming principles would strengthen the paper. More appropriate examples should demonstrate how minimal prompt changes could expose concerning model behaviors, such as safety issues, biases, etc.
4. The choice of traditional PLMs (BERT, BART, T5) as infiller functions (I(.)) raises concerns about semantic coherence in generated texts, as these models often struggle with maintaining semantic correctness in their outputs.

---

confidence: 3
contribution: 2
initial_rating: 3
presentation: 2
questions: Based on the concerns above, I have the following questions:

**Q1:** Could the authors provide a more intuitive motivation for utilizing contrastive explanations? What are the benefits of obtaining these explanations in terms of understanding the mechanisms of LLMs or debugging them?

**Q2:** Could the authors clarify the use case section? It appears that all the responses from LLMs are appropriate, as the contrastive examples are semantically different from the original prompts.

**Q3:** Could the authors conduct a human study to demonstrate that the contrastive examples are genuinely similar to the original prompts while also leading to degradation or unintended behaviors in LLMs?

To summarize, my main concerns revolve around the practical application of CELL. Clarifying how CELL can be effectively used in real-world scenarios would significantly enhance its value in the context of explainable AI. If the authors can address these issues appropriately, I would be inclined to raise the score, as I appreciate the novel algorithm and the detailed analysis they have provided.
soundness: 2
strengths: * The first work studying contrastive explanation for the generation case.
* The algorithm sounds reasonable and the scoring function is clearly defined.
* The authors study the computational budget, which is important in real use cases.
summary: This paper introduces the first contrastive explanation methods for LLMs, known as CELL, designed to clarify why an LLM produces a specific response to a prompt. Unlike previous approaches, CELL operates in a generative setting, utilizing a scoring function rather than relying on a fixed value like a class label. The authors propose two algorithms: CELL, a myopic approach effective for shorter prompts, and CELL-budget, a budgeted algorithm optimized for efficiently identifying contrasts within large contexts while minimizing model calls. The effectiveness of these methods is demonstrated across diverse language tasks, including natural language generation, automated red teaming, and explaining conversational degradation.
weaknesses: * My main concern is with the contrastive prompts found by CELL. While the authors report a 0.99 similarity score between the original and contrastive prompts, many examples in the paper seem semantically different. Using BERT embeddings to assess similarity may be problematic, as they primarily capture word overlap, and changing a word can lead to entirely different meanings. This raises questions about the use cases the author provided:
  1. **Use case of automated red teaming:** A successful adversarial example (contrastive prompts in this context) should have the same meaning for humans while eliciting unusual behavior from LLMs. The responses of LLMs in Figure 4 appear reasonable to me, as the contrastive prompts introduce ambiguity. For instance, "is a competitor" is crucial for LLMs to answer "No," while altering it to "is not part" changes the relationship between the two companies, rendering them non-competitive. I believe responding "Yes" in this scenario is not inappropriate.

  2. **Explaining conversational degradation:** I think neither of the two cases in Figure 5 demonstrates degradation. In the first example, the LLM provided less detail because the CELL altered the assistant's previous response, which was, "That's really funny. Are you going to call this a lesson?" This response had a humorous tone, suggesting that the user's request was overly simple. As a user, I would expect a straightforward answer listing the basic steps. In the second example, the term "harming" was directly changed to "talking," which completely alters the context. I also don't think not calling the emergency services has any problem. If the authors' intention is merely to explain why the LLM responded in that way—although I don't think these explanations significantly enhance our understanding of LLM behavior—then it may be acceptable. However, labeling it as "conversational degradation" suggests that it's a flaw in LLMs that we should aim to fix.

* As the examples provided seem invalid, it raises further concerns about the effectiveness of the evaluation metric. Since the goal is to explain the behavior of LLMs, the most straightforward approach would be to conduct a human study to demonstrate that contrastive explanations align with human expectations. Specifically, do the prompts identified by CELL reveal unintended behaviors from LLMs while maintaining a similar meaning to the original prompts?

* While the algorithms appear valid and the experiments are comprehensive, I'm still unclear about the specific insights that CELL offers and how these explanations can be utilized to fix LLMs. From what I've observed, CELL primarily indicates that LLMs alter their responses based on changes in the prompt. I may be overlooking something, but I don't grasp the advantages of contrastive explanations in this context.

---

confidence: 4
contribution: 2
initial_rating: 5
presentation: 2
questions: Typos:

Abstract: “…why an LLM output” -> outputs
Line 086: “...fed to the LLM result in a contrastive responses that differ from…” -> in contrastive responses…
Line 190: “…score the likelihood that y1 contradicts y1...” -> second y1 should be y2.
Line 258: “...an infiller I(・) that receives an input a string with a <mask>...” -> as input.
soundness: 3
strengths: 1. The authors propose a innovative way to tackle explainability in blackbox gen AI models, in particular LLMs. 

2. Futhermore, they propose two algorithms, the second which tackles optimal prompt perturbation under a budget, that present a novel approach contrastive explainations.

3. The authors showcase their method in two usecases that have important applicability ramifications. 

4. The paper is well writen and well structured. The figures are clear and the methods are nicely examplified.
summary: The authors propose a contrastive explanation method for large language models (LLMs). The method is tailored for black-box gen AI models that are “class-free” in their prediction. In particular, this method explains why an LLM produces a specific output by examining how slight prompt modifications could lead to different responses. The authors introduce two main algorithms: a myopic approach for smaller prompts and a budgeted strategy for larger contexts, balancing accuracy with query limitations. The paper contributes a novel contrastive technique, potentially improving the explainability of LLM responses. Whereas I am sympathetic to the approach developed by the authors, I am rather dubitative towards: (i) generalizability of this approach, (ii) the explainability value of this approach. Nonetheless, I do believe this work is interesting, timely and innovative.
weaknesses: Main weaknesses:

1. The authors propose contrastive explanations for LLMs based on prompt perturbations. Whereas the authors nicely demonstrate use cases of their method, there are important factors that have not been systematically explored to evaluate the robustness and generalizability of this approach. This point is set aside the fact that the scoring function is use case-dependent and thus its formalization changes for every case. In particular, I am curious whether the authors find different results based on the delta parameter value described in equation 1: 

Do the authors find distinct qualitative results depending on that value?
If not, does this also hold across use cases?

Answering these questions would greatly benefit the reader in having more information in terms of applicability of this method.
 
2. I feel like there is a step between finding prompt perturbations that lead to some desirable properties in the output and explaining why these perturbations generate what they generate. I understand this is a tricky issue, but in my opinion a linguistic interpretation of the perturbations has more value than simply providing the perturbation itself. Did the authors try to analyse the linguistic outputs of the perturbed prompt to gain more insights in terms of explainability. It seems that currently, the value of the proposed method rests in applications of making “better” LLM outputs rather than explaining why the models output what they do. I would definitely clarify the contributions of this work in terms of explainability, in order to situate it better within that field. 

Secondary weaknesses:

1. I suggest making the connection between the g function described in equation 1 and the distinct scoring methods described in section 3.1 more explicit. This will increase the clarity of the paper.

2. On line 277, it is confusing to call this a “first iteration”, as in fact there are n iterations at this level. I would call it: “first n iterations”.