Counterfactuals of Counterfactuals: a back-translation-inspired approach to analyse counterfactual editors (2023)

Abstract: In the wake of responsible AI, interpretability methods, which attempt to provide an explanation for the predictions of neural models have seen rapid progress. In this work, we are concerned with explanations that are applicable to natural language processing (NLP) models and tasks, and we focus specifically on the analysis of counterfactual, contrastive explanations. We note that while there have been several explainers proposed to produce counterfactual explanations, their behaviour can vary significantly and the lack of a universal ground truth for the counterfactual edits imposes an insuperable barrier on their evaluation. We propose a new back translation-inspired evaluation methodology that utilises earlier outputs of the explainer as ground truth proxies to investigate the consistency of explainers. We show that by iteratively feeding the counterfactual to the explainer we can obtain valuable insights into the behaviour of both the predictor and the explainer models, and infer patterns that would be otherwise obscured. Using this methodology, we conduct a thorough analysis and propose a novel metric to evaluate the consistency of counterfactual generation approaches with different characteristics across available performance indicators.

## introduction the explainable ai (xai) field has risen to prominence in recent years, spurred on by the success of opaque (black-box) deep learning models, which despite their impressive performance cannot be used in practice in many cases due to ethical and legal @xcite concerns. to mitigate this, multiple explanation methodologies have been proposed for different tasks, data domains and use-cases @xcite . one family of methods for explaining classifiers are counterfactual or contrastive explanations @xcite @xcite ). these answer the question "what should change in an input sample for it to be classified to class b instead of class a" and can be especially useful in real-world applications as they offer recourse to a user. for example someone who was declined a loan by a bank's ai could ask "what is the smallest change i should do for my loan to be approved" @xcite . however, many xai methods have come under scrutiny, as they can often be misleading and cause more problems than they solve @xcite . specifically counterfactual explainers have been shown to suffer from issues of robustness @xcite @xcite , and while there have been attempts to formalise notions of interpretability and its evaluation @xcite , there is no agreement on what constitutes a good explanation @xcite . this ambiguity regarding the desiderata for a good explanation and the fact that they can vary according to the use-case, transcends to the chosen evaluation methods used, which vary accordingly and offer a limited perspective of the editor(s) behaviour. thus it is imperative that we develop further ways to evaluate and compare xai methods in more depth. for counterfactual explanations, existing metrics enable comparisons between different methods, however the absence of a ground truth does not allow us to assess the quality of a single explainer in a standalone mode and evaluate its output on how close it is to a (theoretically attainable) ideal explanation. finding such an ideal explanation in practice is not easy, but we can expand on the idea of evaluation by comparison, and compare a counterfactual system's performance against itself. following this rationale, we propose a methodology that is inspired by back-translation, which has been used for evaluating and improving machine translation systems @xcite . specifically, by feeding the output of the system to itself (a counterfactual of a counterfactual), we would expect the result to be "at least as good as" the original input since we know that the original input exists, is actionable, and feasible, thus it constitutes a "lower bound" for the generated edit, and a proxy for ground truth. this methodology can be applied to obtain a lower bound on several metrics; in this work we focus on its application to minimality since it is the metric that most of the editors attempt to minimise as a primary criterion @xcite . one of the desired characteristics of counterfactual explanations is that they constitute minimal changes to the input sample, and minimality is the metric used to measure the distance between the original and the edited samples. due to the lack of an ideal explanation to be used as ground truth, there is no way to know if a specific value of minimality is optimal (and generally good or bad). given a ground truth explanation, it is possible to calculate the optimal minimality to be obtained, but without it, it is only possible to compare minimality values across different edits/editors. using our proposed methodology we introduce a metric (which we call inconsistency) that uses the editor's previous outputs as reference points to evaluate an editor's capability to produce minimal edits. we feed the output of the editor back to it as input to produce a new edit, and we expect the new edit to be at least as good as the edit of the previous step. for example, in figure 1 we see the outputs through different steps of our feedback loop approach, so when we feed the first edit ("this movie was awful!") back to the counterfactual system, we expect the generated edit to be at least as good as the original text ("this movie was fantastic!"). however, we see that the editor adds an erroneous whitespace to the generated edit (see 2:second edit in figure 1 ) so we know that there exists a better output that the system was not able to find, thus for sure the system did not produce the optimal output. note that a counterfactual system with a non-zero value of inconsistency is guaranteed to be sub-optimal, however a zero value does not indicate that it is optimal. this approach essentially sets a lower bound for the editor, but we do not have access to the higher bound which could be impossible to define automatically. in the rest of the paper we provide a detailed description of our proposed methodology and novel metric and demonstrate its application on several frequently used editors with different characteristics. ## background in this paper we are concerned with systems that attempt to minimally edit a given textual input so that they change the prediction of a classifier; we will henceforth refer to such systems as counterfactual editors, or simply editors. below, we provide a categorisation of such systems, along with related literature, in addition to an overview of how they are being evaluated in related works. ## back-translation for analyzing editors we formalise our problem as follows. we assume access to a classifier g such that g : @xmath0, 1] c , where l the set of text for a specific language and c is the number of different classes. we then consider the counterfactual editors for g as functions f : @xmath1, and we assume that the goal of the editor f is threefold: to examine the degree to which these criteria hold, we analyse the behaviour of editors when they are iteratively fed back with their output, i.e., we are studying the function f (f (f (...f (x)))), and evaluating the three criteria described above after n applications of the editor. specifically, we first define a novel evaluation metric to quantify the second criterion based on the iterative feedback approach, and then we discuss how the first and third criteria can be more thoroughly checked by measuring performance metrics after n steps of feedback (notated as metric@n). ## experimental setup we evaluate our approach on two different datasets and classifiers trained on them. specifically, we use a binary classifier trained for sentiment classification on the imdb dataset @xcite and a multi-class short-document classifier trained on newsgroups @xcite . we provide details and statistics for each dataset in appendix a. using these classifiers, we apply our methodology on the three counterfactual editors described in §4.2 and examine the metrics described in §3 by generating edits, testing the classifiers on them and feeding back the edited text to the editor for @xmath2. for each editor we apply our methodology as follows: for a given input text x, at step n we select from the pool of generated edited texts the one with the minimum minimality that alters the prediction, if such an output exists, otherwise we select the minimum minimality output. we repeat this process until @xmath3. we also study the impact of test-set size on the observed differences between results and the statistical significance of findings. we present detailed results in appendix e. we found that for a test set size greater than 200 texts results converge on both datasets and we obtain statistically significant differences. based on these findings we randomly sample 500 texts from the imdb dataset for our ex-periments, to reduce the computational load. since newsgroups is smaller, we use the full dataset. ## interpreting the inc@n metric in we can observe significant differences for inc@n between the editors, reflecting the differences in the underlying approach. textfooler is the most consistent editor, with low inc@n values that imply very rare increases in minimality between steps. this shows that the controlled nature of textfooler's approach for selecting replacements is beneficial to the generation of consistent explanations. mice and polyjuice on the other hand are less consistent, which could be attributed to the use of large language models in the generation process, which are more sensitive to small perturbations that can alter their output @xcite . the comparatively high value of inconsistency for polyjuice at the first steps for imdb, can be explained by the fact that it has to "guess" the locations it should change in the text without access to the predictor. especially for longer inputs, the search space of polyjuice is exponentially larger. this forces it to make more aggressive edits to achieve the same result, often deleting a large portion of the source text and seems to contribute to polyjuice's reported robustness issue @xcite . for example, polyjuice erased over 70% of the original text for 83% of the first two steps of edits on the imdb dataset (candidates with lower minimality may be produced but failed to change the class and were rejected). an extensive analysis of this tendency across all editors and datasets is presented in the appendix d. this "extreme erasure" pattern disappeared in the next steps, where the input length was significantly smaller (for instance original texts have on average 204 tokens, while the edited ones produced by polyjuice have 29). on the other hand, for shorter texts, for which the search space is smaller, polyjuice is more consistent, without radical changes on the original input. we hence attribute the differences between the first and later steps of the inc@n metric to this tendency of polyjuice to reduce the length of long texts. after the first step since the length of the input texts have already been significantly reduced, its behaviour is more consistent. this pattern is invariant between the two datasets but not visible on the first steps of newsgroups results since it contains texts with 43% fewer tokens than imdb on average. to better understand what these values of inc@n represent, in figures 3a , 3b we show box-plots of minimality and inc@n after each step of feedback for the imdb dataset (box-plots for newsgroups appear in appendix c). while the tendency is for minimality to decrease, implying that the editors tend to perform fewer edits after each feedback step, there are cases where it increases, implying inconsistency. in figure 3b , what stands out is the higher value of inc@n for mice when n is even. since even steps correspond to the attempt to move from the original class to a different one, higher inc@n indicates that it is easier to transition to the original class than from it. this could be attributed to remnants of the original input text pushing the classifier towards the original prediction, and thus requiring fewer edits. for instance in figure 2 we can see an example of an edit produced by mice, where clearly parts of the original text indicating positive sentiment (marked in bold) were left unchanged. on newsgroups, mice is significantly more inconsistent especially in the first steps, probably due to its requirement to select a specific target class in contrast to the other editors (see also appendix c). these observations highlight the need for feedforward evaluations of such systems since the minimality@1 reveals only a limited, dataset-dependent aspect of editors' capabilities and performance. furthermore, they show the effectiveness of additional feedback steps to more accurately quantify the difference between the samples produced by an editor, and obtain a proxy for a global minimum. in it is important to mention that the inconsistency of minimality captures different attributes of the editor than the minimality itself. high minimality means that the editor made more edits in order to alter the label of the input text. this may be due to either a weakness of the editor or the input itself requiring more edits to change class. to exclude the latter hypothesis, it is necessary to find counterfactual examples with lower minimality than the one produced, to confirm that there are better states that the editor could not explore. however, these states must meet the exact same conditions that the editor takes into account. the three editors analysed in this study have a secondary objective of producing realistic counterexamples; hence, for instance, the addition of random characters in the middle of the text is not a desirable goal state, despite the fact that it may result to a label flip with lower minimality. along the same lines, textfooler aims to replace every word with a synonym; therefore, a counterexample that replaces a word with an antonym (e.g., 'love' with 'hate') is not an acceptable goal state for it. to our knowledge, there have been no efficient or impartial methods for finding counterexamples with a lower value of a specified metric (such as minimality) that also meet exactly the same requirements as the editor. the proposed methodology comes to fill this gap, and the inconsistency metric can quantify the weaknesses of the editor in terms of the studied metric, in this case, minimality. in short, a positive inconsistency proves that there are goal states with a lower value of the corresponding metric that the editor should, but did not explore. ## additional insights from counterfactuals of counterfactuals besides measuring minimality and inc@n, we also investigated how the feedback approach can give us additional insights for the other two desiderata for editors, flip-rate and fluency. ## conclusion in this work we introduced a methodology for analysing different aspects of counterfactual editors and obtaining an approximate ground truth by iteratively feeding back their output. combined with evaluation metrics from related literature, our proposed approach provides new ways to understand the counterfactual editors' behaviour and performance based on their intended use case, and thus help develop better editors. we proposed inc@n, a metric to measure the consistency of editors, and we showed how the proposed approach can help diagnose and analyse a diverse set of existing editors and gain new insights on their behaviour, such as those made apparent by observing the discrepancies of odd and even steps of feedback. our findings allow for a more interpretable evaluation of editors that goes beyond mere comparisons between them. the results motivate further research in this direction including experiments on additional evaluation metrics, editors and tasks. apart from expanding the scope of our experiments, we intend to look into using the feedback information to automatically address the weaknesses and inconsistencies of editors during fine-tuning, to obtain more robust and interpretable counterfactual edits. along these lines, we also plan to investigate the benefit of integrating the feedback rationale into the training of counterfactual generation algorithms; for example a back-translation inspired objective could help alleviate the problematic behaviour and boost performance.