SlowGuess/ACL_2020_OCR huggingface dataset:
Decision Boundary Algorithm. In detail, the algorithm takes as input a data point  $x^{*}$ , the classifier  $f$ , a perturbation distribution  $\mathcal{D}(\cdot |x^{*})$ , and a measure of distance between inputs  $d(x_{1},x_{2})$ . We first sample  $\{\tilde{x}\}_{i = 1}^{10,000}$  from the perturbation distribution around  $x^{*}$ . The eligible perturbations to
Figure 4: Counterfactual simulation test procedure. Users see model behavior for an input, then they predict model behavior on an edited version of the input. We isolate the effect of explanations by measuring user accuracy with and without explanations.
choose from are those with the opposite prediction from the original:  $E = \{\tilde{x}_i | f(\tilde{x}_i) \neq f(x^*)\}$ . Then using a distance function  $d$ , we select a counterfactual input as
$$
x ^ {(c)} = \min _ {\tilde {x} _ {i} \in E} d (x ^ {*}, \tilde {x} _ {i})
$$
We provide a path from  $x^{*}$  to  $x^{(c)}$  by greedily picking the single edit from the remaining edits that least changes the model's evidence margin, which is the difference between positive and negative class scores. Our distance function is the count of different features between inputs, plus the squared Euclidean distance between latent representations. The Euclidean distance is on a scale such that it serves as a tie-breaker:
$$
\begin{array}{l} d (x _ {1}, x _ {2}) = \sum_ {j} \mathbb {1} \left(x _ {1 j} \neq x _ {2 j}\right) \\ + | | f (x _ {1}) - f (x _ {2}) | | _ {2} ^ {2}. \\ \end{array}
$$
A.2 Perturbation Distributions
We design perturbation distributions for two points in our experiments: 1) selecting counterfactual inputs in simulation tests, and 2) generating decision boundary explanations. First, we describe our approaches for selecting counterfactual inputs, which are conditioned on the need for a certain prediction type: either the same prediction as the original input or the alternative class. In both data domains, we sample 10,000 local perturbations around the input and then randomly pick a sample that the model predicts to be of the needed prediction type. While working with tabular data, we sample perturbations as follows: we
Figure 5: A screenshot of our user testing interface. This example is of the counterfactual Post test with LIME for text data.
randomly choose to make between 1 and 3 edits, then choose the features to edit uniformly at random, and finally pick new feature values uniformly at random. The only sampling constraint is that a variable cannot be set as its original value.
For text data, we use a strategy that is similar to sampling from the perturbation distribution in Ribeiro et al. (2018), which is to randomly substitute words with their neighbors in GloVe word embedding space, sampling neighbors with probability proportional to their similarity. We make a few changes: we 1) decrease probability of token change with the length of sentence, 2) cap the number of edited words at 5 in the chosen perturbation if possible, and 3) limit edited tokens to be nouns, verbs, adjectives, adverbs, and adpositions. Example perturbations are shown in the example of the user testing interface in Figure 5, which is given for a counterfactual test with text data.
A.3 Simulation Test Design
In Figures 3 and 4, we include additional representations of our experimental design, showing each test separately and in slightly greater detail than in Figure 1.
A.4 Testing Environment
We show a screenshot of our user testing interface in Figure 5. This example is of the counterfactual Post test with LIME for text data. Tests are administered through spreadsheets, wherein users read test material and place responses. Users are guided from file to file by the experimenter.