SlowGuess/NeurIPS_2023_OCR huggingface dataset:
The Utility of “Even if...” Semifactual Explanation to Optimise Positive Outcomes
Eoin M. Kenny*
Cambridge, MA, U.S.A.
ekenny@mit.edu
Weipeng Huang*
Tencent Security Big Data Lab
Shenzhen, Guangdong Province, China
fuzzyhuang@tencent.com
When users receive either a positive or negative outcome from an automated system, Explainable AI (XAI) has almost exclusively focused on how to mutate negative outcomes into positive ones by crossing a decision boundary using counterfactuals (e.g., "If you earn  $2k$  more, we will accept your loan application"). Here, we instead focus on positive outcomes, and take the novel step of using XAI to optimise them (e.g., "Even if you wish to half your down-payment, we will still accept your loan application"). Explanations such as these that employ "even if..." reasoning, and do not cross a decision boundary, are known as semifactuals. To instantiate semifactuals in this context, we introduce the concept of Gain (i.e., how much a user stands to benefit from the explanation), and consider the first causal formalisation of semifactuals. Tests on benchmark datasets show our algorithms are better at maximising gain compared to prior work, and that causality is important in the process. Most importantly however, a user study supports our main hypothesis by showing people find semifactual explanations more useful than counterfactuals when they receive the positive outcome of a loan acceptance.
Explainable AI (XAI) is broadly categorised into factual [4, 27, 30] and contrastive explanations [29, 38]. Within contrastive XAI, despite being neglected in comparison to counterfactuals, semifactuals are a major, fundamental part of human explanation, and have long been studied in psychology [9, 35, 49], philosophy [6, 8, 20], and lately computer science [1-3, 29, 34, 52, 62]. They take the form of "Even if  $x$  happened,  $y$  would still be the outcome". Such reasoning has many potential uses as demonstrated by these prior works, but here we are focused on how semifactuals can help optimise positive outcomes for users, which (to the best of our knowledge) remains completely unexplored.
Our definition of counterfactuals is in line with Wachter et al. [55], where a test instance classified as  $c$  must be mutated to cross a decision boundary into class  $c'$ . Likewise, as established in the literature [1, 29], we define a semifactual as an instance classified as  $c$ , which must be modified in such a way as to not cross a decision boundary (and hence remain class  $c$ ) [29]. In recourse [25, 50], "negative outcomes" (e.g., a loan rejection) are generally mutated to produce "positive outcomes" (e.g., a loan acceptance) for users using counterfactuals. In our setting, we are assuming there was initially a positive outcome, and we are trying to mutate features to produce an even better situation for users, and in doing so not cross the boundary into the negative outcome (i.e., using semifactuals).
Historically, counterfactuals have had obvious applications in computer science, such as explaining how to have a bank loan accepted rather than rejected, but applications for semifactuals as less
*Contributed Equally.
$^{1}$ Code available at: https://github.com/EoinKenny/Semifactual_Recourse_Generation
clear. As such, the usage of semifactuals has often inadvertently defaulted to copying counterfactual research by also explaining negative outcomes (e.g., "Even if you double your savings, your loan will still be rejected" [3, 29, 48]). However, such an application for semifactual explanation perhaps has two main issues. Firstly, it is debatable if these explanations convey useful information [1], whilst a counterfactual explaining how to cross a decision boundary and have a loan accepted has obvious utility [24]. Secondly, such explanations make the user's situation seem helpless [35], in that they cannot possibly have their loan accepted, which raises ethical concerns [1]. However, our proposed framework can be used to not only overcome both of these issues, but actively contribute to fairness.
Firstly, to try offer useful information for users, we flip the usual recourse problem and consider the user starting from a positive (rather than a negative) outcome. In this setting, consider a user that has had their loan accepted, but might prefer to make a smaller down-payment on a loan application. In this situation, our framework could present an explanation such as "Even if you half your downpayment, your loan will still be accepted", which seems to be more useful than explaining negative outcomes (see Section 6). Secondly, because we are starting from a positive outcome, there is no danger of manipulating people into accepting a negative outcome, which guarantees fairness is this regard. Now, with regards to optimising fairness even further, note that banks are not motivated to share such explanations even though they may help people, because (for example) larger downpayments are associated with lower risk on their behalf [7]. So, the usage of semifactuals in this application has clear potential to actively encourage fairness and transparency. As an aside, it is worth noting that although the focus of this paper is on financial applications, this research has broad impact on any domain for which the optimisation of a positive outcome is beneficial. For instance, in medical applications, our framework could present explanations of the form "Even if you half your dose of drug  $x$ , you will still be at a low risk for disease  $y$ ". This is once again important for optimising fairness because people are frequently over-prescribed medicine with adverse side-effects [47], but due to profit Big Pharma has no incentive to actively encourage this type of transparency. Similar usage of semifactuals have also been proposed in smart agriculture to combat climate change [29].
Our main contributions are: (1) the first explicit exploration of how to optimise positive outcomes with XAI, (2) the problem formulation for this which involved augmenting current semifactual research with the concept of Gain (see Section 3.3), and (3) the premiere user test in the XAI literature for semifactuals, showing a clear application in which users find them more useful than counterfactuals.
When using contrastive explanation to explain loan acceptance decisions, to the best of our knowledge, this has only been explored by McGrath et al. [36]. Specifically, they suggest positive counterfactuals, which show "by how much" a user had their loan accepted to help inform them when making future financial decisions. While this is interesting information, we show that users find semifactual explanations more useful in loan acceptance situations than positive counterfactuals (see Section 6).
Semifactual explanation is growing in popularity [3], Kenny & Keane [29] first explored the idea, but focused only on images. Artelt & Hammer [1] used diverse semifactuals to explain why an AI system refuses to make predictions due to having an unacceptably low certainty, but ignore how to explain either positive or negative outcomes. Lu et al. [34] explain spurious patterns with semifactuals using a human-in-the-loop framework in NLP. Zhao et al. [62] proposed a class-to-class variational encoder (C2C-VAR) with low computational cost that can generate semifactual images. Vats et al. [52] used generative models to produce semifactual image explanations for classifications of ulcers. Lastly, for model exploration, Xie et al. [59] sampled semifactual images with a joint Gaussian mixture model, and Dandl et al. [15] proposed deriving semifactual explanations from interpretable region descriptors. In contrast to all these approaches, we are showcasing how semifactuals can be used to optimise positive outcomes for users (notably in causal settings).
From a user perspective, many have discussed the urgent need for comparative tests with semifactuals [29, 37, 40, 48, 56], with Aryal & Keane [3] pointing to the 'paucity of user studies' in the area. However, the only such tests we are aware of are in the psychological literature over two decades ago [35]. Taking to this challenge, we conduct the first such test directly comparing semifactuals to counterfactuals in the XAI literature (see Section 6).
Note there is other work on a-fortiori explanations which have similar computational techniques to semifactuals [14, 19, 45], they are justifications of the form "Because  $x$  it true,  $y$  must also be true".
Our research is related to algorithmic recourse [50] in that we are trying to ensure users are treated fairly by automated systems [24]. In this area, Mothilal et al. [39] explored counterfactual diversity, in that we should be offering users several explanations. In addition, counterfactual robustness has been examined [18], which proposes that generated explanations should be robust to distributional shifts. Lastly, causality has been argued as essential to providing plausible recourse [26]. We see these three facets as being important to our problem setting, and instantiate them in our framework. There are other areas in recourse such as sequential decision making [16, 42], fairness [54], and privacy [43], but we leave their exploration within semifactual explanation for future work.
As an aside, the literature on sufficiency could be conflated with semifactual explanation, as it describes a set of "sufficient" features for a prediction which, in the presence of the other features mutating, mostly doesn't affect the outcome [17, 46, 57]. However the techniques offer no insights for how to generate a meaningful semifactual. More importantly though, if the sufficient features are the only actionable ones, then by definition we can't modify them to create a semifactual.
3 Semifactual Framework
In this section, we describe the basic definitions and assumptions for our semifactual framework to optimise positive outcomes for users, before formalising it under the concept of Gain (i.e., how much a user stands to benefit from the explanation) in a causal setting, neither of which has been considered before. As an aside, we also show how the established concepts of plausibility, robustness, and diversity can be made fit into the objective to offer better explanations. Finally, we reflect on the theoretical properties of the framework.
Let us denote an individual  $\mathbf{x} \in \mathcal{X}$  with  $k$  mutable features  $\mathbf{X} = \{X_1, \ldots, X_k\}$ . Given the individual  $\mathbf{x}$ , a set of actions can be applied to  $\mathbf{x}$  where each action  $a(\mathbf{x})$  is also a  $k$ -dim vector. As in prior work [25], we apply  $a(\mathbf{x})$  and  $a$  exchangeably, since the individual  $\mathbf{x}$  will always be fixed. We explicitly exclude features that are either immutable or non-actionable. Adopting Pearl's  $do()$  operator [44], an action can be defined as  $a(\mathbf{x}) = do(\mathbf{X} := \boldsymbol{\theta})$ , or simply  $do(\boldsymbol{\theta})$ , to force a hard
Figure 1: Semifactual Explanation to Optimise Positive Outcomes: An individual  $\mathbf{x}$  has their loan accepted, but there are several semifactual explanations which can help optimise their outcome. Our algorithm produces a set of semifactual explanations which maximise the distance between  $\mathbf{x}$  and the final explanation  $\mathbb{S}\mathbb{F}(\mathbf{x},a;\mathcal{M})$ . This allows the largest Gain to be achieved so that the user gets the maximum benefit. In contrast, counterfactual algorithms are not suitable because they are designed to target the shortest path across a decision boundary. In addition, the semifactuals are robust to distributional shifts by constraining an  $\epsilon$ -neighborhood between them and the decision boundary. Note  $\mathcal{M}$  is the Structural Causal Model (SCM), see Section 3.
intervention of replacing  $\mathbf{x}$  by  $\pmb{\theta}$  where  $\pmb {\theta}\in \mathcal{X}$ . It implies that, for each feature,  $X_{i}\coloneqq \theta_{i}$  for the individual  $\mathbf{x}$ . If the action  $do(\pmb {\theta})$  imposes no change,  $\mathbf{x} = \pmb{\theta}$  holds. We further denote a set of human-constrained actionable ranges  $\mathcal{A} = \{a(\mathbf{x}) = do(\pmb {\theta}): \forall \pmb {\theta}\in \mathcal{X}\}$ . Note that the actions have to be mutable and explicitly exclude any action which keeps the individual in the same position.
The non-causal semifactual interaction between  $\mathbf{x}$  and  $a(\mathbf{x})$  is defined by  $\mathbb{S}\mathbb{F}:\mathcal{X}\times \mathcal{A}\mapsto \mathcal{X}$ . That is, the individual  $\mathbf{x}$  taking action  $a(\mathbf{x})$  will lead to another representation  $\theta \in \mathcal{X}$  representing that person's recourse. Now, a structural semifactual is defined which considers the dependence between the related features [18, 24]. We denote the structural causal model (SCM) by  $\mathcal{M} = (\mathbf{S},\mathbb{P}_U)$  where  $\mathbf{S}$  are a set of structural equations and  $\mathbb{P}_U$  is the distribution over the exogenous variables  $U\in \mathcal{U}$ . Consider that in a causal graph, there is a set of causal parents for each feature  $x_{i}\in \mathbf{x}$ , denoted by  $\mathrm{Pa}_i$ . We denote the structural equations as  $\mathbf{S} = \{x_i\coloneqq g_i(\mathrm{Pa}_i,U_i):i = 1,\dots ,k\}$  where  $g_{i}(\cdot)$  is a deterministic function that describes the causal relationship for  $x_{i}$ , and depends on the exogenous variable  $U_{i}\in \mathcal{U}$  alongside the corresponding parent set  $\mathrm{Pa}_i$ . Hence,  $\mathbf{S}$  induces a mapping  $\mathbb{S}:\mathcal{U}\mapsto \mathcal{X}^*$  and its inverse mapping  $\mathbb{S}^{-1}:\mathcal{X}\mapsto \mathcal{U}$ . Let  $f\circ g(x) = f(g(x))$  which can be extended to more functions. Hence, we specify the SCM-processed semifactual by  $\mathbb{S}\mathbb{F}(\mathbf{x},do(\pmb {\theta});\mathcal{M})$  to denote the transition between the states by taking a certain action through an SCM  $\mathcal{M}$ , where
$$
\boldsymbol {\theta} ^ {\prime} = \mathbb {S F} (\mathbf {x}, d o (\boldsymbol {\theta}); \mathcal {M}) := \mathbb {S} \circ \mathbb {S} ^ {- 1} \circ \mathbb {S F} (\mathbf {x}, d o (\boldsymbol {\theta}); \mathcal {M}). \tag {1}
$$
If all features are independently manipulable, we have  $\theta' = \theta = \mathbb{S}\mathbb{F}(\mathbf{x},do(\theta);\mathcal{M}) = \mathbb{S}\mathbb{F}(\mathbf{x},do(\theta))$ . Therefore,  $\mathbb{S}\mathbb{F}(\mathbf{x},do(\theta);\mathcal{M})$  is a more generalised formulation which covers the non-causal case. Lastly, we assume a binary model that generates the score for the users is  $h$ , where  $h:\mathcal{X}\mapsto \{0,1\}$  by which we can simply consider that 1 means a positive outcome (e.g., a loan acceptance) and 0 is a negative outcome (e.g., a loan rejection). We set a lower threshold  $\psi$  that separates the decision boundary. For the form defined above,  $\psi = 0.5$  is a reasonable threshold that fits all situations well.
3.2 Framework
We define our semifactual framework as one centering on gain  $(G)$  that is weighted by plausibility  $(P)$ , regularization in the form of diversity  $(R)$ , and hard constraints in the form of robustness  $(H)$ , indexed by  $j$ . All of the components are parameterized with  $\mathbf{x}$  and a subset of suggestions  $\{a_1, \ldots, a_m\}$ . Letting  $f(\cdot)$  be a function composed by gain and some weighting (i.e., plausibility for us), the causal semifactual framework is defined as
$$
\max  _ {a _ {1}, \dots , a _ {m}} \quad \frac {1}{m} \sum_ {i = 1} ^ {m} f (G (\mathbf {x}, a _ {i}), P (\mathbf {x}, a _ {i})) + \gamma R (\{\boldsymbol {\theta} _ {1}, \dots , \boldsymbol {\theta} _ {m} \})
$$
$$
\mathrm {s . t .} \quad \boldsymbol {\theta} _ {i} = \mathbb {S F} (\mathbf {x}, a _ {i}; \mathcal {M}), H _ {j} \left(\boldsymbol {\theta} _ {i}\right) \geq 0, \forall i, j \tag {2}
$$
where the regularisation and hard constraints can be multiple and indexed with  $i$  and  $j$ , respectively. One may define a similar formulation for the non-causal case (see Section 4.2). We defer all details of the components until Section 3.4.
3.3 Optimising Positive Outcomes with Gain
For the core of the objective we appeal to the notion of gain. Note that gain is similar to the idea of cost commonly used in recourse [50], but there are three crucial differences. First, we are trying to maximise gain, rather than minimise cost [24]. Second, gain ideally considers the causal dependencies between features in its function, whilst cost typically only considers the user's action(s) [24]. Third, gain is further subdivided into positive and negative polarities. To elaborate on this last point, take the example of a user who has their loan application for buying a new house accepted. In this situation, if they desired to spend more time away from work with family, they would experience positive gain if they could work less hours per week and still have their loan accepted (see Figure 1). Conversely, if this person increased the number of hours they worked, they would experience negative gain. Notably, positive/negative gain is not necessarily connected to the model's probabilities (see  $a_1$  in Figure 1 moving away from the decision boundary). Similar to actionability constraints which can offer individualised recourse [50], what is positive/negative gain must be manually defined for each individual. As prior work on semifactuals simply maximised the  $L_{2}$  distance between a test instance and explanatory one to define good explanations [1, 29], we introduced the concept of gain to make them more meaningful in application.
More formally, we define the gain function by  $G: \mathcal{X} \times \mathcal{A} \to \mathbb{R}$ . By denoting  $\pmb{\theta} = \mathbb{S}\mathbb{F}(\mathbf{x}, a; \mathcal{M})$ , we decompose the function as follows:
$$
G (\mathbf {x}, a) := \mathcal {P} _ {S F} \circ \delta (\mathbf {x}, \boldsymbol {\theta}) = \mathcal {P} _ {S F} \circ \delta (\mathbf {x}, \mathbb {S} \mathbb {F} (\mathbf {x}, a; \mathcal {M})) \tag {3}
$$
where  $\mathcal{P}(\cdot, \cdot)$  is an oracle function that computes the payoff based on the vectorised difference between  $\mathbf{x}$  and  $\pmb{\theta}$ , i.e.,  $\delta: \mathcal{X} \times \mathcal{X} \mapsto \mathbb{R}^k$  which is a symmetrical difference function between the two feature representations. The subscript of  $\mathcal{P}_{SF}$  denotes a semifactual. In interpretation, the gain function compares two states, (1) the original feature vector  $\mathbf{x}$ , and (2) the SCM-processed end state  $\pmb{\theta}$  which was led to through  $\mathbf{x}$  taking action  $a$ .
Why is Gain not necessarily equivalent to Cost? Formally, to enable the comparison, we write the cost function (denoted by  $C(\mathbf{x}, a)$ ) as
$$
C (\mathbf {x}, a) = - \mathcal {P} _ {C F} \circ \delta (\mathbf {x}, \mathbb {S} \mathbb {F} (\mathbf {x}, a)) \tag {4}
$$
which builds on the fact that cost solely considers the feature change. Note that  $\mathbb{S}\mathbb{F}$  is equivalent to the notion  $\mathbb{C}\mathbb{F}$  in [25], what makes our approach different is the consideration of positive outcomes and gain. Our finding is that gain in semifactuals (SFs) is not necessarily equivalent to cost in counterfactuals (CFs) where the equivalence ignores the sign of both quantities, as formally stated as follows.
Theorem 3.1. Even if  $\mathcal{P}_{SF}(\cdot, \cdot) \equiv \mathcal{P}_{CF}(\cdot, \cdot)$ , gain and cost are not necessarily equivalent ignoring the sign.
Proof. Note that SCMs are also considered in counterfactual recourse [18, 24, 26]. However, in this prior research SCMs are typically applied for enforcing hard plausibility constraints, not in the computation of a user's cost. In contrast, our gain function takes the SCM-processed semifactual  $\theta^{\prime}$  as an input. We employ proof by contradiction here. Assume that cost and gain are equivalent ignoring sign so that, without loss of generality,
$$
\begin{array}{l} | G (\mathbf {x}, a) | = | C (\mathbf {x}, a) | \iff | \mathcal {P} \circ \delta (\mathbf {x}, \mathbb {S F} (\mathbf {x}, a; \mathcal {M})) | = | \mathcal {P} \circ \delta (\mathbf {x}, \mathbb {S F} (\mathbf {x}, a)) | \\ \Longleftrightarrow \delta (\mathbf {x}, \mathbb {S F} (\mathbf {x}, a; \mathcal {M})) = \delta (\mathbf {x}, \mathbb {S F} (\mathbf {x}, a)) \tag {5} \\ \end{array}
$$
holds. However, SCMs can result in possibly more features being changed since some features could be others' causal parents and those causal children will change their values accordingly. By denoting  $\pmb{\theta} = \mathbb{S}\mathbb{F}(\mathbf{x},a)$  and  $\pmb{\theta}' = \mathbb{S}\mathbb{F}(\mathbf{x},a')$ , we consider the general case as follows:
$$
\begin{array}{l} \left| \delta (\mathbf {x}, \boldsymbol {\theta}) \right| - \left| \delta (\mathbf {x}, \boldsymbol {\theta} ^ {\prime}) \right| = \sum_ {i} \left| \delta (\mathbf {x}, \boldsymbol {\theta}) _ {i} \right| - \sum_ {i} \left| \delta (\mathbf {x}, \boldsymbol {\theta} ^ {\prime}) _ {i} \right| \\ = \sum_ {\left\{i: \theta_ {i} = \theta_ {i} ^ {\prime} \right\} \cup \left\{i: \theta_ {i} \neq \theta_ {i} ^ {\prime} \right\}} | \delta (\mathbf {x}, \boldsymbol {\theta}) _ {i} | - | \delta (\mathbf {x}, \boldsymbol {\theta} ^ {\prime}) _ {i} | = 0 + \sum_ {\left\{i: \theta_ {i} \neq \theta_ {i} ^ {\prime} \right\}} | \delta (\mathbf {x}, \boldsymbol {\theta}) _ {i} | - | \delta (\mathbf {x}, \boldsymbol {\theta} ^ {\prime}) _ {i} | \leq 0, \tag {6} \\ \end{array}
$$
which contradicts with Equation (5). Thus, even if the oracle function for calculating the payoff is the same, gain and cost are still not necessarily equivalent. Also, the equality in Equation (6) holds when all features are independently manipulable or the changed features are independently manipulable of the remaining features, so that  $\mathbb{S}\mathbb{F}(\mathbf{x},a) = \mathbb{S}\mathbb{F}(\mathbf{x},a;\mathcal{M})$ . The proof completes here.
3.4 Semifactual Components
Here, we detail how to incorporate the concepts of plausibility, robustness, and diversity into our framework for maximising gain, because they are agreed upon as important in the literature and useful for evaluation. While plausibility and diversity have been explored in semifactual explanation [1, 29], robustness and causality (and indeed an objective balancing all together) have not, yet we argue and show that the subtleties of "even if..." thinking are perhaps better captured in a casual setting.
Plausible Gain We define plausibility here as explanations which are within distribution. For example, an explanation saying a person could earn less and still have their loan accepted should change their "debt-to-income ratio" feature also, or it will lie outside the data manifold. Prior work on semifactuals has only considered euclidean distance to training data as a heuristic for this [29], in contrast we posit (similar to the counterfactual literature [23]) that this is better approached with
SCMs. Hence, we define the plausibility for  $\mathbf{x}$  taking the action  $a$  by  $P(\mathbf{x}, a) = \operatorname{Pr}(a = do(\boldsymbol{\theta})|\mathbf{x})$  where  $\mathbf{x}$  is fixed for an individual and  $\operatorname{Pr}(\cdot)$  is a density function. In our non-causal tests, we use the  $L_{2}$  norm to training data to approximate plausibility (i.e., being in distribution, similar to [29, 32, 51]). However, this issue of plausibility is naturally taken care of in our causal tests thanks to the SCM ensuring plausible feature mutations, so we don't explicitly consider plausibility there going forward.