The Ultimate Guide to Counterfactual Explanations for Classification Models | by Chetana Didugu | Towards Data Science

Let’s say Paul applies for a loan at ABC Bank. He receives a mail from the bank expressing deep regret to inform him that his loan application has been rejected. Rejected!
After being shattered for a while, Paul may be curious to know what happened. So he goes to the bank and asks for the person who takes decisions on loan applications. He is directed to Jane, who takes the decisions on loan applications. Paul asks Jane why his application got rejected. She tells him his asking amount is too high, and his salary is less than minimum, and so on.
But then, he asks her what could he have done to get his loan application approved. She tells him his asking amount needs to be reduced by so much, increase his salary by so much, come back after certain years, so on.
But come on! Paul can’t suddenly change his salary right! Neither can he wait for years for the loan. So he asks Jane what is the smallest change in his application he can do to get his loan approved.
If Jane were replaced by an AI model, what the model would give Paul is called the Counterfactual Explanation.
Counterfactual explanations provide the smallest change in the input feature values required to change the output of an instance to a predetermined/desired output
However, as in Paul’s case, not all features can be changed. So a requirement or constraint on Counterfactual explanations is that they perturb as few features as possible to obtain the desired output.
There are multiple ways of determining counterfactual explanations proposed by multiple researchers. If you are interested in understanding the difference between each of them, you can take a look at this book by Christoph Molnar.
However, a simple method to achieve this is as follows:
Consider the instance and the desired outcome for which you require a counterfactual explanationIdentify the decision boundary between the desired and the current outcomesFind minimum distance between the instance and the decision boundary, with the constraint that minimum number of coordinates (feature values) need to be changed
Consider the instance and the desired outcome for which you require a counterfactual explanation
Identify the decision boundary between the desired and the current outcomes
Find minimum distance between the instance and the decision boundary, with the constraint that minimum number of coordinates (feature values) need to be changed
Caveat: The thing with counterfactual explanations is that it is applicable only to supervised classification tasks. It is not amenable to regression or unsupervised tasks. It is important to know this
To illustrate the use of Counterfactual Explanations, I will be using an illustrated example from the alibi library. The example explains a shallow Neural Network model on the Boston House Pricing dataset. The code for the illustration can be found on the alibi documentation page.
Since the Boston House Dataset is present in the sklearn library, the dataset is loaded from sklearn directly.
Housing dataset is a regression dataset, on which counterfactuals are not effective. Hence we need to convert it into a classification type of dataset. We choose the median of the target value, which is the house price. Any data point (house in the set of houses) with a price less than median is labelled 0 and above the median is labelled 1.
The variant of counterfactual explanation used here is the one that is guided by prototypes. Here, prototypes are counterfactual examples (data points). They are built by building k-d trees or encoders, so that counterfactual explanations can be built fast. It is built based on this paper, which you can refer to for more details on how the algorithm works.
import tensorflow as tffrom tensorflow.keras.layers import Dense, Inputfrom tensorflow.keras.models import Model, load_modelfrom tensorflow.keras.utils import to_categoricalimport matplotlib%matplotlib inlineimport matplotlib.pyplot as pltimport numpy as npimport osfrom sklearn.datasets import load_bostonfrom alibi.explainers import CounterFactualProto
A custom neural network model is built for this illustration:
np.random.seed(42)tf.random.set_seed(42)def nn_model():    x_in = Input(shape=(12,))    x = Dense(40, activation='relu')(x_in)    x = Dense(40, activation='relu')(x)    x_out = Dense(2, activation='softmax')(x)    nn = Model(inputs=x_in, outputs=x_out)    nn.compile(loss='categorical_crossentropy', optimizer='sgd', metrics=['accuracy'])    return nnnn = nn_model()nn.summary()nn.fit(x_train, y_train, batch_size=64, epochs=500, verbose=0)
Now that we have our model, we want to understand a particular outcome. A particular test instance is chosen:
X = x_test[1].reshape((1,) + x_test[1].shape)shape = X.shape
The original label of this data point is 0, which indicates that the price of this house is below median price. Now let us run the counterfactual on it (CounterFactualProto calls counterfactual explanations by prototypes).
# initialize explainer, fit and generate counterfactualcf = CounterFactualProto(nn, shape, use_kdtree=True, theta=10., max_iterations=1000,                         feature_range=(x_train.min(axis=0), x_train.max(axis=0)),                         c_init=1., c_steps=10)cf.fit(x_train)explanation = cf.explain(X)
Let us print out the original and the explained outcomes (both should be different or contrastive to each other).
print(f'Original prediction: {explanation.orig_class}')print('Counterfactual prediction:     {}'.format(explanation.cf['class']))
We can determine how much of which value has changed between the original feature set and the explanation:
orig = X * sigma + mucounterfactual = explanation.cf['X'] * sigma + mudelta = counterfactual - origfor i, f in enumerate(feature_names):    if np.abs(delta[0][i]) > 1e-4:        print('{}: {}'.format(f, delta[0][i]))Output:AGE: -6.526239960695747LSTAT: -4.799340540220259
So this says that for the price to be above median, this house needed to have an age of 6.5 years lesser than it currently has, and an LSTAT value of 4.8 units less than it currently has.
Counterfactuals are the most natural way of explaining model behaviour to humans. However, it has certain limitations, the most important one of which is that it only applies to classification problems.
Another problem is that sometimes it provides explanations which, practically, cannot be fulfilled to reverse the decision. For example, age cannot be reversed. So we try to run counterfactuals by imposing that age cannot be changed.
Note that there can be multiple counterfactual explanations for a single datapoint, since there can be many ways to reach the decision boundary. We can choose the one that respects our business constraints of any other practical constraints.
Having said this, it is a powerful arrow in your XAI quiver!