The How of Explainable AI: Explainable Modelling
By: Bahador Khaleghi
Tags: AI, Explainability, Machine Learning, Trust, Blackbox

Explainable modelling

Achieving explainable modelling is sometimes considered synonymous with restricting the choice of AI model to specific family of models that are considered inherently explainable. We will review this family of AI models. However, our discussion goes far beyond the conventional explainable model families and includes more recent and novel approaches such as joint prediction and explanation, hybrid models, and more.

Ideally we can avoid the black-box problem from the beginning by developing a model that is explainable by design.

Adopting an inherently explainable model family

The traditional approach to achieve explainable modelling is to adopt from a specific family of models that are considered explainable. This family of models often provide one (or more) of the three levels of model transparency proposed by Zack Lipton, namely, simulatability, decomposability, and algorithmic transparency. Examples of these families include linear models, decision trees, rule sets, decision sets, generalized additive models, and case-based reasoning methods.

In practice, however, simply adopting a model from an explainable family does not automatically guarantee explainability. For instance, a linear regression model that is operating on high dimensional input may not be simulatable, and hence, not explainable. To avoid this issue, we could limit the number of important input features during model training using some sort of regularization, e.g. L1 norm as in the Lasso regression. In addition, in the case of feature multicollinearity, that is, features that are correlated due to being caused by, either observed or unobserved confounding feature(s), the coefficients estimated for the linear regression model could be unstable. Further regularization, e.g. L2 norm, could be applied to alleviate this issue.

Simply adopting a model from an explainable family does not automatically guarantee explainability.

While some mitigations are available for specific methods, explainable model families are in general relatively simple and thus inadequate for capturing the complexity of some real-world problems. This observation has led to the so-called explainability vs performance tradeoff, namely, the more performant a model, the less explainable it is, and vice versa.

The alleged explainability vs performance tradeoff: the higher the model performance, the lower its explainability, and vice versa.

Yet a number of researchers have shown that the alleged explainability vs performance tradeoff does not always hold by developing models that are both explainable and performant. The main challenge when developing such a model is to devise it to be simple enough to be understood by its audience, yet sophisticated enough to properly fit the underlying data. There are a relatively large number of related methodologies in this category. So, we group them based on their key underlying, and possibly overlapping, ideas to make our presentation easier to follow.

A number of researchers have shown that the alleged explainability vs performance tradeoff does not always hold by developing models that are both explainable and performant.

Hybrid explainable models

It might be possible to hybridize an inherently explainable modelling approach with a complex black-box method to devise a high-performance and explainable model. This is the underlying intuition of hybrid explainable models.

The deep k-Nearest Neighbours (DkNN) approach proposes to use K-nearest neighbour (kNN) inference on the hidden representation of training dataset learnt through layers of a deep network. The conformal prediction framework is then used to combine kNN predictions of all layers in a mathematically rigorous manner. The DkNN is shown to be efficient and robust while providing example-based explanations for its predictions in terms of nearest training samples used in each layer. However, DkNN requires storing hidden representation of the entire training dataset, which can be infeasible for large datasets.

The Deep Weighted Averaging Classifier (DWAC) method resembles DkNN in terms of making its prediction, within the conformal prediction framework, by relying on labels of training samples similar to given input instance. However, the similarity is computed using only the low-dimensional representation of the final layer.