Feature Instability Search for Multi-Way Explainability
Sean Kelly (skelly26@alumni.nd.edu) and Meng Jiang
University Of Notre DameR2HCAI: The AAAI 2023 Workshop on Representation Learning for Responsible Human-Centric AI

Motivation
Current explainability frameworks (LIME, SHAP):
1) Lack a reliable quantitative definition of explainability2) Aren’t evaluated on a true ground truth measure3) Fail to account for multi-way feature interactions

Contributionsi.

The concept of feature (in)stability as a measure of explainability of the output ofa modelii.

A synthetic model with deterministic ground truth multi-way feature explainability
to evaluate explainability frameworks

iii.

An informed stability descent based search algorithm as an attempt to quantifying
multi-way feature stability, or importance, for a given binary predictioniv.

A feature importance ranking evaluation loss function capable of comparing oneway feature explainability frameworks to more expressive frameworks (DFEST)

Results: Top Feature Interaction Clusters

DFEST: Feature Stability
Descent and Tensor Search
DFEST is a post-hoc XAI method to identify the k-most
unstable feature-interaction clusters through:(1) Uniform Distribution Search, followed by(2) Informed Cluster Search
DFEST Evaluation With Synthetic Ground Truth Model

Feature Instability

Figure A.2
Figure A.1

Uniform Distribution Search

Descending Ranked Feature Clusters Identified by DFEST

The 50 most unstable feature-interaction clusters demonstrates that the clusters with the highest
instability tend to have the same core unstable features, i.e. the MOST unstable features in each cluster

DFEST Feature Importance (Aggregation)
Figure 1a

LIME Feature Importance

Figure 1b• Feature instability is a measure of post-hoc explainability that explains thefeature interactions responsible for a given input source’s prediction• f1 is unstable and f2 is stable w.r.t. the model and any given input (xq) tothe model, representing a 1-way feature interaction, as only 1 feature isrelevant

Synthetic Ground Truth Model
Feature-Interaction Cluster Determination

Figure 3a

Algorithm 1• Search over the decision space has a time complexity of O(nk)• To gain heuristics for an informed search method, sparse solutionsof clusters giving an opposite model output are identified
surrounding the model output to be explained in decision space• Heuristics are identified via generation of an even distribution ofpoints on the surface of an n-sphere

Figure A.3a

Comparison of DFEST & LIME Loss

Algorithm 3• An inherently explainable n-dimensional decision space is generated, suchthat the distance from the origin (0,0) to a unique feature cluster i.e. (1,0)
or (1,1) is predetermined, with distances to the decision boundary
increasing as the block distance from the minimum cluster increases• Provides a ranked list of the most unstable features, with quantitativeinstability values derived from the distance, queryable by XAI methods• Informed cluster search is
implemented as A* search overthe priority queue of solutions
discovered above• Uniformly distributed heuristics
enable random restarts, asadjacent cluster feature stability(gradient) descent toward theinner decision boundary is
performedConclusion• The loss function accounts for both cluster variability
and feature instability magnitude, given continuous
input features

Informed Cluster Search

Figure 2b

Figure A.3b

• The synthetic ground truth model with deterministicfeature instability offers a trustworthy benchmark forthe development and evaluation of future XAI methods

Figure A.2

Equation 3• DFEST demonstrates a method to quantify the impact ofmulti-way feature interactions on a model’s output,
which is inherently out of scope for current feature
attribution methods which perform local linear function
approximation