A Hierarchical Explanation Generation Method Based on Feature Interaction Detection (2023)

Abstract: The opaqueness of deep NLP models has motivated efforts to explain how deep models predict. Recently, work has introduced hierarchical attribution explanations, which calculate attribution scores for compositional text hierarchically to capture compositional semantics. Existing work on hierarchical attributions tends to limit the text groups to a continuous text span, which we call the connecting rule. While easy for humans to read, limiting the attribution unit to a continuous span might lose important long-distance feature interactions for reflecting model predictions. In this work, we introduce a novel strategy for capturing feature interactions and employ it to build hierarchical explanations without the connecting rule. The proposed method can convert ubiquitous non-hierarchical explanations (e.g., LIME) into their corresponding hierarchical versions. Experimental results show the effectiveness of our approach in building high-quality hierarchical explanations.

## introduction the opaqueness of deep natural language processing (nlp) models has increased along with their power @xcite , which has prompted efforts to explain how these "black-box" models work @xcite . this goal is usually approached with attribution method, which assesses the influence of inputs on model predictions @xcite @xcite prior lines of work on attribution explanations usually calculate attribution scores for predefined text granularity, such as word, phrase, or sentence. recently, work has introduced the new idea of hierarchical attribution, which calculates attribution scores for compositional text hierarchically to capture more information for reflecting model predictions @xcite @xcite as shown in fig -ure 1, hierarchical attribution produces a hierarchical composition of words, and provides attribution scores for every text group. by providing compositional semantics, hierarchical attribution can give users a better understanding of the model decisionmaking process. @xcite . however, as illustrated in figure 1 , recent work @xcite @xcite uses continuous text to build hierarchical attributions, which we call the connecting rule. while consistent with human reading habits, using the connecting rule as an additional prior might lose important long-distance compositional semantics. the concerns are summarized as follows: first, modern nlp models such as bert @xcite and gpt @xcite are almost all transformer-based, which use self-attention mechanisms @xcite to capture feature interactions. since all interactions are calculated in parallel in self-attention mechanism, the connecting rule that only considering neighboring text is incompatible with the basic operation principle of these nlp models. second, unlike the example in figure 1 , nlp tasks often require joint reasoning of different parts of the input text (chowdhary, 2020). for example, figure 2 (a) shows an example of natural language interface (nli) task foot_0 , in which 'has a' and 'avail- in this work, we introduce a simple but effective method for generating hierarchical explanations without the connecting rule. moreover, we introduce a novel strategy for detecting feature interactions in order to capture compositional semantics. unlike earlier hierarchical attribution approaches, which use specific algorithms to calculate attribution scores, the proposed method can convert ubiquitous non-hierarchical explanations (e.g., lime) into their corresponding hierarchical versions. we build systems based on two classic non-hierarchical methods: loo (lipton, 2018) and lime @xcite , and the experimental results show that both systems significantly outperform existing methods. furthermore, the ablation experiment additionally reveals detrimental effects of the connecting rule on the construction of hierarchical explanations. our implementation and genenerated explanations are available at an anonymous website: https://github.com/ juyiming/he_examples . ## method this section explains the strategy for feature interaction detecting and the algorithm on building hierarchical explanations. ## experiment we build systems with leave-one-out (loo) (lipton, 2018) and lime @xcite as the basic attribution algorithms, denoted as he loo and he lime . to reduce processing costs, we limit the maximum number of the hierarchical layers to ten in he lim e . ## conclusion in this work, we introduce an effective method for generating hierarchical explanations without the connecting rule, in which a novel strategy is used for detecting feature interactions. the proposed method can convert ubiquitous non-hierarchical explanations into their corresponding hierarchical versions. we build systems based on loo and lime. the experimental results demonstrate the effectiveness of proposed approach.