Improve Interpretability of Neural Networks via Sparse Contrastive Coding (2022)

Abstract: Although explainable artificial intelligence (XAI) has achieved remarkable developments in recent years, there are few efforts have been devoted to the following problems, namely, i) how to develop an explainable method that could explain the black-box in a model-agnostic way? and ii) how to improve the performance and interpretability of the black-box using such explanations instead of pre-collected important attributions? To explore the potential solution, we propose a model-agnostic explanation method termed as Sparse Contrastive Coding (SCC) and verify its effectiveness in text classification and natural language inference. In brief, SCC explains the feature attributions which characterize the importance of words based on the hidden states of each layer of the model. With such word-level explainability, SCC adaptively divides the input sentences into foregrounds and backgrounds in terms of task relevance. Through maximizing the similarity between the foregrounds and input sentences while minimizing the similarity between the backgrounds and input sentences, SSC employs a supervised contrastive learning loss to boost the interpretability and performance of the model. Extensive experiments show the superiority of our method over five state-of-the-art methods in terms of interpretability and classification measurements. The code is available at https://pengxi.me.

## introduction deep neural networks (dnns) have achieved remarkable progress during the past few years. however, relying on stacking somewhat ad-hoc modules, dnns are often referred to as "black-box" methods that lack understanding of the working mechanisms, thus increasing the risk of applying them into real-world applications (ribeiro et al., * the first two authors contributed equally. † corresponding author. improve performance feature attributions great music story great and neural network post-hoc explain improve performance feature attributions great music story great and improve interpretability neural network ## related work this work is closely related to model explainability and interpretable models which will be briefly introduced in this section. ## method this section elaborates on the proposed sparse contrastive coding (scc) which tries to seek a feasible solution to the aforementioned two problems in xai, i.e., i) how to develop an explainable method that could explain the black-box in a model-agnostic way? and ii) how to improve the performance and interpretability of the black-box using such explanations? as illustrated in fig. 2 , scc explains and improves the black-boxes through three jointly optimizing objectives, namely, sparse coding loss l sc , contrastive coding loss l cc , and cross entropy loss ## experiments in this section, we carry out experiments on six widely-used textual datasets. for a comprehensive study, we compare scc with five state-of-the-art approaches on two classification tasks (i.e., sentiment analysis and subjective/objective classification) and natural language inference (nli) task in terms of the metrics of interpretability and classification performance. ## conclusion in this paper, we show a feasible solution to solve two less-touched problems in xai, i.e., how to develop a model-agnostic method to explain the black-box and utilize the explanations to improve the model performance and interpretability. we take text classification and natural language inference as evaluation tasks, and quantitatively and qualitatively show the superiority of our method in terms of interpretability and classification metrics. in the future, we plan to explore the potential of our framework in other applications like medical diagnosis and extend our idea to other data domains like images.