M a N t LE : Model-agnostic Natural Language Explainer (2023)

Abstract: Understanding the internal reasoning behind the predictions of machine learning systems is increasingly vital, given their rising adoption and acceptance. While previous approaches, such as LIME generate algorithmic explanations by attributing importance to input features for individual examples, recent research indicates that practitioners prefer examining language explanations that explain sub-groups of examples (Lakkaraju et al., 2022). In this paper, we introduce MaNtLE, a model-agnostic natural language explainer that analyzes a set of classifier predictions and generates faithful natural language explanations of classifier rationale for structured classification tasks. MaNtLE uses multi-task training on thousands of synthetic classification tasks to generate faithful explanations. Our experiments indicate that, on average, MaNtLE-generated explanations are at least 11% more faithful compared to LIME and Anchors explanations across three tasks. Human evaluations demonstrate that users can better predict model behavior using explanations from MaNtLE compared to other techniques.

## introduction the increasing adoption of black-box machine learning models across various applications @xcite has led to a pressing need for human-understandable explanations of their decision-making process. while such models may yield high predictive accuracies, their underlying reasoning often remains opaque to users. this lack of transparency is a critical barrier to their adoption in critical domains, such as healthcare, law, and medicine. to interpret machine learning model decisions, previous research proposed techniques like feature importances @xcite ), rule f true degree hrs/ week income hs 29 <=50k bs 40 >50k ms 36 >50k bs 50 >50k if user age greater than 20 and user gender is equal to f, then product relevance is certainly true if degree not equal to hs, then income is @xmath0 figure 1 : mantle is a model-agnostic natural language explainer. following massive multi-task learning, mantle can generate explanations of decision-making rationale for new classifiers and datasets. in this paper, we introduce mantle, a modelagnostic natural language explainer that analyzes multiple classifier predictions and generates faithful natural language explanations of the classifier's reasoning for structured classification tasks, as depicted in figure 1 . the goal of mantle is to explain the rationale of classifiers on realworld tasks. to develop mantle, we fine-tune a t5-large model on thousands of synthetic classification tasks, each paired with natural language explanations, in a multi-task learning setup following recent research @xcite @xcite . in §3.5, we discuss inference procedures to improve explanation quality and adapt the model trained on synthetic data for real-world tasks. we test mantle explanations on real-world tasks by assessing their ability to aid explanation-guided zero-shot classifiers in the clues-real benchmark @xcite . our results show that mantle explanations are as helpful as human-written explanations in guiding classifiers ( §5.2). we also compare the faithfulness @xcite and simulatability @xcite of explanations generated by mantle, lime, and anchors for four classification techniques on three real-world tasks ( §5.3). when the number of available examples is comparable, mantle explanations are, on average, 37% more faithful than lime and 11% more faithful than anchors. in user studies ( §5.4), we observe that users without machine learning expertise prefer explanations from mantle over attribution-score-based explanations by lime (overall preference of 3.44 vs 2.16 on a five-point likert scale; @xmath1.001 ttest). further, users can predict model behavior better using mantle explanations in at least 25% more instances than lime and anchors for the adult dataset @xcite . our results corroborate the findings of @xcite on the need for providing natural language explanations that explain subgroups of examples to help practitioners interpret ml models. we find that increasing the number of examples accessible by mantle can enhance explanation quality, highlighting possibilities for using the model in resourcerich settings. finally, we analyze the contributions of the number of multitask-training tasks used and model size on the quality of mantle's generated explanations. in summary, our contributions are: feature attribution techniques @xcite @xcite provide information about how models use certain features to derive classification decisions, utilizing methods such as sparse linear regression models @xcite and rule lists @xcite . these methods have two shortcomings: (a) they explain model behavior in a local region around an example, and (b) they generate explanations by solving a search problem over a set of instances from the original training data or by generating perturbations around an example (typically, ∼ 1000 examples). however, gaining access to model predictions on thousands of examples can be resource-intensive or financially unviable for large models, such as gpt-3 @xcite . recent works like cage @xcite and wt5 @xcite ) explore natural language explanations for language understanding tasks. however, these explanations are specific to individual examples and focus on improving classification performance rather than interpreting model behavior. moreover, training these models often demands a significant number of human-written explanations, which can be impractical. our work diverges from this line of research as we seek to explain model behavior rather than improve classification models, using only a few examples. ## interpreting classifiers with mantle in this section, we compare the quality of mantle explanations to two popular explanation methods in lime and anchors using simulated ( §5.3) and human user studies ( §5.4). before the experiments, we briefly describe the baselines ( §5.1) and datasets ( §5.2) that we use in our experimentation. ## conclusion in this work, we introduce mantle, a modelagnostic natural language explainer, that generates faithful explanations for structured classifiers. we use recent insights in massive multi-task training to train a model that can generate faithful classifier rationale. as mantle can explain classifiers simply by inspecting predictions made by classifiers, it can be used in a model-agnostic fashion similar to popular explanation methods like lime and anchors. in simulation studies and human evaluations, we show that mantle explanations are more faithful than lime and comparable in faithfulness to anchors on multiple datasets. our work suggests the potential for natural language interfaces to enhance end-user trust and promote the safe usage of ml models in critical tasks by providing explanations of the classifier decision-making process. future work can look to extend our work to develop "patches" @xcite for improving classifiers, refining decoding techniques for more faithful and simulatable explanations, and integrating more complex reasoning patterns in generated explanations. further, developing models that can generate language explanations for classifiers trained on other modalities, such as raw text and vision, is an interesting avenue for future work.