Towards Harnessing Natural Language Generation to Explain Black-box Models (2020)

Abstract: The opaque nature of many machine learning techniques prevents the wide adoption of powerful information processing tools for high stakes scenarios. The emerging field eXplainable Artificial Intelligence (XAI) aims at providing justifications for automatic decision-making systems in order to ensure reliability and trustworthiness in the users. For achieving this vision, we emphasize the importance of a natural language textual modality as a key component for a future intelligent interactive agent. We outline the challenges of XAI and review a set of publications that work in this direction.

## introduction in recent times the use of machine learning (ml) has changed many fields across a wide range of domains, revealing the potential for an information processing revolution in our society @xcite . even though there already exist many commercial applications that use ml for delivering products, these are limited by the often opaque nature of the underlying models @xcite . in fact, to produce highly predictive models that reach high-performance metrics on given tasks, commercial products often end up with models whose behavior and rationale in making decisions are not clearly understandable by humans. this is a big issue in all those applications where trust and accountability in the prediction have the highest priority like healthcare, military, finance, or autonomous vehicles. this need for explainable models has made many big institutions, including the european union @xcite , and the us defense advanced research projects agency (darpa) @xcite push for funding research in explainable artificial intelligence (xai), a relatively new and very active research area with the aim of providing human insight into the behavior of informationprocessing tools. the three main xai challenges are: (1) designing explainable models; (2) implementing explanation interfaces; and (3) measuring the effectiveness of the generated explanations. of the many ways of presenting an explanation, natural language is particularly attractive as it allows people with diverse backgrounds and knowledge to interpret it @xcite , thus potentially allowing the interested end-user to understand the model without requiring a detailed background in mathematics and information engineering. this is a mandatory step if we want to make these tools available to the non-technical wider population. the goal of this paper is to provide a general overview of tools and approaches for providing linguistic explanations of ml models to general users. the rest of the paper is organized as follows. in section 2 we present a brief overview of xai field and its challenges. in section 3 we explore how xai can integrate with natural language generation (nlg). finally, we summarize the main conclusions in section 4. ## open challenges in xai as mentioned in the introduction, xai faces three main challenges: models, interfaces and evaluations. in this section, we provide a high-level overview of each of them. ## explaining with natural language an explanation can be laid out using different modalities. the general trend in the literature is to represent results in a graphical visual form, but some researchers are using natural language and measuring an increased benefit for the end-user. nlg-based approaches fall into two broad categories: template-based and end-to-end generation. ## conclusions xai is an emerging field that aims to providing explanations for decision tools that will enable them to gain trust in their users and their wide adoption by the market. in order to achieve this, textual explanations are essential but to date few works have directly addressed this possibility. current trends in explainability push toward making intrinsically more interpretable models or in making opaque models more understandable. there is no agreed-upon definition of explanation and further theoretical work should try to bridge the gap between the large corpus of theoretical speculation coming from social sciences and the empirical work pursued in artificial intelligence. this as yet ill-defined nature of the task leaves much work to do in the standardization of processes for measurement of explanation effectiveness. in this regard, both objective and subjective measures should be considered, especially if evaluation involves human participants. moreover, since the explanation process is dependent on who is receiving the explanation, we envision an interactive agent that is able to dialogue with the user. from this perspective, the nlg community can contribute significantly to this goal by providing a linguistic layer to the many xai methods being proposed so far.