QA 2 E xplanation: Generating and Evaluating Explanations for Question Answering Systems over Knowledge Graph (2020)

Abstract: In the era of Big Knowledge Graphs, Question Answering (QA) systems have reached a milestone in their performance and feasibility. However, their applicability, particularly in specific domains such as the biomedical domain, has not gained wide acceptance due to their “black box” nature, which hinders transparency, fairness, and accountability of QA systems. Therefore, users are unable to understand how and why particular questions have been answered, whereas some others fail. To address this challenge, in this paper, we develop an automatic approach for generating explanations during various stages of a pipeline-based QA system. Our approach is a supervised and automatic approach which considers three classes (i.e., success, no answer, and wrong answer) for annotating the output of involved QA components. Upon our prediction, a template explanation is chosen and integrated into the output of the corresponding component. To measure the effectiveness of the approach, we conducted a user survey as to how non-expert users perceive our generated explanations. The results of our study show a significant increase in the four dimensions of the human factor from the Human-computer interaction community.

## introduction the recent advances of question answering (qa) technologies mostly rely on (i) the advantages of big knowledge graphs which augment the semantics, structure, and accessibility of data, e.g., web of data has published around 150b triples from a variety of domains1, and (ii) the competency of contemporary ai approaches which train sophisticated learning models (statistical models @xcite , neural networks @xcite , and attention models @xcite ) on a large size of training data, and given a variety of 1http ://lodstats.aksw.org/ novel features captured from semantics, structure, and context of the background data. however, similar to other branches of ai applications, the state of the art of qa systems are "black boxes" that fail to provide transparent explanations about why a particular answer is generated. this black box behavior diminishes the confidence and trust of the user and hinders the reliance and acceptance of the black-box systems, especially in critical domains such as healthcare, biomedical, life-science, and self-driving cars @xcite . the running hypothesis in this paper is that the lack of explanation for answers provided by qa systems diminishes the trust and acceptance of the user towards these systems. therefore, by implementing more transparent, interpretable, or explainable qa systems, the end users will be better equipped to justify and therefore trust the output of qa systems @xcite . furthermore, data quality is a critical factor that highly affects the performance of qa systems. in other words, when the background data is flawed or outdated, it undermines the humanlikeness and acceptance of the qa systems if no explanation is provided, especially for non-expert users. for example, the sina engine @xcite failed to answer the simple question "what is the population of canada?" on the dbpedia @xcite version 2013, whereas it succeeded for similar questions such as "what is the population of germany?". the error analysis showed that the expected triple i.e., <dbr2:canada dbo3:population "xxx"> is missing from dbpedia 2013. thus, if the qa system does not provide any explanation about such failures, then the non-expert user concludes the qa system into the demerit points. thus, in general, the errors or failures of the qa systems might be caused by the inadequacies of the underlying data or misunderstanding, misinterpretation, or miscomputation of the employed computational models. in either case, the black-box qa system does not provide any explanations regarding the sources of the error. often the research community obsesses with the technical discussion of qa systems and competes on enhancing the performance of the qa systems, whereas, on the downside of the qa systems, there is a human who plays a vital role in the acceptance of the system. the human-computer interaction (hci) community already targeted various aspects of the human-centered design and evaluation challenges of black-box systems. however, the qa systems over kgs received the least attention comparing to other ai applications such as recommender systems @xcite . ## related work researchers have tackled the problem of question answering in various domains including open domain question answering @xcite , biomedical @xcite , geospatial @xcite , and temporal @xcite . question answering over publicly available kgs is a long-standing field with over 62 qa sys-tems developed since 2010 @xcite . the implementation of various qa systems can be broadly categorized into three approaches @xcite . the first is a semantic parsing based approach such as @xcite that implements a qa system using several linguistic analyses (e.g., pos tagging, dependency parsing) and linked data technologies. the second approach is an end-to-end machine learning based, which uses a large amount of training data to map an input question to its answer directly (e.g., in @xcite ). the third approach is based on modular frameworks @xcite ) which aims at reusing individual modules of qa systems, independent tools (such as entity linking, predicate linking) in building qa systems collaboratively. irrespective of the implementation approach, domain, and the underlying knowledge source (kg, documents, relational tables, etc.), the majority of existing qa systems act as a black box. the reason behind black box behavior is due to either the monolithic tightly coupled modules such as in semantic parsing based qa systems or nested and nonlinear structure of machine learning based algorithms employed in qa systems. the modular framework, on the other hand, provides flexibility to track individual stages of the answer generation process. the rationale behind our choice of the modular framework over monolithic qa systems is a flexible architecture design of such frameworks. it allows us to trace failure at each stage of the qa pipeline. we enrich the output of each step with adequate justification with supporting natural language explanation for the user. hence, as the first step towards explainable qa over knowledge graphs, we propose an automatic approach for generating a description for each stage of a qa pipeline in a state-of-theart modular framework (in our case: frankenstein @xcite ). we are not aware of any work in the direction of explainable question answering over knowledge graphs and we make the first attempt in this paper. although, efforts have been made to explain visual question answering systems. some works generate textual explanations for vqa by training a recurrent neural network (rnn) to mimic examples of human descriptions @xcite directly. the work by @xcite can be considered a closest attempt to our work. the authors proposed a template based approach to translate sparql queries into natural language verbalization. we employ a similar template-based approach to generate an automatic explanation for qa pipelines. in other domains, such as expert systems, the earlier attempts providing explanations to the users can be traced back in the early 70s @xcite . since then, extensive work has been done to include explanations in expert systems followed by recommender systems to explain the system's knowledge of the domain and the reasoning processes these systems employ to produce results (for details, please refer to @xcite @xcite . for a recommender system, work by @xcite is an early attempt to evaluate different implementations of explanation interfaces in "movielens" recommender system. simple statements provided to the customers as explanations mentioning the similarity to other highly rated films or a favorite actor or actress were among the best recommendations of the movielens system compared to the unexplained recommendations. furthermore, applications of explanation are also considered in various sub-domains of artificial intelligence, such as justifying medical decision-making @xcite , explaining autonomous agent behavior @xcite , debugging of machine learning models @xcite , and explaining predictions of classifiers @xcite . ## qa pipeline on knowledge graph one of the implementation approaches for answering questions from interlinked knowledge graphs is typically a multi-stage process which is called qa pipeline @xcite . each stage of the pipeline deals with a required task such as named entity recognition (ner) and disambiguation (ned) (referred as entity linking (el)), relation extraction and linking (rl), and query building (qb). there is an abundance of components performing qa tasks @xcite . these implementations run on the kgs and have been developed based on ai, nlp, and semantic technologies, which accomplish one or more tasks of a qa pipeline @xcite . ask {dbr:nikola_tesla dbo:award dbr:nobel_prize_in_physics.}). the performance values in ## approach a full qa pipeline is required to answer a given question q. such qa pipelines are composed of all the required components performing necessary tasks to transform a user-supplied natural language (nl) question into a formal query language (i.e., sparql). we consider three generic classes for outputs of a full qa pipeline or individual components, namely o @xmath0, noanswer,wrong answer}. concerning a given question, a "success" class is when the qa pipeline (component) successfully provides a correct output, a "no answer" class happens when the full qa pipeline (or an individual component) does not return any output and "wrong answer" class is when the provided output is incorrect. to address rq1, we introduce a scheme for generating explanations for the qa pipeline system. this scheme produces shallow, however automatic 4component is similar to relation linker of https:// github.com/dice-group/nliwod 5component is based on https://github.com/ dice-group/nliwod and @xcite . ## experimental study we direct our experiment in response to our two research questions (i.e., rq1 and rq2) respectively. first, we pursue the following question "how effective is our approach for generating explanations?" this evaluation implies the demonstration of the success of our approach in generating proper explanations. it quantitatively evaluates the effectiveness of our approach. on the contrary, the second discourse of the experiment is an hci study in response to the question "how effective is the perception of the end user on our explanations?" this experi-ment qualitatively evaluates user perception based on the human factors introduced earlier (cf. section 1). in the following subsections, we detail our experimental setups, achieved results, and insights over the outcomes of the evaluation. ## discussion in this paper, we focus on the challenge of explainable qa systems. we mainly target systems that consume data from the kgs. these systems receive a natural language question and then transform that to a formal query. our primary aim is to take the initial steps to break down the full black-box qa systems. thus, we reuse an existing qa pipeline systems since it already decompose the prominent tasks of the qa systems and then integrate individual implementations for each qa task. we based our approach and associated evaluation on the hypothesis that every component integrated into the pipeline should explain the output. it will educate and involve non-expert users and trigger them to trust and accept the system. our findings in section 5 support our hypothesis both on quantitative and qualitative evaluation. the limitation of our approach is that it heavily relies on the performance of the components. in the case of having low performing components, the accuracy of the classifiers is also downgraded. although, on the one hand, this approach is shallow, one the other hand it avoids exposing the user to overwhelming details of the internal functionalities by showing succinct and user-friendly explanations. @xcite noted that for improving the usability of xai systems, it is essential to combine theories from social science and cognitive decision making to validate the intuition of what constitutes a "good explanation." our work in this paper is limited to predefined template based explanations, and does not consider this aspect. also, our work does not focus on the explainability of the behavior of the employed classifier, and the explanations only justify the final output of components. ## conclusion and future direction in this paper, we proposed an approach that is automatic and supervised for generating explanations for a qa pipeline. albeit simple, our approach intuitively expressive for the end user. this approach requires to train a classifier for every integrated component, which is costly in case the components are updated (new release) or replaced by a latest outperforming component. our proposed approach induced in a qa pipeline of a modular framework is the first attempt for explainable qa systems over kgs. it paves the way for future contributions in developing explainable qa systems over kgs. still, there are numerous rooms in this area that require the attention of the research communityfor example, explanations regarding the quality of data, or metadata, or credibility of data publishers. furthermore, recent attempts have been made to provide explanations of machine learning models @xcite . however, the inclusion of the explanations in neural approaches for question answering (such as in @xcite ) is still an open research question, and we plan to extend our work in this direction. the concerning domain of the system is also influential in explanations. for example, biomedical or marketing domains require various levels of details of explanations. in general, all of these concerns affect the acceptance and trust of the qa system by the end user. our ambitious vision is to provide personalized and contextualized explanations, where the user feels more involved and educated.