Is Multi-Hop Reasoning Really Explainable? Towards Benchmarking Reasoning Interpretability (2021)

Abstract: Multi-hop reasoning has been widely studied in recent years to obtain more interpretable link prediction. However, we find in experiments that many paths given by these models are actually unreasonable, while little work has been done on interpretability evaluation for them. In this paper, we propose a unified framework to quantitatively evaluate the interpretability of multi-hop reasoning models so as to advance their development. In specific, we define three metrics, including path recall, local interpretability, and global interpretability for evaluation, and design an approximate strategy to calculate these metrics using the interpretability scores of rules. We manually annotate all possible rules and establish a benchmark. In experiments, we verify the effectiveness of our benchmark. Besides, we run nine representative baselines on our benchmark, and the experimental results show that the interpretability of current multi-hop reasoning models is less satisfactory and is 51.7% lower than the upper bound given by our benchmark. Moreover, the rule-based models outperform the multi-hop reasoning models in terms of performance and interpretability, which points to a direction for future research, i.e., how to better incorporate rule information into the multi-hop reasoning model. We will publish our codes and datasets upon acceptance.

## introduction multi-hop reasoning for knowledge graphs (kgs) has been extensively studied in recent years. it not only infers new knowledge but also provides reasoning paths that can explain the prediction results and make the model trustable. for example, figure 1 shows two inferred triples and their reasoning paths. conventional kg embedding models (e.g., transe @xcite ) implicitly find the target entity piano given the query (bob seger, instrument, ?), while multi-hop reasoning models complete the triple and explicitly output the reasoning path (the purple solid arrows). hence, multi-hop reasoning is expected to be more reliable in real systems, as we can safely add an inferred triple to kg by justifying if the path is reasonable. ## preliminary knowledge graph (kg) is defined as a directed graph @xmath0, r, t }, where e is the set of entities, r is the set of relations and @xmath1, r, t)} ⊆ e × r × e is the set of triples. multi-hop reasoning aims to complete kgs through interpretable link prediction. formally, given a triple query (h, r, ?), it needs to not only predict the correct tail entity t, but also give a path (h, r, t) ← (h, r 1 , e 1 ) ∧ (e 1 , r 2 , e 2 ) ∧ • • • ∧ (e n-1 , r n , t) as an explanation. rule-based reasoning can be considered as generalized multi-hop reasoning and can also be evaluated on our benchmark. given a triple query (h, r, ?), it needs to predict the tail entity t and give some horn rules with confidence as an explanation, where the rule f is of the following form: ## benchmark in order to quantitatively evaluate the interpretability of multi-hop reasoning models, we first construct a dataset based on wikidata (section 4.1). after that, we propose a general evaluation framework (section 4.2). based on this framework, we apply an approximation strategy (section 4.3) and build benchmarks with manual annotation (section 4.4) and mined rules (section 4.5). ## conclusion multi-hop reasoning for kgs has been widely studied in recent years. however, most previous works are conducted on the assumption that the reasoning paths are reasonable. as a result, most of them only care about the model's performance and neglect the evaluation of path interpretability. in this paper, we propose a framework to quantitatively evaluate the interpretability of multi-hop reasoning models. based on this framework, we annotate a dataset to form a benchmark named bimr that can accurately evaluate the interpretability. besides, we also construct benchmarks with generalization ability using mined rules. experimental results show that our manually-annotated benchmark achieves similar results to the golden truth, indicating that it can be used to evaluate the model's interpretability automatically. our experimental results show that the rule-based model anyburl outperforms the current multi-hop reasoning models in terms of link prediction performance and interpretability, which indicates a possible future research direction, i.e., how to better incorporate rule information into the multi-hop reasoning model to improve the performance and interpretability.