# A brief review of explanation in the Semantic Web

Rakebul Hasan, Fabien Gandon

► To cite this version:

Rakebul Hasan, Fabien Gandon. A brief review of explanation in the Semantic Web. ExaCt 2012 - 7th Workshop on Explanation-aware Computing - (20th European Conference on Artificial Intelligence ECAI-2012), Aug 2012, Montpellier, France. hal-00729037

HAL Id: hal-00729037

https://inria.hal.science/hal-00729037v1

Submitted on 27 Sep 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# A Brief Review of Explanation in the Semantic Web

Rakebul Hasan<fnref n="1" /> and Fabien Gandon<fnref n="2" />

**Abstract.** Semantic Web applications use interconnected distributed data and inferential capabilities to compute their results. The users of Semantic Web applications might find it difficult to understand how a result is produced or how a new piece of information is derived in the process. Explanation enables users to understand the process of obtaining results. Explanation adds transparency to the process of obtaining results and enables user trust in the process. The concept of providing explanation was first introduced in expert systems and later studied in different application areas. This paper provides a brief review of existing research on explanation in the Semantic Web.

## 1 INTRODUCTION

Semantic Web applications use interconnected distributed data and inferential capabilities to compute their results. A user might not be able to understand how a Semantic Web application has solved a given query deriving new information and integrating information from data sources across the Web, and therefore the user might not trust the result of such a query. Semantic Web applications should provide explanations about how they obtain results in order to ensure their effectiveness and increase their user acceptance [21]. Semantic Web applications should not only provide explanations about how the answers were obtained, they should also explain and allow users to follow the flows of information between them [20].

Expert systems were among the first software systems to include explanation facilities [13, 22, 28]. Explanation facilities in expert systems have evolved from reasoning trace oriented explanations, primarily useful for developers and knowledge engineers, to more user oriented interactive explanations justifying why a system behavior is correct, to casual explanations generated in a decoupled way from the line of reasoning. The realization of the explanation facilities in expert systems were motivated by enabling transparency in problem solving, imparting an understanding of why and how a given conclusion was reached, and hence enabling trust on the reasoning capabilities of expert systems. These developments motivated adaptation and development of explanation facilities in other fields such as machine learning [10, 27], case-based reasoning [8, 26], recommendation systems [29], and Semantic Web.

In this paper, we provide a brief review of the existing approaches to explanation in the Semantic Web. Our selection criterion for the reviewed works was Semantic Web applications and publications which have contribution in explanation. We have used the keyword search feature in Google Scholar<fnref n="3" /> and also its Cited by feature for the relevant publications for discovering more publications. In addition,

we have examined all the previous publications of the Explanation-aware Computing workshop series. We have also examined the previous publications of the International Semantic Web Conference series. We selected a given work if it is in the domain of the Semantic Web and it has contribution in the field of explanation. Our objective was to extract and analyze the important aspects of explanation-aware Semantic Web systems from these reviewed research, and finally to provide our perspective on these aspects. A detailed version of this paper can be found in the research report in [11].

The paper is organized as follows. In Section 2, we present the requirements and the design principles of explanation-aware Semantic Web applications. In Section 3, we provide an overview of different approaches to represent the metadata which enable support for reasoning and provenance information. In Section 4, we provide an overview of different approaches for generation and presentation of explanations. In Section 5, we focus our discussion on the important aspects of explanation approaches in the Semantic Web. In Section 6, we conclude the paper.

## 2 DESIGNING EXPLANATION-AWARE SEMANTIC WEB APPLICATIONS

McGuinness et al. [20] discuss the requirements for Semantic Web applications from an explanation perspective. The paradigm shift introduced by the Semantic Web applications, from answering queries by retrieving explicitly stored information to using inferential capabilities, generates new requirements to ensure their effective use: “applications must provide explanation capabilities showing how results were obtained”. McGuinness et al. characterize collaboration, autonomy, and the use of ontologies as the important features of Semantic Web applications from an explanation perspective. Given these features, different types of explanations, machine and human consumption of explanations, and trust are the important criteria of Semantic Web applications.

In [21], McGuinness et al. present requirements for distributed and portable justifications, and subsequently present justifications as user-friendly explanations. An explanation infrastructure should support knowledge provenance to allow users to understand the source information used in reasoning processes and hence enable user trust on background reasoners. Support for reasoning information should be also provided to enable users to understand what steps have been performed by reasoners. An explanation infrastructure should support generating human understandable explanation. Explanations should be presented with different degrees of detail taking into account the users’ expertise and problem context.

Forcher et al. present the explanation-aware system design (EASD) principles in [9]. The EASD principles concern two key aspects, namely the development and the runtime of a system. The integration of explanation capabilities in a system during its devel-

<footnote n="1">¹ INRIA Sophia Antipolis – Wimmics, France, email: hasan.rakebul@inria.fr</footnote>

<footnote n="2">² INRIA Sophia Antipolis – Wimmics, France, email: fabien.gandon@inria.fr</footnote>

<footnote n="3">³ http://scholar.google.com/</footnote>
opment should not be too complicated and should not effect system
performance and efficiency. Concerning the runtime aspect, the sys-
tem must be aware of the explanation scenario and must be able pro-
vide explanation accordingly during its runtime. Forcher et al. com-
plement the EASD principles with an abstract architecture of a multi-
layered explanation model considering the main participants in any
explanation scenario.

3 METADTA FOR EXPLANATION

Different approaches use different kinds of metadata for generat-
ing explanations in the context of Semantic Web. McGuinness et
al. [19] describe explanation as “Semantic Web metadata about how
results were obtained”. Provenance metadata concerning information
sources such as how, when, and from whom any given piece of data
is obtained is an important aspect of explanation metadata. Explana-
tions with detailed provenance information provide additional con-
text and enable users to verify a given source of information. Meta-
data representing information manipulation steps and their depen-
dencies are commonly known as justifications. Justifications facili-
tate rich explanation of how a conclusion was drawn. Trust related
metadata is another useful aspect which enables providing explana-
tion with integrated trust information.

In [25], McGuinness et al. present an explanation interlingua called Proof Markup Language (PML)⁴ to represent explanation metadata. PML consists of three OWL ontologies. The PML provenance ontology (PML-P) provides primitives for representing real world things (e.g. information, documents, people) and their properties (e.g. name, creation date-time, description, owners and authors). The PML justification ontology (PML-J) provides primitives for encoding justifications for derivations of conclusions. A justification can be a logical reasoning step, or any kind of computation process, or a factual assertion or assumption. The PML trust ontology (PML-T) provides primitives for representing trust assertions concerning sources and belief assertions concerning information.

The authors in [24] introduce a restricted subset of PML constructs and tools for encoding very basic justifications and performing tasks such as retrieval and browsing of provenance information. The authors present strategies for lightweight use of PML and for simplifying encoding using PML. PML-Lite⁵ is another variant of PML. It is envisioned to construct a simple subset of three PML modules. PML-Lite takes an event based modeling approach. It provides primitives to represent provenance of data flows and data manipulations.

The AIR (Accountability In RDF) [16] rule language uses the AIR Justification Ontology (AIRJ) to represent justifications produced by the AIR reasoner. AIRJ extends the PML-Lite event-based approach. The reasoning steps of AIR reasoner are interpreted as events. AIRJ provides primitives to represent the different events and the operations performed by the AIR reasoner.

4 GENERATION AND PRESENTATION OF
EXPLANATION

As discussed in explanation requirements, what types of explana-
tions are generated and how they are presented to users are important
criteria for success of explanation-aware systems.

OntoNova [2] is an ontology-based question answering system in
chemistry domain. OntoNova provides explanations in natural lan-
guage with its answers. It generates answer justifications in a meta-

inferencing step. The OntoNova inference engine produces log files
which represent proof trees for answers. These files are given as an
input to a second meta-inference step. This second meta-inference
step explains the proof trees in natural language with the description
of how answers were derived. OntoNova allows specifying meta-
inference rules for the original rules for question answering. The
two step method for providing explanation has advantages such as:
(i) provision of additional information with explanations when proof
trees do not contain enough information, (ii) filter explanation paths
in case of redundancies for same results, (iii) provision of explana-
tion with different degrees of detail, (iv) provision of personalized
explanation for different contexts.

Inference Web [19, 20, 21] is an explanation infrastructure which addresses explanation requirements of web services discovery, policy engines, first order logic theorem provers, task execution, and text analytics. Information manipulation traces of these various kinds of systems are encoded as PML proofs. Inference Web provides a set of software tools and services for building, presenting, maintaining, and manipulating PML proofs. The IWBase component of Inference Web provides an interconnected network of distributed repositories of explanation related meta information. IWBase provides a registry-based solution for publishing and accessing information. Content publishers can register metadata and other supporting information such as inference rules, and inference engines. IWBase provides services to populate PML proofs. IWBase exposes the populated metadata as PML documents and provides browsing interfaces to access them. These PML documents can be also accessed by resolving their URI references. The IWSearch component of Inference Web searches for PML documents on the Web and maintains an inventory of these documents. Users can then search for PML documents using different search interfaces offered by IWSearch. Inference Web provides a browser called IWBrowser which can display PML proofs and explanations in number of different formats and styles. The rich presentations include a directed acyclic graph (DAG) view known as global view, a focused view enabling step-by-step navigation between related explanations, a filtered view to show selected parts of an explanation, an abstraction view which shows abstract views of explanations with different degrees of detail, and finally a discourse view which allows follow-up questions. The IWAbtractor component of Inference Web allows users to write abstraction patterns for PML proofs. It matches these patterns against PML proofs to provide an abstract view. Inference Web includes a general trust infrastructure called IWTrust which provides explanations with trust related information and allows filtering out unreliable information. IWTrust includes representation of trust aspects and trust computation services. In addition, it provides a browser with trust view to render the trust annotation. In the trust view, different fragments of a page are rendered in different colors depending on the trustworthiness of them. This allows users to have an understanding of the trustworthiness of rendered information just by looking at a rendered page.

The WIQA - Web Information Quality Assessment Framework [6] provides functionalities for quality-based information filtering. The WIQA framework allows to employ different policies for information filtering. These policies combine content-based, context-based, and rating-based quality assessment metrics. An information consumer's understanding of the employed quality assessment metrics is a major factor that influence whether the information consumer trust or distrust any quality assessment result. The WIQA framework provides detailed explanation of information filtering process for supporting information consumers in their trust decisions. It is able to provide explanation of why a given piece of information satisfies a

⁴ http://tw.rpi.edu/portal/Proof Markup Language

⁵ http://tw.rpi.edu/web/project/TAMI/PML-Lite
given WIQA-PL policy. It provides explanations in natural language for human consumption and explanations in RDF for further processing by software applications. The explanation generation process contains two steps. First, WIQA generates the parts of explanations of why constraints expressed as graph patterns are satisfied. These different parts of explanations are generated using a template mechanism. In the second step, these explanation parts are supplemented with additional explanations of why constraints expressed as extension functions are satisfied. In the RDF-based explanations, WIQA describes the explanation trees (parts and subparts of an explanation) using the Explanation (EXPL) Vocabulary<fnref n="6" />.

The authors in [3, 4] present a nonmonotonic rule system based on defeasible logic which is able to answer queries and provide proof explanations. Defeasible logic enables reasoning with incomplete and inconsistent information. The traces of the underlying logic engine are transformed to defeasible logic proofs. The authors introduce an extension to RuleML<fnref n="7" />, a unifying family of Web rule languages, to enable formal representation of explanations of defeasible logic reasoning. Software agents can consume and verify these proofs described using the RuleML extension. In addition, the authors present graphical user interfaces to visualize the proofs and interact with them. Finally, the authors present an agent interface to enable multi-agent systems to interact with their system.

Horridge et al. present two fine-grained subclasses of justifications called laconic justifications and precise justifications [15]. Laconic justifications consists of axioms that contain no superfluous part. Precise justifications are derived from laconic justifications. Each axioms of a precise justification represents a minimal part of the justification. The authors also present an optimized algorithm to compute laconic justifications showing the feasibility of computing laconic justifications and precise justifications in practice. The authors provide a Protégé ontology editor<fnref n="8" /> plugin as a tool to compute these types of justifications<fnref n="9" />. This tool shows justification-based explanations of entailments. A user can select an entailment from a list of provided entailments and the tool shows the justification-based explanation for the selected entailment.

Kotowski and Bry [18] argue that explanation complements the incremental development of knowledge bases in frequently changing wiki environments. The authors present a semantic wiki called KiWi<fnref n="10" /> which takes a rule-based inconsistency tolerant reasoning approach that has the capability of explaining how a given piece of information was derived. The reasoning approach also allows knowledge base updates in an efficient way by using reason maintenance. The authors argue that providing explanation is important for supporting users' trust and facilitates determining main causes of inconsistencies. KiWi stores the justifications of all the derivations and uses them for providing explanations and also for reason maintenance. KiWi presents explanations as natural language explanations and as tree-based explanations highlighting the derivation paths.

Forcher et al. [9] describe the realization of the EASD approach in a semantic search engine called KOIOS. The KOIOS semantic search engine allows keyword-based search on RDF data. The KOIOS search engine first computes a set of relevant SPARQL queries from a set of given keywords. Users then select the appropriate queries to query a triple store containing RDF data. The search results are provided with explanations about how they are computed. The ex-

planations justify how keywords are mapped to concepts and how concepts are connected. In addition, the explanations interpret the performed queries in an understandable way. The authors introduce a set of ontologies to formally describe the content of explanations provided by KOIOS. The KOIOS Process Language (KPL) is used to describe the behavior of the problem solving process. The Mathematical Graph Language (MGL) is used to realize the graph based view of the process model. Finally, another ontology called VGL is used for visualizing graph based information. In addition, KOIOS includes a set of rules to transform a certain model described in RDF. The trace model is described in RDF and transformed step-by-step to a presentation model using a set of rules to provide different views of explanations. KOIOS provides graphical explanations for keyword-based search with graphical representation of corresponding SPARQL queries and the results of the query. Users can also get a textual explanation of any concept from the graphical representation by clicking on any concept. KOIOS also provides justification-based graphical explanation of search keyword to concept type mapping.

The AIR rule language [16] supports generation of machine consumable explanations. The AIR reasoner annotates all of its performed actions and the dependencies between these actions. These annotations are recursively converted to AIR justifications. To generate natural language explanation, rule authors can specify a natural language description in the definition of a rule itself which can contain variables. These variable values are replaced with the current value during the reasoning process. AIR also provide a feature to declaratively modify justifications. This allows the degrees of detail in AIR justifications to be selectively controlled. The authors of AIR suggest registering AIR with the Inference Web infrastructure and use their toolkit to provide explanations for human consumption.

# 5 DISCUSSION AND PERSPECTIVE

The research we have reviewed exposes several dimensions of explanation in the context of the Semantic Web:

**Infrastructure:** With the increasing growth of sharing Semantic Web data as part of the Linked Data [5] initiatives, it is important that data publishers can publish their data with explanation related metadata with ease. Explanation infrastructures should be able to accommodate common data publishing principles. Semantic Web explanation infrastructures should also address heterogeneous and distributed nature of the Web. Inference Web intent to address these issues. For example, explanation metadata can be described in PML documents and resolved using the URIs of the documents. However, Inference Web provides a centralized solution for publishing and consuming explanation metadata. Explanation metadata should be registered in the Inference Web repository to use their facilities. Moreover, the published metadata using Inference Web facilities have compatibility issues with the Linked Data principles. For instance, not all the resources in PML documents are identifiable by URIs as there are blank nodes in PML documents. Our ongoing work [12] on applying the Linked Data principles intent to address these issues. With regard to diversity of different representation, explanation metadata should be published promoting interoperability. The W3C PROV-DM data model [23] can be used as an interchange data model across different systems in this regard. Different systems can define their explanation metadata model as application-specific and domain-specific extensions of PROV-DM. Applications across the Web can then make sense of explanation metadata in a unified manner. Consumers of these explanation metadata can use explanation presentation and visualization tools according to their needs.

<footnote n="6">⁶ http://www4.wiwiss.fu-berlin.de/bizer/triqlp/</footnote>

<footnote n="7">⁷ http://ruleml.org</footnote>

<footnote n="8">⁸ http://protege.stanford.edu/</footnote>

<footnote n="9">⁹ http://owl.cs.manchester.ac.uk/explanation/</footnote>

<footnote n="10">¹⁰ http://www.kiwi-project.eu/</footnote>
**Target:** Human users and software agents both are target of explanation in Semantic Web applications. In the existing approaches, human users are provided with natural language explanations or graphical explanations promoting easy comprehension. Unlike the expert systems which were used by knowledgeable users and domain experts, the users of Semantic Web applications can have different background, skill level, and knowledge level because of the open nature of the Web. Level of user expertise should be taken into account while providing explanations. The presentations of explanations can change according to user expertise or user scenario context. User profiling approaches might be applied to provide explanations addressing these issues. With regard to software agents, explanations should be described using open standards such as OWL ontologies to make it possible for software agents to make sense of explanations. As pointed out previously, explanations also should be published using common data publishing principles such as Linked Data to enable external software agents to easily consume them.

**What is explained:** The reviewed research discusses explanation of information manipulation steps, operations, and proof trees of derived results. Additional provenance information such as how, when, and who provenance is provided in Inference Web explanations for more context and enable better understanding. Semantic Web applications use distributed interconnected data in their reasoning processes. Explaining the network of data used in the reasoning processes might be useful for users. This would enable users to understand the flow of information used in the reasoning process and have a better understanding of the data integration process performed by an application. Explanation with the details of complex computation processes might always not be as useful for non-expert users as they are for expert users. Exposing problem solving methods in certain scenarios might result in security threat. The existing research does not discuss how explanations, which expose problem solving methods, influence security and confidentiality of Semantic Web systems.

**Representation:** The reviewed vocabularies to represent explanation metadata for machine consumption allow to describe proof trees for answers, processes used to compute answers, different types of provenance information, models for how explanations should be presented to human users, and trust related information. Other important aspects of explanation vocabularies are granularity and existence of blank nodes in the data described using them. Table 1 presents a comparison of reviewed vocabularies taking these aspect into account. Proof trees encode logical deduction of conclusions. PML, PML-Lite, and AIRJ provide primitives to encode proof trees as justifications of answers. KOIOS and EXPL vocabularies do not provide primitives for encoding proof trees as they concern mainly describing computation process and structure of presentation information. Process description concerns describing the information manipulation steps or the algorithms that compute answers. All the reviewed vocabularies except EXPL allow describing processes used to compute answers. Provenance information provide additional context to explanations. PML allows describing how, when, and who provenance. PML-Lite allows one additional provenance, location provenance. AIRJ inherits PML-Lite provenance features. KOIOS and EXPL do not provide any primitive to describe provenance. With respect to granularity, RDF statements can be made at several level of granularity such as triple or graph. PML, PML-Lite, and AIRJ do not strictly define their granularity. The *pml:Information* class instances of PML can refer to a triple, a graph URI, or even to a textual representation of a logical formula. PML-Lite and AIRJ follow a similar approach. For instance, *pml:outputdata* property can point to a graph represented by a set of triples. In [17], the authors introduce new AIRJ properties such as *airj:matchedgraph* to specifically make statements about graphs. The authors of KOIOS vocabularies do not provide details about granularity [9]. EXPL uses RDF reification primitives to provide a triple level fine grained granularity. Presentation model allows describing how and what should presented to the human users as explanations. KOIOS provide VGL vocabulary to describe visualization of explanations. EXPL allows describing the structure and contents of different parts of explanations that are presented to human users. PML, PML-Lite, and AIRJ do not provide any primitive to describe presentation of explanations. Declaratively specifying presentation models enables different types of user interface technologies to render explanation contents. Despite one of the main motivation for providing explanations being trust, only PML allows describing trust related information. Explanation facilities should allow to describe, capture and processing over captured trust. As a Linked Data common practice, blank nodes are avoided while publishing data [14]. Blank nodes add additional complexities in data integration in a global dataspaces. It is not possible to make statements about blank nodes as they do not have identifiers. PML and AIRJ use RDF container concepts such as *NodeSetList*. RDF containers use blank nodes to connect a sequence of items [1]. This approach makes it difficult to publish the data described using PML and AIRJ as Linked Data. In our ongoing work in [12], we present the Ratio4TA<fnref n="11" /> vocabulary to represent justifications supporting graph level granularity without using RDF containers. Graph level granularity gives a flexible control as a graph can contain a single triple or many triples. We define Ratio4TA as an extension of W3C PROV-DM to enable better interoperability.

<table><caption>**Table 1.** Comparison of explanation vocabularies</caption><thead><tr><th></th><th>PML</th><th>PML-Lite</th><th>AIRJ</th><th>KOIOS</th><th>EXPL</th></tr></thead><tbody><tr><td>Proof tree</td><td>Yes</td><td>Yes</td><td>Yes</td><td>No</td><td>No</td></tr><tr><td>Process description</td><td>Yes</td><td>Yes</td><td>Yes</td><td>Yes</td><td>No</td></tr><tr><td>Provenance</td><td>How, when, who</td><td>How, when, who, where</td><td>How, when, who, where</td><td>N/A</td><td>No</td></tr><tr><td>Granularity</td><td>Not strictly defined</td><td>Not strictly defined</td><td>Coarse grained /graph</td><td>N/A</td><td>Fine grained /triple</td></tr><tr><td>Presentation model</td><td>No</td><td>No</td><td>No</td><td>Yes</td><td>Yes</td></tr><tr><td>Trust information</td><td>Yes</td><td>No</td><td>No</td><td>No</td><td>No</td></tr><tr><td>Blank node</td><td>Yes</td><td>No</td><td>Yes</td><td>N/A</td><td>No</td></tr></tbody></table>

**Presentation:** Explanations are presented to human users as natural language or as graphical explanation in the reviewed approaches. Different kinds of graphical representation have been used to present proof trees, queries, or the steps performed by information manipulation algorithms. As discussed previously, users with different levels of expertise should be considered for Semantic Web applications. How to present complex information manipulation processes to the end users in an understandable way and how much details is useful in the context of Semantic Web need to be researched more. The EASD approach discusses providing context dependent explanations. Existing approaches such as [7] on context-aware data consumption can

<footnote n="11">¹¹ http://ns.inria.fr/ratio4ta/</footnote>
be also applied to address providing different types of explanations
depending on different types of users. Presentation models can be
declaratively defined and associated with different context related in-
formation. Different user interface rendering methodologies can then
be applied on the defined presentation models to provide the final ex-
planation user interfaces.

**Interaction:** Explanations should be provided with navigation support to enable end users to discover more related information. Other interaction models such as follow up or feedback mechanisms might also be useful in certain contexts. Inference Web provides explanation with navigation and follow up support. How users can interact trust information based on the provided explanations would be another interesting area to explore.

**Trust:** The reviewed research lacks studies about understanding how explanations influence user trust in the context of Semantic Web. In contrast to the expert systems, Semantic Web applications have new dimensions such as openness and distributed. Furthermore, Semantic Web applications have much broader and diverse user base than expert systems. How these aspects of Semantic Web influence trust needs to be studied more. In the existing work, only Inference Web provides an infrastructure for trust which includes a vocabulary to describe trust related information. However, how users trust can be captured and processed for further trust assertions is not discussed. Another interesting area to explore would be how explanation approaches can be applied to explain trust itself. For instance, explanation can be provided about who have trusted a given resource or about trust rating calculations.

# 6 CONCLUSION

In this paper, we have presented an overview of the design principles of explanation-aware Semantic Web systems, representation and usage of explanation related metadata, and finally how explanations are generated and presented to end users. We have also presented a discussion on the important aspects of ongoing research relating to explanation in the context of Semantic Web.

# ACKNOWLEDGEMENTS

The work presented in this paper is supported by the CONTINT programme of French National Agency for Research (ANR) under the Kolflow project (ANR-2010-CORD-021-02).

# REFERENCES

[1] P. Hayes and B. McBride, eds., 'RDF semantics', W3C recommendation, (2004).

[2] J. Angele, E. Moench, H. Oppermann, S. Staab, and D. Wenke, 'Ontology-based query and answering in chemistry: Ontonova project halo', in *The Semantic Web - ISWC 2003*, eds., D. Fensel, K. Sycara, and J. Mylopoulos, volume 2870 of *Lecture Notes in Computer Science*, 913–928, Springer Berlin / Heidelberg, (2003).

[3] G. Antoniou, A. Bikakis, N. Dimaresis, M. Genetzakis, G. Georgalis, G. Governatori, E. Karouzaki, N. Kazepis, D. Kosmadakis, M. Kritso-takis, G. Lilis, A. Papadogiannakis, P. Pediaditis, C. Terzakis, R. Theodosaki, and D. Zeginis, 'Proof explanation for the semantic web using defeasible logic', in *Knowledge Science, Engineering and Management*, eds., Z. Zhang and J. Siekmann, volume 4798 of *Lecture Notes in Computer Science*, 186–197, Springer Berlin / Heidelberg, (2007).

[4] N. Bassiliades, G. Antoniou, and G. Governatori, 'Proof explanation in the DR-DEVICE system', in *Proc. of 1st Int'l Conference on Web Reasoning and Rule Systems*, pp. 249–258. Springer, (2007).

[5] T. Berners-Lee. Linked data. W3C Design Issues http://www.w3.org/DesignIssues/LinkedData.html, 2006.

[6] C. Bizer, *Quality-Driven Information Filtering in the Context of Web-Based Information Systems*, Ph.D. dissertation, Freie Universität Berlin, Universitätsbibliothek, 2007.

[7] L. Costabello, 'DC proposal: PRISSMA, towards mobile adaptive presentation of the web of data', in *The Semantic Web ISWC 2011*, eds., L. Aroyo, C. Welty, H. Alani, J. Taylor, A. Bernstein, L. Kagal, N. Noy, and E. Blomqvist, volume 7032 of *Lecture Notes in Computer Science*, 269–276, Springer Berlin / Heidelberg, (2011).

[8] D. Doyle, A. Tsymbal, and P. Cunningham, 'A review of explanation and explanation in case-based reasoning', Technical Report TCD-CS-2003-41, Trinity College Dublin, (2003).

[9] B. Forcher, M. Sintek, T. Roth-Berghofer, and A. Dengel, 'Explanation-aware system design of the semantic search engine koios', in *Proc. of the 5th Int'l. Workshop on Explanation-aware Computing*, (2010).

[10] A. Glass, *Explanation of Adaptive Systems*, Ph.D. dissertation, Stanford University, 2011.

[11] R. Hasan and F. Gandon, 'Explanation in the Semantic Web: a survey of the state of the art', Research Report RR-7974, INRIA, (2012).

[12] R. Hasan and F. Gandon, 'Linking justifications in the collaborative semantic web applications', in *Proc. of the 21st Int'l Conference Companion on World Wide Web*, WWW '12 Companion, pp. 1083–1090. ACM, (2012).

[13] S.R. Haynes, *Explanation in Information Systems: A Design Rationale Approach*, Ph.D. dissertation, The London School of Economics, 2001.

[14] T. Heath and C. Bizer, *Linked Data: Evolving the Web into a Global Data Space*, Morgan & Claypool, 1st edn., 2011.

[15] M. Horridge, B. Parsia, and U. Sattler, 'Laconic and precise justifications in OWL', in *Proc. of the 7th Int'l Conference on the Semantic Web*, ISWC '08, pp. 323–338. Springer-Verlag, (2008).

[16] L. Kagal, I. Jacobi, and A. Khandelwal, 'Gasping for AIR – why we need linked rules and justifications on the semantic web', Technical Report MIT-CSAIL-TR-2011-023, MIT, (2011).

[17] A. Khandelwal, L. Ding, I. Jacobi, L. Kagal, and D.L. McGuinness. PML based AIR justification. http://tw.rpi.edu/proj/tami/PML_Based_AIR_Justification. 2011.

[18] J. Kotowski and F. Bry, ‘A perfect match for reasoning, explanation and reason maintenance: OWL 2 RL and semantic wikis’, in *Proc. of 5th Semantic Wiki Workshop*, (2010).

[19] D.L. McGuinness, Li Ding, A. Glass, C. Chang, H. Zeng, and V. Furtado, 'Explanation interfaces for the semantic web: Issues and models', in *Proc. of the 3rd Int'l Semantic Web User Interaction Workshop*, (2006).

[20] D.L. McGuinness, V. Furtado, P. Pinheiro da Silva, L. Ding, A. Glass, and C. Chang, 'Explaining semantic web applications.', in *Semantic Web Engineering in the Knowledge Society*, (2008).

[21] D.L. McGuinness and P. Pinheiro da Silva, 'Explaining answers from the semantic web: the inference web approach', *Web Semantics: Science, Services and Agents on the World Wide Web*, 1(4), 397 – 413, (2004).

[22] J.D. Moore and W.R. Swartout, 'Explanation in expert systems: A survey', Research Report ISI/RR-88-228, University of Southern California, (1988).

[23] L. Moreau and P. Missier, 'PROV-DM: The PROV data model', *World Wide Web Consortium*, Fourth Public Working Draft, (2012).

[24] P. Pinheiro da Silva, D.L. McGuinness, N. Del Rio, and L. Ding, 'Inference web in action: Lightweight use of the proof markup language', in *Proc. of the 7th Int'l Semantic Web Conference*, ISWC '08, pp. 847–860, (2008).

[25] P. Pinheiro da Silva, D.L. McGuinness, and R. Fikes, 'A proof markup language for semantic web services', *Information Systems*, 31(4-5), 381–395, (2006).

[26] T. Roth-Berghofer, 'Explanations and case-based reasoning: Foundational issues', in *Advances in Case-Based Reasoning*, eds., P. Funk and P.A.G. Calero, pp. 389–403. Springer-Verlag, (2004).

[27] S. Stumpf, V. Rajaram, L. Li, M. Burnett, T. Dietterich, E. Sullivan, R. Drummond, and J. Herlocker, 'Toward harnessing user feedback for machine learning', in *Proc. of the 12th international conference on Intelligent user interfaces*, IUI '07, pp. 82–91. ACM, (2007).

[28] W. Swartout, C. Paris, and J. Moore, 'Explanations in knowledge systems: design for explainable expert systems', *IEEE Expert*, 6(3), 58–64, (1991).

[29] N. Tintarev and J. Masthoff, 'A survey of explanations in recommender systems', *ICDE07 Workshop on Recommender Systems and Intelligent User Interfaces*, (2007).