Web Semantics: Science, Services and Agents on the World Wide Web 7 (2009) 1–10

Contents lists available at ScienceDirect

Web Semantics: Science, Services and Agentson the World Wide Web
journal homepage: www.elsevier.com/locate/websem

Quality-driven information ﬁltering using the WIQA policy framework
Christian Bizer a,∗ , Richard Cyganiak babFreie Universität Berlin, Germany
Digital Enterprise Research Institute, NUI Galway, Irelanda r t i c l e

i n f oArticle history:
Received 2 August 2007
Received in revised form
30 November 2007
Accepted 17 February 2008
Available online 21 April 2008
Keywords:
Information quality
Quality-driven information ﬁltering
Semantic Web

a b s t r a c t
Web-based information systems, such as search engines, news portals, and community sites, provide
access to information originating from numerous information providers. The quality of provided information varies as information providers have different levels of knowledge and different intentions. Usersof web-based systems are therefore confronted with the increasingly difﬁcult task of selecting highquality information from the vast amount of web-accessible information. How can information systems
support users to distinguish high-quality from low-quality information? Which ﬁltering mechanisms canbe used to suppress low-quality information? How can ﬁltering decisions be explained to the user? This
article identiﬁes information quality problems that arise in the context of web-based systems, and gives
an overview of quality indicators as well as information quality assessment metrics for web-based systems. Afterwards, we introduce the WIQA—Information Quality Assessment Framework. The framework
enables information consumers to apply a wide range of policies to ﬁlter information. The framework
employs the Named Graphs data model for the representation of information together with qualityrelated meta-information. The framework uses the WIQA-PL policy language for expressing information
ﬁltering policies against this data model. WIQA-PL policies are expressed in the form of graph patterns
and ﬁlter conditions. This allows the compact representation of policies that rely on complex metainformation such as provenance chains or combinations of provenance information and backgroundinformation about information providers. In order to facilitate the information consumers’ understanding of ﬁltering decisions, the framework generates explanations of why information satisﬁes a speciﬁc
policy.
© 2008 Elsevier B.V. All rights reserved.

1. Introduction
The World Wide Web is a global information space consisting of information from a multitude of autonomous information
providers [17]. Web-based information systems provide accessto this information space. They integrate information frommultiple providers and present integrated information to theirusers.
The key success factor of the web is the vast amount of webaccessible information. On the other hand, its openness and theautonomy of information providers make the web vulnerable toinaccurate, misleading, or outdated information. Information quality problems arise in various application domains of web-based
information systems:
Search engines provide access to billions of web documents and anincreasing number of structured information sources. The quality∗ Corresponding author. Tel.: +49 30 838 54057.
E-mail addresses: chris@bizer.de (C. Bizer), richard@cyganiak.de (R. Cyganiak).
1570-8268/$ – see front matter © 2008 Elsevier B.V. All rights reserved.
doi:10.1016/j.websem.2008.02.005

of provided information varies widely and the huge amount ofaccessible information obscures relevant information.
News portals aggregate news articles from a wide range of newspapers and news agencies and assemble them according to user’s
interests. As news providers have different views of the world anddifferent levels of knowledge, news may be biased or inaccurate.
Financial information portals integrate stock quotes, ﬁnancial news,
company proﬁles, and analyst reports from multiple information
sources. The expertise of information providers on speciﬁc markets and companies varies widely and investors are confronted
with conﬂicting advice.
Online communities like MySpace, Facebook, Flickr, or YouTube areused by large numbers of information providers to share information. The quality of provided information varies widely, and again
the amount of accessible information blurs relevant information.
Semantic Web applications. The Semantic Web [15] is a global
information space consisting of Linked Data [4]. Semantic Web
applications enable users to navigate and query this information
space. Assuring information quality is problematic within Semantic Web applications as they operate on an unbound, dynamic setof autonomous data sources.

2

C. Bizer, R. Cyganiak / Web Semantics: Science, Services and Agents on the World Wide Web 7 (2009) 1–10

2. Problem statement

3. Information quality

Information providers have different levels of knowledge, different views of the world, and different intentions. Therefore,
provided information may be wrong, biased, outdated, incomplete,and inconsistent.
Before information from the web is used to accomplish a speciﬁc
task, its quality should be assessed according to task-speciﬁc criteria. Based on the assessment result, information may be acceptedor rejected for a speciﬁc task.
In everyday life, we use a wide range of different policies to assess the quality of information: we might accept
information from a friend on restaurants, but distrust him
on computers; regard scientiﬁc papers only as relevant, if
they have been published within the last 2 years; or believe
foreign news only when they are reported by several independent sources. Which policy is chosen depends on thespeciﬁc task at hand, our subjective preferences, and theavailability of information quality-related meta-information,such as ratings or background information about information
providers.
This article introduces an innovative solution to quality-driven
information ﬁltering in the context of web-based information systems. Instead of having the designer of an information system
decide for the user on a single, ﬁxed method to distinguish highquality from low-quality information, the user is empowered to
employ a similar wide range of ﬁltering policies as she is using in
the off-line world.
The article makes the following contributions to the research on
policy frameworks for web-based information systems:

Compared to concepts like data integrity and security whichhave been studied in detail since the introduction of relational
database technology, the notion of information quality is relatively young and its general conceptualization as well as themethods developed to assess information quality are still highly
diverse.
The concept of information quality is a domain-speciﬁc subconcept of the general concept of quality. A popular deﬁnition
for quality is given by Joseph Juran. He deﬁnes quality as“ﬁtness for use” [20]. Juran’s deﬁnition has been adopted bymost authors working on information quality. Information quality is commonly deﬁned as the ﬁtness for use of information[38,37,29,13,23]. This deﬁnition implies two important aspects:- While there is a lot of work on policy frameworks for access
control and privacy protection, the article highlights the needof web-based systems for information ﬁltering policies. The
article gives an overview of different types of quality indicators that are relevant in the context of web-based systems
and discusses information quality assessment metrics for thesesystems.
- Information ﬁltering may rely on various quality indicators, suchas provenance meta-information or ratings. Therefore, information ﬁltering frameworks need a ﬂexible means to represent
information together with quality-related meta-information.
The article proposes to employ the Named Graphs [8] data
model to fulﬁll this requirement and to represent information
together with quality-related meta-information as an integrated
model.
- The article proposes a new approach to expressing information
ﬁltering policies. Instead of relying on a speciﬁc policy ontology ora generic rules language, WIQA-PL policies are expressed as graphpatterns and ﬁlter conditions. This approach closely couples
information representation and policy formulation and allows
the compact representation of policies that rely on complex
meta-information such as provenance chains or combinationsof provenance information and background information about
information providers.
The article is structured as follows. Sections 3 and 4 givean overview of the concept of information quality and discuss information quality assessment metrics. Chapter 6 describes
the WIQA information ﬁltering framework and the WIQA-PL
policy language. Chaper 7 shows how the WIQA frameworkis used to extend a web browser with information ﬁltering
capabilities. Chapter 8 describes the evaluation of the WIQA framework, while Chapter 9 compares the framework with related
work.

- Information quality is task-dependent. A user might consider thequality of a piece of information appropriate for one task but not
sufﬁcient for another task.
- Information quality is subjective, as a second less qualityconcerned user might consider the quality of the same piece of
information appropriate for both tasks.
Information quality is commonly conceived as a multidimensional construct [38,29,36,32,9], as the “ﬁtness for use”
may depend on various factors such as accuracy, timeliness,completeness, relevancy, objectivity, believability, understandability, consistency, conciseness, availability, and veriﬁability[38,29,10].
These information quality dimensions are not independent ofeach other and typically only a subset of the dimensions is relevant
in a speciﬁc situation. Which quality dimensions are relevant andwhich levels of quality are required for each dimension is determined by the speciﬁc task at hand and the subjective preferencesof the information consumer [29,38].
4. Information quality assessment
Information quality assessment is the process of evaluating if apiece of information meets the information consumer’s needs in a
speciﬁc situation [29,10]. Information quality assessment involves
measuring the quality dimensions that are relevant to the information consumer and comparing the assessment results with theinformation consumer’s quality requirements.
An information quality assessment metric is a procedure formeasuring an information quality dimension. Assessment metrics
rely on quality indicators and calculate an assessment score fromthese indicators using a scoring function. Assessment metrics areheuristics that are designed to ﬁt a speciﬁc assessment situation[34,39].
The types of information which may be used as quality indicators are highly diverse. Besides the information to be assessed
itself, assessment metrics may rely on meta-information about thecircumstances in which information was created, on backgroundinformation about the information provider, or on ratings provided
by the information consumer herself, other information consumers,or domain experts. Fig. 1 shows an abstract view on an informationexchange situation. All types of information that may be used as
quality indicators are shaded gray.
Information quality assessment metrics can be classiﬁed into
three categories according to the type of information that is used
as quality indicator: (1) information content itself; (2) information
about the context in which information was claimed; (3) ratings
about information itself or the information provider.

C. Bizer, R. Cyganiak / Web Semantics: Science, Services and Agents on the World Wide Web 7 (2009) 1–10

3

working for certain companies.”), or his former activities (“Believe
only information from authors who have already published several
times on a topic.” or “Believe only reports from stock analysts whose
former predictions proved correct to a certain percentage.”).
4.3. Rating-based metrics

Fig. 1. Abstract view on an information exchange situation.

4.1. Content-based metrics
Content-based metrics use information to be assessed itself asquality indicator. The metrics analyze the information content orcompare information with related information. Which metrics areapplicable depends on the type of information to be analyzed.
In the case of natural language texts or loosely structured documents, such as HTML pages, it is possible to apply various text
analysis methods for information quality assessment. In general,these methods derive assessment scores by matching terms orphrases against a document and/or by analyzing the structure of the
document. Within deployed web-based information systems, text
analysis methods are widely used to assess the relevancy of documents, to detect spam, and to scan websites for offensive content.
In the case of formalized information, one possibility is to apply
simple rule-based metrics. For instance, a metric to assess thebelievability of a sales offer could be to check if the price lies above a
speciﬁc boundary. If the price is too low, the offer might be considered bogus. Frequently, formalized information contains values thatare grossly different or inconsistent with the remaining set. There
are various statistical methods to identify such outliers [24]. Within
the context of information quality assessment, outlier detection
methods can be used as heuristics to assess quality dimensions suchas accuracy or believability.
4.2. Context-based metrics
Context-based metrics employ meta-information about the
information content and the circumstances in which information
was created, e.g., who said what and when, as quality indicator. For
instance, an important quality indicator for assessing the believability of information is meta-information about the identity of
the information provider. That is, assumptions about the believability of information providers are extended to information they
provide. An example for a simple heuristic to assess the believabilityof information is to check whether an information provider is contained in a list of trusted providers. Other meta-information thatmight inﬂuence believability are the identities of the contributors
and the publisher of information as well as the source from which
information is retrieved.
Instead of relying solely on meta-information, information quality assessment metrics can also combine meta-information withbackground information about the application domain. Anothermetric for assessing the believability dimension could, for instance,
be based on the role of an information provider in the application
domain (“Prefer product descriptions published by the manufacturer over descriptions published by a vendor” or “Disbelieve
everything a vendor says about its competitor.”), his membership in a speciﬁc group (“Believe only information from authors

Rating-based metrics rely on explicit ratings about information
itself, information sources, or information providers. Ratings may
originate from the information consumer herself, other information
consumers, or domain experts.
The design of rating systems has been widely studied in computer science [19]. Seen from an abstract perspective, rating-based
quality assessment involves two processes: The acquisition of ratings and the calculation of assessment scores from these ratings[10]. Within the assessment process, a scoring function calculates
assessment scores from the collected ratings. The scoring function decides which ratings are taken into account and might assign
different weights to ratings. Designing scoring functions is a popular research topic and various authors have proposed different
algorithms. Approaches to classifying the proposed algorithms arepresented by Zhang et al. [40] and Ziegler [41].
4.4. Accuracy of assessment results
A general problem of information quality assessment is thatassessment results are often imprecise [29,33,10]. This is especiallytrue for the the context of web-based systems where the availability and the quality of quality indicators is often not optimal. For
instance, because of the decentralized nature of the web and theautonomy of information providers, meta-information about web
content is often incomplete. The quality of ratings is often uncertain, as the expertise of raters is unknown in many cases. Because of
these problems, information quality assessment often has to tradeaccuracy for practicability [29].
The imprecision of assessment results is relativized by the factthat users of web-based information systems are accustomed totolerating a certain amount of low-quality information. For them,
the beneﬁt of having access to a huge information-base is often
higher than the costs of having some noise in the answers [28].
Therefore, the goal of practical information quality assessment is
to ﬁnd heuristics which can be applied in a given situation andthat are sufﬁciently precise to be useful from the perspective of theinformation consumer.
5. Information ﬁltering policies
Quality-based information ﬁltering policies are heuristics for
deciding whether to accept or reject information to accomplish a
speciﬁc task [10].
A information ﬁltering policy consists of a set of assessment
metrics, for assessing the quality dimensions that are relevant for
the task at hand and a decision function which aggregates theresulting assessment scores into an overall decision on whether
information satisﬁes the information consumer’s quality requirements. Each assessment metric relies on a set of quality indicatorsand speciﬁes a scoring function to calculate an assessment scorefrom these indicators. The decision function weights assessmentscores depending on the relevance of the different quality dimensions for the task at hand.
Information consumers may choose a wide range of different
policies to decide whether to accept or reject information. Whenchoosing a policy that ﬁts a speciﬁc situation, an information consumer has to answer the following questions: Which information
quality dimensions are relevant in the context of the task at hand?
Which information quality assessment metric should be used to4

C. Bizer, R. Cyganiak / Web Semantics: Science, Services and Agents on the World Wide Web 7 (2009) 1–10

assess each dimension? How should the assessment results becompiled into an overall decision on whether to accept or reject
information?
The relevance of the different quality dimensions is determinedby the task at hand. The choice of suitable assessment metrics for
speciﬁc quality dimensions is restricted by several factors:
Availability of quality indicators. Whether an assessment metriccan be used in a speciﬁc situation depends on the availability ofthe quality indicators that are required by the metric. Assessing
quality dimensions like timeliness is possible in many cases, as
the required quality indicators are often available. Accessing otherdimensions like accuracy or objectivity often proves difﬁcult, as
it might involve the information consumer or experts verifying or
rating information.
Quality of quality indicators. The choice of assessment metrics is
also inﬂuenced by the quality of the available quality indicators.
If an information consumer is in doubt about the quality of certain indicators, she might prefer to choose a different assessment
metric which relies on other indicators.
Understandability. The key factor for an information consumer totrust assessment results is his understanding of the assessment
process. Therefore, relatively simple, easily understandable, andtraceable assessment metrics are often preferred [19].
Subjective preferences. The information consumer might have subjective preferences for speciﬁc assessment metrics. She might, forexample, consider speciﬁc quality indicators and scoring functions
more reliable than others. Thus, there is never a single best policy
for a speciﬁc task, as the subjectively best policy differs from user
to user.
6. The WIQA framework
The WIQA—Information Quality Assessment Framework is a set ofsoftware components that can be employed by applications which
process information of uncertain quality and want to enable theirusers to ﬁlter information using a wide range of different qualitybased information ﬁltering policies.
The framework has been designed to fulﬁll the following
requirements: (1) ﬂexible representation of information together
with quality-related meta-information; (2) enable users to employ
different information ﬁltering policies; (3) ability to generate explanations about the ﬁltering process.
The WIQA framework consists of two components: a Named
Graph store for representing information together with qualityrelated meta-information, and a ﬁltering and explanation engine
which enables applications to ﬁlter information and to retrieve
explanations about ﬁltering decisions.
Fig. 2 gives an overview of the components of the WIQA
framework. An application deposits unﬁltered information in theframework’s Named Graph store. The application also provides a set

Fig. 3. The WIQA ﬁltering process.

of ﬁltering policies which ﬁt the speciﬁc requirements of the application domain. When the application requests the ﬁltering engine
to apply one of these policies, the ﬁltering engine provides a viewon the Named Graph store which contains only information that
fulﬁlls the policy. Fig. 3 illustrates this ﬁltering process.
6.1. Representing quality-related information
The WIQA framework uses Named Graphs [8] as a ﬂexible data
model for representing both quality-related meta-information andinformation to be ﬁltered itself. Named Graphs are an extension of
the RDF data model [22]. Like in RDF, the atomic units of information
are subject–predicate–object triples. A set of triples forms an RDF
graph. But unlike in RDF, these graphs are named with URI references. Graph names can be used as the subject and object of triples,
which allows the expression of meta-information about graphs.
Within the WIQA framework, each Named Graph is assumed tocontain information published by a speciﬁc information provider at
a certain point in time. Provenance information about each graph isrepresented within other graphs using the Semantic Web Publishing (SWP) Vocabulary [10]. In SWP, graph provenance is recorded in
the form of a warrant attached to the graph. A warrant is authorized
by an actor, called its authority. The warrant expresses a propositional attitude towards one or more Named Graphs, e.g., assertionor quotation [10]. Warrants can be enriched with additional metainformation and may be digitally signed.
This approach to representing primary information together
with quality-related meta-information as an integrated model provides for ﬁne-grained provenance tracking, the representation ofprovenance chains, and the representation of provenance information together with background information about information
providers.
The example graph set shown in Fig. 4 demonstrates how the
Named Graphs data model is used to represent partial descriptionsof a resource while keeping track of the provenance of different
pieces of information. The example also demonstrates how thedata model is used to represent provenance chains. The example
uses the TriG syntax [5]. Namespace declarations are omitted. The
graph set consists of four Named Graphs. The ﬁrst graph originatesfrom Intel and contains general information about the company(lines 1–5). The second graph originates from an individual andcontains a newsgroup posting and several trust ratings for companies and other individuals (lines 7–16). The third graph originatesfrom Yahoo Finance (lines 18–26). It contains additional information about Intel as well as provenance information about the second
graph. The fourth graph originates from an information aggregation service and contains provenance information about the ﬁrst
and third graph (lines 28–38). The third and the forth graph taken
together represent the provenance chain that according to Yahoo
Finance, the newsgroup posting described in the second graph was
authored by Mark Scott.
6.2. Expressing policies

Fig. 2. Overview of the WIQA framework.

Information consumers use a wide range of different policies
for determining whether to accept or reject information. The WIQA
framework allows the expression of such policies using the declar-

C. Bizer, R. Cyganiak / Web Semantics: Science, Services and Agents on the World Wide Web 7 (2009) 1–10

5

A named graph pattern is a pair (n, P), where n ∈ (U ∪ V ) and P
is a graph pattern. A graph pattern, for the purposes of this paper,
is either a tuple from (T ∪ V ) × (U ∪ V ) × (T ∪ V ) (a triple pattern);
or an expression (P1 ANDP2 ), where P1 and P2 are graph patterns;
or an expression (P FILTERR), where P is a graph pattern, and R is aboolean ﬁlter condition composed of constant RDF terms, variables,
comparison and boolean operators, and function calls.
6.2.2. Semantics of WIQA policies
A WIQA policy consists of one or more named graph patterns
P1 . . . Pi , and zero or more additional ﬁlter conditions R1 . . . Rj . The
policy pattern is deﬁned as the expression
P1 AND . . . AND Pi FILTER R1 . . . FILTER Rj
The connectives AND and FILTER over named graph patterns aredeﬁned in analogy to those given in Ref. [31] over graph patterns.
For the policy to be meaningful, at least one of the named graphpatterns must involve at least one of the referring variables?GRAPH,?SUBJ, ?PRED, and ?OBJ, which are used to express conditions that
triples have to fulﬁll to be accepted by the policy.
In Ref. [31], the evaluation of a graph pattern P over an RDF graphG, denoted by [[P]]G , is deﬁned as a set of mappings from variables
to RDF terms that, informally stated, fulﬁll the conditions imposedby the graph pattern. We will now extend the notion of evaluation
to cover Named Graphs.
Let (n, P) be a named graph pattern, and N a set of Named Graphs.
The evaluation of (n, P) over N, denoted by [[(n, P)]]N is deﬁned asfollows:{|(n, G) ∈ N and  ∈ [[P]]G },{ |(g, G) ∈ N and  ∈ [[P]]G },if n ∈ U
if n ∈ V

where  is the singleton mapping {n → g}.
Let Q be a WIQA policy, PQ its policy pattern, and N a set of Named
Graphs. Then a triple (s, p, o) in Named Graph (g, G) ∈ N is said tobe accepted by Q if there exists a mapping m such that m ∈ [[PQ ]]N
and m compatible to the mapping{?GRAPH → g, ?SUBJ → s, ?PRED → p, ?OBJ → o}Less formally speaking, a triple is accepted if its origin graph,subject, predicate and object, respectively, fulﬁll the conditions
that the patterns and ﬁlter conditions of the policy impose on thevariables ?GRAPH, ?SUBJ, ?PRED and ?OBJ.
Fig. 4. Example set of Named Graphs.

ative WIQA-PL policy language. The policy language is able to: (1)use whatever quality-related information that is available in theapplication context as quality indicators; (2) express a wide range ofcontext-, content-, and rating-based assessment metrics over these
indicators; (3) combine those assessments into an overall ﬁltering
decision.
The following sections deﬁne the semantics of WIQA policies. A
WIQA policy speciﬁes conditions that a triple from a set of Named
Graphs has to fulﬁll in order to be accepted by the policy (Fig. 3).
We rely on the SPARQL semantics given by Pérez et al. [31] for basic
deﬁnitions.
6.2.1. Named Graphs and named graph patterns
Let U be the set of all URIs, B the set of blank nodes, L the setof RDF literals, and V the set of variables. U, B, L and V are pairwise
disjoint. We denote by T the set of RDF terms, the union U ∪ B ∪ L.
A triple (s, p, o) ∈ T × U × T is called an RDF triple. An RDF graphis a set of RDF triples. A named graph is a tuple (g, G), where g, thegraph name, is an URI, and G is an RDF graph.

6.2.3. The WIQA-PL syntax
WIQA policies are expressed using the WIQA-PL syntax. The syntax is based on the syntax of the SPARQL query language [35] inorder to make it easier for people who already know SPARQL tolearn WIQA-PL. The complete grammar of the WIQAP-PL syntax
given in Ref. [10].
Fig. 5 shows an example of a WIQA-PL policy. Lines 1–4 specify the policy name and description. The PATTERN clause restricts
information to originate from analysts who achieved a StarMinescore above 80. It consists of two named graph patterns.
The ﬁrst named graph pattern requires provenanceinformation about graphs to be contained in the graphfd:GraphFromAggregator. It contains two triple patterns
which require provenance information to be expressed using the
SWP properties swp:assertedBy and swp:authority. The ﬁrst
pattern binds the names of asserted graphs to the referring variable?GRAPH. The second pattern binds URIs that identify authorities tothe variable ?authority. The second triple pattern is connectedto the ﬁrst by sharing the variable ?warrant.
The second graph pattern requires authorities to be an instanceof the class fin:Analyst and to have a fin:benchmark prop-

6

C. Bizer, R. Cyganiak / Web Semantics: Science, Services and Agents on the World Wide Web 7 (2009) 1–10

Fig. 5. WIQA-PL policy: accept only information that has been asserted by analysts
who achieved a StarMine score above 80.

erty whose value is bound to ?benchmark. The FILTER clausein line 12 restricts the benchmark score to values above 80.
The triples that describe authorities have to occur in the graph
fd:BackgroundInformation.
6.2.4. Extension functions
Quality-based information ﬁltering policies rely on a wide
range of different, application domain speciﬁc assessment metrics.
Therefore, WIQA-PL provides an extension mechanism for invoking arbitrary, application domain speciﬁc functions. For instance,
rating-based ﬁltering policies may use various scoring algorithms
to calculate a score for an entity from a network of ratings. Contentbased ﬁltering policies may rely on natural language processing
methods to analyze text or may use various statistical methods
to compare a piece of information with related information. By
including domain speciﬁc functions, the WIQA framework can beextended to ﬁt the requirements of different application domains.
There are two types of extension functions. Conditional extension functions are used as part of FILTER conditions and canbe combined with the built-in logical operators. They computeboolean values from mappings, and are used in determining if thecondition holds for a particular mapping.
The second type are result set ﬁlters. They are applied to theentire set of mappings that satisfy a policy’s patterns and conditions. Result set ﬁlters can perform arbitrary modiﬁcations to theset, though typical applications are ranking and counting within
the result mappings. Both types of extensions have access to theoriginal unﬁltered dataset.
Three example extension functions have been implemented:
The wiqa:morePositiveRatings and the wiqa:tidalTrust[18]conditional functions implement different rating-based scoringalgorithms; the wiqa:count result set ﬁlter allows the formulationof quantity constraints. A WIQA-PL policy that uses the wiqa:countfunction is given in Fig. 6.
6.3. Explaining assessment results
The accuracy of information quality assessment results is often
uncertain due to the limited availability of quality indicatorsand due to the uncertain quality of the quality indicators themselves. Therefore, the user’s ﬁnal decision whether to trust or
distrust assessment results depends on her understanding of theassessment metrics and quality indicators that were used in theassessment process. Information systems can support users in thistrust decision by providing explanations of why information satisﬁes a given ﬁltering policy.

Fig. 6. WIQA-PL policy using the wiqa:count extension function and including
explanation templates.

Making information ﬁltering decisions comprehensible andtraceable requires diverse forms of explanations. The content ofsuitable explanations depends on the assessment metrics that areused within a policy and on the current task of the user. For lessimportant tasks, the user will be contented with short, simple tocomprehend explanations. For other, more important tasks the user
will require explanations to contain detailed information about theassessment process and the quality indicators that were used in theprocess.
Explanations for rating-based metrics. Ratings might be subjective and raters may try to inﬂuence rating systems by providing
unfair ratings. Therefore, explanations for rating-based assessment metrics should contain the ratings that were used in theevaluation and explain the calculation steps of the scoring algorithm. More detailed explanations might provide provenanceinformation about ratings and background information about
raters.
Explanations for context-based metrics. An explanation for a policythat relies on provenance information should list the information providers. The explanation might also contain additional
background information about information providers in order toC. Bizer, R. Cyganiak / Web Semantics: Science, Services and Agents on the World Wide Web 7 (2009) 1–10

7

6.4. Implementation
A Java implementation of the WIQA framework is available.
It builds on the NG4J—Named Graphs API,1 the Jena—Semantic
Web framework2 and the ARQ SPARQL processor.3 As the WIQA
framework relies on ARQ for graph pattern matching, it can takeadvantage of performance gains of ARQ’s graph pattern matchingalgorithm.
The implementation of the WIQA framework is available under
the terms of the GNU General Public License and can be downloaded
from the WIQA website.4
7. The WIQA browser

Fig. 7. Example explanation.

support information consumers in judging their trustworthiness.
Explanations for content-based metrics detail why information content itself satisﬁes the requirements of an assessment metric. For
instance, an explanation for a statistical metric should contain thedata that was used in the calculation and describe the calculation
process. An explanation for a text analysis method might list relevant keywords and explain how the overall score for a text wascalculated.

The WIQA framework combines two explanation generation
mechanisms. First, a template mechanism is used to generate theparts of an explanation which explain why constraints that are
expressed as graph patterns are satisﬁed. When a user requests an
explanation why an accepted triple fulﬁlls the policy, explanation
templates that are part of the WIQA-PL policy syntax are instantiated with variable bindings from the matching solutions that led
to the acceptance of the triple. As a convention, instantiated templates should form a phrase that completes the sentence: “The pieceof information fulﬁlls the policy because . . .” Second, the explanation is supplemented with explanation parts that explain why
constraints that are expressed using WIQA extension functions are
satisﬁed. Extension functions may conduct complex calculations
such as rating-based scoring or statistical evaluations. In orderto make their calculations comprehensible, extension functions
can generate custom, function-speciﬁc explanations. The complete
algorithm that is used by the WIQA framework to combine template and extension function generated explanation parts is given
in Ref. [10].
Fig. 6 shows a WIQA policy containing explanation templates.
Explanation templates are marked by the EXPL keyword. The policy requires that information is stated by an analyst, and the analyst
must have received at least three positive ratings. The explanation will display: (1) the source of the information; (2) who claims
that the source is an analyst; (3) the raters and their afﬁliation. An
example explanation generated by this policy is given in Fig. 7.

The WIQA browser is an example application that uses the
WIQA framework. The browser demonstrates how information
quality ﬁltering capabilities can be integrated into a standard webbrowser. The browser enables users to extract structured information from web pages. Extracted information from different web
pages is stored in a local repository and can be browsed, sorted,and searched. The content of the local repository can be ﬁlteredusing quality-based information ﬁltering policies. In order to help
users to understand the ﬁltering decisions, the browser can display explanations of why a piece of information satisﬁes a selected
policy.
The WIQA browser is based on the Piggy Bank extension for theFirefox web browser developed by the SIMILE project at the Massachusetts Institute of Technology [16]. The WIQA browser uses
Piggy Bank functionality to extract structured information fromweb pages and to display and navigate extracted information. The
WIQA browser stores information together with provenance metainformation as a set of Named Graphs. It uses the WIQA—Filteringand Explanation Engine to ﬁlter stored information and to generate
explanations about ﬁltering decisions.
While a user browses the web, the WIQA browser runs in the
background and analyzes the visited web pages. Whenever thebrowser can extract structured information items from a page, it
displays a data coin icon in the status bar, indicating that the user
can switch to an information item view of the web page. This view
lists all information items that have been extracted from the page.
Next to each item is a save button that stores the item in the local
repository.
Whenever the user saves information from a webpage into the
local repository, the browser creates a new Named Graph for thisvisit of the page and stores the current timestamp, the URL of thepage, and the authority (website URL) together with the actual
information. The new graph is named with an UUID [25]. Provenance meta-information is represented using the Semantic Web
Publishing vocabulary and the Dublin Core [30] vocabulary.
In a 1997 note on web browser user interfaces, World Wide
Web inventor Tim Berners-Lee envisioned that browsers could offer
what he called the “Oh, yeah?”-button [3]. Whenever a surfer questions the quality and trustworthiness of displayed information, sheshould press this button and the browser would display an explanation of why information should be considered trustworthy. The
WIQA browser realizes the “Oh, yeah?”-button.
The user can load a WIQA policy suite into the browser. After
selecting a policy from a selection panel, the content of the local
repository is ﬁltered and the view is updated to show only information matching the policy. As shown in Fig. 8, a small button is1

http://sites.wiwiss.fu-berlin.de/suhl/bizer/ng4j/.
http://jena.sourceforge.net/.
3
http://jena.sourceforge.net/ARQ/.
4
http://www.wiwiss.fu-berlin.de/suhl/bizer/wiqa/.
2

8C. Bizer, R. Cyganiak / Web Semantics: Science, Services and Agents on the World Wide Web 7 (2009) 1–10

Fig. 8. When the user selects a policy from the policy selection panel on the right-hand side, the left-hand view updates to show only matching information. The “Oh,yeah?”-buttons open new windows with explanations of why a piece of information satisﬁes the selected policy.

located next to each piece of information. Pressing any of these buttons opens a new window that displays an explanation generatedusing the active policy.
8. Evaluation
The WIQA framework and the WIQA browser are working
prototypes. In order to verify the utility of the Named Graphs
data model to represent information together with quality-related
meta-information and to demonstrate the utility of the WIQAPL policy language, we did a preliminary evaluation of the
WIQA browser against a ﬁnancial information integration use
case.
We gathered a test dataset consisting of ﬁnancial news, discussion forum post and information about analyst reports from Yahoo
Finance5 and Google Finance6 using screen scraping techniques.
The resulting dataset represents provenance information using thesame techniques as the example graph set shown in Fig. 4. The
dataset was enhanced with artiﬁcial ratings for users and analysts.
The dataset is available on the WIQA browser website.7
We developed a set of information ﬁltering policies thattake advantage of the different types of quality-related metainformation that are available in the test dataset. This policy suiteis also found on the WIQA browser website.
The evaluation of the Named Graphs data model and the WIQAPL policy language against the use case provided a ﬁrst indicator
for the utility of both technologies. Using the Named Graphs data
model, it was possible to represent primary information, quality-

5

http://ﬁnance.yahoo.com/.
http://ﬁnance.google.com/.
7
http://sites.wiwiss.fu-berlin.de/suhl/bizer/wiqa/browser/.
6

related meta-information as well as background information about
information providers as a compact, integrated model. WIQA-PL’sgraph pattern approach proved suitable for expressing complex
information provenance oriented policies against the model. The
extension function mechanism proved suitable for extending the
WIQA framework with different rating-based scoring functions thatwere required by the use case.
As a next step, we plan to evaluate the WIQA framework against
further use cases. Current candidates are news portals, searchengines within knowledge management systems, and online communities that are used by large numbers of information providers
to share information.
9. Related work
This chapter compares the WIQA framework with related work.
9.1. Database views
Accepted graphs within the WIQA framework can be comparedto views in the context of relational databases. WIQA’s explanation capabilities relate to work within the database community
on explaining data lineage and view generation. An approach to
explaining view generation has been developed by Cui and Wisdomin the context of the Stanford University WHIPS data warehousing
system [12]. For a given data item in a materialized view, the authors
propose a lineage tracing algorithm to identify the set of source
data items that produced the view item. The algorithm is applicable to aggregate-select-project-join views and can be employed bydata warehouse analysis tools to provide drill-down functionality
from view items to source data items. What distinguishes the WIQA
framework from the work within the relational data base community is the underlying data model. By employing a variation of theC. Bizer, R. Cyganiak / Web Semantics: Science, Services and Agents on the World Wide Web 7 (2009) 1–10

RDF data model, the WIQA framework is tailored towards the integration of heterogeneous information from the web. For instance,
integrating two partial descriptions of the same object while keeping track of the provenance of different pieces of information can
simply be achieved within the Named Graphs data model, but istricky using the relational data model.
9.2. Inference Web
A related approach to explaining information quality in the context of web-based information systems has been developed by theInference Web project at the Stanford University Knowledge Systems Laboratory [27]. The project aims at making query answers
more transparent by providing explanations about information
sources as well as inference processes that are used to derive query
results. The Inference Web infrastructure includes a registry containing details on information sources, reasoners, languages, andrewrite rules; a portable proof speciﬁcation language; and a proof
and explanation browser. Inference Web and the WIQA framework
assume different application scenarios. While the WIQA framework
is tailored towards a simple web-based information integration
scenario, Inference Web assumes an agent community consisting of actively reasoning agents that cooperatively derive queryanswers from shared knowledge. Therefore, Inference Web focuses
on explaining distributed reasoning paths [26], while the WIQA
framework generates explanations of why subjective information
ﬁltering policies are satisﬁed.
9.3. TRELLIS
A further system that employs the RDF data model and provides information quality assessment functionality is the TRELLIS
information analysis tool [14] developed at the University of Southern California. TRELLIS aims at supporting intelligence analysts inselecting quality information within a military setting. As an analyst makes a decision on which sources to dismiss and which tobelieve, TRELLIS captures the derivation of the decision in a semantic markup. The system then uses these annotations to derive an
information quality assessment of the source based on the annotations of many individuals. Compared with the WIQA framework,
TRELLIS supports only one ﬁxed ontology for capturing qualityrelated meta-information and a single hard-coded assessment
policy, while the WIQA framework is ontology independent andmay employ a wide range of different ﬁltering policies.
9.4. REI, KAoS and PROTUNEThere are several well established frameworks for enforcing
access control policies within Semantic Web settings: REI [21] provides for positive and negative authorization as well as obligation
policies and supports remote policy management. The KAoS framework [2] is aimed at agent environments and provides a service
architecture for constraining the execution of actions in relation tovarious aspects of the situation. Within both frameworks, policies
are expressed using a speciﬁc policy ontology. The PROTUNE framework [7] uses a rule-based policy language for specifying access
control policies, privacy policies and certain classes of businessrules.
All three frameworks assume a different application scenario
from the WIQA framework. The frameworks are designed for situations where agents actively exchange messages in order todetermine whether certain actions can be performed or certain
information should be accessible. In contrast, the WIQA framework is designed for classic Web scenarios where information has
been published by different information providers and where infor-

9mation consumers want to determine the subset of the available
information that fulﬁlls their information quality requirements.
Information quality assessment relies on different types ofmeta-information such as ﬁne-grained provenance information orprovenance chains. The WIQA framework provides for the representation of complex meta-information by employing the Named
Graphs data model. REI, KAoS and PROTUNE use pure triple-based
data models for the internal representation of information. This
makes it difﬁcult to represent meta-information about speciﬁc setsof triples. For instance, it is difﬁcult to represent partial descriptions of the same object while keeping track of the provenanceof different pieces of information or to model provenance chains
which require the representation of meta-information about metainformation.
9.5. Almendra and Schwabe
An approach for translating abstract information quality
requirements into concrete ﬁltering policies is presented by Almendra and Schwabe from the Pontiﬁcia Universidade Catolica do Rio
de Janeiro in Ref. [1]. Their work is based on the work presented
in this article. It also employs the Named Graphs data model, theSemantic Web Publishing Vocabulary and the TriQL.P policy language [6], an earlier version of the WIQA-PL policy language. In
addition to our work, where each policy must explicitly specify all
the conditions that triples must fulﬁll to be accepted, they propose an ontology for expressing information quality requirements(TrustPoints) and an algorithm that automatically derives information ﬁltering policies from these requirements by combining policy
fragments. Given adequate tool support, their translation mechanism provides a valuable extension to our work as it reduces thetechnical knowledge required from a policy author.
10. Conclusion
This article highlighted the need of web-based information systems for task-speciﬁc information ﬁltering policies and gave an
overview of different metrics that can be used to assess information quality in the context of web-based information systems. We
demonstrated how the Named Graphs data model can be employed
to represent information together with quality related-meta information as an integrated model. We developed the WIQA-PL policy
language which closely couples information representation andpolicy formulation and provides for expressing content-, contextand rating-based assessment policies.
As future work, we plan to integrate the WIQA framework
into further applications besides the WIQA browser. Interesting
candidates for being upgraded with information quality ﬁltering
capabilities are news portals and newsfeed aggregators, searchengines within knowledge management systems, and online communities that are used by large numbers of information providers
to share information.
A further, more visionary application domain for the WIQA
framework is the Semantic Web as a whole. A growing numberof data sources have started to expose their content as Linked Data[11]. As this number further increases, technologies for selecting
high-quality information from this web of data are needed and it
would be exciting to integrate the WIQA framework into a Semantic
Web search engine, which crawls data from different sources andprovides query capabilities over crawled data.
References[1] V. Almendra, D. Schwabe, Real-world Trust Policies, in: Proceedings of the
Semantic Web Policy Workshop, 2005.

10

C. Bizer, R. Cyganiak / Web Semantics: Science, Services and Agents on the World Wide Web 7 (2009) 1–10

[23] S.-A. Knight, J. Burn, Developing a framework for assessing information[2] A. Uszok, et al., Kaos policy and domain services: toward a description-logic
quality on the World Wide Web, Informing Science Journal 8 (2005) 160–
approach to policy representation, deconﬂiction, and enforcement, in: Proceed172.
ings of 4th IEEE International Workshop on Policies for Distributed Systems and[24] E. Knorr, R. Ng, V. Tucakov, Distance-based outliers: algorithms and applicaNetworks (POLICY 2003), 2003.
tions, International Journal on Very Large Data Bases 8 (3–4) (2000) 237–253,[3] T. Berners-Lee, Cleaning up the User Interface, Section—The “Oh, yeah?”-Button,
citeseer.ist.psu.edu/knorr00distancebased.html.
1997, http://www.w3.org/DesignIssues/UI.html.
[4] T. Berners-Lee, Linked Data, 2006, http://www.w3.org/DesignIssues/LinkedData.html.[25] P. Leach, M. Mealling, R. Salz, RFC 4122: A Universally Unique IDentiﬁer (UUID)URN Namespace, 2005, http://tools.ietf.org/html/4122.
[5] C. Bizer, The TriG Syntax, 2005, http://www.wiwiss.fu-berlin.de/suhl/[26] D. McGuinness, Explaining reasoning in description logics, Ph.D. Thesis,
bizer/TriG/.
Rutgers—The State University of New Jersey, 1996, URL citeseer.ist.psu.edu/[6] C. Bizer, R. Cyganiak, T. Gauss, O. Maresch, The TriQL.P Browser: ﬁltering informcguinness96explaining.html.
mation using context-, content- and rating-based trust policies, in: Semantic[27] D. McGuinness, P. da Silva, Infrastructure for web explanations, citeWeb and Policy Workshop at the 4th International Semantic Web Conference,
seer.ist.psu.edu/mcguinness03infrastructure.html, in: Proceedings of the 2nd
2005.
International Semantic Web Conference, 2003.
[7] P.A. Bonatti, D. Olmedilla, Policy language speciﬁcation. project deliverable d2,[28] F. Naumann, From databases to information systems—information quality
working group i2, eu noe rewerse, Tech. rep., 2005.
makes the difference, citeseer.ist.psu.edu/naumann01from.html, in: Pro[8] J. Carroll, C. Bizer, P. Hayes, P. Stickler, Named Graphs, Journal of Web Semantics
ceedings of the 6th International Conference on Information Quality,
3 (4) (2005) 247–267.
2001.
[9] Y. Chen, Q. Zhu, N. Wang, Query processing with quality control in the World[29] F. Naumann, Quality-Driven Query Answering for Integrated Information SysWide Web, World Wide Web Journal 1 (4) (1998) 241–255.
tems, Springer, Berlin/Heidelberg/New York, 2002.
[10] C. Bizer, Quality-driven information ﬁltering in the context of web-based infor[30] M. Nilsson, A. Powell, P. Johnston, A. Naeve, Expressing Dublin Core Metadatamation systems, Ph.D. Thesis, Freie Universität Berlin, 2007.
Using the Resource Description Framework (RDF)—Dublin Core Working Draft,[11] C. Bizer, et al., Interlinking open data on the web, in: Poster at 4th European
2006, http://dublincore.org/documents/dc-rdf/.
Semantic Web Conference, 2007.
[31] J. Pérez, M. Arenas, C. Gutierrez, Semantics and complexity of sparql, in: Inter[12] Y. Cui, J. Widom, Practical lineage tracing in data warehouses, citenational Semantic Web Conference, 2006.
seer.ist.psu.edu/article/cui99practical.html, in: Proceedings of the 16th Inter[32] B. Pernici, M. Scannapieco, Data quality in web information systems, Journal on
national Conference on Data Engineering, 2000.
Data Semantics 1 (2003) 48–68.
[13] M. Eppler, P. Muenzenmayer, Measuring information quality in the web context:[33] L. Pipino, Y. Lee, R. Wang, Data quality assessment, Communications of the ACM
a survey of state-of-the-art instruments and an application methodology, in:
45 (2002) 211–218.
International Conference on Information Quality, 2002.
[34] L. Pipino, R. Wang, D. Kopcso, W. Rybold, Developing Measurement Scales for[14] Y. Gil, V. Ratnakar, Trusting information sources one citizen at a time, in: ProData-Quality Dimensions, M.E. Sharpe, New York, 2005.
ceedings of the 1st International Semantic Web Conference, 2002.
[35] E. Prud’hommeaux, A. Seaborne, SPARQL Query Language for RDF, 2005,[15] I. Herman, W3C Semantic Web Activity, 2006, http://www.w3.org/2001/sw/.
http://www.w3.org/TR/2005/WD-rdf-sparql-query-20050217/.
[16] D. Huynh, S. Mazzocchi, D. Karger, Piggy Bank: experience the semantic web[36] T. Redman, Data Quality for the Information Age, Artech House, London, 1996.
inside your web browser, in: Proceedings of the 4th International Semantic Web[37] D. Strong, Y. Lee, R. Wang, Data quality in context, Communications of the ACM
Conference, 2005.
40 (5) (1997) 103–110.
[17] I. Jacobs, N. Walsh, Architecture of the World Wide Web, Volume One, W3C[38] R. Wang, D. Strong, Beyond accuracy: what data quality means to data
Recommendation, 2004, http://www.w3.org/TR/webarch/.
consumers, Journal of Management Information Systems 12 (4) (1996)[18] J. Golbeck, Computing and applying trust in Web-based Social Networks, Ph.D.
5–33.
Thesis, University of Maryland, 2005.
[39] R. Wang, M. Ziad, Y. Lee, Data Quality, Kluwer Academic Publishers, Norwell,[19] A. Jøsang, R. Ismail, C. Boyd, A Survey of Trust and Reputation Systems for Online
2000.
Service Provision, 2006, URL citeseer.ist.psu.edu/738255.html.
[40] Q. Zhang, T. Yu, K. Irvin, A Classiﬁcation scheme for trust functions in reputation[20] J. Juran, The Quality Control Handbook, 3rd ed., McGraw-Hill, New York, 1974.
based trust management, in: Proceedings of the Workshop on Trust, Security,[21] L. Kagal, T. Finin, A. Joshi, A policy-based approach to security for the semantic
and Reputation on the Semantic Web, 2004.
web, in: Proceedings of 2nd International Semantic Web Conference (ISWC’03),[41] C.-N. Ziegler, Towards decentralized recommender systems, Ph.D. Thesis, UniLNCS 2870, Springer, 2003.
versität Karlsruhe, 2005.
[22] G. Klyne, J. Carroll, Resource Description Framework (RDF): Concepts andAbstract Syntax—W3C Recommendation, 2004, http://www.w3.org/TR/rdfconcepts/.