The amount of information produced in the world increases by 30% every year and this rate will only go up. With advanced network technology, more and more sources are available either over the Internet or in enterprise intranets. Modern data management applications, such as setting up Web portals, managing enterprise data, managing community data, and sharing scientific data, often require integrating available data sources and providing a uniform interface for users to access data from different sources; such requirements have been driving fruitful research on data integration over the last two decades [11, 13] . Data integration systems face two folds of challenges. First, data from disparate sources are often heterogeneous. Heterogeneity can exist at the schema level, where different data sources often describe the same domain using different schemas; it can also exist at the instance level, where different sources can represent the same real-world entity in different ways. There has been rich body of work on resolving heterogeneity in data, including, at the schema level, schema mapping and matching [14] Second, different sources can provide conflicting data. Conflicts can arise because of incomplete data, erroneous data, and out-of-date data. Returning incorrect data in a query result can be misleading and even harmful: one may contact a person by an out-of-date phone number, visit a clinic at a wrong address, and even make poor business decisions. It is thus critical for data integration systems to resolve conflicts from various sources and identify true values. This problem becomes especially prominent with the ease of publishing and spreading false information on the Web. This tutorial focuses on data fusion, which addresses the second challenge by fusing records on the same real-world entity into a single record and resolving possible conflicts from different data sources. Data fusion plays an important Permission to copy without fee all or part of this material is granted provided that the copies are not made or distributed for direct commercial advantage, the VLDB copyright notice and the title of the publication and its date appear, and notice is given that copying is by permission of the Very Large Data Base Endowment. To copy otherwise, or to republish, to post on servers or to redistribute to lists, requires a fee and/or special permission from the publisher, ACM. VLDB '09, August 24-28, 2009, Lyon, France Copyright 2009 VLDB Endowment, ACM 000-0-00000-000-0/00/00. role in data integration systems: it detects and removes dirty data and increases correctness of the integrated data. The main objective of the tutorial is to gather models, techniques, and systems of the wide but yet unconsolidated field of data fusion and present them in a concise and consolidated manner. In the tutorial we provide an overview of the causes and challenges of data fusion. We cover a wide set of both simple and advanced techniques to resolve data conflicts in different types of settings and systems. Finally, we provide a classification of existing information management systems with respect to their ability to perform data fusion. The tutorial is based on a recent survey on data fusion [3] and various techniques proposed for truth discovery, including, but not limited to, [2, 5, 6, 16, 18] . Our tutorial begins with an overview of the importance of data fusion in data integration and possible reasons for data conflicts. We then present a classification of existing data fusion techniques and introduce relational operations for conflict resolution. After that, we describe several advanced techniques for finding the best (true) values in presence of data conflicts. We end our tutorial with surveying data fusion techniques in existing data integration systems and suggesting future research directions. Data integration has three broad goals: increasing the completeness, conciseness, and correctness of data. Completeness measures the amount of data, in terms of both the number of tuples and the number of attributes. Conciseness measures the uniqueness of object representations in the integrated data, in terms of both the number of unique objects and the number of unique attributes of the objects. Finally, correctness measures correctness of data; that is, whether the data conform to the real world. Data fusion, which is the focus of this tutorial, aims at resolving conflicts from data and increasing correctness of data. We distinguish two kinds of data conflicts: uncertainty and contradiction. Uncertainty is a conflict between a nonnull value and one or more null values that are all used to describe the same property of a real-world entity. Uncertainty is caused by missing information, such as null values in a source or a completely missing attribute in a source. Contradiction is a conflict between two or more different non-null values that are all used to describe the same property of the same entity. Contradiction is caused by different
Entity consolidation or data fusion aims at resolving data conflicts in clusters of duplicate records [11, REF .
Data fusion - Resolving Data Conflicts for Integration
PVLDB
PVLDB
Computer Science