Enabling data interoperability with provenance and schema integration
Authors: Wang-Chiew Tan
Venue:  (2008)

Interoperability is the ability of a system to work with other systems without special effort on the part of the user. One cause of complexity in providing support for data interoperability is the high heterogeneity of the data. The goal of this dissertation is to enable the interoperability of heterogeneous data sources.We study the problem of tracing the provenance of data in the context of two commonly used mapping formalisms: SQL queries, and respectively, schema mappings. In the DBNotes system [22, 23, 33], we develop and implement an approach for tracing data provenance over SQL queries. In the SPIDER system [8, 32], we develop and implement an approach for tracing data provenance over schema mappings.DBNotes is an annotation management system for relational databases where each attribute value of each tuple in a relation is associated with zero or more annotations, and annotations propagate along with the data as data is being transformed through an SQL query. In the default propagation scheme, annotations propagate based on provenance. Hence, the provenance of data in the result of a query can be determined by analyzing its annotations. The default annotation propagation behavior is sensitive to query rewriting. Hence, DBNotes provides an alternative propagation scheme, called default-all, which propagates annotations according to all equivalent rewritings of a given query. A main result of this dissertation is that DBNotes can always simulate the default-all propagation behavior by executing finitely many queries, although there may be infinitely many equivalent rewritings. DBNotes also provides a third propagation scheme, called custom, which allows a user to specify how annotations should propagate.In contrast to DBNotes, which traces data provenance over mappings expressed as SQL queries, the SPIDER system traces data provenance over mappings expressed using a different formalism: the formalism of schema mappings. Schema mappings are high-level, declarative, assertions for specifying relationships between schemas. When compared to SQL queries, schema mappings have an advantage in that they operate at a higher level of abstraction. Thus, they are easier to analyze and reason about. A fundamental difference between DBNotes and SPIDER is the method by which they compute provenance.A primary feature of SPIDER is the novel notion of routes. Routes are a form of provenance: they describe the relationships between data in the source and the output databases that are established by the schema mapping. A main difficulty that arises in designing algorithms for computing routes is that schema mappings may be recursive. In this case, the number of routes may be exponential. Hence, it was initially unclear whether there exists a polynomial-time algorithm for computing routes. This difficulty has been resolved in SPIDER, which features two complete, polynomial-time algorithms for computing one or all routes for selected data, through chains of schema mappings that may be recursive. A main result of this dissertation states that SPIDER produces a compact, polynomial-size representation of all routes in polynomial time, even though there may be exponentially many routes. This part of the dissertation is based on joint work with Wang-Chiew Tan [32]. A demonstration of the SPIDER system is joint work with Bogdan Alexe and Wang-Chiew Tan [8].Another contribution of this dissertation is a principled method and system for performing schema integration. Schema integration is the problem of designing a unified integrated schema based on a set of overlapping source schemas. This aspect of the dissertation enables data interoperability in two ways. First, the integrated schema provides a standard representation of the source data, thus offering a way to deal with the heterogeneity in these databases. Furthermore, our system facilitates the process of designing mappings to relate the sources and the integrated schema, in that it automatically generates these mappings and guarantees they have precise information-preservation properties. The system features a principled method for enumerating multiple integrated schemas, and easy-to-use capabilities for refining the enumerated schemas via user interaction. (Abstract shortened by UMI.)