Many real world applications involve not just a single dataset, but a view of multiple datasets. These datasets may be collected from different sources and/or at different time instances. In such scenarios, comparing patterns or features from different datasets and understanding their relationships can be an extremely important part of the KDD process. This paper considers the problem of optimizing a mining task over multiple datasets, when it has been expressed using a highlevel interface. Specifically, we make the following contributions: 1) We present an SQL-based mechanism for querying frequent patterns across multiple datasets, and establish an algebra for these queries. 2) We develop a systematic method for enumerating query plans and present several algorithms for finding optimized query plan which reduce execution costs. 3) We evaluate our algorithms on real and synthetic datasets, and show up to an order of magnitude performance improvements.
Jin and Agrawal REF established new operators for querying frequent patterns across multiple databases and presented an algorithm for finding an optimized query plan across multiple databases.
Systematic Approach for Optimizing Complex Mining Tasks on Multiple Databases
22nd International Conference on Data Engineering (ICDE'06)
22nd International Conference on Data Engineering (ICDE'06)
Computer Science