Patent ID: 7613682
Filing Date: 2009-11-03
Classification: G06F,Y10S

Abstract:
1. A method, on an information processing system, for collecting statistics associated with data in a database, the method comprising: determining an amount of memory needed to collect statistics for data associated with a defined data type in a relational database, wherein the amount of memory also includes an amount of memory needed for a set of reservoirs used to collect specific statistics for the data associated with the defined data type and a work area associated with each of the reservoirs, and an amount of memory needed for a set of filters to be used in collecting statistics for data associated with the defined data type, wherein the defined data type is based upon a mark-up language using a tree structure with one or more root-to-node paths therein; allocating the amount of memory as determined for collecting the statistics for the data of the defined data type; performing a collection of statistics for the data of the defined data type in a single pass through the database and within the amount of memory which has been allocated, wherein performing a collection of statistics includes: counting a total number of instances of at least one path-identifier within a given set of documents; determining at least one of a node count and a document count for the path-identifier based on counting the total number of instances of the at least one path-identifier, wherein the at least one path-identifier is one of a given number of most frequently occurring path-identifiers in the given set of documents; counting, for each document within a set of given documents, a total number of path-identifiers that are each associated with a unique path, thereby resulting in a total number of path-identifiers in the set of given documents that are associated with the unique path; counting a total number of nodes within the set of given documents; and counting, for each path-identifier that is associated with a unique path, a total number of documents in the set of given documents that include the path-identifier, thereby resulting in a sum of document counts; sampling at a document level, wherein the sampling comprises applying a first bloom filter in the set of filters to each path-identifier prior to passing the path-identifier to the second reservoir, thereby preventing a duplicate path-identifier within the particular document from being passed to the second reservoir; summarizing the first reservoir for identifying a node count for determining a given number of most frequently occurring path-identifiers; and summarizing the second reservoir for identifying a document count for determining the given number of most frequently occurring path-identifiers.