space complexity of hierarchical heavy hitters in multi-dimensional data streams
Heavy hitters, which are items occurring with frequency above a given threshold, are an important aggregation and summary tool when processing data streams or data warehouses. Hierarchical heavy hitters (HHHs) have been introduced as a natural generalization for hierarchical data domains, including multi-dimensional data. An item x in a hierarchy is called a ?- HHH if its frequency after discounting the frequencies of all its descendant hierarchical heavy hitters exceeds ? n , where ? is a user-specified parameter and n is the size of the data set. Recently, single-pass schemes have been proposed for computing ?- HHHs using space roughly O (1/? log(? n )). The frequency estimates of these algorithms, however, hold only for the total frequencies of items, and not the discounted frequencies; this leads to false positives because the discounted frequency can be significantly smaller than the total frequency. This paper attempts to explain the difficulty of finding hierarchical heavy hitters with better accuracy. We show that a single-pass deterministic scheme that computes ?- HHHs in a d -dimensional hierarchy with any approximation guarantee must use ?(1/? d +1 ) space. This bound is tight: in fact, we present a data stream algorithm that can report the ?- HHHs without false positives in O (1/? d +1 ) space.
space complexity hierarchic data streaming data threshold aggregate tool process data warehouse general domain hierarchies use size scheme computation log algorithm false positive paper accuracy approximation user