Approximating ranks, quantiles, and distributions over streaming data is a central task in data analysis and monitoring. Given a stream of n items from a data universe U equipped with a total order, the task is to compute a sketch (data structure) of size \mathrm{poly (\log(n), 1/\varepsilon). Given the sketch and a query item y \in U, one should be able to approximate its rank in the stream, i.e., the number of stream elements smaller than  y. Most works to date focused on additive \varepsilon n error approximation, culminating in the KLL sketch that achieved optimal asymptotic behavior. This paper investigates multiplicative (1\pm\varepsilon)-error approximations to the rank. The motivation stems from practical demands to understand the tails of distributions, and hence for sketches to be more accurate near extreme values. The most space-efficient algorithms that can be derived from prior work store either O(\log(\varepsilon^2 n)/\varepsilon^2) or O(\log^3(\varepsilon n)/\varepsilon) universe items. This paper presents a randomized sketch of size O(\log^{1.5}(\varepsilon n)/\varepsilon) that achieves a 1%DIFDELCMD < \pm%%% \varepsilon multiplicative error guarantee, without prior knowledge of the stream length or dependence on the size of the data universe. This is within a O(\log(\varepsilon n)) factor of optimal. Moreover, our sketch is fully mergeable, which makes it suitable for a parallel or distributed environment .
Approximating ranks, quantiles, and distributions over streaming data is a central task in data analysis and monitoring. Given a stream of n items from a data universe U equipped with a total order, the task is to compute a sketch (data structure) of size poly (\log(n), 1/\varepsilon). Given the sketch and a query item y \in U, one should be able to approximate its rank in the stream, i.e., the number of stream elements smaller than or equal to y. Most works to date focused on additive \varepsilon n error approximation, culminating in the KLL sketch that achieved optimal asymptotic behavior. This paper investigates multiplicative (1\pm\varepsilon)-error approximations to the rank. Practical motivation for multiplicative error stems from demands to understand the tails of distributions, and hence for sketches to be more accurate near extreme values. The most space-efficient algorithms due to prior work store either O(\log(\varepsilon^2 n)/\varepsilon^2) or O(\log^3(\varepsilon n)/\varepsilon) universe items. This paper presents a randomized algorithm storing O(\log^{1.5}(\varepsilon n)/\varepsilon) %DIFDELCMD < \pm%%% items, which is within an O(\log(\varepsilon n)) factor of optimal. The algorithm does not require prior knowledge of the stream length and is fully mergeable, rendering it suitable for parallel and distributed computing environments .
R
\mathrm{poly
poly
A
or equal to
R
The motivation stems from practical
Practical motivation for multiplicative error stems from
R
that can be derived from
due to
R
sketch of size
algorithm storing
D
that achieves a 1
R
\varepsilon multiplicative error guarantee, without prior knowledge of the stream length or dependence on the size of the data universe. This is within a
items, which is within an
R
Moreover, our sketch
The algorithm does not require prior knowledge of the stream length and
R
which makes
rendering
R
a parallel or distributed environment
parallel and distributed computing environments