We study the classical problem of moment estimation of an underlying vector whose n coordinates are implicitly defined through a series of updates in a data stream. We show that if the updates to the vector arrive in the random-order insertion-only model , then there exist space efficient algorithms with improved dependencies on the approximation parameter \varepsilon . In particular, for any real p > 2, we first obtain an algorithm for F_p moment estimation using%DIFDELCMD < \mathcal{O}\left(%%% \frac{1{\varepsilon^{4/p}}\cdot n^{1-2/p}}%DIFDELCMD < \right) %%% bits of memory. Our techniques also give algorithms for F_p moment estimationwith p>2 on arbitrary order insertion-only and turnstile streams, using%DIFDELCMD < \mathcal{O}\left(%%% \frac{1{\varepsilon^{4/p}}\cdot n^{1-2/p}}%DIFDELCMD < \right) %%% bits of space and two passes, which is the first optimal multi-pass F_p estimation algorithm up to \log n factors. Finally, we give an improved lower bound of \Omega%DIFDELCMD < \left(%%% \frac{1{\varepsilon^2}\cdot n^{1-2/p}}%DIFDELCMD < \right) %%% for one-pass insertion-only streams. Our results separate the complexity of this problem both between random and non-random orders, as well as one-pass and multi-pass streams.
In the adversarially robust streaming model, a stream of elements is presented to an algorithm and is allowed to depend on the output of the algorithm at earlier times during the stream. In the classic insertion-only model %DIFDELCMD < \mathcal{O}\left(%%% {\varepsilon^{4/p}}\cdot n^{1-2/p}}%DIFDELCMD < \right) %%% %DIFDELCMD < \mathcal{O}\left(%%% {\varepsilon^{4/p}}\cdot n^{1-2/p}}%DIFDELCMD < \right) %%% of data streams, Ben-Eliezer et. al. (PODS 2020, best paper award) show how to convert a non-robust algorithm into a robust one with a roughly 1/\varepsilon factor overhead. This was subsequently improved to a 1/\sqrt{\varepsilon the first optimal %DIFDELCMD < \left(%%% {\varepsilon^2}\cdot n^{1-2/p}}%DIFDELCMD < \right) %%% dependence on the approximation factor. As in previous work, we obtain a general transformation that applies to any non-robust streaming algorithm and depends on the so-called flip number. However, the key technical innovation is that we apply the transformation to what we call a difference estimator for the streaming problem, rather than an estimator for the streaming problem itself. We then develop the first difference estimators for a wide range of problems. Our difference estimator methodology is not only applicable to the adversarially robust model, but to other streaming models where temporal properties of the data play a central role. (Abstract shortened to meet arXiv limit.)
R
We study the classical problem of moment estimation of an underlying vector whose n coordinates are implicitly defined through a series of updates in a data stream. We show that if the updates to the vector arrive in the random-order
In the adversarially robust streaming model, a stream of elements is presented to an algorithm and is allowed to depend on the output of the algorithm at earlier times during the stream. In the classic
D
, then there exist space efficient algorithms with improved dependencies on the approximation parameter \varepsilon . In particular, for any real p > 2, we first obtain an algorithm for F_p moment estimation using
D
\frac{1
D
bits of memory. Our techniques also give algorithms for F_p moment estimationwith p>2 on arbitrary order insertion-only and turnstile streams, using
D
\frac{1
R
bits of space and two passes, which is
of data streams, Ben-Eliezer et. al. (PODS 2020, best paper award) show how to convert a non-robust algorithm into a robust one with a roughly 1/\varepsilon factor overhead. This was subsequently improved to a 1/\sqrt{\varepsilon
D
multi-pass F_p estimation algorithm up to \log n factors. Finally, we give an improved lower bound of \Omega
D
\frac{1
R
for one-pass insertion-only streams. Our results separate the complexity of this problem both between random and non-random orders, as well as one-pass and multi-pass streams.
dependence on the approximation factor. As in previous work, we obtain a general transformation that applies to any non-robust streaming algorithm and depends on the so-called flip number. However, the key technical innovation is that we apply the transformation to what we call a difference estimator for the streaming problem, rather than an estimator for the streaming problem itself. We then develop the first difference estimators for a wide range of problems. Our difference estimator methodology is not only applicable to the adversarially robust model, but to other streaming models where temporal properties of the data play a central role. (Abstract shortened to meet arXiv limit.)