SlowGuess/NeurIPS_2023_OCR huggingface dataset:
The algorithm and analysis are given in Section H. Replacing the threshold in Line 7 of Algorithm 2 with the output of Algorithm 6 (more precisely the square root of the value) readily gives us the following worst case guarantees. Lemma 3.3 states that the expected error of the estimates outputted by Algorithm 2 using  $B$ , regardless of the true frequency distribution, is no worse than that of a standard CountSketch table using slightly smaller  $O(B / \log \log n)$  space.
Lemma 3.3. Suppose  $B \geq \log n$ . Let  $\{\hat{f}_i\}_{i=1}^n$  denote the estimates of Algorithm using  $B/2$  space and with Line 7 replaced by the square root of the estimate of Algorithm also using  $B/2$  space. Suppose the condition of Lemma holds. Let  $\{\hat{f}_i'\}_{i=1}^n$  denote the estates computed by a CountSketch table with  $\frac{cB}{\log\log n}$  columns for a sufficiently small constant  $c$ . Then,  $\mathbb{E}[|\hat{f}_i - f_i|] \leq \mathbb{E}[|\hat{f}_i' - f_i'|]$ .
Remark 3.1. The learned version of the algorithm automatically inherits any worst case guarantees from the unlearned (without predictions) version. This is because we only set aside half the space to explicitly track the frequency of some elements, which has worst case guarantees, while the other half is used for the unlearned version, also with worst case guarantees.
We experimentally evaluate our algorithms with and without predictions on real and synthetic datasets and demonstrate that the improvements predicted by theory hold in practice. Comprehensive additional figures are given in Appendix J.
Algorithm Implementations In the setting without predictions, we compare our algorithm to CountSketch (CS) (which was shown to have favorable empirical performance compared to CountMin (CM) in [36] and better theoretical performance due to our work). In the setting with predictions, we compare the algorithm of [36], using CS as the base sketch and dedicated half of the space for items
Recall  $B^{\prime} = B / T$  in Algorithm2.
Figure 1: Log-log plots of the sorted frequencies of the first day/minute of the CAIDA/AOL datasets. Both data distributions are heavy-tailed with few items accounting for much of the total stream.
Figure 2: Comparison of weighted error without predictions on the CAIDA dataset. The left plot compares the performance of various algorithms (including our algorithm with different choices of  $C$ ) for a fixed dataset and varying space. The right plot compares algorithms over time across separate streams for each minute of data for a specific choice of space being 750.
which are predicted to be heavy by the learned oracle. For all implementations, we use three rows in the CS table and vary the number of columns. We additionally augment each of these baselines with a version that truncates all negative estimated frequencies to zero as none of our datasets include stream deletions. This simple change does not change the asymptotic  $(\varepsilon, \delta)$  classic sketching guarantees but does make a big difference when measuring empirical weighted error.
We implement a simplified and practical version of our algorithm which uses a single CS table. If the median estimate of an element is below a threshold of  $\frac{Cn}{w}$  for domain size  $n$ , sketch width  $w$  (a third of the total space), and a tunable constant  $C$ , the estimate is instead set to 0. As all algorithms use a single CS table as the basic building block with different estimation functions, for each trial we randomly sample hash functions for a single CS table and only vary the estimation procedure used.
We evaluate algorithms according the weighted error as in Equation (1) but also according to unweighted error which is simply the sum over all elements of the absolute estimation error, given by  $\sum_{i} |f_{i} - \tilde{f}_{i}|$ . Space is measured by the size of the sketch table, and all errors are averaged over 10 independent trials with standard deviations shown shaded in.
Datasets We compare our algorithm with prior work on three datasets. We use the same two real-world datasets and predictions from [36]: the CAIDA and AOL datasets. The CAIDA dataset [12] contains 50 minutes of internet traffic data. For each minute of data, the stream is formed of the IP addresses associated with packets going through a Tier1 ISP. A typical minute of data contains 30 million packets accounted for by 1 million IPs. The AOL dataset [55] contains 80 days of internet search queries with a typical day containing  $\approx 3 \cdot 10^5$  total queries and  $\approx 10^5$  unique queries. As shown in Figure [1], both datasets approximately follow a power law distribution. For both datasets, we use the predictions from prior work [36] formed using recurrent neural networks. We also generate synthetic data following a Zipfian distribution with  $n = 10^7$  elements and where the  $i$ th element has frequency  $n / i$ .
Results Across the board, our algorithm outperforms the baselines. On the CAIDA and AOL datasets without predictions, our algorithm consistently outperforms the standard CS with up to  $4\mathbf{x}$  smaller error with space 300. This gap widens when we compare our algorithm with predictions
Figure 3: Comparison of weighted error with predictions on the CAIDA dataset.
Figure 4: Comparison of weighted error without predictions on the AOL dataset.
to that of [36] with a gap of up to  $17\mathbf{x}$  with space 300. In all cases, the performance of CS and [36] is significantly improved by the simple trick of truncating negative estimates to zero. However, our algorithm still outperforms these "nonneg" baselines. The longitudinal plots which compare algorithms over time show that our algorithm consistently outperforms the state-of-the-art with and without predictions.
In the case of the CAIDA dataset, predictions do not generally improve the performance of any of the algorithms. This is consistent with the findings of [36] where the prediction quality for the CAIDA dataset was relatively poor. However, for the AOL which has a more accurate learned oracle, our algorithm in particular is significantly improved when augmented with predictions. Intuitively, the benefit of our algorithm comes from removing error due to noise for low frequency elements. Conversely, good predictions help to obtain very good estimates of high frequency elements. In combination, this yields very small total weighted error.
In Appendix J we display comprehensive experiments of the performance of the algorithms across the CAIDA and AOL datasets with varying space and for both weighted and unweighted error as well as results for synthetic Zipfian data. In all cases, our algorithm outperforms the baselines. On synthetic Zipfian, the gap between our algorithm and the non-negative CS for weighted error is relatively small compared to that for the real datasets. While we mainly focus on weighted error in this work, the benefits of our algorithm are even more significant for unweighted error as setting estimates below the noise floor to zero is especially impactful for this error measure. In general, we see the trend, matching our theoretical results, that as space increases, the gap between the different algorithms shrinks as the estimates of the base CS become more accurate.
Anders Aamand is supported by DFF-International Postdoc Grant 0164-00022B from the Independent Research Fund Denmark and a Simons Investigator Award. Justin Chen is supported by an NSF Graduate Research Fellowship under Grant No. 174530. Huy Nguyen is supported by NSF Grants 2311649 and 1750716.
Figure 5: Comparison of weighted error with predictions on the AOL dataset.
[1] Anders Aamand, Justin Y. Chen, and Piotr Indyk. (Optimal) Online Bipartite Matching with Degree Information. In Advances in Neural Information Processing Systems, volume 35, 2022.
[2] Noga Alon, Yossi Matias, and Mario Szegedy. The space complexity of approximating the frequency moments. In Proceedings of the twenty-eighth annual ACM symposium on Theory of computing, pages 20-29, 1996.
[3] Amirali Aghazadeh and Ryan Spring and Daniel LeJeune and Gautam Dasarathy and Anshumali Shrivastava and Richard G. Baraniuk. Mission: Ultra large-scale feature selection using countsketches. In International Conference on Machine Learning, pages 80-88. PMLR, 2018.
[4] Keerti Anand, Rong Ge, and Debmalya Panigrahi. Customizing ml predictions for online algorithms. In International Conference on Machine Learning, pages 303-313, 2020.
[5] Spyros Angelopoulos, Christoph Durr, Shendan Jin, Shahin Kamali, and Marc Renault. Online computation with untrusted advice. In 11th Innovations in Theoretical Computer Science Conference (ITCS 2020). Schloss Dagstuhl-Leibniz-Zentrum für Informatik, 2020.
[6] Antonios Antoniadis, Christian Coester, Marek Eliás, Adam Polak, and Bertrand Simon. Online metric algorithms with untrusted predictions. ACM Transactions on Algorithms, 19(2):1-34, 2023.
[7] Maria-Florina Balcan, Travis Dick, Tuomas Sandholm, and Ellen Vitercik. Learning to branch. In International Conference on Machine Learning, pages 353–362, 2018.
[8] George Bennett. Probability inequalities for the sum of independent random variables. Journal of the American Statistical Association, 57(297):33-45, 1962.
[9] Aditya Bhaskara, Ashok Cutkosky, Ravi Kumar, and Manish Purohit. Logarithmic regret from sublinear hints. In Advances in Neural Information Processing Systems 34: Annual Conference on Neural Information Processing Systems 2021, pages 28222-28232, 2021.
[10] Oscar Boykin, Avi Bryant, Edwin Chen, ellchow, Mike Gagnon, Moses Nakamura, Steven Noble, Sam Ritchie, Ashutosh Singhal, and Argyris Zymnis. Algebird. https://twitter.github.io/algebird/, 2016.
[11] Vladimir Braverman, Stephen R Chestnut, Nikita Ivkin, Jelani Nelson, Zhengyu Wang, and David P Woodruff. Bptree: an 12 heavy hitters algorithm using constant memory. In Proceedings of the 36th ACM SIGMOD-SIGACT-SIGAI Symposium on Principles of Database Systems, pages 361-376, 2017.
[12] CAIDA. Caida internet traces, chicago. http://www.caida.org/data/monitors/passive-equinix-chicago.xml, 2016.
[13] Emmanuel J Candès, Justin Romberg, and Terence Tao. Robust uncertainty principles: Exact signal reconstruction from highly incomplete frequency information. IEEE Transactions on information theory, 52(2):489-509, 2006.
[49] Ahmed Metwally, Divyakant Agrawal, and Amr El Abbadi. Efficient computation of frequent and top-k elements in data streams. In International Conference on Database Theory, pages 398-412. Springer, 2005.
[50] Gregory T Minton and Eric Price. Improved concentration bounds for count-sketch. In Proceedings of the twenty-fifth annual ACM-SIAM symposium on Discrete algorithms, pages 669-686. Society for Industrial and Applied Mathematics, 2014.
[51] Jayadev Misra and David Gries. Finding repeated elements. Science of computer programming, 2(2):143-152, 1982.
[52] Michael Mitzenmacher. A model for learned bloom filters and optimizing by sandwiching. In Advances in Neural Information Processing Systems, pages 464-473, 2018.
[53] Michael Mitzenmacher. Scheduling with predictions and the price of misprediction. In 11th Innovations in Theoretical Computer Science Conference (ITCS 2020). Schloss Dagstuhl-Leibniz-Zentrum für Informatik, 2020.
[54] Thy Nguyen, Anamay Chaturvedi, and Huy Le Nguyen. Improved learning-augmented algorithms for  $k$ -means and  $k$ -medians clustering. 2023.
[55] Greg Pass, Abdur Chowdhury, and Cayley Torgeson. A picture of search. In Proceedings of the 1st international conference on Scalable information systems, pages 1-es, 2006.
[56] Manish Purohit, Zoya Svitkina, and Ravi Kumar. Improving online algorithms via ml predictions. In Advances in Neural Information Processing Systems, pages 9661-9670, 2018.
[57] Sandeep Silwal, Sara Ahmadian, Andrew Nystrom, Andrew McCallum, Deepak Ramachandran, and Seyed Mehran Kazemi. Kwikbucks: Correlation clustering with cheap-weak and expensive-strong signals. In The Eleventh International Conference on Learning Representations, 2023.
[58] Partha Talukdar and William Cohen. Scaling graph-based semi supervised learning to large number of labels using count-min sketch. In Artificial Intelligence and Statistics, pages 940-947. PMLR, 2014.
[59] Jun Wang, Wei Liu, Sanjiv Kumar, and Shih-Fu Chang. Learning to hash for indexing big data - a survey. Proceedings of the IEEE, 104(1):34-57, 2016.
[60] Alexander Wei and Fred Zhang. Optimal robustness-consistency trade-offs for learning-augmented online algorithms. Advances in Neural Information Processing Systems, 33:8042-8053, 2020.
[61] David P Woodruff et al. Sketching as a tool for numerical linear algebra. Foundations and Trends® in Theoretical Computer Science, 10(1-2):1-157, 2014.
[62] Minlan Yu, Lavanya Jose, and Rui Miao. Software defined traffic measurement with opensketch. In NSDI, volume 13, pages 29-42, 2013.
[63] Matei Zaharia, Reynold S Xin, Patrick Wendell, Tathagata Das, Michael Armbrust, Ankur Dave, Xiangrui Meng, Josh Rosen, Shivaram Venkataraman, Michael J Franklin, et al. Apache spark: a unified engine for big data processing. Communications of the ACM, 59(11):56-65, 2016.