SlowGuess/NeurIPS_2023_OCR huggingface dataset:
The algorithm of Lemma 3.2 for constant probability proceeds by sampling pools of  $k = \widetilde{O}\left(\frac{n}{RT}\right)$  experts and running majority vote on the pool, while iteratively deleting poorly performing experts until no experts remain in the pool, at which a new pool of  $k$  experts is randomly sampled. The main intuition is that either the pool of experts will perform well and achieve low regret, or the pool will be continuously re-sampled until the best expert is sampled multiple times, after which point it will not be deleted from the pool. Unfortunately, it is not evident that this algorithm is robust to adaptive inputs because an adversary can potentially learn the experts in each sampled pool and force the experts to make mistakes only on days in which they are sampled by the algorithm. To boost the algorithm to high probability of success, we take the deterministic majority vote of  $O(\log n)$  independent instances of the algorithm with constant success probability.
Towards adaptive robustness, we use differential privacy to hide the internal randomness of the algorithm, and in particular, the identity of the experts that are sampled by each pool. We first run  $\widetilde{O}(\sqrt{T})$  copies of the algorithm and then output the private median of the  $\widetilde{O}(\sqrt{T})$  copies, guaranteeing roughly  $\left(\frac{1}{\widetilde{O}(\sqrt{T})}, 0\right)$ -differential privacy because we use  $\widetilde{O}(\sqrt{T})$  copies of the algorithm. Advanced composition, i.e., Theorem 2.3, then ensures  $(O(1), 1 / \mathrm{poly}(n))$ -differential privacy, so that correctness then follows from the generalization properties of DP, i.e., Theorem 2.4.
We give our algorithm in full in Algorithm 1.
Algorithm 1 Randomized, robust streaming algorithm for the experts problem
Input: A stream of length  $T$  with  $n$  experts and a target regret  $R$
Output: A sequence of predictions with regret  $R$
1: Run  $m = O\left(\sqrt{T}\log (nT)\right)$  independent instances of DISCPRED with regret  $\frac{R}{4}$
2: Run PRIVMED on the  $m$  instances with privacy parameter  $\varepsilon = O\left(\frac{1}{\sqrt{T}\log(nT)}\right)$  and failure probability  $\delta = \frac{1}{\mathrm{poly}(n,T)}$
3: At each time  $t \in [T]$ , select the output of PRIVMED
Next, we show the correctness of our algorithm on adaptive inputs.
Theorem 3.3 (Algorithm for adaptive inputs). Let  $R > \sqrt{\frac{2048\log^2n}{T}}$ , and suppose the best expert makes at most  $M \leq \frac{R^2T}{1280\log^2n}$  mistakes. Then there exists an algorithm for the discrete prediction with experts problem that uses  $\widetilde{O}\left(\frac{n}{R\sqrt{T}}\right)$  space and achieves regret at most  $R$ , with probability at least  $1 - \frac{1}{\mathrm{poly}(n,T)}$ .
Proof. Suppose we run  $m = O\left(\sqrt{T}\log (nT)\right)$  independent instances of DISCPRED with regret  $\frac{R}{4}$ .
Note that for  $R > \sqrt{\frac{2048\log^2n}{T}}$ , we have  $\frac{R}{4} > \sqrt{\frac{128\log^2n}{T}}$ , which is a valid input to DISCPRED in Lemma 3.2. By Lemma 3.2, each instance succeeds on an arbitrary-order stream with probability at least  $1 - 1 / \mathrm{poly}(n,T)$ . By a union bound over the  $m$  instances, all instances succeed with probability at least  $1 - 1 / \mathrm{poly}(n,T)$ . In particular, each instance has regret at most  $R / 4$ , so that the total number of mistakes by each instance is at most  $M + RT / 4$ . Thus, the total number of mistakes by all instances is at most  $m(M + RT / 4)$ .
To consider an adaptive stream, observe that PRIVMED is called with privacy parameter  $O\left(1 / \sqrt{T}\log (nT)\right)$  and failure probability  $1 / \mathrm{poly}(n,T)$ . By Theorem 2.3, the mechanism permits  $T$  adaptive interactions and guarantees privacy  $O(1)$  with failure probability  $1 / \mathrm{poly}(n,T)$ . By Theorem 2.4, we have that with high probability, if the output of the algorithm is incorrect, then at least  $m / 3$  of the instances DISCPRED are also incorrect. Since the total number of mistakes by all instances is at most  $m(M + RT / 4)$ , then the total number of mistakes by the algorithm is at most  $3(M + RT / 4) \leq M + RT$ , since  $M \leq \frac{R^2T}{1280\log^2n}$ . Hence, the algorithm achieves  $R$  regret with high probability.
By Lemma 3.2, each instance of DISCPRED uses  $\widetilde{O}\left(\frac{n}{RT}\right)$  space. Since we use  $m = O\left(\sqrt{T}\log (nT)\right)$  independent instances of DISCPRED, then the total space is  $\widetilde{O}\left(\frac{n}{R\sqrt{T}}\right)$ .
4 Lower Bound for Arbitrary-Order Streams
In this section, we provide a space lower bound for randomized algorithms with a high probability of success. Together with Theorem 1.1, the lower bound completely characterizes the complexity of deterministic algorithms for the online learning with experts problem. We restate Theorem 1.3, give a proof sketch and defer the full analysis to Appendix D.
Theorem 4.1 (Memory lower bound for high-probability algorithms). For  $n = o(2^T)$ , any randomized algorithm algorithm that achieves  $R$  regret with probability at least  $1 - 2^{-\Omega(T)}$  for the discrete prediction with experts problem must use  $\Omega\left(\frac{nM}{RT}\right)$  space when the best expert makes  $M$  mistakes.
Proof sketch of Theorem 4.1. We consider the communication problem of  $\varepsilon$ -DIFFDIST. It combines  $n$  instances of the distributed detection problem given by [14]. This was also used by the prior work of [49] to prove space lower bounds for expert learning in random-order stream.
Specifically, for fixed  $T$ , the  $\varepsilon$ -DIFFDIST problem with  $\varepsilon = \frac{M}{T}$  consists of  $T$  players, who each hold  $n$  bits, indexed from 1 to  $n$ . The players must distinguish between:
(1) the NO case  $\mathcal{D}_{\mathrm{NO}}^{(n)}$ , in which every bit for every player is drawn i.i.d. from a fair coin and
(2) the YES case  $\mathcal{D}_{\mathrm{YES}}^{(n)}$ , in which an index  $L \in [n]$  is selected arbitrarily and the  $L$ -th bit of each player is chosen i.i.d. from a Bernoulli distribution with parameter  $\left(1 - \frac{M}{T}\right)$ , while all other bits for every player are chosen i.i.d. from a fair coin.
At a high level, the proof proceeds in two steps:
(1) First, we show that the  $\varepsilon$ -DIFFDIST problem can be reduced to the expert prediction problem in the streaming setting.
(2) Second, we prove a communication complexity lower bound for  $\varepsilon$ -DIFFDIST against any protocol that succeeds with probability  $1 - 2^{-\Theta (T)}$ , which includes deterministic protocols.
The first step is straightforward. In the reduction, each player in an instance of  $\varepsilon$ -DIFFDIST corresponds to a day of the expert problem. The  $n$  bit input held by each player correspond to the  $n$  expert predictions of each day. Therefore, in the NO case, each expert is correct on roughly half of the days. In the YES case, there is a single expert  $L \in [n]$  that is correct on roughly  $1/2 + \delta$  of the days (for  $\delta = 1/2 - M/T$ ), while all other experts randomly guess each day. Suppose that there is a streaming
algorithm for the expert prediction problem with average regret  $\delta /2$ . Then roughly speaking, in the YES case, the algorithm is correct approximately on  $1 / 2 + \delta /2$  of the days, while in the NO case where every expert is randomly guessing, the algorithm is correct on less than  $1 / 2 + \delta /2$  of the days. This distinguishes the YES and NO case and thus solves  $\varepsilon$ -DIFFDIST.
For the second step, we show that solving the  $\varepsilon$ -DIFFDIST problem with probability at least  $1 - 2^{-\Theta(T)}$  requires  $\Omega(nM)$  total communication. We give a sketch of the argument below.
Observe that if the input is viewed as a  $T \times n$  matrix, then  $\mathcal{D}_{\mathrm{NO}}^{(n)}$  is a product distribution across columns that can be written as  $\zeta^n$ , where  $\zeta$  is the distribution over a single column such that all entries of the column are i.i.d. Bernoulli with parameter  $\frac{1}{2}$ . We view  $\mathcal{D}_{\mathrm{NO}}^{(n)}$  as a hard distribution and applies an information complexity analysis. By a direct sum argument, it suffices to show that the single column problem, i.e., distinguishing between  $\mathcal{D}_{\mathrm{NO}}^{(1)}$  and  $\mathcal{D}_{\mathrm{YES}}^{(1)}$  (i.e., for  $n = 1$ ), requires  $\Omega(M)$  total communication.
Let  $(C_1, C_2, \ldots, C_T)$  be a single column drawn from the hard distribution—namely, the NO case where each player holds one i.i.d. Bernoulli with parameter  $1/2$ . Let  $A$  be a fixed protocol with success probability at least  $1 - \exp(-\Theta(T))$ . For all  $i < T$ , let  $M_i$  denote the message sent from player  $P_i$  to player  $P_{i+1}$  and  $M_{<i} = \{M_j : j < i\}$ . Let  $\Pi = \Pi(C_1, \dots, C_T)$  be the communication transcript of  $A$  given the input  $(C_i)_{i=1}^T$ . A standard information complexity argument [8] implies that the total communication is at least the information cost, defined as  $I(C_1, \dots, C_T; \Pi(C_1, \dots, C_T))$ , where  $I(X, Y)$  denotes the mutual information between random variables  $X$  and  $Y$ .
The key step now is to lower bound the information cost by  $\Omega(M)$ . The main ideas are the following. For any  $i \in [T]$ , we say that  $(M_i, M_{<i})$  is informative for  $i$  with respect to the input  $C$  and the transcript  $\Pi = (M_1, M_2, \ldots, M_T)$  if
$$
\left| \Pr \left(C _ {i} = 0 \mid M _ {i}, M _ {<   i}\right) - \Pr \left(C _ {i} = 1 \mid M _ {i}, M _ {<   i}\right) \right| \geq c \tag {4.1}
$$
for some constant  $c > 0$ . Otherwise, we say that  $M_{i}$  is uninformative. Informally, an informative message  $M_{i}$  reveals sufficiently large information about  $C_i$  so that the mutual information  $I(M_{i},C_{i}\mid M_{< i})$  would be large. Let  $p_i$  be the probability that  $(M_{i},M_{< i})$  is informative. Intuitively, we need that  $\sum_{i}p_{i}$  is large, because then there would be sufficiently many informative messages, and so the information cost is high. To formalize this approach, we claim two key lemmas. First, by Lemma D.9
$$
I \left(\Pi ; C _ {1}, C _ {2}, \dots , C _ {T}\right) = \sum_ {j = 1} ^ {T} I \left(M _ {j}; C _ {j} \mid M _ {<   j}\right) \geq \Omega \left(\sum_ {j = 1} ^ {T} p _ {j}\right).
$$
Conceptually, this shows that the information cost is at least the expected number of informative messages. Furthermore, by Lemma D.10, the latter is indeed high, and in particular,  $\sum_{j}p_{j}\geq \Omega (M)$ . Much of the technical work is dedicated to prove these lemmas. This finishes the proof since the communication complexity is lower bounded by the information cost.
In this section, we perform experimental evaluations as a simple proof-of-concept demonstrating the importance of deterministic algorithms against adversarial input.
Experimental setup. We assume a white-box adversary with access to the internal state of the algorithm. We evaluate the natural deterministic algorithm that iteratively selects pools of  $k = \widetilde{O}\left(\frac{nM}{RT}\right)$  experts, discarding any expert that has erred, and refreshing the pool with the next batch of  $k$  experts once the pool is emptied. As a baseline, we compare to a randomized algorithm that repeatedly samples pools of  $k = \widetilde{O}\left(\frac{nM}{RT}\right)$  experts, discarding any expert that has erred, and refreshing the pool with the next batch of  $k$  sampled experts once the pool is emptied.
Provided that the best expert has not yet made  $M$  mistakes, the adversary simply compels the experts in each pool to err. Once all experts have made at least  $M$  mistakes, the adversary gives up and permits all subsequent predictions to be correct. It can be theoretically verified that against such an adversary, the deterministic algorithm is the optimal algorithm, in the sense that it achieves the smallest number of errors.
Experimental details. We first evaluate our experiments on the setting  $n = 10000$ ,  $M = 20$ , and  $T = 1000$  across various values of  $R \in \{0.05, 0.1, 0.15, 0.2, 0.25, 0.3, 0.35\}$ . For each setting of  $R$ , we ran the experiment 20 times, recording the runtime and number of errors by the algorithms in each repetition. We then computed the minimum, mean, and maximum number of errors by the randomized algorithm across all 20 repetitions. We then repeated the experimental setup for a  $10x$  larger setting of  $T$ , i.e.,  $n = 10000$ ,  $M = 20$ , and  $T = 1000$ . Our experiments were performed on a 64-bit operating system using an AMD Ryzen 75700U CPU with 8.00 GB RAM and 8 cores with base clock 1.80 GHz.
Results. Our experiments show that the deterministic algorithm performs significantly better than the randomized algorithm. On average across the 20 trials for each setting, the randomized algorithm made several times more mistakes than the deterministic algorithm, ranging from  $1.98\mathrm{x}$  times more mistakes for the setting  $n = 100000$ ,  $M = 20$ ,  $T = 1000$ ,  $R = 0.05$  to  $3.06\mathrm{x}$  times more mistakes for the setting  $n = 100000$ ,  $M = 10$ ,  $T = 10000$ ,  $R = 0.3$ . Even the best performance by a randomized algorithm over all trials, which occurred at the setting  $n = 100000$ ,  $M = 20$ ,  $T = 1000$ ,  $R = 0.05$ , the randomized algorithm made  $1.9\mathrm{x}$  times more mistakes than the deterministic algorithm. Meanwhile, the worst performance by a randomized algorithm over all trials, which occurred at the setting  $n = 100000$ ,  $M = 10$ ,  $T = 10000$ ,  $R = 0.25$ , the randomized algorithm made  $3.29\mathrm{x}$  times more mistakes than the deterministic algorithm. The average runtime was 98 seconds for each batch of 20 experiments for the setting of  $n = 100000$ ,  $M = 20$ ,  $T = 1000$ ,  $R = 0.05$  while the average runtime was 98 seconds for each batch of 20 experiments for the setting of  $n = 100000$ ,  $M = 10$ ,  $T = 10000$ ,  $R = 0.3$  was roughly 350 seconds. See Figure 1 for a summary.
(a)  $n = 100000, M = 20, T = 1000$
(b)  $n = 100000, M = 10, T = 10000$
Figure 1: Comparison of errors made by deterministic algorithm and average number of errors made by randomized algorithm across 20 repetitions for each trial, across various values of input target regret  $R$ . Minimum and maximum numbers of errors by randomized algorithm across each trial are also reported.
In this work, we provide robust streaming algorithms for learning with experts. We provide a deterministic algorithm parametrized by the number of mistakes made by the best expert. We also give a randomized algorithm with a different space-regret trade-off, based on differential privacy. We complement our algorithms with a lower bound for high-probability success algorithms. This gives tight memory lower bound for deterministic algorithms. We then show the importance of robust algorithmic design by empirically comparing the performance of the natural deterministic algorithm and the state-of-the-art randomized algorithm when the inputs are adaptive.
We remark that our results do not rule out space-efficient robust algorithms that match the bounds of the oblivious randomized algorithm of [49] for constant probability of success. We believe whether or not there exists such an algorithm is a fascinating question for future work.
David P. Woodruff and Samson Zhou were partially supported by a Simons Investigator Award and by the National Science Foundation under Grant No. CCF-1815840. This work was done in part while Fred Zhang was at Google Research, and Samson Zhou was at Carnegie Mellon University, UC Berkeley, and Rice University.
[1] Miklós Ajtai, Vladimir Braverman, T.S. Jayram, Sandeep Silwal, Alec Sun, David P. Woodruff, and Samson Zhou. The white-box adversarial data stream model. In Proceedings of the 41st ACM SIGMOD-SIGACT-SIGAI Symposium on Principles of Database Systems (PODS), 2022. A
[2] Raman Arora, Teodor Vanislavov Marinov, and Mehryar Mohri. Bandits with feedback graphs and switching costs. In Advances in Neural Information Processing Systems (NeurIPS), 2019. A
[3] Sepehr Assadi, Amit Chakrabarti, Prantar Ghosh, and Manuel Stoeckl. Coloring in graph streams via deterministic and adversarially robust algorithms. CoRR, abs/2212.10641, 2022. A
[4] Idan Attias, Edith Cohen, Moshe Shechner, and Uri Stemmer. A framework for adversarial streaming via differential privacy and difference estimators. In 14th Innovations in Theoretical Computer Science Conference, ITCS, pages 8:1-8:19, 2023. 1.2, A
[5] Peter Auer, Nicolò Cesa-Bianchi, Yoav Freund, and Robert E. Schapire. The nonstochastic multiarmed bandit problem. SIAM J. Comput., 32(1):48-77, 2002. A
[6] Dmitrii Avdiukhin, Slobodan Mitrovic, Grigory Yaroslavtsev, and Samson Zhou. Adversarily robust submodular maximization under knapsack constraints. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, KDD, pages 148-156, 2019. A
[7] Baruch Awerbuch and Robert D. Kleinberg. Adaptive routing with end-to-end feedback: distributed learning and geometric approaches. In Proceedings of the 36th Annual ACM Symposium on Theory of Computing (STOC), 2004. A
[8] Ziv Bar-Yossef, T. S. Jayram, Ravi Kumar, and D. Sivakumar. An information statistics approach to data stream and communication complexity. J. Comput. Syst. Sci., 68(4):702-732, 2004. 4, B.5, B.6
[9] Raef Bassily, Kobbi Nissim, Adam D. Smith, Thomas Steinke, Uri Stemmer, and Jonathan R. Ullman. Algorithmic stability for adaptive data analysis. SIAM J. Comput., 50(3), 2021. 2.4
[10] Amos Beimel, Haim Kaplan, Yishay Mansour, Kobbi Nissim, Thatchaphol Saranurak, and Uri Stemmer. Dynamic algorithms against an adaptive adversary: generic constructions and lower bounds. In 54th Annual ACM SIGACT Symposium on Theory of Computing (STOC), 2022. 1.2, A
[11] Omri Ben-Eliezer, Talya Eden, and Krzysztof Onak. Adversarially robust streaming via dense-sparse trade-offs. In 5th Symposium on Simplicity in Algorithms (SOSA@SODA), 2022. A
[12] Omri Ben-Eliezer, Rajesh Jayaram, David P. Woodruff, and Eylon Yogev. A framework for adversarially robust streaming algorithms. SIGMOD Rec., 50(1):6–13, 2021. 1.1, A
[13] Allan Borodin and Ran El-Yaniv. Online computation and competitive analysis. Cambridge University Press, 1998. A
[14] Mark Braverman, Ankit Garg, Tengyu Ma, Huy L Nguyen, and David P Woodruff. Communication lower bounds for statistical estimation problems via a distributed data processing inequality. In Proceedings of the Forty-eighth Annual ACM Symposium on Theory of Computing (STOC), 2016. 4
[52] Vladimir Vovk. Aggregating strategies. In Proceedings of the Third Annual Workshop on Computational Learning Theory (COLT), 1990. 1.2
[53] Vladimir Vovk. A game of prediction with expert advice. J. Comput. Syst. Sci., 56(2):153-173, 1998.
[54] Vladimir Vovk. Derandomizing stochastic prediction strategies. Machine Learning, 35(3):247-282, 1999.
[55] Vladimir Vovk. Defensive prediction with expert advice. In 16th International Conference on Algorithmic Learning Theory (ALT), 2005. 1.2
[56] David Wajc. Rounding dynamic matchings against an adaptive adversary. In 52nd Annual ACM SIGACT Symposium on Theory of Computing (STOC), 2020. A
[57] David P. Woodruff and Samson Zhou. Tight bounds for adversarially robust streams and sliding windows via difference estimators. In 62nd IEEE Annual Symposium on Foundations of Computer Science (FOCS), 2021. A
A Additional Related Work on Adaptive Inputs
Motivated by non-independent inputs and adversarial attacks, adaptive inputs have recently been considered in the centralized model [19, 41, 20, 21], in the streaming model [6, 12, 34, 57, 15, 18, 11, 1, 3, 22, 4, 28], and in the dynamic model [56, 10]. In particular, algorithms robust to inputs that can depend on the previous outputs by the algorithm, i.e., black-box attacks, are also robust to situations in which future inputs may be dependent on previous outputs. This is especially relevant in applications such as forecasting, in which a prediction on day  $i$  can lead to a series of actions that might impact outcomes and expert predictions on day  $i + 1$  and beyond.
Adaptive adversaries have received considerable attention in literature for online learning when the goal is simply to achieve the best possible regret [13, 17, 44]. Building off a line of results on multi-armed bandit problems [5, 7, 40], the work of [45] first considered the experts setting against memory-bounded adaptive adversaries, giving an algorithm with regret  $O\left(T^{2/3}\right)$ . An early paper of [25] introduced a family of algorithms for adaptive inputs, but provided guarantees using concepts not quite related to the standard definitions of regret. More recent works have explored online learning with additional considerations, such as alternative quantities to optimize [27], additional switching costs [16, 26, 48], and feedback graphs [2]. The closest work to our setting is the recent result by [47] showing that no algorithm using space sublinear in  $n$  can achieve regret sublinear in  $T$  when the input is chosen by an adversary with access to the internal state of the algorithm, i.e., a white-box adversary.
B Additional Technical Preliminaries
For any  $p \in [0,1]$ , we slightly abuse notation and let  $H(p) = -p\log_2p - (1 - p)\log_2(1 - p)$  be the binary entropy function. The following is a standard upper and lower bound of  $H(p)$ .
Lemma B.1 (Bound on the binary entropy function; see e.g. [51]). For  $p \in [0, 1]$ , the binary entropy function satisfies
$$
4 p (1 - p) \leq H (p) \leq 2 (p (1 - p)) ^ {1 / \ln 4}.
$$
B.2 Communication Complexity
Definition B.2 (Mutual information). Let  $X$  and  $Y$  be a pair of random variables with joint distribution  $p(x, y)$ . Then the mutual information is defined as  $I(X; Y) := \sum_{x, y} p(x, y) \log \frac{p(x, y)}{p(x)p(y)}$ , for marginal distributions  $p(x)$  and  $p(y)$ .
In a multi-party communication problem of  $t$  players, each player is given  $x_{i}\in \mathcal{X}_{t}$ . They communicate according to fixed protocol to compute a function  $f:\mathcal{X}_t\times \dots \times \mathcal{X}_t\to \mathcal{Y}$ . A protocol  $\Pi$  is called a  $\delta$ -error protocol for  $f$  if there exists a function  $\Pi_{\mathrm{out}}$  such that  $\operatorname*{Pr}[\Pi_{\mathrm{out}}(\Pi (x,y)) = f(x,y)]\geqslant 1 - \delta$ . For a (multi-party) communication problem, we denote the transcript of all communication in a protocol as  $\Pi \in \{0,1\}^*$ . The communication cost of a protocol, as a result, is the bit length of the transcript. Let  $R_{\delta}(f)$  denote the minimum communication cost across all  $\delta$ -error protocols for  $f$ .
Definition B.3 (Information cost). Let  $\Pi$  be a randomized protocol that produces a random variable  $\Pi(X_1, \ldots, X_T)$  as a transcript on inputs  $X_1, \ldots, X_T$  drawn from a distribution  $\mu$ . Then the information cost of  $\Pi$  with respect to  $\mu$  is defined as  $I(X_1, \ldots, X_T; \Pi(X_1, \ldots, X_T))$ .
Definition B.4 (Information complexity). The information complexity of a function  $f$  with respect to a distribution  $\mu$  and failure probability  $\delta$  is the minimum information cost of a protocol for  $f$  with respect to  $\mu$  that fails with probability at most  $\delta$  on every input and denoted by  $\mathsf{IC}_{\mu, \delta}(f)$ .
Lemma B.5 (Information cost decomposition lemma, Lemma 5.1 in [8]). Let  $\mu$  be a mixture of product distributions and suppose  $\Pi$  is a protocol for inputs  $(X_{1},\ldots ,X_{T})\sim \mu^{n}$ . Then  $I(X_{1},\dots,X_{T};\Pi (X_{1},\dots,X_{T}))\geq \sum_{i = 1}^{n}I(X_{1,i},\dots,X_{T,i};\Pi (X_{1},\dots,X_{T}))$ , where  $X_{i,j}$  denotes the  $j$ -th component of  $X_{i}$ .
Lemma B.6 (Information complexity lower bounds communication complexity; Proposition 4.3 [8]). For any distribution  $\mu$  and error  $\delta$ ,  $R_{\delta}(f) \geq \mathsf{IC}_{\mu, \delta}(f)$ .
C Proof of Theorem 3.1 and Formal Algorithm
We give a formal description of our deterministic robust algorithm in pseudocode.