SlowGuess/NeurIPS_2023_OCR huggingface dataset:
1. The adversary  $\mathcal{B}$  chooses two neighboring datasets  $S_0$  and  $S_{1}$ .
2. The dataset  $S_{b}$  is given to the mechanism  $\mathcal{M}$ .
3. For  $i = 1,2,\ldots$
(a) The adversary  $\mathcal{B}$  chooses a query  $q_{i} \in Q$ .
(b) The mechanism  $\mathcal{M}$  is given  $q_{i}$  and returns  $a_{i}$
(c)  $a_{i}$  is given to  $\mathcal{B}$
4. When  $\mathcal{M}$  or  $\mathcal{B}$  halts, output  $\mathcal{B}$ 's view of the interaction.
Theorem 2.8 ([Dwork et al., 2009, Hardt and Rothblum, 2010, Kaplan et al., 2021]). Algorithm AboveThreshold is  $(\varepsilon, \delta)$ -differentially private.
A private counter. In the setting of algorithm AboveThreshold, the dataset is fixed in the beginning of the execution, and the queries arrive sequentially one by one. Dwork et al. [2010a] and Chan et al. [2010] considered a different setting, in which the data arrives sequentially. In particular, they considered the counter problem where in every time step  $i \in [T]$  we obtain an input bit  $x_{i} \in \{0,1\}$  (representing the data of user  $i$ ) and must immediately respond with an approximation for the current sum of the bits. That is, at time  $i$  we wish to release an approximation for  $x_{1} + x_{2} + \dots + x_{i}$ .
Similarly to our previous discussion, this is an interactive setting, and privacy is defined via a game between a mechanism  $\mathcal{M}$  and an adversary  $\mathcal{B}$  that adaptively determines the inputs for the mechanism.
Definition 2.9 (DP under adaptive inputs [Dwork et al., 2006, 2010a, Chan et al., 2010, Kaplan et al., 2021, Jain et al., 2021]). Let  $\mathcal{M}$  be a mechanism that in every round  $i$  obtains an input point  $x_{i}$  (representing the information of user  $i$ ) and outputs a response  $a_{i}$ . Mechanism  $\mathcal{M}$  is  $(\varepsilon, \delta)$ -differentially private if for every adversary  $\mathcal{B}$  we have that  $\text{AdaptiveInput}_{\mathcal{M},\mathcal{B}}$  (defined below) is  $(\varepsilon, \delta)$ -differentially private (w.r.t. its input bit  $b$ ).
Algorithm 3 AdaptiveInput  $\mathcal{M},\mathcal{B}$  [Jain et al., 2021]
1. For  $i = 1,2,\ldots$
(a) The adversary  $\mathcal{B}$  outputs a bit  $c_{i} \in \{0,1\}$ , under the restriction that  $\sum_{j=1}^{i} c_{j} \leq 1$ .
$\%$  The round  $i$  in which  $c_{i} = 1$  is called the challenge round. Note that there could be at most one challenge round throughout the game.
(b) The adversary  $\mathcal{B}$  chooses two input points  $x_{i,0}$  and  $x_{i,1}$ , under the restriction that if  $c_{i} = 0$  then  $x_{i,0} = x_{i,1}$ .
(c) Algorithm  $\mathcal{M}$  obtains  $x_{i,b}$  and outputs  $a_{i}$ .
(d)  $a_{i}$  is given to  $\mathcal{B}$
2. When  $\mathcal{M}$  or  $\mathcal{B}$  halts, output  $\mathcal{B}$ 's view of the interaction.
Theorem 2.10 (Private counter [Dwork et al., 2010a, Chan et al., 2010, Jain et al., 2021]). There exists an  $(\varepsilon, 0)$ -differentially private mechanism  $\mathcal{M}$  (as in Definition 2.9) that in each round  $i \in [T]$  obtains an input bit  $x_i \in \{0, 1\}$  and outputs a response  $a_i \in \mathbb{N}$  with the following properties. Let  $s$  denote the random coins of  $\mathcal{M}$ . Then there exists an event  $E$  such that: (1)  $\operatorname*{Pr}[s \in E] \geq 1 - \beta$ , and (2) Conditioned on every  $s \in E$ , for every input sequence  $(x_1, \ldots, x_T)$ , the answers  $(a_1, \ldots, a_T)$  satisfy  $\left|a_i - \sum_{j=1}^{i} x_i\right| \leq O\left(\frac{1}{\varepsilon} \log(T) \log\left(\frac{T}{\beta}\right)\right)$ .
3 Challenge Differential Privacy
We now introduce the privacy definition we consider in this work is. Intuitively, the requirement is that even an adaptive adversary controlling all of the users except Alice, cannot learn much information about the interaction Alice had with the algorithm.
Definition 3.1 (Extension of Naor et al. [2023]). Consider an algorithm  $\mathcal{M}$  that, in each round  $i\in [T]$  obtains an input point  $x_{i}$ , outputs a "predicted" label  $\hat{y}_i$ , and obtains a "true" label  $y_{i}$ . We say that algorithm  $\mathcal{M}$  is  $(\varepsilon ,\delta)$ -challenge differentially private if for any adversary  $\mathcal{B}$  we have that OnlineGame $_{\mathcal{M},\mathcal{B},T}$ , defined below, is  $(\varepsilon ,\delta)$ -differentially private (w.r.t. its input bit  $b$ ).
Algorithm 4 OnlineGame $_{\mathcal{M}, \mathcal{B}, T, g}$
Setting:  $T \in \mathbb{N}$  denotes the number of rounds and  $g \in \mathbb{N}$  is a "group privacy" parameter. If not explicitly stated we assume that  $g = 1$ .  $\mathcal{M}$  is an online algorithm and  $\mathcal{B}$  is an adversary that determines the inputs adaptively.
Input of the game: A bit  $b \in \{0,1\}$ . (The bit  $b$  is unknown to  $\mathcal{M}$  and  $\mathcal{B}$ .)
1. For  $i = 1,2,\ldots ,T$
(a) The adversary  $\mathcal{B}$  outputs a bit  $c_{i} \in \{0,1\}$ , under the restriction that  $\sum_{j=1}^{i} c_{j} \leq g$ .
$\%$  We interpret rounds  $i$  in which  $c_{i} = 1$  as challenge rounds. Note that there could be at most  $g$  challenge rounds throughout the game.
(b) The adversary  $\mathcal{B}$  chooses two labeled inputs  $(x_{i,0},y_{i,0})$  and  $(x_{i,1},y_{i,1})$ , under the restriction that if  $c_{i} = 0$  then  $(x_{i,0},y_{i,0}) = (x_{i,1},y_{i,1})$ .
(c) Algorithm  $\mathcal{M}$  obtains  $x_{i,b}$ , then outputs  $\hat{y}_i$ , and then obtains  $y_{i,b}$ .
(d) If  $c_{i} = 0$  then set  $\tilde{y}_i = \hat{y}_i$ . Otherwise set  $\tilde{y}_i = \bot$ .
(e) The adversary  $\mathcal{B}$  obtains  $\tilde{y}_i$
$\%$  Note that the adversary  $\mathcal{B}$  does not get to see the outputs of  $\mathcal{M}$  in challenge rounds.
2. Output  $\mathcal{B}$ 's view of the game, that is  $\tilde{y}_1,\dots ,\tilde{y}_T$  and the internal randomness of  $\mathcal{B}$ .
$\%$  Note that from this we can reconstruct all the inputs specified by  $\mathcal{B}$  throughout the game.
Remark 3.2. Naor et al. [2023] studied a special case of this definition, suitable to their stochastic setting. More specifically, they considered a setting where the algorithm initially gets a dataset containing labeled examples. Then, on every time step, a new user arrives and submits its unlabeled example to the algorithm, and the algorithm responds with a predicted label. We extend the definition to capture settings in which every user interacts with the algorithm (rather than just submitting its input). In the concrete application we consider (online learning) this corresponds to the user submitting its input, then obtaining the predicted label, and then submitting the "true" label. Our generalized definition (Section A) captures this as a special case and allows for arbitrary interactions with each user.
Composition and post-processing. Composition and post-processing for challenge-DP follows immediately from their analogues for (standard) DP. Formally, composition is defined via the following game, called CompositionGame, in which a "meta adversary"  $\mathcal{B}^*$  is trying to guess an unknown bit  $b \in \{0,1\}$ . The meta adversary  $\mathcal{B}^*$  is allowed to (adaptively) invoke  $k$  executions of the game specified in Algorithm 4, where all of these  $k$  executions are done with the same (unknown) bit  $b$ . See Algorithm 5.
Algorithm 5 CompositionGame $_{\mathcal{B}^*,m,\varepsilon,\delta}$
Input of the game: A bit  $b \in \{0,1\}$ . (The bit  $b$  is unknown to  $\mathcal{B}^*$ .)
1. For  $\ell = 1,2,\ldots ,m$
(a) The adversary  $\mathcal{B}^*$  outputs an  $(\varepsilon, \delta)$ -challenge-DP algorithm  $\mathcal{M}_{\ell}$ , an adversary  $\mathcal{B}_{\ell}$ , and an integer  $T_{\ell}$ .
(b) The adversary  $\mathcal{B}^*$  obtains the outcome of OnlineGame $_{\mathcal{M}_{\ell}, \mathcal{B}_{\ell}, T_{\ell}}(b)$ .
2. Output  $\mathcal{B}^*$ 's view of the game (its internal randomness and all of the outcomes of OnlineGame it obtained throughout the execution).
The following theorem follows immediately from standard composition theorems for differential privacy [Dwork et al., 2010b].
Theorem 3.3 (special case of [Dwork et al., 2010b]). For every  $\mathcal{B}^*$ , every  $m \in \mathbb{N}$  and every  $\varepsilon, \delta, \delta' \geq 0$  it holds that CompositionGame $_{\mathcal{B}^*, m, \varepsilon, \delta}$  is  $(\varepsilon', m\delta + \delta')$ -differentially private (w.r.t. the input bit  $b$ ) for  $\varepsilon' = \sqrt{2m\ln(1/\delta')} \varepsilon + m\varepsilon(e^{\varepsilon} - 1)$ .
Group privacy. We show that challenge-DP is closed under group privacy. This is more subtle than the composition argument. In fact, we first need to define what do we mean by "group privacy" in the context of challenge-DP, which we do using the parameter  $g$  in algorithm 0OnlineGame. Recall that throughout the execution of 0OnlineGame, the adversary is allowed  $g$  challenge rounds. We show that if an algorithm satisfies challenge-DP when the adversary is allowed only a single challenge round, then it also satisfies challenge-DP (with related privacy parameters) when the adversary is allowed  $g > 1$  challenge rounds. This is captured by the following theorem; see Appendix B for the proof.
Theorem 3.4. Let  $\mathcal{M}$  be an algorithm that in each round  $i\in [T]$  obtains an input point  $x_{i}$ , outputs a "predicted" label  $\hat{y}_i$ , and obtains a "true" label  $y_{i}$ . If  $\mathcal{M}$  is  $(\varepsilon ,\delta)$ -challenge-DP then for every  $g\in \mathbb{N}$  and every adversary  $\mathcal{B}$  (posing at most  $g$  challenges) we have that OnlineGame $_{\mathcal{M},\mathcal{B},T,g}$  is  $(g\varepsilon, g\cdot e^{\varepsilon g}\cdot \delta)$ -differentially private.
4 Algorithm ChallengeAT
Towards presenting our private online learner, we introduce a variant of algorithm AboveThreshold with additional guarantees, which we call ChallengeAT. Recall that AboveThreshold "hides" arbitrary modifications to a single input point. Intuitively, the new variant we present aims to hide both an arbitrary modification to a single input point and an arbitrary modification to a single query throughout the execution. Consider algorithm ChallengeAT.
Algorithm 6 ChallengeAT
Input: Dataset  $S \in X^{*}$ , privacy parameters  $\varepsilon, \delta$ , threshold  $t$ , number of positive reports  $r$ , and an adaptively chosen stream of queries  $f_{i}: X^{*} \to \mathbb{R}$  each with sensitivity  $\Delta$
Tool used: An  $(\varepsilon, 0)$ -DP algorithm, PrivateCounter, for counting bits under continual observation, guaranteeing error at most  $\lambda$  with probability at least  $1 - \delta$
1. Instantiate PrivateCounter
2. Denote  $\gamma = O\left(\frac{\Delta}{\varepsilon}\sqrt{r + \lambda}\ln \left(\frac{r + \lambda}{\delta}\right)\right)$
3. In each round  $i$ , when receiving a query  $f_{i}$ , do the following:
(a) Let  $\hat{f}_i\gets f_i(S) + \mathrm{Lap}(\gamma)$
(b) If  $\hat{f}_i\geq t$  , then let  $\sigma_{i} = 1$  and otherwise let  $\sigma_{i} = 0$
(c) Output  $\sigma_{i}$
(d) Feed  $\sigma_{i}$  to PrivateCounter and let count  $^{\mathrm{i}}$  denote its current output
(e) If  $\mathrm{count}_i\geq r$  then HALT
Remark 4.1. When we apply ChallengeAT, it sets  $\lambda = O\left(\frac{1}{\varepsilon}\log (T)\log \left(\frac{T}{\beta}\right)\right)$ . Technically, for this it has to know  $T$  and  $\beta$ . To simplify the description this is not explicit in our algorithms.
The utility guarantees of ChallengeAT are straightforward. The following theorem follows by bounding (w.h.p.) all the noises sampled throughout the execution (when instantiating ChallengeAT with the private counter from Theorem 2.10).
Theorem 4.2. Let  $s$  denote the random coins of ChalengeAT. Then there exists an event  $E$  such that: (1)  $\operatorname*{Pr}[s \in E] \geq 1 - \beta$ , and (2) Conditioned on every  $s \in E$ , for every input dataset  $S$  and every sequence of  $T$  queries  $(f_1, \ldots, f_T)$  it holds that
1. Algorithm Challenge does not halt before the  $r$ th time in which it outputs  $\sigma_{i} = 1$ ,
2. For every  $i$  such that  $\sigma_{i} = 1$  it holds that  $f_{i}(S)\geq t - O\left(\frac{\Delta}{\varepsilon}\sqrt{r + \lambda}\ln (\frac{r + \lambda}{\delta})\log (\frac{T}{\beta})\right),$
3. For every  $i$  such that  $\sigma_{i} = 0$  it holds that  $f_{i}(S)\leq t + O\left(\frac{\Delta}{\varepsilon}\sqrt{r + \lambda}\ln \left(\frac{r + \lambda}{\delta}\right)\log \left(\frac{T}{\beta}\right)\right)$
where  $\lambda = O\left(\frac{1}{\varepsilon}\log (T)\log \left(\frac{T}{\beta}\right)\right)$  is the error of the counter of Theorem 2.10.
The event  $E$  in Theorem 4.2 occurs when all the Laplace noises of the counter and ChallengeAT are within a factor of  $\log (T / \beta)$  of their expectation. The privacy guarantees of ChallengeAT are more involved. They are defined via a game with an adversary  $\mathcal{B}$  whose goal is to guess a secret bit  $b$ . At the beginning of the game, the adversary chooses two neighboring datasets  $S_0, S_1$ , and ChallengeAT is instantiated with  $S_b$ . Then throughout the game the adversary specifies queries  $f_i$  and observes the output of ChallengeAT on these queries. At some special round  $i^*$ , chosen by the adversary, the adversary specifies two queries  $f_{i^*}^0, f_{i^*}^1$ , where only  $f_{i^*}^b$  is fed into ChallengeAT. In round  $i^*$  the adversary does not get to see the answer of ChallengeAT on  $f_{i^*}^b$  (otherwise it could easily learn the bit  $b$  since  $f_{i^*}^0, f_{i^*}^1$  may be very different). We show that any such adversary  $\mathcal{B}$  has only a small advantage in guessing the bit  $b$ . The formal details are given in Appendix C.
5 Online Classification under Challenge Differential Privacy
We are now ready to present our private online prediction algorithm. Consider algorithm P0P.
Theorem 5.1. When executed with a learner  $\mathcal{A}$  that makes at most  $d$  mistakes and with parameters  $k = \tilde{O}\left(\frac{d}{\varepsilon^2}\log^2\left(\frac{1}{\delta}\right)\log^2\left(\frac{T}{\beta}\right)\right)$  and  $r = O\left(dk + \ln \left(\frac{1}{\beta}\right)\right)$ , then with probability at least  $(1 - \beta)$  the number of mistakes made by algorithm  $POP$  is bounded by  $\tilde{O}\left(\frac{d^2}{\varepsilon^2}\log^2\left(\frac{1}{\delta}\right)\log^2\left(\frac{T}{\beta}\right)\right)$ .
Algorithm 7 POP (Private Online Procedure)
Setting:  $T \in \mathbb{N}$  denotes the number of rounds in the game.  $\mathcal{A}$  is a non-private online-algorithm.
Parameters:  $k$  determines the number of copies of  $\mathcal{A}$  we maintain.  $r$  determines the number of positive reports we aim to receive from ChallengeAT.
1. Instantiate  $k$  copies  $\mathcal{A}_1, \ldots, \mathcal{A}_k$  of algorithm  $\mathcal{A}$
2. Instantiate algorithm ChallengeAT on an empty dataset with threshold  $t = -k / 4$ , privacy parameters  $\varepsilon, \delta$ , number of positive reports  $r$ , and sensitivity parameter  $\Delta = 1$ .
3. For  $i = 1,2,\ldots ,T$
(a) Obtain input  $x_{i}$
(b) Let  $\mathcal{A}_1^{\mathrm{temp}},\ldots ,\mathcal{A}_k^{\mathrm{temp}}$  be duplicated copies of  $\mathcal{A}_1,\dots ,\mathcal{A}_k$
(c) Let  $\ell_i\in [k]$  be chosen uniformly at random
(d) Let  $\hat{y}_{i,\ell_i} \gets \mathcal{A}_{\ell_i}(x_i)$ . For  $j \in [k] \setminus \{\ell_i\}$  let  $\hat{y}_{i,j} \gets \mathcal{A}_j^{\mathrm{temp}}(x_i)$
(e) Feed ChallengeAT the query  $f_{i} \equiv -\left|\frac{k}{2} - \sum_{j \in [k]} \hat{y}_{i,j}\right|$  and obtain an outcome  $\sigma_{i}$ . (If ChallengeAT halts then POP also halts.)
$\%$  Recall that  $\sigma_{i} = 1$  indicates that  $-\left|\frac{k}{2} - \sum_{j \in [k]} \hat{y}_{i,j}\right| \gtrsim -\frac{k}{4}$ , meaning that there is "a lot" of disagreement among  $\hat{y}_{i,1}, \ldots, \hat{y}_{i,k}$ .
(f) If  $\sigma_{i} = 1$  then sample  $\hat{y}_i\in \{0,1\}$  at random. Else let  $\hat{y}_i = \mathrm{majority}\{\hat{y}_{i,1},\dots ,\hat{y}_{i,k}\}$
(g) Output the bit  $\hat{y}_i$  as a prediction, and obtain a "true" label  $y_i$
(h) Feed  $y_{i}$  to  $\mathcal{A}_{\ell_i}$
$\%$  Note that  $\mathcal{A}_{\ell}$  is the only copy of  $\mathcal{A}$  that changes its state during this iteration
Informally, the privacy guarantees of POP follow from those of ChallengeAT. We give the formal details in Appendix D, obtaining the following theorem.
Theorem 5.2. Algorithm  $POP$  is  $(O(\varepsilon), O(\delta))$ -Challenge-DP. That is, For every adversary  $\mathcal{B}$  it holds that OnlineGame $POP, \mathcal{B}$  is  $(O(\varepsilon), O(\delta))$ -DP w.r.t. the bit  $b$  (the input of the game).
We proceed with the utility guarantees of POP. See Appendix F for an extension to the agnostic setting.
Theorem 5.3. When executed with a learner  $\mathcal{A}$  that makes at most  $d$  mistakes and with parameters  $k = \tilde{O}\left(\frac{d}{\varepsilon^2}\log^2\left(\frac{1}{\delta}\right)\log^2\left(\frac{T}{\beta}\right)\right)$  and  $r = O\left(dk + \ln \left(\frac{1}{\beta}\right)\right)$ , then with probability at least  $(1 - \beta)$  the number of mistakes made by algorithm POP is bounded by  $\tilde{O}\left(\frac{d^2}{\varepsilon^2}\log^2\left(\frac{1}{\delta}\right)\log^2\left(\frac{T}{\beta}\right)\right)$ .
Proof. By Theorem 4.2, with probability  $(1 - \beta)$  over the internal coins of ChallengeAT, for every input sequence, its answers are accurate up to error of
$$
\mathrm {e r r o r} _ {\mathrm {C A T}} = O \left(\frac {\Delta}{\varepsilon} \sqrt {r + \lambda} \ln (\frac {r + \lambda}{\delta}) \log (\frac {T}{\beta})\right),
$$
where in our case, the sensitivity  $\Delta$  is 1, and the error of the counter  $\lambda$  is at most  $O\left(\frac{1}{\varepsilon}\log (T)\log \left(\frac{T}{\delta}\right)\right)$  by Theorem 2.10. We continue with the proof assuming that this event occurs. Furthermore, we set  $k = \Omega$  (errorCAT), large enough, such that if less than  $\frac{1}{5}$  the experts disagree with the other experts, then algorithm POP returns the majority vote with probability 1.
Consider the execution of algorithm POP and define 1/5-Err be a random variable that counts the number of time steps in which at least 1/5th of the experts make an error. That is
$$
1 / 5 - \operatorname {E r r} = \left| \left\{i \in [ T ]: \sum_ {j \in [ k ]} \mathbb {1} \left\{\hat {y} _ {i, j} \neq y _ {i} \right\} > k / 5 \right\} \right|.
$$
We also define the random variable
$$
\operatorname {e x p e r t A d v a n c e} = | \left\{i \in [ T ]: y _ {i} \neq \hat {y} _ {i, \ell_ {i}} \right\} |.
$$
That is expertAdvance counts the number of times steps in which the random expert we choose (the  $\ell_i$ th expert) errs. Note that the  $\ell_i$ th expert is the expert that gets the "true" label  $y_i$  as feedback. As we run  $k$  experts, and as each of them is guaranteed to make at most  $d$  mistakes, we get that
$$
\begin{array}{l} \text {e x p e r t A d v a n c e} \leq k d. \end{array}
$$
We now show that with high probability 1/5-Err is not much larger than expertAdvance. Let  $i$  be a time step in which at least  $1/5$  fraction of the experts err. As the choice of  $\ell_i$  (the expert we update) is random, then with probability at least  $\frac{1}{5}$  the chosen expert also errs. It is therefore unlikely that 1/5-Err is much larger than expertAdvance, which is bounded by  $kd$ . Specifically, by standard concentration arguments (see Appendix E for the precise version we use) it holds that
$$
\Pr \left[ 1 / 5 - \operatorname {E r r} > 1 8 d k + 1 8 + \ln \left(\frac {1}{\beta}\right) \right] \leq \beta .
$$
Note that when at least  $1/5$  of the experts disagree with other experts then at least  $1/5$  of the experts err. It follows that 1/5-Err upper bounds the number of times in which algorithm ChallengeAT returns an "above threshold" answer. Hence, by setting  $r > 18dk + 18 + \ln\left(\frac{1}{\beta}\right)$  we ensure that w.h.p. algorithm ChallengeAT does not halt prematurely (and hence POP does not either).
Furthermore our algorithm errs either when there is a large disagreement between the experts or when all experts err. It follows that 1/5-Err also upper bounds the number of times which our algorithm errs.
Overall, by setting  $r = O\left(dk + \ln \left(\frac{1}{\beta}\right)\right)$  we ensure that POP does not halt prematurely, and by setting  $k = O\left(\frac{\Delta}{\varepsilon}\sqrt{r + \lambda}\ln \left(\frac{r + \lambda}{\delta}\right)\log \left(\frac{T}{\beta}\right)\right)$  we ensure that POP does not err too many times throughout the execution. Combining the requirement on  $r$  and on  $k$ , it suffices to take
$$
k = \tilde {O} \left(\frac {d}{\varepsilon^ {2}} \log^ {2} (\frac {1}{\delta}) \log^ {2} (\frac {T}{\beta}) + \frac {1}{\varepsilon \cdot d} \log (T) \log \left(\frac {T}{\delta}\right)\right),
$$
in which case algorithm P0P makes at most  $\tilde{O}\left(\frac{d^2}{\varepsilon^2}\log^2\left(\frac{1}{\delta}\right)\log^2\left(\frac{T}{\beta}\right)\right)$  with high probability.
Our work presents a new privacy model for online classification, together with an efficiency preserving transformation from non-private online classification, that exhibits a doubly exponential improvement in the error compared to prior works on this topic. We leave open the possibility that such an improvement could also be achieved in the setting of Golowich and Livni [2021], i.e., under the more restrictive notion of privacy where the sequence of predictors does not compromise privacy.
Haim Kaplan was artially supported by Israel Science Foundation (grant 1595/19), and the Blavatnik Family Foundation.
Yishay Mansour was partially funded from the European Research Council (ERC) under the European Union's Horizon 2020 research and innovation program (grant agreement No. 882396), by the Israel Science Foundation (grant number 993/17), Tel Aviv University Center for AI and Data Science (TAD), and the Yandex Initiative for Machine Learning at Tel Aviv University.
Shay Moran is a Robert J. Shillman Fellow; he acknowledges support by ISF grant 1225/20, by BSF grant 2018385, by an Azrieli Faculty Fellowship, by Israel PBC-VATAT, by the Technion Center for Machine Learning and Intelligent Systems (MLIS), and by the European Union (ERC, GENERALIZATION, 101039692). Views and opinions expressed are however those of the author(s) only and do not necessarily reflect those of the European Union or the European Research Council Executive Agency. Neither the European Union nor the granting authority can be held responsible for them.
Kobbi Nissim was partially funded by NSF grant No. CNS 2001041 and by a gift to Georgetown University.
Uri Stemmer was partially supported by the Israel Science Foundation (grant 1871/19) and by Len Blavatnik and the Blavatnik Family foundation.