SlowGuess/NeurIPS_2023_OCR huggingface dataset:
- Setting of  $(x, \Delta)$  (observing  $x_{t}$  before choosing  $f_{t}$  and observing  $\Delta_{t}$  after): For online learning, we propose an variant of the Halving algorithm, called Strategic Halving (Algorithm 1), which can eliminate half of the remaining hypotheses when making a mistake. The algorithm depends on observing  $x_{t}$  before choosing the predictor  $f_{t}$ . Then by applying the standard technique of converting mistake bound to PAC bound, we are able to achieve sample complexity of  $\mathcal{O}(\frac{\log(|\mathcal{H}|) \log \log(|\mathcal{H}|)}{\varepsilon})$ .
- Setting of  $(\bot, (x, \Delta))$  (observing both  $x_{t}$  and  $\Delta_{t}$  after selecting  $f_{t}$ ): We prove that, there exists an example of  $(\mathcal{H}, \mathcal{Q})$  s.t. the mistake bound is lower bounded by  $\Omega(|\mathcal{H}|)$ . This implies that no algorithm can perform significantly better than sequentially trying each hypothesis, which would make at most  $|\mathcal{H}|$  mistakes before finding the correct hypothesis. However, unlike the construction of mistake lower bounds in classic online learning, where all mistakes can be forced to occur in the initial rounds, we demonstrate that we require  $\Theta(|\mathcal{H}|^{2})$  rounds to ensure that all mistakes occur. In
the PAC setting, we first show that, any learning algorithm with proper output  $f_{\mathrm{out}}$ , i.e.,  $f_{\mathrm{out}} \in \mathcal{H}$ , needs a sample size of  $\Omega(\frac{|\mathcal{H}|}{\varepsilon})$ . We can achieve a sample complexity of  $O\left(\frac{\log^2(|\mathcal{H}|)}{\varepsilon}\right)$  by executing Algorithm 2, which is a randomized algorithm with improper output.
- Setting of  $(\perp, \Delta)$  (observing only  $\Delta_t$  after selecting  $f_t$ ): The mistake bound of  $\Omega(|\mathcal{H}|)$  also holds in this setting, as it is known to be harder than the previous setting. For the PAC learning, we show that any conservative algorithm, which only depends on the information from the mistake rounds, requires  $\Omega(\frac{|\mathcal{H}|}{\varepsilon})$  samples. The optimal sample complexity is left as an open problem.
- Setting of  $(\bot, \bot)$  (observing neither  $x_{t}$  nor  $\Delta_{t}$ ): Similarly, the mistake bound of  $\Omega(|\mathcal{H}|)$  still holds. For the PAC learning, we show that the sample complexity is  $\Omega\left(\frac{|\mathcal{H}|}{\varepsilon}\right)$  by reducing the problem to a stochastic linear bandit problem.
Then we move on to non-ball manipulations. However, we show that even in the simplest setting of observing  $x_{t}$  before choosing  $f_{t}$  and observing  $\Delta_t$  after, there is an example of  $(\mathcal{H},\mathcal{Q})$  such that the sample complexity is  $\widetilde{\Omega} (\frac{|\mathcal{H}|}{\varepsilon})$ . This implies that in all four settings of different revealed information, we will have sample complexity of  $\widetilde{\Omega} (\frac{|\mathcal{H}|}{\varepsilon})$  and mistake bound of  $\widetilde{\Omega} (|\mathcal{H}|)$ . We summarize our results in Table 1.
setting	mistake bound	sample complexity
ball	(x,Δ)	Θ(log(|H|)) (Thm 1)	O(log(|H|)/ε)a(Thm 2), Ω(log(|H|)/ε)
(⊥,(x,Δ))	O(min(√log(|H|)T, |H|)) (Thm 4)	O(log2(|H|)/ε) (Thm 6), Ω(log(|H|)/ε)
Ω(min(T/|H|log(|H|), |H|)) (Thm 3)	SCprop = Ω(|H|)/ε) (Thm 5)
(⊥,Δ)	Θ(|H|) (implied by Thm 3)	SCcsv = Ω(|H|)/ε) (Thm 7)
(⊥,⊥)	Θ(|H|) (implied by Thm 3)	O(|H|)/ε), Ω(|H|)/ε) (Thm 8)
nonball	all	Ω(|H|)(Cor 1), O(|H|)	O(|H|)/ε), Ω(|H|)/ε) (Cor 1)
A factor of  $\log \log (|\mathcal{H}|)$  is neglected.
Table 1: The summary of results.  $\widetilde{\mathcal{O}}$  and  $\widetilde{\Omega}$  ignore logarithmic factors on  $|\mathcal{H}|$  and  $\frac{1}{\varepsilon}$ . The superscripts prop stands for proper learning algorithms and csv stands for conservative learning algorithms. All lower bounds in the non-strategic setting also apply to the strategic setting, implying that  $\mathrm{MB}_{C,F} \geq \Omega (\log (|\mathcal{H}|))$  and  $\mathrm{SC}_{C,F} \geq \Omega (\frac{\log(|\mathcal{H}|)}{\varepsilon})$  for all settings of  $(C,F)$ . In all four settings, a mistake bound of  $\mathcal{O}(|\mathcal{H}|)$  can be achieved by simply trying each hypothesis in  $\mathcal{H}$  while the sample complexity can be achieved as  $\widetilde{\mathcal{O}} (\frac{|\mathcal{H}|}{\varepsilon})$  by converting the mistake bound of  $\mathcal{O}(|\mathcal{H}|)$  to a PAC bound using standard techniques.
4 Ball manipulations
In ball manipulations, when  $\mathcal{B}(x;r)\cap \mathcal{X}_{h, + }$  has multiple elements, the agent will always break ties by selecting the one closest to  $x$ , i.e.,  $\Delta (x,h,r) = \arg \min_{x^{\prime}\in \mathcal{B}(x;r)\cap \mathcal{X}_{h, + }}d(x,x^{\prime})$ . In round  $t$ , the learner deploys predictor  $f_{t}$ , and once he knows  $x_{t}$  and  $\widehat{y}_t$ , he can calculate  $\Delta_t$  himself without needing knowledge of  $r_t$  by
$$
\Delta_ {t} = \left\{ \begin{array}{l l} \arg \min  _ {x ^ {\prime} \in \mathcal {X} _ {f _ {t}, +}} d (x _ {t}, x ^ {\prime}) & \text {i f} \widehat {y} _ {t} = + 1  , \\ x _ {t} & \text {i f} \widehat {y} _ {t} = - 1  . \end{array} \right.
$$
Thus, for ball manipulations, knowing  $x_{t}$  is equivalent to knowing both  $x_{t}$  and  $\Delta_t$ .
4.1 Setting  $(x,\Delta)$ : Observing  $x_{t}$  Before Choosing  $f_{t}$
Online learning We propose a new algorithm with mistake bound of  $\log (|\mathcal{H}|)$  in setting  $(x,\Delta)$ . To achieve a logarithmic mistake bound, we must construct a predictor  $f_{t}$  such that if it makes a mistake, we can reduce a constant fraction of the remaining hypotheses. The primary challenge is that we do not have access to the full information, and predictions of other hypotheses are hidden. To extract the information of predictions of other hypotheses, we take advantage of ball manipulations, which
induces an ordering over all hypotheses. Specifically, for any hypothesis  $h$  and feature vector  $x$ , we define the distance between  $x$  and  $h$  by the distance between  $x$  and the positive region by  $h$ ,  $\mathcal{X}_{h, + }$ , i.e.,
$$
d (x, h) := \min  \left\{d \left(x, x ^ {\prime}\right) \mid x ^ {\prime} \in \mathcal {X} _ {h, +} \right\}. \tag {2}
$$
At each round  $t$ , given  $x_{t}$ , the learner calculates the distance  $d(x_{t}, h)$  for all  $h$  in the version space (meaning hypotheses consistent with history) and selects a hypothesis  $f_{t}$  such that  $d(x_{t}, f_{t})$  is the median among all distances  $d(x_{t}, h)$  for  $h$  in the version space. We can show that by selecting  $f_{t}$  in this way, the learner can eliminate half of the version space if  $f_{t}$  makes a mistake. We refer to this algorithm as Strategic Halving, and provide a detailed description of it in Algorithm 1.
Theorem 1. For any feature-ball manipulation set space  $\mathcal{Q}$  and hypothesis class  $\mathcal{H}$ , Strategic Halving achieves mistake bound  $\mathrm{MB}_{x,\Delta} \leq \log (|\mathcal{H}|)$ .
Algorithm 1 Strategic Halving
1: Initialize the version space  $\mathrm{VS} = \mathcal{H}$
2: for  $t = 1, \dots, T$  do
3: pick an  $f_{t} \in \mathrm{VS}$  such that  $d(x_{t},f_{t})$  is the median of  $\{d(x_t,h)|h \in \mathrm{VS}\}$ .
4: if  $\widehat{y}_t \neq y_t$  and  $y_t = +\text{then} \mathrm{VS} \gets \mathrm{VS} \setminus \{h \in \mathrm{VS} | d(x_t, h) \geq d(x_t, f_t)\}$ ;
5: else if  $\widehat{y}_t \neq y_t$  and  $y_t = -\text{then VS} \gets \text{VS} \setminus \{h \in \text{VS} | d(x_t, h) \leq d(x_t, f_t)\}$ .
6: end for
To prove Theorem 1, we only need to show that each mistake reduces the version space by half. Supposing that  $f_{t}$  misclassifies a true positive example  $(x_{t}, r_{t}, +1)$  by negative, then we know that  $d(x_{t}, f_{t}) > r_{t}$  while the target hypothesis  $h^{*}$  must satisfy that  $d(x_{t}, h^{*}) \leq r_{t}$ . Hence any  $h$  with  $d(x_{t}, h) \geq d(x_{t}, f_{t})$  cannot be  $h^{*}$  and should be eliminated. Since  $d(x_{t}, f_{t})$  is the median of  $\{d(x_{t}, h) | h \in \mathrm{VS}\}$ , we can eliminate half of the version space. It is similar when  $f_{t}$  misclassifies a true negative. The detailed proof is deferred to Appendix B.
PAC learning We can convert Strategic Halving to a PAC learner by the standard technique of converting a mistake bound to a PAC bound (Gallant, 1986). Specifically, the learner runs Strategic Halving until it produces a hypothesis  $f_{t}$  that survives for  $\frac{1}{\varepsilon}\log \left(\frac{\log(|\mathcal{H}|)}{\delta}\right)$  rounds and outputs this  $f_{t}$ . Then we have Theorem 2, and the proof is included in Appendix C.
Theorem 2. For any feature-ball manipulation set space  $\mathcal{Q}$  and hypothesis class  $\mathcal{H}$ , we can achieve  $\mathrm{SC}_{x,\Delta}(\varepsilon, \delta) = \mathcal{O}\left(\frac{\log(|\mathcal{H}|)}{\varepsilon} \log\left(\frac{\log(|\mathcal{H}|)}{\delta}\right)\right)$  by combining Strategic Halving and the standard technique of converting a mistake bound to a PAC bound.
4.2 Setting  $(\bot ,(x,\Delta))$  : Observing  $x_{t}$  After Choosing  $f_{t}$
When  $x_{t}$  is not revealed before the learner choosing  $f_{t}$ , the algorithm of Strategic Halving does not work anymore. We demonstrate that it is impossible to reduce constant fraction of version space when making a mistake, and prove that the mistake bound is lower bounded by  $\Omega(|\mathcal{H}|)$  by constructing a negative example of  $(\mathcal{H}, \mathcal{Q})$ . However, we can still achieve sample complexity with poly-logarithmic dependency on  $|\mathcal{H}|$  in the distributional setting.
4.2.1 Results in the Online Learning Model
To offer readers an intuitive understanding of the distinctions between the strategic setting and standard online learning, we commence by presenting an example in which no deterministic learners, including the Halving algorithm, can make fewer than  $|\mathcal{H}| - 1$  mistakes.
Example 1. Consider a star shape metric space  $(\mathcal{X},d)$ , where  $\mathcal{X} = \{0,1,\dots,n\}$ ,  $d(i,j) = 2$  and  $d(0,i) = 1$  for all  $i,j\in [n]$  with  $i\neq j$ . The hypothesis class is composed of singletons over  $[n]$ , i.e.,  $\mathcal{H} = \{2\mathbb{1}_{\{i\}} - 1|i\in [n]\}$ . When the learner is deterministic, the environment can pick an agent  $(x_{t},r_{t},y_{t})$  dependent on  $f_{t}$ . If  $f_{t}$  is all-negative, then the environment picks  $(x_{t},r_{t},y_{t}) = (0,1,+1)$ , and then the learner makes a mistake but no hypothesis can be eliminated. If  $f_{t}$  predicts 0 by positive, the environment will pick  $(x_{t},r_{t},y_{t}) = (0,0,-1)$ , and then the learner makes a mistake but no hypothesis can be eliminated. If  $f_{t}$  predicts some  $i\in [n]$  by positive, the environment will pick  $(x_{t},r_{t},y_{t}) = (i,0,-1)$ , and then the learner makes a mistake with only one hypothesis  $2\mathbb{1}_{\{i\}} - 1$  eliminated. Therefore, the learner will make  $n - 1$  mistakes.
In this work, we allow the learner to be randomized. When an  $(x_{t},r_{t},y_{t})$  is generated by the environment, the learner can randomly pick an  $f_{t}$ , and the environment does not know the realization of  $f_{t}$  but knows the distribution where  $f_{t}$  comes from. It turns out that randomization does not help much. We prove that there exists an example in which any (possibly randomized) learner will incur  $\Omega(|\mathcal{H}|)$  mistakes.
Theorem 3. There exists a feature-ball manipulation set space  $\mathcal{Q}$  and hypothesis class  $\mathcal{H}$  s.t. the mistake bound  $\mathrm{MB}_{\perp,(x,\Delta)} \geq |\mathcal{H}| - 1$ . For any (randomized) algorithm  $\mathcal{A}$  and any  $T \in \mathbb{N}$ , there exists a realizable sequence of  $(x_t, r_t, y_t)_{1:T}$  such that with probability at least  $1 - \delta$  (over randomness of  $\mathcal{A}$ ),  $\mathcal{A}$  makes at least  $\min \left(\frac{T}{5|\mathcal{H}|\log(|\mathcal{H}| / \delta)}, |\mathcal{H}| - 1\right)$  mistakes.
Essentially, we design an adversarial environment such that the learner has a probability of  $\frac{1}{|\mathcal{H}|}$  of making a mistake at each round before identifying the target function  $h^*$ . The learner only gains information about the target function when a mistake is made. The detailed proof is deferred to Appendix D. Theorem 3 establishes a lower bound on the mistake bound, which is  $|\mathcal{H}| - 1$ . However, achieving this bound requires a sufficiently large number of rounds, specifically  $T = \widetilde{\Omega}(|\mathcal{H}|^2)$ . This raises the question of whether there exists a learning algorithm that can make  $o(T)$  mistakes for any  $T \leq |\mathcal{H}|^2$ . In Example 1, we observed that the adversary can force any deterministic learner to make  $|\mathcal{H}| - 1$  mistakes in  $|\mathcal{H}| - 1$  rounds. Consequently, no deterministic algorithm can achieve  $o(T)$  mistakes.
To address this, we propose a randomized algorithm that closely resembles Algorithm 1, with a modification in the selection of  $f_{t}$ . Instead of using line 3, we choose  $f_{t}$  randomly from VS since we lack prior knowledge of  $x_{t}$ . This algorithm can be viewed as a variation of the well-known multiplicative weights method, applied exclusively during mistake rounds. For improved clarity, we present this algorithm as Algorithm 3 in Appendix E due to space limitations.
Theorem 4. For any  $T \in \mathbb{N}$ , Algorithm 3 will make at most  $\min(\sqrt{4\log(|\mathcal{H}|)T}, |\mathcal{H}| - 1)$  mistakes in expectation in  $T$  rounds.
Note that the  $T$ -dependent upper bound in Theorem 4 matches the lower bound in Theorem 3 up to a logarithmic factor when  $T = |\mathcal{H}|^2$ . This implies that approximately  $|\mathcal{H}|^2$  rounds are needed to achieve  $|\mathcal{H}| - 1$  mistakes, which is a tight bound up to a logarithmic factor. Proof of Theorem 4 is included in Appendix E.
4.2.2 Results in the PAC Learning Model
In the PAC setting, the goal of the learner is to output a predictor  $f_{\mathrm{out}}$  after the repeated interactions. A common class of learning algorithms, which outputs a hypothesis  $f_{\mathrm{out}} \in \mathcal{H}$ , is called proper. Proper learning algorithms are a common starting point when designing algorithms for new learning problems due to their natural appeal and ability to achieve good performance, such as ERM in classic PAC learning. However, in the current setting, we show that proper learning algorithms do not work well and require a sample size linear in  $|\mathcal{H}|$ . The formal theorem is stated as follows and the proof is deferred to Appendix F.
Theorem 5. There exists a feature-ball manipulation set space  $\mathcal{Q}$  and hypothesis class  $\mathcal{H}$  s.t.  $\mathrm{SC}_{\perp, \Delta}^{prop}(\varepsilon, \frac{7}{8}) = \Omega(\frac{|\mathcal{H}|}{\varepsilon})$ , where  $\mathrm{SC}_{\perp, \Delta}^{prop}(\varepsilon, \delta)$  is the  $(\varepsilon, \delta)$ -PAC sample complexity achievable by proper algorithms.
Theorem 5 implies that any algorithm capable of achieving sample complexity sub-linear in  $|\mathcal{H}|$  must be improper. As a result, we are inspired to devise an improper learning algorithm. Before presenting the algorithm, we introduce some notations. For two hypotheses  $h_1, h_2$ , let  $h_1 \lor h_2$  denote the union of them, i.e.,  $(h_1 \lor h_2)(x) = +1$  iff.  $h_1(x) = +1$  or  $h_2(x) = +1$ . Similarly, we can define the union of more than two hypotheses. Then for any union of  $k$  hypotheses,  $f = \vee_{i=1}^{k} h_i$ , the positive region of  $f$  is the union of positive regions of the  $k$  hypotheses and thus, we have  $d(x, f) = \min_{i \in [k]} d(x, h_i)$ . Therefore, we can decrease the distance between  $f$  and any feature vector  $x$  by increasing  $k$ . Based on this, we devise a new randomized algorithm with improper output, described in Algorithm 2.
Theorem 6. For any feature-ball manipulation set space  $\mathcal{Q}$  and hypothesis class  $\mathcal{H}$ , we can achieve  $\mathrm{SC}_{\perp,(x,\Delta)}(\varepsilon, \delta) = \mathcal{O}(\frac{\log^2(|\mathcal{H}|) + \log(1/\delta)}{\varepsilon} \log(\frac{1}{\delta}))$  by combining Algorithm 2 with a standard confidence boosting technique. Note that the algorithm is improper.
Algorithm 2
1: Initialize the version space  $\mathrm{VS}_0 = \mathcal{H}$
2: for  $t = 1, \dots, T$  do
3: randomly pick  $k_{t} \sim \mathrm{Unif}(\{1,2,2^{2},\dots,2^{\lfloor \log_{2}(n_{t}) - 1\rfloor}\})$  where  $n_t = |\mathrm{VS}_{t - 1}|$ ;
4: sample  $k_{t}$  hypotheses  $h_{1},\ldots ,h_{k_{t}}$  independently and uniformly at random from  $\mathrm{VS}_{t - 1}$ ;
5: let  $f_{t} = \vee_{i=1}^{k_{t}} h_{i}$ .
6: if  $\widehat{y_t} \neq y_t$  and  $y_t = +\text{then} \mathrm{VS}_t = \mathrm{VS}_{t-1} \setminus \{h \in \mathrm{VS}_{t-1} | d(x_t, h) \geq d(x_t, f_t)\}$ ;
7: else if  $\widehat{y}_t \neq y_t$  and  $y_t = -\text{then} \, \mathrm{VS}_t = \mathrm{VS}_{t-1} \setminus \{h \in \mathrm{VS}_{t-1} | d(x_t, h) \leq d(x_t, f_t)\}$ ;
8: else  $\mathrm{VS}_t = \mathrm{VS}_{t - 1}$
9: end for
10: randomly pick  $\tau$  from  $[T]$  and randomly sample  $h_1, h_2$  from  $\mathrm{VS}_{\tau - 1}$  with replacement.
11: output  $h_1 \lor h_2$
Now we outline the high-level ideas behind Algorithm 2. In correct rounds where  $f_{t}$  makes no mistake, the predictions of all hypotheses are either correct or unknown, and thus, it is hard to determine how to make updates. In mistake rounds, we can always update the version space similar to what was done in Strategic Halving. To achieve a poly-logarithmic dependency on  $|\mathcal{H}|$ , we aim to reduce a significant number of misclassifying hypotheses in mistake rounds. The maximum number we can hope to reduce is a constant fraction of the misclassifying hypotheses. We achieve this by randomly sampling a  $f_{t}$  (lines 3-5) s.t.  $f_{t}$  makes a mistake, and  $d(x_{t},f_{t})$  is greater (smaller) than the median of  $d(x_{t},h)$  for all misclassifying hypotheses  $h$  for true negative (positive) examples. However, due to the asymmetric nature of manipulation, which aims to be predicted as positive, the rate of decreasing misclassifications over true positives is slower than over true negatives. To compensate for this asymmetry, we output a  $f_{\mathrm{out}} = h_1 \lor h_2$  with two selected hypotheses  $h_1, h_2$  (lines 10-11) instead of a single one to increase the chance of positive prediction.
We prove that Algorithm 2 can achieve small strategic loss in expectation as described in Lemma 1. Then we can achieve the sample complexity in Theorem 6 by boosting Algorithm 2 to a strong learner. This is accomplished by running Algorithm 2 multiple times until we obtain a good predictor. The proofs of Lemma 1 and Theorem 6 are deferred to Appendix G.
Lemma 1. Let  $S = (x_{t}, r_{t}, y_{t})_{t=1}^{T} \sim \mathcal{D}^{T}$  denote the i.i.d. sampled agents in  $T$  rounds and let  $\mathcal{A}(S)$  denote the output of Algorithm 2 interacting with  $S$ . For any feature-ball manipulation set space  $\mathcal{Q}$  and hypothesis class  $\mathcal{H}$ , when  $T \geq \frac{320\log^2(|\mathcal{H}|)}{\varepsilon}$ , we have  $\mathbb{E}_{\mathcal{A},S}\left[\mathcal{L}^{str}(\mathcal{A}(S))\right] \leq \varepsilon$ .
4.3 Settings  $(\perp, \Delta)$  and  $(\perp, \perp)$
Online learning As mentioned in Section 2, both the settings of  $(\perp, \Delta)$  and  $(\perp, \perp)$  are harder than the setting of  $(\perp, (x, \Delta))$ , all lower bounds in the setting of  $(\perp, (x, \Delta))$  also hold in the former two settings. Therefore, by Theorem 3, we have  $\mathrm{MB}_{\perp, \perp} \geq \mathrm{MB}_{\perp, \Delta} \geq \mathrm{MB}_{\perp, (x, \Delta)} = |\mathcal{H}| - 1$ .
PAC learning In the setting of  $(\perp, \Delta)$ , Algorithm 2 is not applicable anymore since the learner lacks observation of  $x_{t}$ , making it impossible to replicate the version space update steps in lines 6-7. It is worth noting that both PAC learning algorithms we have discussed so far fall under a general category called conservative algorithms, depend only on information from the mistake rounds. Specifically, an algorithm is said to be conservative if for any  $t$ , the predictor  $f_{t}$  only depends on the history of mistake rounds up to  $t$ , i.e.,  $\tau < t$  with  $\widehat{y}_{\tau} \neq y_{\tau}$ , and the output  $f_{\mathrm{out}}$  only depends on the history of mistake rounds, i.e.,  $(f_{t}, \widehat{y}_{t}, y_{t}, \Delta_{t})_{t: \widehat{y}_{t} \neq y_{t}}$ . Any algorithm that goes beyond this category would need to utilize the information in correct rounds. As mentioned earlier, in correct rounds, the predictions of all hypotheses are either correct or unknown, which makes it challenging to determine how to make updates. For conservative algorithms, we present a lower bound on the sample complexity in the following theorem, which is  $\widetilde{\Omega} \left( \frac{|\mathcal{H}|}{\varepsilon} \right)$ , and its proof is included in Appendix H. The optimal sample complexity in the setting  $(\perp, \Delta)$  is left as an open problem.
Theorem 7. There exists a feature-ball manipulation set space  $\mathcal{Q}$  and hypothesis class  $\mathcal{H}$  s.t.  $\mathrm{SC}_{\perp,\Delta}^{csv}(\varepsilon,\frac{7}{8}) = \widetilde{\Omega}(\frac{|\mathcal{H}|}{\varepsilon})$ , where  $\mathrm{SC}_{\perp,\Delta}^{csv}(\varepsilon,\delta)$  is  $(\varepsilon,\delta)$ -PAC the sample complexity achievable by conservative algorithms.
In the setting of  $(\perp, \perp)$ , our problem reduces to a best arm identification problem in stochastic bandits. We prove a lower bound on the sample complexity of  $\widetilde{\Omega}\left(\frac{|\mathcal{H}|}{\varepsilon}\right)$  in Theorem 8 by reduction to stochastic linear bandits and applying the tools from information theory. The proof is deferred to Appendix I.
Theorem 8. There exists a feature-ball manipulation set space  $\mathcal{Q}$  and hypothesis class  $\mathcal{H}$  s.t.  $\mathrm{SC}_{\perp,\perp}(\varepsilon, \frac{7}{8}) = \widetilde{\Omega}(\frac{|\mathcal{H}|}{\varepsilon})$ .
5 Non-ball Manipulations
In this section, we move on to non-ball manipulations. In ball manipulations, for any feature vector  $x$ , we have an ordering of hypotheses according to their distances to  $x$ , which helps to infer the predictions of some hypotheses without implementing them. However, in non-ball manipulations, we don't have such structure anymore. Therefore, even in the simplest setting of observing  $x_{t}$  before  $f_{t}$  and  $\Delta_t$ , we have the PAC sample complexity lower bounded by  $\widetilde{\Omega}\left(\frac{|\mathcal{H}|}{\varepsilon}\right)$ .
Theorem 9. There exists a feature-manipulation set space  $\mathcal{Q}$  and hypothesis class  $\mathcal{H}$  s.t.  $\mathrm{SC}_{x,\Delta}(\varepsilon, \frac{7}{8}) = \widetilde{\Omega}(\frac{|\mathcal{H}|}{\varepsilon})$ .
The proof is deferred to Appendix J. It is worth noting that in the construction of the proof, we let all agents to have their original feature vector  $x_{t} = \mathbf{0}$  such that  $x_{t}$  does not provide any information. Since  $(x,\Delta)$  is the simplest setting and any mistake bound can be converted to a PAC bound via standard techniques (see Section A.2 for more details), we have the following corollary.
Corollary 1. There exists a feature-manipulation set space  $\mathcal{Q}$  and hypothesis class  $\mathcal{H}$  s.t. for all choices of  $(C,F)$ ,  $\mathrm{SC}_{C,F}(\varepsilon, \frac{7}{8}) = \widetilde{\Omega}(\frac{|\mathcal{H}|}{\varepsilon})$  and  $\mathrm{MB}_{C,F} = \widetilde{\Omega}(|\mathcal{H}|)$ .