SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$$
\begin{array}{l} \mathbb {E} \left[ \mathcal {R} (M, T) \right] \leq 2 0 \sqrt {2} \left(d \sqrt {M T} + 2 \min  \{\varepsilon d M T, d M \sqrt {T} \}\right) \xi_ {T} ^ {2} \\ \leq 3 2 0 \sqrt {2} \left(d \sqrt {M T} + 2 \min  \{\varepsilon d M T, d M \sqrt {T} \}\right) \log^ {2} (M T) \\ \end{array}
$$
B Proof of Lower Bound
B.1 Proof of Theorem 4.3
Before proving the theorem, we give a formal definition of  $\varepsilon$ -MALCB problem for  $p$ -norm,
Definition B.1. Given an instance of multi-agent linear bandits, it belongs to the class of  $p$ -norm  $\varepsilon$ -MALCB, which we denote as  $\mathcal{I}_p(\varepsilon)$ , if for all  $i,j\in [M]$ ,  $\| \theta_i - \theta_j\| _p\leq \varepsilon$ .
To prove Theorem 4.3, we use the results of the two following lemmas on max-norm  $\varepsilon$ -MALCB instances.
Lemma B.1. Let  $\mathcal{I}_{\infty}(\varepsilon)$  be the class of max-norm  $\varepsilon$ -MALCB instances that satisfy the Assumption 3.1. For  $\varepsilon \geq 0$ , we have the following,
$$
\inf  _ {\mathcal {A}} \sup  _ {\mathcal {I} \in \mathcal {I} _ {\infty} (\varepsilon)} \mathcal {R} _ {\mathcal {A}, \mathcal {I}} (M, T) = \Omega (d \sqrt {M T}).
$$
Lemma B.2. Let  $\mathcal{I}_{\infty}(\varepsilon)$  be the class of max-norm  $\varepsilon$ -MALCB instances that satisfy the Assumption 3.1. Assume  $d \leq 2T$ ,  $\frac{d^2}{48} \leq T$ . For  $\varepsilon \geq 0$ , we have the following,
$$
\inf  _ {\mathcal {A}} \sup  _ {\mathcal {I} \in \mathcal {I} _ {\infty} (\varepsilon)} \mathcal {R} _ {\mathcal {A}, \mathcal {I}} (M, T) = \Omega \left(\min  \left\{\varepsilon M T \sqrt {d}, d M \sqrt {T} \right\}\right).
$$
Proof of Theorem 4.3. Combining the results of Lemma B.1 and Lemma B.2, we have the following results for the class of max-norm  $\varepsilon$ -MALCB, for  $\varepsilon \geq 0$
$$
\inf  _ {\mathcal {A}} \sup  _ {\mathcal {I} \in \mathcal {I} _ {\infty} (\varepsilon)} \mathcal {R} _ {\mathcal {A}, \mathcal {I}} (M, T) = \Omega \left(d \sqrt {M T} + \min  \left\{\varepsilon M T \sqrt {d}, d M \sqrt {T} \right\}\right).
$$
In addition,  $\| x \|_{\infty} \leq \varepsilon$  implies  $\| x \|_2 \leq \varepsilon \sqrt{d}$ , therefore,  $\mathcal{I}_{\infty}(\varepsilon) \subseteq \mathcal{I}_2(\varepsilon \sqrt{d})$ . In other words, for  $\varepsilon \geq 0$ ,
$$
\inf_{\mathcal{A}}\sup_{\mathcal{I}\in \mathcal{I}_{2}(\varepsilon)}\mathcal{R}_{\mathcal{A},\mathcal{I}}(M,T)\geq \inf_{\mathcal{A}}\sup_{\mathcal{I}\in \mathcal{I}_{\infty}\left(\frac{\varepsilon}{\sqrt{d}}\right)}\mathcal{R}_{\mathcal{A},\mathcal{I}}(M,T) = \Omega \left(d\sqrt{MT} +\min \left\{\varepsilon MT,dM\sqrt{T}\right\}\right).
$$
Proof of Lemma B.1. In this lemma, we prove that solving any  $\varepsilon$ -MALCB instance for  $T$  rounds is at least as hard as solving a (single-agent) linear bandits for  $MT$  rounds. We prove the lemma by contradiction, which is based on linear bandits lower bound of Lemma C.1. We have that  $\mathcal{I}_{\infty}(\varepsilon') \subseteq \mathcal{I}_{\infty}(\varepsilon)$ , for  $0 \leq \varepsilon' \leq \varepsilon$ . This implies, for  $\varepsilon \geq 0$ ,
$$
\inf  _ {\mathcal {A}} \sup  _ {\mathcal {I} \in \mathcal {I} _ {\infty} (\varepsilon)} \mathcal {R} _ {\mathcal {A}, \mathcal {I}} (M, T) \geq \inf  _ {\mathcal {A}} \sup  _ {\mathcal {I} \in \mathcal {I} _ {\infty} (0)} \mathcal {R} _ {\mathcal {A}, \mathcal {I}} (M, T).
$$
We completes the proof by proving  $\inf_{\mathcal{A}}\sup_{\mathcal{I}\in \mathcal{I}_{\infty}(0)}\mathcal{R}_{\mathcal{A},\mathcal{I}}(M,T) = \Omega \left(d\sqrt{MT}\right)$ .
Now, we assume there exists an algorithm  $\mathcal{A}$  which achieves  $\sup_{\mathcal{I} \in \mathcal{I}_{\infty}(0)} \mathcal{R}_{\mathcal{A}, \mathcal{I}}(M, T) < \frac{d\sqrt{MT}}{16\sqrt{3}}$ .
We observe that  $\mathcal{I}_{\infty}(0)$  is the class of multi-agent solving exactly the same linear bandits problem. We simulate the algorithm  $\mathcal{B}$  on a single agent linear bandits for  $MT$  rounds by the protocol of  $M$  agents solving an identical linear bandits for  $T$  rounds. Therefore, if we have  $\mathcal{R}_{\mathcal{A},\mathcal{I}}(M,T) < \frac{d\sqrt{MT}}{16\sqrt{3}}$  then we also achieve  $\mathcal{R}_{\mathcal{B},\mathcal{I}}(MT) < \frac{d\sqrt{MT}}{16\sqrt{3}}$ , which contradicts Lemma C.1. Thus, we have  $\sup_{\mathcal{I} \in \mathcal{I}_{\infty}(0)} \mathcal{R}_{\mathcal{A},\mathcal{I}}(M,T) \geq \frac{d\sqrt{MT}}{16\sqrt{3}}$ , which completes the proof.
Proof of Lemma B.2. We extend Lemma C.1 to multi-agent linear contextual bandit by constructing the following max-norm  $\varepsilon$ -MALCB instance.
Max-norm  $\varepsilon$ -MALCB instance. The set of linear parameter of  $M$  agents  $\{\theta_m\}_{m=1}^M$  belongs to a  $d$ -dimensional hypercube  $\theta_m \in \{\pm \varepsilon\}^d$ , where  $\varepsilon \in [0, \frac{1}{\sqrt{d}}]$ . Let  $\mathcal{D} = \{x \in \mathbb{R}^d : \|x\|_2 \leq 1\}$  be the action set given to agents at every round. The reward when agent  $m$  picks action  $x$  is defined as  $r_{m,x} = \theta_m^\top x + \eta_{m,x}$ , where the noise samples from a standard normal distribution,  $\eta_{m,x} \sim \mathcal{N}(0,1)$ .
We first verify if the instance satisfies the Assumption 3.1. It satisfies the context assumption since we also restrict the context vector to lie in the unit ball. It also satisfies the  $\| \theta_{m}\| \leq 1$  assumption because we restrict  $\varepsilon \in [0,\frac{1}{\sqrt{d}} ]$ . Finally,  $\eta_{m,x}\sim \mathcal{N}(0,1)$  is 1-subGaussian distribution. This instance belongs to  $\mathcal{I}_{\infty}(2\varepsilon)$  since  $\| \theta_{i} - \theta_{j}\|_{\infty}\leq 2\varepsilon$  for all  $i,j\in [M]$ .
Now, we proceed to prove Lemma B.2. Let  $\{\theta_m\}_{m=1}^M$  be a set of parameters of the max-norm  $\varepsilon$ -MALCB instance. For brevity, we omit the commas in the subscripts when it is clear from the context, e.g.,  $\theta_{mi} = \theta_{m,i}$  or  $x_{tmi} = x_{t,m,i}, \forall t \in [T], m \in [M], i \in [d]$ . Given  $m \in [M], i \in [d]$ , we define the stopping time  $\tau_{mi} = T \wedge \min \{t : \sum_{s=1}^{t} x_{smi}^2 \geq t / d\}$ , and the function  $U_{mi}(z) = \sum_{t=1}^{\tau_{mi}} \left( \frac{1}{\sqrt{d}} - x_{tmi} \cdot z \right)^2$ . We then have the following result for  $U_{mi}(1)$ :
$$
U _ {m i} (1) = \sum_ {t = 1} ^ {\tau_ {m i}} \left(\frac {1}{\sqrt {d}} - x _ {t m i}\right) ^ {2} \leq 2 \sum_ {t = 1} ^ {\tau_ {m i}} \frac {1}{d} + 2 \sum_ {t = 1} ^ {\tau_ {m i}} x _ {t m i} ^ {2} \leq \frac {4 T}{d} + 2. \tag {5}
$$
Let  $x_{m}^{\star}$  be the optimal action of agent  $m$ . We then have the following for the group regret
$$
\begin{array}{l} \mathcal {R} _ {\mathcal {A}} (M, T) = \mathbb {E} _ {\left\{\theta_ {m} \right\} _ {m = 1} ^ {M}} \left[ \sum_ {t = 1} ^ {T} \sum_ {m = 1} ^ {M} \sum_ {i = 1} ^ {d} \left(x _ {m i} ^ {\star} \theta_ {m i} - x _ {t m i} \theta_ {m i}\right) \right] \\ = \varepsilon \cdot \mathbb {E} _ {\left\{\theta_ {m} \right\} _ {m = 1} ^ {M}} \left[ \sum_ {t = 1} ^ {T} \sum_ {m = 1} ^ {M} \sum_ {i = 1} ^ {d} \left(\frac {1}{\sqrt {d}} - x _ {t m i} \operatorname {s i g n} \left(\theta_ {m i}\right)\right) \right] \\ \geq \frac {\varepsilon \sqrt {d}}{2} \cdot \mathbb {E} _ {\left\{\theta_ {m} \right\} _ {m = 1} ^ {M}} \left[ \sum_ {t = 1} ^ {T} \sum_ {m = 1} ^ {M} \sum_ {i = 1} ^ {d} \left(\frac {1}{\sqrt {d}} - x _ {t m i} \operatorname {s i g n} \left(\theta_ {m i}\right)\right) ^ {2} \right] \\ \geq \frac {\varepsilon \sqrt {d}}{2} \cdot \sum_ {m = 1} ^ {M} \sum_ {i = 1} ^ {d} \mathbb {E} _ {\{\theta_ {m} \} _ {m = 1} ^ {M}} \left[ \sum_ {t = 1} ^ {\tau_ {m i}} \left(\frac {1}{\sqrt {d}} - x _ {t m i} \mathrm {s i g n} (\theta_ {m i})\right) ^ {2} \right], \\ \end{array}
$$
where we use the fact that optimal action  $x_{m}^{\star} = \left[\frac{\theta_{m1}}{\|\theta_{m}\|},\dots ,\frac{\theta_{md}}{\|\theta_{m}\|}\right]^{\top}$ , which is a unit vector and has the same direction with  $\theta_{m}$ , the first inequality holds due to  $\| x_{tm}\| ^2\leq 1$ .
Now, let  $\{\theta_m^{\prime}\}_{m = 1}^{M}$  be another set of linear parameters that are different at only one coordinate of the linear parameter of only one agent compared to  $\{\theta_{m}\}_{m = 1}^{M}$ . Specifically, fix  $g\in [M],i\in [d]$ , we have  $\theta_{g,i} = -\theta_{g,i}^{\prime}$ ; otherwise,  $\theta_{k,j} = \theta_{k,j}^{\prime}$ , for  $(k,j)\neq (g,i),\forall k\in [M],j\in [d]$ . To simplify the notion, we define  $\Phi ,\Phi^{\prime}$  as  $Md$  dimensional vectors, where  $\Phi ,\Phi^{\prime}\in \{\pm \varepsilon \}^{Md}$ . That is,  $\Phi ,\Phi^{\prime}$  represent  $\{\theta_m\}_{m = 1}^M,\{\theta_m^{\prime}\}_{m = 1}^M$ , respectively. We let  $\mathbb{P}$  and  $\mathbb{P}'$  be the law of  $U_{gi}(z)$  w.r.t. the interaction of the multi-agent linear bandits induced by  $\Phi ,\Phi^{\prime}$ , respectively. We then get
$$
\begin{array}{l} \mathbb {E} _ {\Phi} \left[ U _ {g i} (1) \right] \geq \mathbb {E} _ {\Phi^ {\prime}} \left[ U _ {g i} (1) \right] - \left(\frac {4 T}{d} + 2\right) \sqrt {\frac {1}{2} D _ {K L} \left(\mathbb {P} | | \mathbb {P} ^ {\prime}\right)} \\ \geq \mathbb {E} _ {\Phi^ {\prime}} [ U _ {g i} (1) ] - \frac {\varepsilon}{2} \left(\frac {4 T}{d} + 2\right) \sqrt {\mathbb {E} \left[ \sum_ {t = 1} ^ {\tau_ {g i}} x _ {t g i} ^ {2} \right]} \\ \geq \mathbb {E} _ {\Phi^ {\prime}} [ U _ {g i} (1) ] - \frac {\varepsilon}{2} \left(\frac {4 T}{d} + 2\right) \sqrt {\frac {T}{d} + 1} \\ \geq \mathbb {E} _ {\Phi^ {\prime}} \left[ U _ {g i} (1) \right] - 4 \sqrt {3} \varepsilon \left(\frac {T}{d}\right) ^ {3 / 2}, \\ \end{array}
$$
where the first inequality holds due to Lemma C.5 and Equation (5), the second inequality follows the stopping time arguments from Lattimore and Csaba [2020], the last inequality holds due to the assumption that  $d \leq 2T$ . Then,
$$
\begin{array}{l} \mathbb {E} _ {\Phi} \left[ U _ {g i} (1) \right] + \mathbb {E} _ {\Phi^ {\prime}} \left[ U _ {g i} (- 1) \right] \geq \mathbb {E} _ {\Phi^ {\prime}} \left[ U _ {g i} (1) + U _ {g i} (- 1) \right] - 4 \sqrt {3} \varepsilon \left(\frac {T}{d}\right) ^ {3 / 2} \\ = 2 \mathbb {E} _ {\Phi^ {\prime}} \left[ \frac {\tau_ {g i}}{d} + \sum_ {t = 1} ^ {\tau_ {g i}} x _ {t g i} ^ {2} \right] - 4 \sqrt {3} \varepsilon \left(\frac {T}{d}\right) ^ {3 / 2} \\ \geq \frac {2 T}{d} - 4 \sqrt {3} \varepsilon \left(\frac {T}{d}\right) ^ {3 / 2} \geq \frac {T}{d}, \\ \end{array}
$$
where the last inequality holds for  $\varepsilon \in [0, \frac{1}{4\sqrt{3}}\sqrt{\frac{d}{T}}]$ , this satisfies the requirement  $\varepsilon \in [0, \frac{1}{\sqrt{d}}]$  of the instance construction due to the assumption that  $\frac{d^2}{48} \leq T$ . Furthermore, we denote  $\Phi_{-mi}$  as a  $(Md - 1)$ -dimensional vector, which is obtained by excluding  $\Phi_{mi}$  from vector  $\Phi$ . Let  $\mathcal{R}_{\Phi}(M,T)$  be the regret w.r.t.  $\Phi$ , and applying randomization hammer, we have the following,
$$
\begin{array}{l} \sum_ {\Phi \in \{\pm \varepsilon \} ^ {M d}} \mathcal {R} _ {\Phi} (M, T) \geq \frac {\varepsilon \sqrt {d}}{2} \sum_ {m = 1} ^ {M} \sum_ {i = 1} ^ {d} \sum_ {\Phi \in \{\pm \varepsilon \} ^ {M d}} \mathbb {E} _ {\Phi} [ U _ {m i} (\operatorname {s i g n} (\varepsilon_ {m i})) ] \\ = \frac {\varepsilon \sqrt {d}}{2} \sum_ {m = 1} ^ {M} \sum_ {i = 1} ^ {d} \sum_ {\Phi_ {- m i} \in \{\pm \varepsilon \} ^ {M d - 1}} \sum_ {\Phi_ {m i} \in \{\pm \varepsilon \}} \mathbb {E} _ {\Phi} \left[ U _ {m i} \left(\operatorname {s i g n} \left(\varepsilon_ {m i}\right)\right) \right] \\ \geq \frac {\varepsilon \sqrt {d}}{2} \sum_ {m = 1} ^ {M} \sum_ {i = 1} ^ {d} \sum_ {\Phi_ {- m i} \in \{\pm \varepsilon \} ^ {M d - 1}} \frac {T}{d} = 2 ^ {M d - 2} \varepsilon M T \sqrt {d}. \\ \end{array}
$$
Therefore, we conclude that there exists an instance with the parameter set of  $\{\theta_m\}_{m=1}^M$  such that  $\mathcal{R}(M,T) \geq \frac{\varepsilon MT\sqrt{d}}{4}$ , for  $\varepsilon \in [0,\frac{1}{4\sqrt{3}}\sqrt{\frac{d}{T}}]$ . Notice that this proof holds for  $\mathcal{I}_{\infty}(2\varepsilon)$  class. Scaling down by 2, we have the following for  $\mathcal{I}_{\infty}(\varepsilon)$  class,
$$
\mathcal {R} _ {\mathcal {I} _ {\infty} (\varepsilon)} (M, T) = \mathcal {R} _ {\mathcal {I} _ {\infty} \left(2 \cdot \frac {\varepsilon}{2}\right)} (M, T) \geq \frac {\varepsilon M T \sqrt {d}}{8}, \text {f o r} \varepsilon \in \left[ 0, \frac {1}{2 \sqrt {3}} \sqrt {\frac {d}{T}} \right].
$$
Observe that  $\frac{\varepsilon M T \sqrt{d}}{8}$  is a strictly increasing function w.r.t.  $\varepsilon$ . It is at most  $\frac{d M \sqrt{T}}{16 \sqrt{3}}$ , when  $\varepsilon = \frac{1}{2 \sqrt{3}} \sqrt{\frac{d}{T}}$ . In other words,  $\mathcal{R}_{\mathcal{I}_{\infty}(\varepsilon)}(M, T) \geq \min \left(\frac{\varepsilon M T \sqrt{d}}{8}, \frac{d M \sqrt{T}}{16 \sqrt{3}}\right)$  for  $\varepsilon \in [0, \frac{1}{2 \sqrt{3}} \sqrt{\frac{d}{T}}]$ .
Recall that  $\inf_{\mathcal{A}}\sup_{\mathcal{I}\in \mathcal{I}_{\infty}(\varepsilon)}\mathcal{R}(M,T)\geq \inf_{\mathcal{A}}\sup_{\mathcal{I}\in \mathcal{I}_{\infty}(\varepsilon ')}\mathcal{R}(M,T)$ , for any  $\varepsilon \geq \varepsilon^{\prime}\geq 0$ . Thus, we conclude,  $\forall \varepsilon \geq 0$ , the following holds,
$$
\inf_{\mathcal{A}}\sup_{\mathcal{I}\in \mathcal{I}_{\infty}(\varepsilon)}\mathcal{R}_{\mathcal{A},\mathcal{I}}(M,T)\geq \min (\frac{\varepsilonMT\sqrt{d}}{8},\frac{dM\sqrt{T}}{16\sqrt{3}}),
$$
C Supporting Lemmas
Lemma C.1. ([Lattimore and Csaba, 2020, Theorem 24.2]). Assume  $d \leq 2n$  and let  $\mathcal{D} = \{x \in \mathbb{R}^d : \| x \|_2 \leq 1\}$ . Then there exists a parameter vector  $\theta \in \mathbb{R}^d$  with  $\| \theta \|_2^2 = d^2 / (48n)$  such that  $R_n(\mathcal{D}, \theta) \geq d\sqrt{n} / (16\sqrt{3})$ .
Lemma C.2. ([Abbasi-Yadkori et al., 2011, Lemma 12]). Let  $A$ ,  $B$  and  $C$  be positive semi-definite matrices such that  $A = B + C$ . Then, we have that
$$
\sup  _ {x \neq 0} \frac {x ^ {\top} A x}{x ^ {\top} B x} \leq \frac {\det  (A)}{\det  (B)}.
$$
Lemma C.3. ([Abbasi-Yadkori et al., 2011, Lemma 11]). Let  $\{X_{t}\}_{t = 1}^{\infty}$  be a sequence in  $\mathbb{R}^d$ ,  $V$  a  $d\times d$  positive definite matrix and define  $\bar{V}_t = V + \sum_{s = 1}^t X_sX_s^\top$ . Then, we have that
$$
\log \left(\frac {\operatorname* {d e t} \left(\bar {V} _ {n}\right)}{\operatorname* {d e t} (V)}\right) \leq \sum_ {t = 1} ^ {n} \| X _ {t} \| _ {\bar {V} _ {t - 1} ^ {- 1}} ^ {2}.
$$
$$
\sum_ {t = 1} ^ {n} \min  \left\{1, \| X _ {t} \| _ {\bar {V} _ {t - 1} ^ {- 1}} ^ {2} \right\} \leq 2 \left(\log \det  \left(\bar {V} _ {n}\right) - \log \det  V\right) \leq 2 \left(d \log \left(\left(\operatorname {t r a c e} (V) + n L ^ {2}\right) / d\right) - \log \det  V\right)
$$
Lemma C.4. (Self-Normalized Bound for Vector-Valued Martingales, [Abbasi-Yadkori et al., 2011, Theorem 1]). Let  $\{F_t\}_{t=0}^{\infty}$  be a filtration. Let  $\{\eta_t\}_{t=1}^{\infty}$  be a real-valued stochastic process such that  $\eta_t$  is  $F_t$ -measurable and  $\eta_t$  is conditionally  $R$ -sub-Gaussian for some  $R \geq 0$  i.e.
$$
\forall \lambda \in \mathbb {R} \quad \mathbf {E} \left[ e ^ {\lambda \eta_ {t}} \mid F _ {t - 1} \right] \leq \exp \left(\frac {\lambda^ {2} R ^ {2}}{2}\right).
$$
Let  $\{X_{t}\}_{t = 1}^{\infty}$  be an  $\mathbb{R}^d$ -valued stochastic process such that  $X_{t}$  is  $F_{t - 1}$ -measurable. Assume that  $V$  is a  $d\times d$  positive definite matrix. For any  $t\geq 0$ , define  $\bar{V}_t = V + \sum_{s = 1}^{t}X_sX_s^\top$ . Then, for any  $\delta >0$  with probability at least  $1 - \delta$ , for all  $t\geq 0$ ,
$$
\left\| \sum_ {s = 1} ^ {t} \eta_ {s} X _ {s} \right\| _ {\bar {V} _ {t} ^ {- 1}} ^ {2} \leq 2 R ^ {2} \log \left(\frac {\det  (\bar {V} _ {t}) ^ {1 / 2} \det  (V) ^ {- 1 / 2}}{\delta}\right).
$$
Lemma C.5. (Pinsker's inequality [Lattimore and Csaba, 2020, Equation 14.12])
For measures  $P$  and  $Q$  on the same probability space  $(\Omega, \mathcal{F})$ , we have the following,
$$
\sup _ {A \in \mathcal {F}} P (A) - Q (A) \leq \sqrt {\frac {1}{2} D _ {K L} (P | | Q)}.
$$