SlowGuess/NeurIPS_2023_OCR huggingface dataset:
As sketched in Section 4.3, we devised an iterative search method based on the following ranking heuristic (formally defined in Eq. (6)): the more one client assists in increasing the server's determinant, the more valuable its contribution is, and thus we should motivate the most valuable clients to participate. Denote  $n \leq k$  (initialized as 1) as the number of clients to be selected from the invalid list  $\{i_1, \ldots, i_k\}$ , and initialize the participant set  $S_t = \widehat{S}_t$ . In each round  $n$ , we rank the remaining  $k - n + 1$  clients based on their potential contribution to the server by Eq. (6), and add the most valuable one to  $S_t$  (Line 3-4). With the latest  $S_t$  committed, we then proceed to determine additional data-incentivized participants by Eq. (4) (Line 5-8), and compute the total payment by Eq. (5) (Line 9). If having  $n$  clients results in the total cost  $\mathcal{I}_{n,t}^m > \mathcal{I}_{\text{last}}^m$ , then we terminate the search and resort to our last resort (Line 10-11). Otherwise, if the resulting  $S_t$  enables the server to satisfy the  $\beta$  gap requirement, then we successfully find a better solution than last resort and can terminate the search. However, if having  $n$  clients is insufficient for the server to pass the  $\beta$  gap requirement, we increase  $n = n + 1$  and repeat the search process (Line 12-14). In particular, if the above process fails to terminate (i.e., having all  $m$  clients still not suffices, we will still use the last resort. Note that, by utilizing matrix computation to calculate the contribution list in each round, this method only incurs a linear time complexity of  $O(N)$ , when  $n = m = N$ .
Lemma 5 (Lemma H.3 of [40]) With probability  $1 - \delta$ , single step pseudo-regret  $r_t = \langle \theta^*, \mathbf{x}^* - \mathbf{x}_t \rangle$  is bounded by
$$
r _ {t} \leq 2 \left(\sqrt {2 \log \left(\frac {\operatorname* {d e t} (V _ {i _ {t} , t}) ^ {1 / 2} \operatorname* {d e t} (\lambda I) ^ {- 1 / 2}}{\delta}\right)} + \lambda^ {1 / 2}\right) \| \mathbf {x} _ {t} \| _ {V _ {i _ {t}, t} ^ {- 1}} = O \left(\sqrt {d \log \frac {T}{\delta}}\right) \| \mathbf {x} _ {t} \| _ {V _ {i _ {t}, t} ^ {- 1}}.
$$
Lemma 6 (Lemma 11 of [1]) Let  $\{X_{t}\}_{t = 1}^{\infty}$  be a sequence in  $\mathbb{R}^d$ ,  $V$  is a  $d\times d$  positive definite matrix and define  $V_{t} = V + \sum_{s = 1}^{t}X_{s}X_{s}^{\top}$ . Then we have that
$$
\log \left(\frac {\det  \left(V _ {n}\right)}{\det  (V)}\right) \leq \sum_ {t = 1} ^ {n} \| X _ {t} \| _ {V _ {t - 1} ^ {- 1}} ^ {2}.
$$
Further, if  $\| X_t\| _2\leq L$  for all  $t$ , then
$$
\sum_ {t = 1} ^ {n} \min  \left\{1, \| X _ {t} \| _ {V _ {t - 1} ^ {- 1}} ^ {2} \right\} \leq 2 \left(\log \det  \left(V _ {n}\right) - \log \det  V\right) \leq 2 \left(d \log \left(\left(\operatorname {t r a c e} (V) + n L ^ {2}\right) / d\right) - \log \det  V\right).
$$
Lemma 7 (Lemma 12 of [1]) Let  $A, B$  and  $C$  be positive semi-definite matrices such that  $A = B + C$ . Then, we have that
$$
\sup  _ {\mathbf {x} \neq \boldsymbol {\theta}} \frac {\mathbf {x} ^ {\top} A \mathbf {x}}{\mathbf {x} ^ {\top} B \mathbf {x}} \leq \frac {\det  (A)}{\det  (B)}.
$$
Our proof relies on the following lower bound result for federated linear bandits established in [13].
Lemma 8 (Theorem 5.3 of [13]) Let  $p_i$  denote the probability that an agent  $i \in [N]$  will communicate with the server at least once over time horizon  $T$ . Then for any algorithm with
$$
\sum_ {i = 1} ^ {N} p _ {i} \leq \frac {c}{2 C} \cdot \frac {N}{\log (T / N)} \tag {7}
$$
there always exists a linear bandit instance with  $\sigma = L = S = 1$ , such that for  $T \geq Nd$ , the expected regret of this algorithm is at least  $\Omega(d\sqrt{NT})$ .
In the following, we will create a situation, where Eq. (7) always holds true for the payment-free incentive mechanism. Specifically, recall that the payment-free incentive mechanism (Section 4.2) motivates clients to participate using only data, i.e., the determinant ratio defined in Eq. (4) that indicates how much client  $i$ 's confidence ellipsoid can shrink using the data offered by the server. Based on matrix determinant lemma [6], we know that  $\mathcal{I}_{i,t} \leq (1 + \frac{L^2}{\lambda})^T$ . Additionally, by applying the determinant-trace inequality (Lemma 10 of [1]), we have  $\mathcal{I}_{i,t} \leq (1 + \frac{TL^2}{\lambda d})^d$ . Therefore, as long as  $D_i^p > \min \{(1 + \frac{L^2}{\lambda})^T, (1 + \frac{TL^2}{\lambda d})^d\}$ , where the tighter choice between the two upper bounds depends on the specific problem instance (i.e., either  $d$  or  $T$  being larger), it becomes impossible for the server to incentivize client  $i$  to participate in the communication. Now based on Lemma 8, if the number of clients that satisfy  $D_i^p \leq \min \{(1 + \frac{L^2}{\lambda})^T, (1 + \frac{TL^2}{\lambda d})^d\}$  is smaller than  $\frac{c}{2C} \cdot \frac{N}{\log(T / N)}$ , a sub-optimal regret of the order  $\Omega(d\sqrt{NT})$  is inevitable for payment-free incentive mechanism, which finishes the proof.
To prove this theorem, we first need the following lemma.
Lemma 9 (Communication Frequency Bound) By setting the communication threshold  $D_{c} = \frac{T}{N^{2}d\log T} - \sqrt{\frac{T^{2}}{N^{2}dR\log T}}\log \beta$ , the total number of epochs defined by the communication rounds satisfies,
$$
P = O (N d \log T)
$$
where  $R = \left\lceil d\log \left(1 + \frac{T}{\lambda d}\right)\right\rceil = O(d\log T)$
Proof of Lemma 9. Denote  $P$  as the total number of epochs divided by communication rounds throughout the time horizon  $T$ , and  $V_{g,t_p}$  as the aggregated covariance matrix at the  $p$ -th epoch. Specifically,  $V_{g,t_0} = \lambda I$ ,  $\widetilde{V}_T$  is the covariance matrix constructed by all data points available in the system at time step  $T$ .
Note that according to the incentivized communication scheme in INC-FEDUCB, not all clients will necessarily share their data in the last epoch, hence  $\operatorname*{det}(V_{g,t_P})\leq \operatorname*{det}(\widetilde{V}_T)\leq \left(\frac{tr(\widetilde{V}_T)}{d}\right)\leq (\lambda +T / d)^d$  Therefore,
$$
\log \frac {\det  (V _ {g , t _ {P}})}{\det  (V _ {g , t _ {P - 1}})} + \log \frac {\det  (V _ {g , t _ {P - 1}})}{\det  (V _ {g , t _ {P - 2}})} + \dots + \log \frac {\det  (V _ {g , t _ {1}})}{\det  (V _ {g , t _ {0}})} = \log \frac {\det  (V _ {g , t _ {P}})}{\det  (V _ {g , t _ {0}})} \leq \left\lceil d \log (1 + \frac {T}{\lambda d}) \right\rceil
$$
Let  $\alpha \in \mathbb{R}^{+}$  be an arbitrary positive value, for epochs with length greater than  $\alpha$ , there are at most  $\left\lceil \frac{T}{\alpha} \right\rceil$  of them. For epochs with length less than  $\alpha$ , say the  $p$ -th epoch triggered by client  $i$ , we have
$$
\Delta t _ {i, t _ {p}} \cdot \log \frac {\det  (V _ {i , t _ {p}})}{\det  (V _ {i , t _ {\text {l a s t}}})} > D _ {c}
$$
Combining the  $\beta$  gap constraint defined in Section 4.3 and the fact that the server always downloads to all clients at every communication round, we have  $\Delta t_{i,t_p} \leq \alpha$  and hence
$$
\log \frac {\det (g , V _ {t _ {p}})}{\det (V _ {g , t _ {p - 1}})} \geq \log \frac {\beta \cdot \det (\widetilde {V} _ {t _ {p}})}{\det (V _ {g , t _ {p - 1}})} \geq \log \frac {\beta \cdot \det (V _ {i , t _ {p}})}{\det (V _ {g , t _ {p - 1}})} \geq \log \frac {\beta \cdot \det (V _ {i , t _ {p}})}{\det (V _ {i , t _ {\mathrm {l a s t}}})} \geq \frac {D _ {c}}{\alpha} + \log \beta
$$
Let  $R = \left\lceil d\log (1 + \frac{T}{\lambda d})\right\rceil = O(d\log T)$ , therefore, there are at most  $\left\lceil \frac{R}{\frac{D_c}{\alpha} + \log\beta}\right\rceil$  epochs with length less than  $\alpha$  time steps. As a result, the total number of epochs  $P \leq \left\lceil \frac{T}{\alpha}\right\rceil + \left\lceil \frac{D_c}{\alpha} + \log \beta \right\rceil$ . Note that  $\left\lceil \frac{T}{\alpha}\right\rceil + \left\lceil \frac{R}{\frac{D_c}{\alpha} + \log\beta}\right\rceil \geq 2\sqrt{\frac{TR}{D_c + \alpha\log\beta}}$ , where the equality holds when  $\alpha = \sqrt{\frac{T(D_c + \alpha\log\beta)}{R}}$ .
Furthermore, let  $D_{c} = \frac{T}{N^{2}d\log T} - \alpha \log \beta$ , then  $\alpha = \sqrt{\frac{T^2}{N^2dR\log T}}$ , we have
$$
P = O \left(\sqrt {\frac {T R}{D _ {c} + \alpha \log \beta}}\right) = O \left(N \sqrt {d R \log T}\right) = O (N d \log T) \tag {8}
$$
This concludes the proof of Lemma 9.
Communication Cost: The proof of communication cost upper bound directly follows Lemma 9. In each epoch, all clients first upload  $O(d^{2})$  scalars to the server and then download  $O(d^{2})$  scalars. Therefore, the total communication cost is  $C_{T} = P \cdot O(Nd^{2}) = O(N^{2}d^{3}\log T)$
Monetary Incentive Cost: Under the clients' committed data sharing cost  $D^{p} = \{D_{1}^{p},\dots ,D_{N}^{p}\}$ , during each communication round at time step  $t_p$ , we only pay clients in the participant set  $S_{t_p}$ . Specifically, the payment (i.e., monetary incentive cost)  $\mathcal{I}_{i,t_p}^m = 0$  if the data incentive is already sufficient to motivate the client to participate, i.e., when  $\mathcal{I}_{i,t_p}^d\geq D_i^p$ . Otherwise, we only need to pay the minimum amount of monetary incentive such that Eq. (1) is satisfied, i.e.,  $\mathcal{I}_{i,t_p}^m = D_i^p -\mathcal{I}_{i,t_p}^d$ . Therefore, the accumulative monetary incentive cost is
$$
\begin{array}{l} M _ {T} = \sum_ {p = 1} ^ {P} \sum_ {i = 1} ^ {N} \mathcal {I} _ {i, t _ {p}} ^ {m} = \sum_ {p = 1} ^ {P} \sum_ {i = 1} ^ {N} \max \left\{0, D _ {i} ^ {p} - \mathcal {I} _ {i, t _ {p}} ^ {d} \right\} \cdot \mathbb {I} (\Delta V _ {i, t _ {p}} \in S _ {t _ {p}}) \\ \leq \sum_ {p = 1} ^ {P} \sum_ {i = 1} ^ {N} \max \left\{0, \max _ {i \in [ N ]} \{D _ {i} ^ {p} \} - \mathcal {I} _ {i, t _ {p}} ^ {d} \right\} \cdot \mathbb {I} (\Delta V _ {i, t _ {p}} \in S _ {t _ {p}}) \\ \leq \sum_ {p = 1} ^ {P} \sum_ {i \in \mathcal {N} _ {p}} \left(\max _ {i \in [ N ]} \{D _ {i} ^ {p} \} - \mathcal {I} _ {i, t _ {p}} ^ {d}\right) \cdot \mathbb {I} (\Delta V _ {i, t _ {p}} \in S _ {t _ {p}}) \\ \leq \max _ {i \in [ N ]} \{D _ {i} ^ {p} \} \sum_ {p = 1} ^ {P} \sum_ {i = 1} ^ {N} \mathbb {I} (\Delta V _ {i, t _ {p}} \in S _ {t _ {p}}) - \sum_ {p = 1} ^ {P} \sum_ {i \in \bar {\mathcal {N}} _ {p}} \mathcal {I} _ {i, t _ {p}} ^ {d} \cdot \mathbb {I} (\Delta V _ {i, t _ {p}} \in S _ {t _ {p}}) \\ = \max _ {i \in [ N ]} \{D _ {i} ^ {p} \} \sum_ {p = 1} ^ {P} N _ {p} - \sum_ {i = 1} ^ {N} \sum_ {p \in \mathcal {P} _ {i}} \mathcal {I} _ {i, t _ {p}} ^ {d} \\ \end{array}
$$
where  $P$  and  $N$  represent the number of epochs and clients,  $N_{p}$  is the number of participants in  $p$ -th epoch,  $\bar{\mathcal{N}}_p$  is the set of money-incentivized participants in the  $p$ -th epoch,  $\bar{\mathcal{P}}_i$  is the set of epochs where client  $i$  gets monetary incentive, whose size is denoted as  $P_{i} = |\bar{\mathcal{P}}_{i}|$ . Denote  $D_{\mathrm{max}}^{p} = \max_{i\in [N]}\{D_{i}^{p}\}$  to simplify our later discussion.
Recall the definition of data incentive and  $D_{i,t_p}(S_{t_p}) = \sum_{j:\{\Delta V_{j,t_p} \in S_{t_p}\} \wedge \{j \neq i\}} \Delta V_{j,t_p} + \Delta V_{-i,t_p}$  introduced in Eq. (4), we can show that
$$
\begin{array}{l} \mathcal {I} _ {i, t _ {p}} ^ {d} = \frac {\det \left(D _ {i , t _ {p}} (S _ {t _ {p}}) + V _ {i , t _ {p}}\right)}{\det (V _ {i , t _ {p}})} - 1 \\ \geq \frac {\det  (V _ {g , t _ {p}})}{\det  (V _ {i , t _ {p}})} - 1 \\ \end{array}
$$
$$
\begin{array}{l} M _ {T} \leq D _ {\mathrm {m a x}} ^ {p} \cdot \sum_ {p = 1} ^ {P} N _ {p} + \sum_ {i = 1} ^ {N} \sum_ {p \in \bar {\mathcal {P}} _ {i}} 1 - \sum_ {i = 1} ^ {N} \sum_ {p \in \bar {\mathcal {P}} _ {i}} \frac {\det (V _ {g , t _ {p}})}{\det (V _ {i , t _ {p}})} \\ \leq D _ {\max } ^ {p} \cdot \sum_ {p = 1} ^ {P} N _ {p} + \sum_ {i = 1} ^ {N} P _ {i} - \sum_ {i = 1} ^ {N} P _ {i} \cdot \left(\frac {\det  \left(V _ {g , t _ {1}}\right)}{\det  \left(V _ {i , t _ {1}}\right)} \cdot \frac {\det  \left(V _ {g , t _ {2}}\right)}{\det  \left(V _ {i , t _ {2}}\right)} \dots \frac {\det  \left(V _ {g , t _ {P _ {i}}}\right)}{\det  \left(V _ {i , t _ {P _ {i}}}\right)}\right) ^ {\frac {1}{P _ {i}}} \\ \leq D _ {\max} ^ {p} \cdot \sum_ {p = 1} ^ {P} N _ {p} + \sum_ {i = 1} ^ {N} P _ {i} - \sum_ {i = 1} ^ {N} P _ {i} \cdot \left(\frac {\det (V _ {g , t _ {1}})}{\det (V _ {i , t _ {1}})} \cdot \frac {\det (V _ {i , t _ {1}})}{\det (V _ {i , t _ {2}})} \dots \frac {\det (V _ {i , t _ {P _ {i}} - 1})}{\det (V _ {i , t _ {P _ {i}}})}\right) ^ {\frac {1}{P _ {i}}} \\ = D _ {\max } ^ {p} \cdot \sum_ {p = 1} ^ {P} N _ {p} + \sum_ {i = 1} ^ {N} P _ {i} - \sum_ {i = 1} ^ {N} P _ {i} \cdot \left(\frac {\det  \left(V _ {g , t _ {1}}\right)}{\det  \left(V _ {i , t _ {P _ {i}}}\right)}\right) ^ {\frac {1}{P _ {i}}} \\ \leq \left(1 + D _ {\max } ^ {p}\right) \cdot P \cdot N - \sum_ {i = 1} ^ {N} P _ {i} \cdot \left(\frac {\det (\lambda I)}{\det (V _ {T})}\right) ^ {\frac {1}{P _ {i}}} \\ \end{array}
$$
where the second step holds by Cauchy-Schwarz inequality and the last step follows the facts that  $P_{i} \leq P, N_{p} \leq N, \det(V_{g,t_{1}}) \geq \det(\lambda I),$  and  $\det(V_{i,t_{P_i}}) \leq \det(V_T)$ .
Specifically, by setting the communication threshold  $D_{c} = \frac{T}{N^{2}d\log T} -\sqrt{\frac{T^{2}}{N^{2}dR\log T}}\log \beta$ , where  $R = \lceil d\log (1 + \frac{T}{\lambda d})\rceil$ , we have the total number of epochs  $P = O(Nd\log T)$  (Lemma 9). Therefore,
$$
\begin{array}{l} M _ {T} \leq (1 + D _ {\max} ^ {p}) \cdot O (N ^ {2} d \log T) - \sum_ {i = 1} ^ {N} P _ {i} \cdot \left(\frac {\det (\lambda I)}{\det (V _ {T})}\right) ^ {\frac {1}{P _ {i}}} \\ = O \left(N ^ {2} d \log T\right) \\ \end{array}
$$
Regret: To prove the regret upper bound, we first need the following lemma.
Lemma 10 (Instantaneous Regret Bound) Under threshold  $\beta$ , with probability  $1 - \delta$ , the instantaneous pseudo-regret  $r_t = \langle \theta^*, \mathbf{x}^* - \mathbf{x}_t \rangle$  in  $j$ -th epoch is bounded by
$$
r _ {t} = O \left(\sqrt {d \log \frac {T}{\delta}}\right) \cdot \| \mathbf {x} _ {t} \| _ {\tilde {V} _ {t - 1} ^ {- 1}} \cdot \sqrt {\frac {1}{\beta} \cdot \frac {\det (V _ {g , t _ {j}})}{\det (V _ {g , t _ {j - 1}})}}
$$
Proof of Lemma 10. Denote  $\widetilde{V}_t$  as the covariance matrix constructed by all available data in the system at time step  $t$ . As introduced in Eq. (3), the instantaneous regret of client  $i$  is upper bounded by
$$
r _ {t} \leq 2 \alpha_ {i _ {t}, t - 1} \sqrt {\mathbf {x} _ {t} ^ {\top} \widetilde {V} _ {t - 1} ^ {- 1} \mathbf {x} _ {t}} \cdot \sqrt {\frac {\det (\widetilde {V} _ {t - 1})}{\det (V _ {i _ {t} , t - 1})}} = O \left(\sqrt {d \log \frac {T}{\delta}}\right) \cdot \| \mathbf {x} _ {t} \| _ {\widetilde {V} _ {t - 1} ^ {- 1}} \cdot \sqrt {\frac {\det (\widetilde {V} _ {t - 1})}{\det (V _ {i _ {t} , t - 1})}}
$$
Suppose the client  $i_{t}$  appears at the  $j$ -th epoch, i.e.,  $t_{j-1} \leq t \leq t_{j}$ . As the server always downloads the aggregated data to every client in each communication round, we have
$$
\frac {\det  (\widetilde {V} _ {t})}{\det  (V _ {i _ {t} , t})} \leq \frac {\det  (\widetilde {V} _ {t})}{\det  (V _ {i _ {t} , t _ {j - 1}})} \leq \frac {\det  (\widetilde {V} _ {t})}{\det  (V _ {g , t _ {j - 1}})}
$$
Combining the  $\beta$  gap constraint defined in Section 4.3, we can show that
$$
\frac {\det  (\widetilde {V} _ {t})}{\det  (V _ {i _ {t} , t})} \leq \frac {\det  (\widetilde {V} _ {t})}{\det  (V _ {g , t _ {j - 1}})} \leq \frac {\det  (V _ {g , t _ {j}}) / \beta}{\det  (V _ {g , t _ {j - 1}})} = \frac {1}{\beta} \cdot \frac {\det  (V _ {g , t _ {j}})}{\det  (V _ {g , t _ {j - 1}})}
$$
Lastly, plugging the above inequality into Eq. (3), we have
$$
r _ {t} = O \left(\sqrt {d \log \frac {T}{\delta}}\right) \cdot \| \mathbf {x} _ {t} \| _ {\tilde {V} _ {t - 1} ^ {- 1}} \cdot \sqrt {\frac {1}{\beta} \cdot \frac {\det  (V _ {g , t _ {j}})}{\det  (V _ {g , t _ {j - 1}})}}
$$
which finishes the proof of Lemma 10.
Now, we are ready to prove the accumulative regret upper bound. Similar to DisLinUCB [40], we group the communication epochs into good epochs and bad epochs.
Good Epochs: Note that for good epochs, we have  $1 \leq \frac{\operatorname*{det}(V_{g,t_j})}{\operatorname*{det}(V_{g,t_{j-1}})} \leq 2$ . Therefore, based on Lemma 10, the instantaneous regret in good epochs is
$$
r _ {t} = O \left(\sqrt {d \log \frac {T}{\delta}}\right) \cdot \| \mathbf {x} _ {t} \| _ {\tilde {V} _ {t - 1} ^ {- 1}} \cdot \sqrt {\frac {2}{\beta}}
$$
Denote the accumulative regret among all good epochs as  $REG_{good}$ , then using the Cauchy-Schwarz inequality we can see that
$$
\begin{array}{l} R E G _ {g o o d} = \sum_ {p \in P _ {g o o d}} \sum_ {t \in \mathcal {B} _ {p}} r _ {t} \\ \leq \sqrt {T \cdot \sum_ {p \in P _ {\text {g o o d}}} \sum_ {t \in \mathcal {B} _ {p}} r _ {t} ^ {2}} \\ \leq O \left(\sqrt {T \cdot d \log \frac {T}{\delta} \cdot \frac {2}{\beta} \sum_ {p \in P _ {\text {g o o d}}} \sum_ {t \in \mathcal {B} _ {p}} \| \mathbf {x} _ {t} \| _ {\widetilde {V} _ {t - 1} ^ {- 1}} ^ {2}}\right) \\ \end{array}
$$
Combining the fact  $x \leq 2\log (1 + x), \forall x \in [0,1]$  and Lemma 6, we have
$$
\begin{array}{l} R E G _ {g o o d} \leq O \left(\sqrt {T \cdot \frac {d}{\beta} \log \frac {T}{\delta} \sum_ {p \in P _ {g o o d}} \sum_ {t \in \mathcal {B} _ {p}} 2 \log \left(1 + \| \mathbf {x} _ {t} \| _ {\hat {V} _ {t - 1} ^ {- 1}} ^ {2}\right)}\right) \\ \leq O \left(\sqrt {T \cdot \frac {d}{\beta} \log \frac {T}{\delta} \cdot \sum_ {p \in P _ {\text {g o o d}}} \log \frac {\det (\tilde {V} _ {t _ {p}})}{\det (\tilde {V} _ {t _ {p - 1}})}}\right) \\ \leq O \left(\sqrt {T \cdot \frac {d}{\beta} \log \frac {T}{\delta} \sum_ {p \in P _ {A l l}} \log \frac {\det (\widetilde {V} _ {t _ {p}})}{\det (\widetilde {V} _ {t _ {p - 1}})}}\right) \\ = O \left(\sqrt {T \cdot \frac {d}{\beta} \log \frac {T}{\delta} \cdot \log \frac {\det (\widetilde {V} _ {t _ {P}})}{\det (\widetilde {V} _ {t _ {0}})}}\right) \\ \leq O \left(\sqrt {T \cdot \frac {d}{\beta} \log \frac {T}{\delta} \cdot d \log \left(1 + \frac {T}{\lambda d}\right)}\right) \\ = O \left(\frac {d}{\sqrt {\beta}} \cdot \sqrt {T} \cdot \sqrt {\log \frac {T}{\delta} \cdot \log T}\right) \\ \end{array}
$$
Bad Epochs: Now moving on to the bad epoch. For any bad epoch starting from time step  $t_s$  to time step  $t_e$ , the regret in this epoch is
$$
R E G = \sum_ {t = t _ {s}} ^ {t _ {e}} r _ {t} = \sum_ {i = 1} ^ {N} \sum_ {\tau \in \mathcal {N} _ {i} (t _ {e}) \backslash \mathcal {N} _ {i} (t _ {s})} r _ {\tau}
$$
where  $\mathcal{N}_i(t) = \{1\leq \tau \leq t:i_\tau = i\}$  denotes the set of time steps when client  $i$  interacts with the environment up to  $t$ . Combining the fact  $r_{\tau}\leq 2$  and Lemma 5, we have
$$
r _ {\tau} \leq \min \{2, 2 \alpha_ {i _ {\tau}, \tau - 1} \sqrt {\mathbf {x} _ {\tau} ^ {\top} V _ {i _ {\tau} , \tau - 1} ^ {- 1} \mathbf {x} _ {\tau}} \} = O \left(\sqrt {d \log \frac {T}{\delta}}\right) \min \{1, \| \mathbf {x} _ {\tau} \| _ {V _ {i _ {\tau}, \tau - 1} ^ {- 1}} \}
$$
$$
\begin{array}{l} R E G \leq O \left(\sqrt {d \log \frac {T}{\delta}}\right) \sum_ {i = 1} ^ {N} \sum_ {\tau \in \mathcal {N} _ {i} (t _ {e}) \backslash \mathcal {N} _ {i} (t _ {s})} \min  \{1, \| \mathbf {x} _ {\tau} \| _ {V _ {i, \tau - 1} ^ {- 1}} \} \\ \leq O \left(\sqrt {d \log \frac {T}{\delta}}\right) \sum_ {i = 1} ^ {N} \sqrt {\Delta t _ {i , t _ {e}} \sum_ {\tau \in \mathcal {N} _ {i} (t _ {e}) \backslash \mathcal {N} _ {i} (t _ {s})} \min  \{1 , \| \mathbf {x} _ {\tau} \| _ {V _ {i , \tau - 1} ^ {- 1}} ^ {2} \}} \\ \leq O \left(\sqrt {d \log \frac {T}{\delta}}\right) \sum_ {i = 1} ^ {N} \sqrt {\Delta t _ {i , t _ {e}} \sum_ {\tau \in \mathcal {N} _ {i} (t _ {e}) \backslash \mathcal {N} _ {i} (t _ {s})} \log \left(1 + \| \mathbf {x} _ {\tau} \| _ {V _ {i , \tau - 1} ^ {- 1}} ^ {2}\right)} \\ = O \left(\sqrt {d \log \frac {T}{\delta}}\right) \sum_ {i = 1} ^ {N} \sqrt {\Delta t _ {i , t _ {e}} \sum_ {\tau \in \mathcal {N} _ {i} (t _ {e}) \backslash \mathcal {N} _ {i} (t _ {s})} \log \left(\frac {\det  (V _ {i , \tau})}{\det  (V _ {i , \tau - 1})}\right)} \\ \leq O \left(\sqrt {d \log \frac {T}{\delta}}\right) \sum_ {i = 1} ^ {N} \sqrt {\Delta t _ {i , t _ {e}} \cdot \log \frac {\det  (V _ {i , t _ {e}})}{\det  (V _ {i , t _ {\text {l a s t}}})}} \\ \leq O \left(\sqrt {d \log \frac {T}{\delta}}\right) N \cdot \sqrt {D _ {c}}. \\ \end{array}
$$
where the second step holds by the Cauchy-Schwarz inequality, the third step follows from  $x \leq 2\log(1 + x), \forall x \in [0,1]$ , the fourth step utilizes the elementary algebra, and the last two steps follow the fact that no client triggers the communication before  $t_e$ .
Recall that, as introduced in Lemma 9, the number of bad epochs is less than  $R = \lceil d\log (1 + \frac{T}{\delta})\rceil = O(d\log T)$ , therefore the regret across all bad epochs is
$$
\begin{array}{l} R E G _ {b a d} = O \left(\sqrt {d \log \frac {T}{\delta}}\right) N \cdot \sqrt {D _ {c}} \cdot O (d \log T) \\ = O \left(N d ^ {1. 5} \sqrt {D _ {c} \cdot \log \frac {T}{\delta}} \log T\right) \\ \end{array}
$$
Combining the regret for all good and bad epochs, we have accumulative regret
$$
\begin{array}{l} R _ {T} = R E G _ {\text {g o o d}} + R E G _ {\text {b a d}} \\ = O \left(\frac {d}{\sqrt {\beta}} \cdot \sqrt {T} \cdot \sqrt {\log \frac {T}{\delta} \cdot \log T}\right) + O \left(N d ^ {1. 5} \sqrt {D _ {c} \cdot \log \frac {T}{\delta}} \log T\right) \\ \end{array}
$$
According to Lemma 10, the above regret bound holds with high probability  $1 - \delta$ . For completeness, we also present the regret when it fails to hold, which is bounded by  $\delta \cdot \sum r_t \leq 2T \cdot \delta$  in expectation. And this can be trivially set to  $O(1)$  by selecting  $\delta = 1 / T$ . In this way, we can primarily focus on analyzing the following regret when the bound holds.
$$
R _ {T} = O \left(\frac {d}{\sqrt {\beta}} \sqrt {T} \log T\right) + O \left(N d ^ {1. 5} \log^ {1. 5} T \cdot \sqrt {D _ {c}}\right)
$$
With our choice of  $D_{c} = \frac{T}{N^{2}d\log T} -\sqrt{\frac{T^{2}}{N^{2}dR\log T}}\log \beta$  in Lemma 9, we have
$$
R _ {T} = O \left(\frac {d}{\sqrt {\beta}} \sqrt {T} \log T\right) + O \left(N d ^ {1. 5} \log^ {1. 5} T \cdot \sqrt {\frac {T}{N ^ {2} d \log T} - \sqrt {\frac {T ^ {2}}{N ^ {2} d R \log T}} \log \beta}\right)
$$
Plugging in  $R = \left\lceil d\log \left(1 + \frac{T}{\lambda d}\right)\right\rceil = O(d\log T)$ , we get
$$
R _ {T} = O \left(\frac {d}{\sqrt {\beta}} \sqrt {T} \log T\right) + O \left(N d ^ {1. 5} \log^ {1. 5} T \cdot \sqrt {\frac {T}{N ^ {2} d \log T} + \frac {T}{N d \log T} \log \frac {1}{\beta}}\right)
$$
Furthermore, by setting  $\beta > e^{-\frac{1}{N}}$ , we can show that  $\frac{T}{N^2 d \log T} > \frac{T}{Nd \log T} \log \frac{1}{\beta}$ , and therefore
$$
R _ {T} = O \left(\frac {d}{\sqrt {\beta}} \sqrt {T} \log T\right) + O \left(d \sqrt {T} \log T\right) = O \left(d \sqrt {T} \log T\right)
$$
E Detailed Experimental Results
In addition to the empirical studies on the synthetic datasets reported in Section 5, we also conduct comprehensive experiments on the real-world recommendation dataset MovieLens [12]. Following [3, 22], we pre-processed the dataset to align it with the linear bandit problem setting, with feature dimension  $d = 25$  and arm set size  $K = 25$ . Specifically, it contains  $N = 54$  users and 26567 items (movies), where items receiving non-zero ratings are considered as having positive feedback, i.e., denoted by a reward of 1; otherwise, the reward is 0. In total, there are  $T = 214729$  interactions, with each user having at least 3000 observations. By default, we set all clients' costs of data sharing as  $D_{i}^{p} = D_{\star}^{p} \in \mathbb{R}, \forall i \in [N]$ .
E.1 Payment-free vs. Payment-efficient incentive mechanism (Supplement to Section 5.1)
Figure 3: Comparison between payment-free vs. payment-efficient incentive designs.