SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Li Fan
lf2by@virginia.edu
Chao Tian
chao.tian@tamu.edu
Ruida Zhou
ruida@tamu.edu
Cong Shen
cong@virginia.edu
We study a federated linear bandits model, where  $M$  clients communicate with a central server to solve a linear contextual bandits problem with finite adversarial action sets that may be different across clients. To address the unique challenges of adversarial finite action sets, we propose the FedSupLinUCB algorithm, which extends the principles of SupLinUCB and OFUL algorithms in linear contextual bandits. We prove that FedSupLinUCB achieves a total regret of  $\tilde{O}(\sqrt{dT})$ , where  $T$  is the total number of arm pulls from all clients, and  $d$  is the ambient dimension of the linear model. This matches the minimax lower bound and thus is order-optimal (up to polylog terms). We study both asynchronous and synchronous cases and show that the communication cost can be controlled as  $O(dM^2 \log(d) \log(T))$  and  $O(\sqrt{d^3 M^3} \log(d))$ , respectively. The FedSupLinUCB design is further extended to two scenarios: (1) variance-adaptive, where a total regret of  $\tilde{O}(\sqrt{d \sum_{t=1}^{T} \sigma_t^2})$  can be achieved with  $\sigma_t^2$  being the noise variance of round  $t$ ; and (2) adversarial corruption, where a total regret of  $\tilde{O}(\sqrt{dT} + dC_p)$  can be achieved with  $C_p$  being the total corruption budget. Experiment results corroborate the theoretical analysis and demonstrate the effectiveness of FedSupLinUCB on both synthetic and real-world datasets.
In the canonical formulation of contextual bandits, a single player would repeatedly make arm-pulling decisions based on contextual information with the goal of maximizing the long-term reward. With the emerging federated learning paradigm (McMahan et al., 2017) where multiple clients and a server jointly learn a global model with each client locally updating the model with its own data and server only aggregating the local models periodically, researchers have started exploring contextual bandits algorithms in such federated learning setting (Dubey and Pentland, 2020; Huang et al., 2021; Li and Wang, 2022a,b). This federated contextual bandits framework broadens the applicability of contextual bandits to practical scenarios such as recommender systems, clinical trials, and cognitive radio. In these applications, although the goal is still to maximize the cumulative reward for the overall system, decision-making and observations are naturally distributed at the participating clients.
Several intrinsic challenges arise with the federated contextual bandit formulation. One important issue is that besides regret, we should also take into account the communication cost, which is usually the system bottleneck. To reduce the communication cost while maintaining the same regret guarantee, the clients should transmit the necessary information to the server only when the local
information has accumulated to the extent that it would affect the decision-making. Compared with the centralized contextual bandits, which have a linearly growing communication cost, algorithms for federated contextual bandits attempt to achieve a comparable regret with sub-linear communication cost.
Second, most existing studies on federated contextual bandits focus on the synchronous communication scenario (Huang et al., 2021; Li and Wang, 2022b), in which all participating clients first upload local information and then download updated global information from the server in each communication round. This stringent communication requirement is often not met in practice. A recent work of Li and Wang (2022a) studies the asynchronous federated linear bandit problem. However, communications for different clients are not independent in their approach because the upload from one client may trigger the server to perform downloads for all clients. To address this issue, He et al. (2022a) proposes FedLinUCB, which enables independent synchronizations between clients and the server.
Third, the majority of prior studies on federated linear bandits focused on the infinite-arm setting (Li and Wang, 2022b,a; He et al., 2022a) (see Section 2 for a detailed literature review). From a methodology point of view, these papers largely build on the OFUL principle (Abbasi-Yadkori et al., 2011). One notable exception is Huang et al. (2021), which studies synchronous communication with fixed contexts and proposes the Fed-PE algorithm based on the phased elimination G-optimal design (Lattimore and Szepesvári, 2020). To the best of our knowledge, no prior result exists for federated linear bandits with finite arms and time-evolving adversarial contexts, which is the focus of our work.
Table 1: Comparison of this paper with related works
System	Action	Algorithm	Regret	Communication
Single-player	infinite arm	OFUL (Abbasi-Yadkori et al., 2011)	d√T log T	N/A
Single-player	finite fixed arm	PE + G-optimal (Lattimore and Szepesvári, 2020)	O(√dT log T)	N/A
Single-player	finite adversarial arm	SupLinUCB (Chu et al., 2011)	O(√dT log3 T)	N/A
Federated (Async)	infinite arm	FedLinUCB (He et al., 2022a)	O(d√T log T)	O(dM2 log T)
Federated (Async)	infinite arm	Async-LinUCB (Li and Wang, 2022a)	O(d√T log T)	O(dM2 log T)
Federated (Sync)	infinite arm	DisLinUCB (Wang et al., 2019)	O(d√T log2 T)	O(dM3/2)
Federated (Sync)	finite fixed arm	Fed-PE (Huang et al., 2021)	O(√dT log T)	O(d2MK log T)
Federated (Async)	finite adversarial arm	FedSupLinUCB (This work)	O(√dT log3 T)	O(dM2 log d log T)
Federated (Sync)	finite adversarial arm	FedSupLinUCB (This work)	O(√dT log3 T)	O(d3/2 M3/2 log(d))
$d$ : the dimension of the unknown parameter,  $M$ : the number of clients,  $K$ : the number of finite actions,  $T$ : the total arm pulls from all clients.
Main contributions. Our main contributions are summarized as follows.
- We develop a general federated bandits framework, termed FedSupLinUCB, for solving the problem of federated linear contextual bandits with finite adversarial actions. FedSupLinUCB extends SupLinUCB (Chu et al., 2011; Ruan et al., 2021) and OFUL (Abbasi-Yadkori et al., 2011), two important principles in (single-player, centralized) linear bandits, to the federated bandits setting with a carefully designed layered successive screening.
- We instantiate FedSupLinUCB with both asynchronous and synchronous client activities. For the former setting, we propose Async-FedSupLinUCB where communication is triggered only when the cumulative local information impacts the exploration uncertainty to a certain extent. We prove that Async-FedSupLinUCB achieves  $\tilde{O}(\sqrt{dT})$  regret with  $O(dM^2\log d\log T)$  communication cost, which not only reduces the regret by  $\sqrt{d}$  compared with previous results on asynchronous federated linear bandits with infinite arms, but also matches the minimax lower bound up to polylog terms, indicating that Async-FedSupLinUCB achieves order-optimal regret.
- For synchronous communications, we propose Sync-FedSupLinUCB, which has a refined communication design where only certain layers are communicated, as opposed to the complete information. Sync-FedSupLinUCB achieves order-optimal regret of  $\tilde{O}(\sqrt{dT})$  with horizon-independent communication cost  $O(\sqrt{d^3M^3}\log d)$ . Compared with the best previous result (Huang et al., 2021) which achieves the same order-optimal regret but only for fixed actions, we show that it is the finite actions that fundamentally determines the regret behavior in the federated linear bandits setting.
- We further develop two extensions of FedSupLinUCB: (1) Variance-adaptive FedSupLinUCB, for which a total regret of  $\tilde{O} (\sqrt{d\sum_{t = 1}^{T}\sigma_{t}^{2}})$  is achieved, where  $\sigma_t^2$  is the noise variance at round  $t$ . (2)
Adversarial corruption FedSupLinUCB, for which a total regret of  $\tilde{O} (\sqrt{dT} +dC_p)$  is achieved, where  $C_p$  is the total corruption budget.
The linear bandit model, as a generalization of finite armed bandits with linear contextual information, has been extensively studied. The setting of infinite arm sets solved by LinUCB was analyzed in (Dani et al., 2008; Abbasi-Yadkori et al., 2011), which achieves regret  $\tilde{O}(d\sqrt{T})$  with appropriate confidence width (Abbasi-Yadkori et al., 2011) and matches the lower bound (Dani et al., 2008) up to logarithmic factors. In contrast, algorithms like SupLinRel (Auer, 2002) and SupLinUCB (Chu et al., 2011) achieve  $\tilde{O}(\sqrt{dT})$  in the setting of finite time-varying adversarial arm sets under  $K \ll 2^d$ , with a lower bound  $\Omega(\sqrt{dT})$  (Chu et al., 2011). The SupLinUCB algorithm was later optimized and matches the lower bound up to iterated logarithmic factors in Li et al. (2019). As a special case of the finite arm setting, if the arm set is time-invariant, an elimination-based algorithm (Lattimore and Szechesváři, 2020) via G-optimal design can be applied to achieve similar optimal performance.
The federated linear bandits problems were studied under the settings of infinite arm set (Dubey and Pentland, 2020; Li et al., 2020; Li and Wang, 2022a) and time-invariant finite arm set (Huang et al., 2021), while the time-varying finite arm set setting has not been well explored. A finite time-varying arm set has many meaningful practical applications such as recommendation system (Li et al., 2010; Chu et al., 2011), and the distributed (federated) nature of the applications naturally falls in the federated linear bandits problem with finite time-varying arms. The paper fills this gap by generalizing the SupLinUCB algorithm to the federated setting.
We study both the asynchronous setting (Li and Wang, 2022a) (He et al., 2022a), where clients are active on their own and full participation is not required, and the synchronous setting (Shi et al., 2021; Dubey and Pentland, 2020), where all the clients make decisions at each round and the communication round requires all the clients to upload new information to the server and download the updated information. We design algorithms so as to reduce the communication cost while maintaining optimal regret. Technically, the communication cost is associated with the algorithmic adaptivity, since less adaptivity requires fewer updates and thus fewer communication rounds. The algorithmic adaptivity of linear bandits algorithms was studied in the single-player setting (Han et al., 2020) (Ruan et al., 2021). It was also considered in the federated setting (Wang et al., 2019; Huang et al., 2021; Salgia and Zhao, 2023).
3 System Model and Preliminaries
We consider a federated linear contextual bandits model with  $K$  finite but possibly time-varying arms. The model consists of  $M$  clients and one server in a star-shaped communication network. Clients jointly solve a linear bandit problem by collecting local information and communicating with the central server through the star-shaped network in a federated manner, with no direct communications among clients. The only function of the server is to aggregate received client information and to send back updated information to clients. It cannot directly play the bandits game.
Specifically, some clients  $I_{t} \subseteq [M]$  are active at round  $t$ . Client  $i \in I_{t}$  receives  $K$  arms (actions to take) associated with contexts  $\{x_{t,a}^{i}\}_{a \in [K]} \subset \mathbb{R}^{d}$  with  $\| x_{t,a}^{i}\|_{2} \leq 1$ . Here we adopt the oblivious adversarial setting, where all contexts are chosen beforehand, and not dependent on previous game observation. Client  $i$  then pulls an arm  $a_{t}^{i} \in [K]$  based on the information collected locally as well as previously communicated from the server. A reward  $r_{t,a_{t}^{i}}^{i} = \theta^{\top} x_{t,a_{t}^{i}}^{i} + \epsilon_{t}$  is revealed privately to client  $i$ , where  $\theta \in \mathbb{R}^{d}$  is an unknown weight vector with  $\| \theta \|_{2} \leq 1$  and  $\epsilon_{t}$  is an independent 1-sub-Gaussian noise. At the end of round  $t$ , depending on the communication protocol, client  $i$  may exchange the collected local information with the server so that it can update the global information.
We aim to design algorithms to guide the clients' decision-making and overall communication behaviors. We analyze two patterns of client activity. 1) Synchronous: all  $M$  clients are active at each round. 2) Asynchronous: one client is active at each round. For the latter case, we further assume that client activity is independent of data and history. Denote by  $T_{i}$  the number of times client
$i$  is active. In the former case,  $T_{i} = T_{j}, \forall i, j \in [M]$ , while in the latter case,  $T_{i}$  may be different among clients. We define  $T = \sum_{i=1}^{M} T_{i}$  as the total number of arm pulls from all clients.
The performance is measured under two metrics - total regret and communication cost, which concern the decision-making effectiveness and the communication efficiency respectively. Denote by  $P_{T}^{i} = \{t\in [T]\mid i\in I_{t}\}$  the set of time indices at which client  $i$  is active, with  $|P_T^i| = T_i$ . The total regret is defined as
$$
R _ {T} = \sum_ {i = 1} ^ {M} R _ {T} ^ {i} = \sum_ {i = 1} ^ {M} \mathbb {E} \left[ \sum_ {t \in P _ {T} ^ {i}} r _ {t, a _ {t} ^ {i, *}} ^ {i} - r _ {t, a _ {t} ^ {i}} ^ {i} \right], \tag {1}
$$
where  $a_{t}^{i,*} = \arg \max_{a\in [K]}\theta^{\top}x_{t,a}^{i}$ . We define the communication cost as the total number of communication rounds between clients and the server.
Information encoding. In the linear bandits setting (federated or not), the information a client acquires is usually encoded by the gram matrix and the action-reward vector. Specifically, when the client has observed  $n$  action-reward pairs  $\{(x_{t},r_{t})\}_{t = 1}^{n}$ , the information is encoded by matrix  $A_{n} = \sum_{t = 1}^{n}x_{t}x_{t}^{\top}$  and vector  $b_{n} = \sum_{t = 1}^{n}r_{t}x_{t}$ . Denote by  $\mathrm{Encoder}(\cdot)$  this encoding function, i.e.,  $A_{n},b_{n}\gets \mathrm{Encoder}(\{x_{t},r_{t}\}_{t = 1}^{n})$ .
Communication criterion. Communication in our proposed framework is data-dependent, in the same spirit as the "doubling trick" introduced in Abbasi-Yadkori et al. (2011) to reduce the computation complexity in single-player linear bandits. The key idea is that communication is triggered only when the cumulative local information, represented by the determinant of the gram matrix  $A_{n}$ , affects the exploration uncertainty to a great extent and hence the client needs to communicate with the server. Detailed communication protocols will be presented in each algorithm design.
Synchronization procedure. Denote by  $\mathrm{Sync}(.)$  a routine that  $n$  clients (client 1, ..., client  $n$ ) first communicate their local gram matrices and action-reward vectors to the server, and the server then aggregates the matrices (and vectors) into one gram matrix (and action-reward vector) and transmits them back to the  $n$  clients. Specifically, each client  $i$  holds newly observed local information  $(\Delta A^i, \Delta b^i)$ , which is the difference between the client's current information  $(A^i, b^i)$  and the information after the last synchronization. In other words,  $(\Delta A^i, \Delta b^i)$  is the information that has not been communicated to the server. The server, after receiving the local information  $\{(\Delta A^i, \Delta b^i)\}_{i=1}^n$ , updates the server-side information  $(A^{ser}, b^{ser})$  by  $A^{ser} \gets A^{ser} + \sum_{i=1}^n \Delta A^i$ ,  $b^{ser} \gets b^{ser} + \sum_{i=1}^n \Delta b^i$  and sends them back to each of the  $n$  clients. Each client  $i$  will then update the local information by  $A^i \gets A^{ser}$ ,  $b^i \gets b^{ser}$ . The procedure is formally presented in Algorithm 1.
Algorithm 1 Sync(s, server, client 1, ... client  $n$ )
1: for  $i = 1,2,\dots ,n$  do  $\triangleright$  Client-side local information upload
2: Client  $i$  sends the local new layer  $s$  information  $(\Delta A_s^i, \Delta b_s^i)$  to the server
3: end for
4: Update server's layer  $s$  information:  $\triangleright$  Server-side information aggregation and distribution
$$
A _ {s} ^ {s e r} \gets A _ {s} ^ {s e r} + \sum_ {i = 1} ^ {n} \Delta A _ {s} ^ {i}, b _ {s} ^ {s e r} \gets b _ {s} ^ {s e r} + \sum_ {i = 1} ^ {n} \Delta b _ {s} ^ {i}
$$
5: Send server information  $A_{s}^{ser}$ ,  $b_{s}^{ser}$  back to all clients
6: for  $i = 1,2,\dots ,n$  do
7:  $A_{s}^{i}\gets A_{s}^{ser},b_{s}^{i}\gets b_{s}^{ser},\Delta A_{s}^{i}\gets 0,\Delta b_{s}^{i}\gets 0$ $\triangleright$  Client  $i$  updates the local information
8: end for
4 The FedSupLinUCB Framework
In this section, we present a general framework of federated bandits for linear bandits with finite oblivious adversarial actions. Two instances (asynchronous and synchronous) of this general framework will be discussed in subsequent sections.
Building block: SupLinUCB. As the name suggests, the proposed FedSupLinUCB framework is built upon the principle of SupLinUCB (Chu et al., 2011; Ruan et al., 2021). The information  $(A,b)$  is useful in the sense that the reward corresponding to an action  $x$  can be estimated within confidence interval  $x^{\top}\hat{\theta} \pm \alpha \| x\|_{A^{-1}}$ , where  $\hat{\theta} = A^{-1}b$ . It is shown in Abbasi-Yadkori et al. (2011) that in linear bandits (even with an infinite number of actions) with  $\alpha = \tilde{O}(\sqrt{d})$ , the true reward is within the confidence interval with high probability. Moreover, if the rewards in the action-reward vector  $b$  are mutually independent,  $\alpha$  can be reduced to  $O(1)$ . The former choice of  $\alpha$  naturally guarantees  $\tilde{O}(d\sqrt{T})$  regret. However, to achieve regret  $\tilde{O}(\sqrt{dT})$ , it is critical to keep  $\alpha = O(1)$ . This is fulfilled by the SupLinUCB algorithm (Chu et al., 2011) and then recently improved by Ruan et al. (2021). The key intuition is to successively refine an action set that contains the optimal action, where the estimation precision of sets is geometrically strengthened. Specifically, the algorithm maintains  $(S + 1)$  layers of information pairs  $\{(A_s, b_s)\}_{s=0}^S$ , and the rewards in the action-reward vectors are mutually independent, except for layer 0. The confidence radius for each layer  $s$  is  $w_s = 2^{-s}d^{1.5}/\sqrt{T}$ .
Algorithm 2 S-LUCB
1: Initialization:  $S = \lceil \log d\rceil$ $\overline{w}_0 = d^{1.5} / \sqrt{T},\overline{w}_s\gets 2^{-s}\overline{w}_0,\forall s\in [1:S].$    
2:  $\alpha_0 = 1 + \sqrt{d\ln(2M^2T / \delta)},\alpha_s\gets 1 + \sqrt{2\ln(2KMT\ln d / \delta)},\forall s\in [1:S]$    
3: Input: Client  $i$  (with local information  $A^i,b^i,\Delta A^i,\Delta b^i)$  , contexts set  $\{x_{t,1}^i,\dots,x_{t,K}^i\}$    
4:  $A_{t,s}^{i}\gets A_{s}^{i} + \Delta A_{s}^{i},b_{t,s}^{i}\gets b_{s}^{i} + \Delta b_{s}^{i}$  or  $A_{t,s}^{i}\gets A_{s}^{i},b_{t,s}^{i}\gets b_{s}^{i}$  for lazy update   
5:  $\hat{\theta}_{s}\leftarrow (A_{t,s}^{i})^{-1}b_{t,s}^{i},\hat{r}_{t,s,a}^{i} = \hat{\theta}_{s}^{\top}x_{t,a}^{i},w_{t,s,a}^{i}\gets \alpha_{s}\| x_{t,a}^{i}\|_{(A_{t,s}^{i})^{-1}},\forall s\in [0:S],\forall a\in [K]$    
6:  $s\gets 0;\mathcal{A}_0\gets \{a\in [K]\mid \hat{r}_{t,0,a}^i +w_{t,0,a}^i\geq \max_{a\in [K]}(\hat{r}_{t,0,a}^i -w_{t,0,a}^i)\}$  ▷ Initial screening   
7: repeat ▷ Layered successive screening   
8: if  $s = S$  then   
9: Choose action  $a_t^i$  arbitrarily from  $\mathcal{A}_S$    
10: else if  $w_{t,s,a}^{i}\leq \overline{w}_{s}$  for all  $a\in \mathcal{A}_s$  then   
11:  $\mathcal{A}_{s + 1}\gets \{a\in \mathcal{A}_s\mid \hat{r}_{t,s,a}^i\geq \max_{a'\in \mathcal{A}_s}(\hat{r}_{t,s,a'}^i) - 2\overline{w}_s\}; s\gets s + 1$    
12: else   
13:  $a_t^i\gets \arg \max_{\{a\in \mathcal{A}_s,w_{t,s,a}^i >\overline{w}_s\}}w_{t,s,a}^i$    
14: end if   
15: until action  $a_t^i$  is found   
16: Take action  $a_t^i$  and and receive reward  $r_{t,a_t^i}^i$    
17:  $\Delta A_s^i\gets \Delta A_s^i +x_{t,a_t^i}^i x_{t,a_t^i}^{i\top},\Delta b_s^i\gets \Delta b_s^i +r_{t,a_t^i}^i x_{t,a_t^i}^i$  ▷ Update local information   
18: Return layer index  $s$
FedSupLinUCB. S-LUCB, presented in Algorithm 2, combines the principles of SupLinUCB and OFUL (Abbasi-Yadkori et al., 2011) and is the core subroutine for FedSupLinUCB. We maintain  $S = \lceil \log d \rceil$  information layers, and the estimation accuracy starts from  $d^{1.5} / \sqrt{T}$  of layer 0 and halves as the layer index increases. Finally, it takes  $\Theta(\log d)$  layers to reach the sufficient accuracy of  $\sqrt{d/T}$  and achieves the minimax-optimal regret.
When client  $i$  is active, the input parameters  $(A^i, b^i)$  contain information received from the server at the last communication round, and  $(\Delta A^i, \Delta b^i)$  is the new local information collected between two consecutive communication rounds.  $\{x_{t,1}^i, \ldots, x_{t,K}^i\}$  is the set of contexts observed in this round. Client  $i$  can estimate the unknown parameter  $\theta$  either with all available information or just making a lazy update. This choice depends on the communication protocol and will be elaborated later. During the decision-making process, client  $i$  first makes arm elimination at layer 0 to help bootstrap the accuracy parameters. Then, it goes into the layered successive screening in the same manner as the SupLinUCB algorithm, where we sequentially eliminate suboptimal arms depending on their empirical means and confidence widths. After taking action  $a_t^i$  and receiving the corresponding reward  $r_{t,a_t^i}^i$ , client  $i$  updates its local information set  $(\Delta A_s^i, \Delta b_s^i)$  by aggregating the context into layer  $s$  in which we take the action, before returning layer  $s$ .
5 Asynchronous FedSupLinUCB
In the asynchronous setting, only one client is active in each round. Note that global synchronization and coordination are not required, and all inactive clients are idle.
We first initialize the information for all clients and the server (gram matrix and action-reward vector) in each layer  $s \in [0:S]$ . We assume only one client  $i_t$  is active at round  $t$ . It is without loss of generality since if multiple clients are active, we can queue them up and activate them in turn. More discussion of this equivalence can be found in He et al. (2022a); Li and Wang (2022a). The active client chooses the action, receives a reward, updates local information matrices of layer  $s$  with a lazy update according to S-LUCB, and decides whether communication with the server is needed by the criterion in Line 7 of Algorithm 3. If communication is triggered, we synchronize client  $i_t$  with the server by Algorithm 1.
Algorithm 3 Async-FedSupLinUCB
1: Initialization:  $T, C, S = \lceil \log d \rceil$   
2:  $\{A_s^{ser} \gets I_d, b_s^{ser} \gets 0 \mid s \in [0:S]\}$   
3:  $\{A_s^i \gets I_d, \Delta A_s^i, b_s^i, \Delta b_s^i \gets 0 \mid s \in [0:S], i \in [M]\}$   
4: for  $t = 1, 2, \dots, T$  do  
5: Client  $i_t = i$  is active, and observes  $K$  contexts  $\{x_{t,1}^i, x_{t,2}^i, \dots, x_{t,K}^i\}$   
6:  $s \gets S$ -LUCB ( $\text{client } i, \{x_{t,1}^i, x_{t,2}^i, \dots, x_{t,K}^i\}$ ) with lazy update  
7: if  $\frac{\operatorname*{det}(A_s^i + \Delta A_s^i)}{\operatorname*{det}(A_s^i)} > (1 + C)$  then  
8: Sync(s, server, clients  $i$ ) for each  $s \in [0:S]$   
9: end if  
10: end for
Theorem 5.1. For any  $0 < \delta < 1$ , if we run Algorithm 3 with  $C = 1 / M^2$ , then with probability at least  $1 - \delta$ , the regret of the algorithm is bounded as  $R_T \leq \tilde{O}\left(\sqrt{d\sum_{i=1}^{M}T_i}\right) = \tilde{O}\left(\sqrt{dT}\right)$ . Moreover, the corresponding communication cost is bounded by  $O(dM^2\log d\log T)$ .
Remark 1. The minimax lower bound of the expected regret for linear contextual bandits with  $K$  adversarial actions is  $\Omega(\sqrt{dT})$ , given in Chu et al. (2011). Theorem 5.1 indicates that AsyncFedSupLinUCB achieves order-optimal regret (up to polylog term) with  $O(dM^2\log d\log T)$  communication cost. To the best of our knowledge, this is the first algorithm that achieves the (near) optimal regret in federated linear bandits with finite adversarial actions.