Title: Federated Multi-armed Bandits with Efficient Bit-Level Communications
Venue: NeurIPS 2025
Accepted: yes
Keywords: federated multi-armed bandit

communication-efficient algorithm
Abstract: In this work, we study the federated multi-armed bandit (FMAB) problem, where a set of distributed agents collaboratively aim to minimize cumulative regret while interacting with a shared set of arms. Unlike traditional centralized bandit models, agents in FMAB settings are connected via a communication graph and cannot share data freely due to bandwidth limitations or privacy constraints. This raises a fundamental challenge: how to achieve optimal learning performance under stringent communication budgets.
We propose a novel communication-efficient algorithm that decouples the learning process into two phases: one for eliminating suboptimal arms through early and frequent communication of key decisions, and another for refining global estimates using buffered, quantized, and differentially transmitted statistics. By carefully balancing the communication frequency and precision of shared information, our algorithm achieves the optimal individual regret bound $O(N^{-1}\log T)$ while significantly reducing the total number of communication rounds and transmitted bits.
Theoretically, we derive tight upper bounds on both individual cumulative regret and group regret, and prove that our method asymptotically matches the lower bound of regret in federated settings. Experimental results on synthetic data validate the effectiveness of the proposed approach in various graph topologies and under heterogeneous feedback.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper studies the federated multi-armed bandit (FMAB) problem where N agents connected via a communication graph collaborate to identify the optimal arm while minimizing communication costs. The paper focuses on the heterogeneous setting where local reward distributions differ across agents. The main contribution is EpoInc-SE (Incremental Epoch-based Successive Elimination), an algorithm that combines epoch-based exploration, two-phase communication, and adaptive differential encoding to achieve near-optimal regret bounds while reducing communication to $O(\\sqrt{log T})$ total bits with $O(1)$ bits per message in expectation. The authors provide both upper and lower bounds for individual and group regret, claiming their algorithm is near-optimal.\n\nSTRENGTHS AND WEAKNESSES:\n# Strengths:\n\n- The paper addresses a practically relevant setting combining federated learning with communication efficiency, which is crucial for distributed systems with bandwidth constraints.\n- The adaptive differential encoding technique that reduces message size from $O(\\sqrt{log T})$ to $O(1)$ bits (in expectation) is creative and represents a genuine technical contribution to the federated bandits literature, to the best of my knowledge.\n- The separation of urgent information (arm elimination) from routine updates (estimates) is an elegant design principle that balances latency and efficiency.\n\n# weakness\n\n1. The paper could benefit from a clearer articulation of the precise problem it aims to solve. Currently, it addresses multiple aspects—communication efficiency, heterogeneity handling, optimal regret, and consensus estimation—without clearly identifying which is the primary contribution versus supporting components. This scattered presentation makes it challenging to evaluate the paper’s specific goals and their contribution to the existing literature. To strengthen the work, the authors are encouraged to explicitly state the primary problem, separate necessary technical elements from the key scholarly advancements, and include a comparison of communication costs with prior methods to highlight the approach’s improvements.\n2. in lines 40-42, Srikant & Ying study service‐fairness in queueing networks—not learning agents’ regret—so the citation is a domain mismatch. And the statement itself is not rigorous as \"fairness\" has been formalized in several distinct ways in the bandits literature.\n3. The O(1) bit communication claim is only in expectation, not worst-case, which is not clearly stated in the main results.\n4. The lower bound proof uses \"replica subsystems\" that fundamentally change the problem by giving each agent access to N times more information, making the comparison invalid.\n5. Experiments only use synthetic data with small problem instances (N=8, K=10). And there is no direct validation of the O(1) bit communication claim. The authors are encouraged to conduct ablation studies as the work involves multiple components.\n\nMinor\n\n- multiple grammar issues and typos (e.g., \"were\" vs “are” in line 50; \"with non-server\" in line 51; a space missing in line 52, etc.)\n\nQUESTIONS:\n1. what is the computational complexity?\n2. what are the implications of these different communication schemes on privacy?\n3. What is the worst-case number of bits required per message?\n\nLIMITATIONS:\nYes, they provided discussions to some limitations. More on computational complexity and privacy implications should also be discussed", "TITLE": "Official Review", "RECOMMENDATION": 3, "REVIEWER_CONFIDENCE": 2, "CLARITY": 1, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-06-12"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper addresses the Federated Multi-Armed Bandit (FMAB) problem. Under this problem setting, multiple agents aim to minimize cumulative regret while constrained by limited communication cost and privacy restrictions. In contrast to centralized bandit models, FMAB agents operate in a decentralized network, communicating only with neighbors, which makes effective coordination challenging. The paper proposes a new algorithm, EpoInc-SE (Incremental Epoch-based Successive Elimination), which is designed to balance communication cost with learning performance by incorporating early arm elimination, epoch-based exploration, and a quantized, differentially transmitted statistics mechanism. The paper shows that the algorithm achieves near-optimal regret both at the group and individual levels, and matches the theoretical lower bounds on regret in heterogeneous federated settings. EpoInc-SE is the first method to consider communication bit cost in such settings, introducing a two-phase communication strategy to prioritize important information and reduce overhead. Additionally, the authors propose a consensus estimator subroutine (EBCES) that allows agents to estimate global arm rewards accurately under distributed communication constraints. Theoretical analysis and empirical evaluations show the algorithm’s effectiveness, demonstrating good performance in both regret minimization and communication efficiency compared to existing approaches.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**\n\n**Quality**\n\nThe paper has a fine quality. The proposed algorithm, EpoInc-SE, is well-motivated and specifically designed to deal with the communication efficiency in federated bandit. The theoretical analysis covers both regret upper and lower bounds, and shows that the algorithm is near-optimal. \n\n**Clarity**\n\nThe paper is largely well-structured. On the theoretical part, mathematical concepts and definitions are well introduced. Section 3 presents a good introduction of the propose algorithm. \n\n**Significance**\n\nThe work addresses a meaningful and increasingly important problem—efficient distributed learning in the presence of communication constraints and heterogeneity. Federated bandit and distributed decision-making is an important research topic and applicable to various areas such as finance and network optimization. The paper's theoretical contribution is a good one. The practical implications of reducing communication to O(1) bits per message while preserving regret performance are substantial.\n\n**Originality**\n\nThe paper introduces several novel elements. It is, to the authors’ knowledge, the first to optimize bit-level communication in heterogeneous FMAB settings and to propose a lower bound on regret that explicitly incorporates agent heterogeneity. \n\n**Weaknesses**\n\n**Lack of Novel Analysis Techniques**\n\nThe theoretical analysis relies entirely on standard techniques like Hoeffding’s inequality, spectral graph theory, and existing lower bound arguments from prior work. It does not introduce new analytical tools or proof techniques. The regret decomposition into quantization, consensus, and concentration errors is standard, and the lower bounds are derived using known results in a relaxed model. For a theory-focused paper, this lack of technical novelty in the analysis is a notable weakness.\n\n**Minor Weakness**\n\nThe empirical evaluation is limited to synthetic experiments; performance on real-world federated datasets would make the results more compelling.\n\nQUESTIONS:\n1. What is the notable technical challenge in the theoretical analysis of the upper and/or lower bounds?\n\n2. The experiments are run on a small network (i.e., 8 agents). Any comment on how communication and regret behave in more realistic, large-scale deployments?\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-06-21"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors consider the problem of cumulative regret minimization in decentralized heterogeneous federated bandits with non-complete communication graphs. Agents can only communicate with their neighbors on the graph and aim to find the optimal arm across agents while having access to potentially diverse local expected rewards. They propose an algorithm called EpoInc-SE which features several aspects: epoch-based increasing uniform sampling of arms, modulated communication with distinct phases depending on the type of transmitted information, computation of quantized estimates and transmission of differential estimates across neighbors to reduce the communication cost, and refined elimination criterion to take into account quantization and consensus estimation errors in addition to the standard concentration inequality. They show that their algorithmic contribution has an improved upper bound on both the individual and group regrets compared to the state-of-the-art in heterogeneous bandits and asymptotically matching the lower bound, while considerably reducing the communication cost. Finally, experiments are performed on synthetic data compared to the baselines in heterogeneous and homogeneous, centralized and decentralized bandits.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths:**\n- Quality: The paper looks technically solid. The limitations mentioned in the paper make sense but should perhaps be featured in the main paper instead of the appendix.\n- Clarity: The paper is well-written, and its algorithmic and theoretical contributions are clear. \n- Significance: The proposed algorithm matches the lower bound for cumulative individual and group regret minimization (which the authors rederive), while considerably reducing the upper bound on the communication cost. These results seem confirmed by the experimental results in Section 5. In particular, this is the first algorithm for heterogeneous federated bandits matching those bounds.\n- Originality: The quantization and consensus algorithm for estimation are interesting, rather new contributions which required refined confidence bounds for the arm elimination procedure.\n\n**Weaknesses:**\n- Significance: The importance and the source of the agents’ weights in the communication graph is difficult to assess. Reading the appendix (Section B), it seems that this matrix is roughly fixed, up to a parameter $\\beta$ which exact value in practice is unspecified. I think this information should be present in the main text.\n\nQUESTIONS:\n- Can you discuss the importance and the source of the agents’ weights in the communication graph and its potential impact on the regret upper bounds? \n\nMinor concern: Ideally, the code should feature a README file to document which commands should be run for each figure in the paper.\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 4, "IMPACT": 3, "ORIGINALITY": 4, "DATE": "2025-06-26"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper addresses the Federated Multi-Armed Bandit (FMAB) problem in fully distributed networks with communication constraints. The main contribution is EpoInc-SE, a communication-efficient successive elimination algorithm that balances regret minimization with reduced bit-level communication. The algorithm incorporates (1) an epoch-based exploration strategy, (2) an adaptive differential communication scheme that transmits only quantized incremental updates, and (3) a consensus estimation routine to aggregate local statistics. Theoretically, the paper establishes tight upper and lower bounds on both individual and group regret, showing that EpoInc-SE achieves near-optimal performance. Empirically, it outperforms or matches baselines in regret while significantly reducing communication cost.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths:**\n- **Quality:** The paper’s main claims are supported by rigorous and well-structured proofs, demonstrating a solid theoretical foundation. Additionally, the paper includes a basic experimental evaluation that demonstrates the superiority of the proposed methods over existing baselines.\n- **Clarity:** The proposed algorithm and accompanying proof techniques are presented with intuitive explanations in the main text, and their effectiveness is validated through a basic experimental evaluation.\n- **Significance:** This work advances the state of the art by introducing a distributed algorithm for the heterogeneous FMAB setting with the lowest known communication cost, and with tight individual and group regret upper bounds. \n- **Originality:** Although the authors build on established techniques such as quantized communication [1], epoch-based communication [2] and successive elimination-based implicit synchronization [3], they employ them in a novel manner in the heterogeneous setting to achieve an algorithm with tight bounds and low communication cost. \n\n**Weaknesses:**\nRather than separating into broad quality, clarity, significance and originality categories, I will outline my main concerns in a more detailed manner below.\n- My primary concern with this work lies in its level of novelty. The contributions appear to be incremental, building upon established techniques in cooperative bandit literature—such as epoch-based communication [2], successive elimination with implicit synchronization [3], and running-consensus estimation [4]. That said, my current score reflects the view that this is a complete study in a new heterogeneous setting. It rigorously analyzes both group and individual regret under communication constraints, matches known lower bounds, and includes a basic experimental validation.\n- Implementing the proposed algorithm appears to require agents to have access to global properties of the communication graph, such as its diameter, the eigenvalues of the weight matrix, and whether the graph is balanced. This raises concerns regarding the practicality of the approach, particularly in fully distributed settings where such information may be unavailable or costly to obtain. These limitations should be explicitly discussed in the paper.\n- Furthermore, the analysis of the proposed algorithm relies heavily on implicit synchronization among agents when selecting arms. This dependence raises additional concerns about the algorithm’s applicability in more realistic settings, such as those involving unreliable communication or asynchronous decision-making. \n\nOther issues –\n- The authors highlight low communication cost as a key contribution of their work. However, it remains unclear whether the stated $O(\\sqrt{\\log T})$ bit complexity is optimal, as the paper does not provide a corresponding lower bound. Including such a bound or discussing the potential for further improvement would strengthen the contribution.\n- While the intuitive explanation of the algorithm in Section 3 is helpful, the use of numerous similar notations in this section makes the presentation difficult to follow. Additionally, Section 2 would benefit from revision to improve clarity.\n- Unfortunately, the main text does not include proof outlines for Theorems 1, 2, and 3. Given that this is primarily a theoretical contribution, providing such outlines is important for guiding readers through the core ideas of the analysis. I strongly recommend including them in a revised version of the paper.\n- Additional proofreading would improve the paper. For example, the meaning of the parameter $a$ should be explained at its first appearance (line 196). In line 197, “epochs” should be used instead of “epoch.” Referring to the global estimator in line 203 before its formal definition is confusing and should be avoided. In equation (5), the quantized estimators of neighbors should be used instead of the true estimators, which are not available to individual agents. Line 260 describes a sample count bound, but Lemma 1 provides a concentration bound for the means. Lines 266–267 contain unnecessary repetition, and the reference to Section 3 in line 300 appears to be misplaced. Finally, in line 305, the notation “C” should be used instead of “c”.\n\n[1] Shi, Chengshuai, et al. \"Heterogeneous multi-player multi-armed bandits: Closing the gap and generalization.\" Advances in neural information processing systems 34 (2021): 22392-22404.\\\n[2] Wang, Yuanhao, et al. \"Distributed bandit learning: Near-optimal regret with efficient communication.\" arXiv preprint arXiv:1904.06309 (2019).\\\n[3] Yang, Lin, et al. \"Cooperative multi-agent bandits: Distributed algorithms with optimal individual regret and constant communication costs.\" arXiv preprint arXiv:2308.04314 (2023).\\\n[4] Martínez-Rubio, David, Varun Kanade, and Patrick Rebeschini. \"Decentralized cooperative stochastic bandits.\" Advances in Neural Information Processing Systems 32 (2019).\n\nQUESTIONS:\n1. Given my concerns regarding the novelty of the work, could you clarify the main technical challenges involved in proving the regret and communication upper bounds presented in Theorems 1 and 2?\n2. Additionally, in Theorem 3, what distinguishes your proof of the regret lower bounds from the arguments of a similar lower bound for the single-agent case in [1], which your approach seems to closely follow? What are the key technical hurdles specific to your setting?\n3. What are the main obstacles to establishing a lower bound for the communication cost? Do you believe the current upper bound of $O(\\sqrt{\\log T})$ is tight, or is there potential for further improvement?\n4. In Section 5, you mention incorporating a consensus estimation module into Fed2-UCB and UCB-TCOM, but the details are not elaborated. Could you provide more information on this?\n5. In lines 111–112, the global rewards are defined as sums over agents' local rewards, whereas the global means are defined as averages of the agents’ local means. Is this intentional?\n\n[1] Lattimore, Tor, and Csaba Szepesvári. Bandit algorithms. Cambridge University Press, 2020.\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-06-30"}