Paper: ZGS-Based Event-Driven Algorithms for Bayesian Optimization in Fully Distributed Multi-Agent Systems (ICLR-2024)

Reviewer comment (Experiments):
**Experiments do not compare to the most relevant baselines, and results are not presented well.** The paper makes mention of [1] and [2], and the idea of using RFF to share information in a parametric manner is borrowed from [1]. In a BO context, [1] is clearly the closest work. However, the experiments do not compare against the algorithms from [1] and [2], and instead compare against existing algorithms in the distributed learning (DL) literature. This is a problematic choice given that, in the introduction, the authors claim that "the existing DL works ... do not consider the expensive black-box optimization problems utilizing limited data", so why only use them for the experiments and exclude the directly relevant algorithms from [1] and [2]? This weakens the empirical support for the proposed algorithm. Furthermore, the authors present the number $R_T/T$ without error bars in tables instead of graphs of the per iteration cumulative regret as is standard for BO works in order to compare the convergence over time.

Author response:
(2). Thanks a lot for reading the reference paper "Dai et al. (2020)", which is [1] in your mark. The reference paper [2] you mark is "Dai et al. (2021)" in our reference list. Although both tree-structured federated learning (FL) and decentralized (fully) distributed learning (DL) are distinct distributed machine learning paradigms, FBO in FL and DBO in DL are distinct paradigms. FBO/FL algorithms cannot be implemented in the DBO/DL setting. We reference Dai et al. (2021; 2020), however, it's not feasible to conduct comparative experiments using the algorithms presented in those references. We introduce and investigate the novel DBO paradigm, and we present the first DBO algorithm, ZGS-ED-DLCB. No prior DBO algorithms have been developed before. Similar to the comparative experiments in Dai et al. (2020), well-known DL algorithms including the DCL, ATC-LMS, CTA-LMS and ADMM-based algorithms are adjusted to adapt to the DBO setting as the baselines in the comparative experiments. 

Moreover, in the FBO experiments, none of the baselines compared in the experiments are federated/distributed BO. The following is a direct quote from the NeurIPS reference paper Dai et al. (2020): "Although FTS is the first algorithm for the FBO setting, some algorithms for meta-learning in BO, such as ranking-weighted GP ensemble (RGPE) [16] and transfer acquisition function (TAF) [55], can be adapted to the FBO setting through a heuristic combination with RFF approximation.". Drawing inspiration from the experimental design in Dai et al. (2020), we have constructed comparative experiments for the first DBO algorithm.

Reviewer comment (Writing):
I have to say the paper is not well written. It suffers from a lack of clarity in terms of writing, notation, and explanation, making it challenging to follow. From the problem formulation to the theoretical results, several essential details are omitted. Understanding the context is difficult, necessitating multiple visits to the appendices. I would suggest the authors maintain the core message within the main paper while relocating unnecessary sections to the appendix.

Author response:
We appreciate your feedback regarding our writing and presentation. We have revisited our manuscript with a fresh perspective and revised it. If you are interested, the final version will deliever the revised manuscript.

Reviewer comment (Theory):
Furthermore, the main technical contributions remain poorly explained, with insufficient references to existing literature. High-level intuitions are well presented. It is hard for me to evaluate the theoretical contribution.

Author response:
We have revisited our manuscript with a fresh perspective and revised it. DBO is a novel paradigm that we introduce and investigate, no prior DBO algorithms or exist. Could you please display the additional reference papers you think that we should reference to us?

Reviewer comment (Experiments):
Regarding the contributions you claim, it is unclear how your algorithm tracks expensive-to-evaluate black-box optimization. Additionally, the paper does not adequately demonstrate how your algorithm handles limited data and how it compares to existing approaches. The privacy preservation aspects are mentioned, but the extent and specific mechanisms are not detailed.

Author response:
In our paper, the data samples used in the whole experiments are limited to each agent. Compared with the traditional centralized BO, the decentralized learning framework in the DBO setting has the following advantages in terms of privacy: 
  (a) Data Localization: In fully distributed learning, each node processes its own data locally and shares only model updates (like weight updates or gradients), not the raw data. This means that raw data doesn't need to propagate through the network, reducing the risk of data leaks.
  (b) Reduced Centralization Risk: In centralized learning, all data is brought to a central node for processing. If this central node is compromised, all data could potentially be exposed. In fully distributed learning, there's no such central node, reducing the overall attack surface of the system.
  (c) Regulatory Compliance: In some cases, the storage and processing of data may be subject to strict regulatory requirements, such as GDPR. In these cases, distributed learning allows for data to be processed locally, making it easier to meet these regulatory requirements.

Reviewer comment (Writing):
The line immediately following equation (11) introduces $\hat{W}_t^i$, but this term is not defined.

Author response:
The latest transferred knowledge of agent $i$ is denoted as $\hat{{W_t^i}}(k)=W_{t}^{i}(k_{t,i}^{m_{i}})$. We have revisited our manuscript with a fresh perspective and revised it.

Reviewer comment (Evaluation):
The introduction of the notion of "regret" raises questions as the paper does not present the associated results.

Author response:
We are the first to present nontions of regret in the DBO setting. In the synthetic experiments, the evaluation metric $R_T/T$, which is associated with the global cumulative regret $R_T$, is used to compare the learning performance of ZGS-ED-DLCB with the baselines is in the paper.

Reviewer comment (Writing):
Section 3.2, I think the presentation of this section can be further polished. Below equation (11), what is \( \hat{W} \)? The estimated \( W \) since the last information exchange? In the next paragraph, what is a "sub-iteration"? It is never mentioned that the proposed method has an inner loop with sub-iterations. Remark 2: again it's confusing to me why the parameters are time-varying.

Author response:
$\hat{{W_t^i}}(k)$ is the local transferred weight vector at the $k$th sub-iteration step during the $t$th iteration step. To illustrate our ZGS-ED-DLCB algorithm more detailedly, the pseudocode is shown in App. C. From the pseudocode, we can see the the update of $W_{t}^{i}(k)$ includes $T$ iterations from 1 to $t$, of which each includes $K$ sub-iterations from 0 to $K$ in Stage 1. Thus, $t$ indexes the iteration, $k$ indexes the sub-iteration. The parameters are time-varying because lcoal data samples are actually increased after each iteration step indexed by $t$ and the parameters corresponding to the calculation process of ZGS-ED-DLCB in each iteration step are changed with the index $t$.

Reviewer comment (Writing):
Section 3.3: I find this section not easy to understand. Perhaps it will help understanding if more intuitions/descriptions are given in addition to the equations.

Author response:
In response to your concerns, we have revisited our manuscript with a fresh perspective and revised it. If you are interested, the final version will deliever the revised manuscript.

Reviewer comment (Theory):
Stage 2 in Section 3.3: it says here that based on the \( W^{i^*}_t \), each agent can calculate the Gaussian process posterior mean and variance. But in my understanding, with a single parameter vector \( W \), we can only calculate a single function value at an input (see equation 4). To be able to calculate the GP posterior mean and variance, we need the entire distribution of \( W \) (see Dai et al., (2020)). Please give the exact equations using which the GP posterior mean and variance are calculated.

Author response:
RFF are utilized to approximate a typical, so the approximated GP posterior mean and variance are calculated in this paper. Similar to the work Dai et al., (2020), the posterior mean and variance are calculated according to formulae (6) and (7) in App .B of Dai et al., 2020. The corresponding computing formula for our ZGS-ED-DLCB in the DBO setting will be shown in our revised manuscript.

Reviewer comment (Writing):
Section 4.1: I also find this section not easy to understand.

Author response:
In response to your concerns, we have revisited our manuscript with a fresh perspective and revised it. If you are interested, the final version will deliever the revised manuscript.