Differentially Private Federated Bayesian Optimization with Distributed Exploration

Abstract: Bayesian optimization (BO) has recently been extended to the federated learning (FL) setting by the federated Thompson sampling (FTS) algorithm, which has promising applications such as federated hyperparameter tuning. However, FTS is not equipped with a rigorous privacy guarantee which is an important consideration in FL. Recent works have incorporated differential privacy (DP) into the training of deep neural networks through a general framework for adding DP to iterative algorithms. Following this general DP framework, our work here integrates DP into FTS to preserve user-level privacy. We also leverage the ability of this general DP framework to handle different parameter vectors, as well as the technique of local modeling for BO, to further improve the utility of our algorithm through distributed exploration (DE). The resulting differentially private FTS with DE (DP-FTS-DE) algorithm is endowed with theoretical guarantees for both the privacy and utility and is amenable to interesting theoretical insights about the privacy-utility trade-off. We also use real-world experiments to show that DP-FTS-DE achieves high utility (competitive performance) with a strong privacy guarantee (small privacy loss) and induces a trade-off between privacy and utility.


Reviews and author responses:

This paper proposes a differentially private version of FTS-DE algorithm. Theoretically, the authors provide not only the privacy guarantee of the proposed algorithm, but also provide the privacy-utility trade-off. Experimentally, the impact of different parameters is verified. The results match the privacy-utility trade-off theorem in the theoretical section.  Since I am not an expert in Federated Learning or Bayesian Optimization. My evaluation focus on the differential privacy part of this paper.

The main theorems of this paper express clear messages about the proposed algorithms. With this theorem, engineering scientists can set their desired privacy level according to the utility requirement, which is meaningful.

The related works are not well-discussed. There is a lot of literature on federated learning + DP. For example,
1. Federated learning with differential privacy: Algorithms and performance analysis
2. Local differential privacy-based federated learning for internet of things
3. Personalized federated learning with differential privacy
4. A hybrid approach to privacy-preserving federated learning
How is your work related to/different from those works?

I tried my best to understand the BO and FBO part in the background section, but I am still not able to understand them. In line 95, what does $f_{t+1}$ means and how is it related with the objective function $f$?  In line 100, how is the random feature $\phi$ generated? It seems like $\phi$ is a function of $x$ and cannot be totally at random. In the FBO, the notation $f^{n}$ also looks confusing. How is it related with $f_t$ and $f$? I totally gave up on understanding the settings of FBO here.


--------------------------
I have read the feedback from the authors. I will change my score from 4 to 5 because writing is not a big problem for the expert in the community of FL, but I think the presentation still needs to be improved if the authors seek broader impacts. I think there is no potential negative social impact of their work

 Thank you for the clarifications, in particular about $\phi_{max}$ and the averaging contribution.



The paper studies the Bayesian optimization problem in the differentially private federated learning setting. The authors integrate DP into the federated Thompson sampling (FTS) algorithm and  propose a distributed exploration technique to accelerate exploration in the early stages.
  Clarity: Though the paper is generally well written, the writing of the algorithm part can be improved. I find it hard to fully understand the algorithm.

Strong points:
The DP extension of FTS is of interest to the ML community though I do not know much about Bayesian Optimization.

Weak points:
The paper does not have sufficient algorithmic novelty compared to the prior work [10], and the extension to the DP setting seems to be straightforward under the DP-FedAvg framework. The privacy guarantee comes from DP-FedAvg directly, and the utility guarantee builds upon the proof from [10]. From my understanding, the main novelty compared to [10] is the distributed exploration. 

The second concern is about the correctness of privacy guarantees. On page 3 line 113, the authors claim that ``all participating agents share the same set of random features follow [10]’’. However, [10] assumes sharing the first layer of a neural network between all clients (see Appendix A in [10]), which violates the privacy requirement. 

Given the limited algorithmic novelty and the privacy guarantee issues (please correct me if my understand is wrong), I tend to reject the paper. 
 The authors have addressed the negative societal impact.

 Thanks for the clarifications on the correctness.  If I understand it correctly, the $\phi(\cdot)$ is a data-independent feature extractor. If this is the case, my privacy concern is solved. I have updated the score to reflect it. 

 We would like to thank the reviewer for appreciating our contributions and your thoughtful comments.

***

&nbsp;


> (1) In the evaluation, the privacy budget is as high as 20. Such high privacy cost does not provide strong privacy guarantee. Is it possible to achieve similar performance with smaller privacy budget?

We agree with the reviewer that a privacy budget of 20 is not practical. However, in our real-world experiments (Fig. 3 (a-c)), the vast majority of the settings we have evaluated have privacy losses within the single-digit range (i.e., $\epsilon < 10$), which provide reasonable privacy protection [1]. Moreover, all of these settings lead to competitive performances because they are able to outperform standard local TS by a large margin.

&nbsp;

> (2) I am not sure if the comparison in the evaluation is fair. Seems the federated version has access to more data points? How does it compare to centralized version with all the data points? It will be good to provide it as a baseline to compare with.

Rather than having access to more data points, our algorithm can access and exploit the additional information that is **naturally available in the federated setting**. In particular, the federated setting allows information to be shared among different agents without requiring the agents to share their raw data, which is satisfied by our algorithm. Therefore, our empirical comparisons are fair because they demonstrate that our algorithm can leverage these naturally available information to achieve the benefit of the federated setting. The centralized version with all the data points is infeasible because (a) it requires the sharing of the raw data of every client which violates the requirement in the federated setting as mentioned above, and (b) the objective functions of different clients can be (potentially highly) different so it will require some special treatments to the centralized BO algorithm to account for the differences.


 We would like to thank the reviewer for your careful review and feedback.
***
> Clarity: Though the paper is generally well written, the writing of the algorithm part can be improved. I find it hard to fully understand the algorithm.

We will revise the algorithm part to make it easier to understand.

&nbsp;

> Weak points: The paper does not have sufficient algorithmic novelty compared to the prior work [10], and the extension to the DP setting seems to be straightforward under the DP-FedAvg framework. The privacy guarantee comes from DP-FedAvg directly, and the utility guarantee builds upon the proof from [10]. From my understanding, the main novelty compared to [10] is the distributed exploration.

Although the FTS algorithm and the general DP framework have been proposed by [10] and DP-FedAvg [28] respectively, one of our major novel contributions in fact lies in the **theoretical analysis (Section 4) and empirical demonstration (Section 5) of the privacy-utility trade-off** of our proposed DP-FTS-DE algorithm, which are non-trivial and have never been done by previous works. Moreover, as the reviewer has also pointed out, another major novel contribution of our work is the method of distributed exploration (DE), which arises from **the interesting coupling between FTS and the general DP framework** and has been shown to be critical for our algorithms to achieve competitive performances.

&nbsp;

> The second concern is about the correctness of privacy guarantees. On page 3 line 113, the authors claim that “all participating agents share the same set of random features follow [10]”. However, [10] assumes sharing the first layer of a neural network between all clients (see Appendix A in [10]), which violates the privacy requirement.

Regarding the correctness of the privacy guarantee, we would like to clarify that [10] does not assume sharing the first layer of a neural network between all clients. In fact, [10] also assumes sharing the same set of random features (which is the same as our paper), and Appendix A in [10] only claims that the sharing of random features is **equivalent to** sharing the first layer of a neural network. Therefore, the claim made by [10] only aims to draw **an analogy between the sharing of random features and neural networks**, in order to justify that sharing random features among agents can be easily achieved. As a result, sharing random features does not violate the privacy requirement because our algorithm aims to preserve the user-level privacy and the random features contain no identifying information about any user/client. We will revise the paper and make this more explicit to avoid any potential confusion.

&nbsp;

***

Thank you again for your feedback and we hope our clarifications of your misunderstandings could improve your evaluation of our paper.


 We would like to thank the reviewer for your careful review and insightful comments, which we will follow to improve our paper.

***
&nbsp;

> For the clipping value p. 6, does the bound have to depend on $\varphi_{\max}$, or could it be on something like $|| \varphi ||_2$, which should be smaller?

Regarding the clipping value on page 6, if we understand correctly, by $|| \varphi ||\_2$, the reviewer refers to viewing the weights for all agents and all sub-regions as a vector $\varphi$ and calculating its L2 norm. We would like to clarify that because $\varphi_{\max}$ corresponds to the infinity norm of this vector $\varphi$, so $\varphi_{\max}$ is in fact smaller than its L2 norm $|| \varphi ||_2$.

&nbsp;

> What are the assumptions under which the utility guarantee is derived? It would be nice to state and discuss them in the core of the paper, or at least in the appendix in a more centralized fashion before the proofs. 

Our assumptions in the theoretical analysis are all common assumptions in the theoretical analysis of BO, and our most important assumption is that the objective functions of all agents have a bounded norm induced by the RKHS associated with the kernel $k$ (lines 115-116). This is a common smoothness assumption in the analysis of BO algorithms [6], and has also been adopted by the previous work on FBO [10]. We will discuss our assumptions in a more centralized fashion following your suggestion.

&nbsp;

> The utility guarantee also seems to mostly ignore (even degrade) due to the federated approach, and hence mostly rely on local GP results. This is a bit disappointing, as in practice the federated part is supposed to help. It’d be useful to see an analysis of when the federated learning can improve utility, and under which conditions it happens. 

The reason why our utility guarantee does not reflect the benefit of the federated approach is because in order to derive a theoretical guarantee that is robust against the heterogeneity of agents, we have upper-bounded the worst-case error induced by the use of any set of agents (including those agents who are extremely heterogeneous and hence not helpful for the convergence at all) in our theoretical analysis (lines 255-260). This is consistent with most existing works proving the convergence of FL algorithms [24, 25] and makes our theoretical results more generally applicable because the presence of heterogeneous agents is one of the most prominent challenges in the federated setting. We will explore the possibility of extending our theoretical results to prove the benefit of the federated approach in future works, as the reviewer has suggested.

&nbsp;

> It would also be nice to at least add the intuition of why the utility guarantee holds despite averaging other contributions in the core of the paper, and a discussion of why this is useful in practice (and when it might not be), as opposed to the typical approach of choosing a contribution at random.

The utility guarantee holds despite averaging over the contributions from different clients because in the theoretical analysis, we have managed to upper-bound the errors introduced by the use of the weighted average, which is achieved by considering the difference between the target agent and every agent involved in the weighted averaging. If we understand correctly, by “the typical approach of choosing a contribution at random”, the reviewer refers to randomly sampling an agent and only relying on the contribution from the sampled agent, which is what the original FTS [10] has done. The weighted averaging is in fact an essential step in the general DP framework (line 134), and is hence required in order for the general DP framework to be applicable in our setting. As a result, the original FTS [10] which chooses a contribution at random is not applicable in our setting since it is not amenable to a straightforward integration of the general DP framework (lines 147-150). This is because the original FTS requires an agent to receive the contributions from all agents, so the transformations (including the weighted averaging) required by the general DP framework cannot be incorporated.

&nbsp;


> What is the finial outcome for each client? Do they just keep their last posterior, and the federated approach was used exclusively to guide sampling/exploration? If that is indeed the case, it would be useful to show the computational improvement empirically in the evaluation (that is, when does TS reaches the same level of performance as FTS, or at least the DP versions). The evaluation should then give the epsilon/delta values used is different experiments, and this could be contrasted with the computational gains.

After the completion of our DP-FTS-DE algorithm, every client can choose to continue running their local  BO task using standard TS. In this case, our federated approach can be viewed as accelerating the convergence in the early stage by guiding exploration. As you have suggested, we added an empirical evaluation to compare the computational improvement and the privacy loss. Specifically, using the human activity recognition experiment (corresponding to Fig. 3 (b) and (e)), we continued running the standard TS after the termination of DP-FTS-DE after 60 iterations, and evaluated how many additional iterations is required for TS to reach the same level of average performance of some settings of DP-FTS-DE. We have set $\delta=1/N^{1.1} \approx 0.030$ for all settings (line 302), and the table below presents the value of $\epsilon$ (i.e., the privacy loss) as well as the number of required additional iterations for TS (i.e., the computational improvement). The table shows that a large number of additional iterations (around $40$) is required for TS to reach the same level of performance as DP-FTS-DE, which demonstrates a large advantage of DP-FTS-DE in terms of computational improvement. Moreover, it also shows that a smaller privacy loss in general leads to smaller computational gains (i.e., less required additional iterations for TS), which is reasonable since a smaller privacy loss usually results in worse utility for DP-FTS-DE.

| Method | $\epsilon$ | required additional iterations for TS |
| :---: | :---: | :---: |
| DP-FTS-DE ($q=0.35,z=1.6$) | $7.15$ | $40$ |
| DP-FTS-DE ($q=0.35,z=2.0$) | $5.16$ | $38$ |
| DP-FTS-DE ($q=0.35,z=3.0$) | $3.06$ | $46$ |
| DP-FTS-DE ($q=0.35,z=4.0$) | $2.17$ | $33$ |

&nbsp;



> In the evaluation, it would be interesting to see more complex problems (e.g. deep learning hyper-parameter tuning). How will the proposed solution fare under higher dimensional input spaces? Similarly, what data sizes / change in the local loss can be supported before improvements vanish?

We agree with the reviewer that it is interesting to apply our algorithm to more complex problems.   
For problems with higher-dimensional input spaces, we can incorporate existing techniques on high-dimensional BO into our algorithm. For example, we can project the high-dimensional input space into lower dimensions following existing works on high-dimensional BO via random embedding (such as reference [a] below), or assume an additive structure of the high-dimensional objective function [b]. In fact, same as our paper, reference [b] is also based on random features and TS, so it is indeed an interesting topic to explore the interplay between our algorithm and high-dimensional BO methods such as [b] as the reviewer has suggested.

If we understand correctly, by “data sizes”, the reviewer is referring to when the size of the local data of an agent is large such that the benefit of running the federated algorithm vanishes. This is in fact only applicable in FL yet not applicable in our FBO setting, because having more local data cannot improve the efficiency of local BO for hyperparameter tuning.

For “change in the local loss”, if our interpretation is accurate, the reviewer is referring to when the objective functions of different clients are significantly different. Our theoretical result (Theorem 1) ensures our robustness against the heterogeneity of agents, i.e., our algorithm is no-regret even if the objective functions of different agents are highly different. It is indeed interesting to evaluate this via empirical experiments and we will explore this in future works.

[a] Re-Examining Linear Embeddings for High-Dimensional Bayesian Optimization.   
[b] Efficient High Dimensional Bayesian Optimization with Additivity and Quadrature Fourier Features.

&nbsp;

***

Thanks again for your comments and we hope that our clarifications and additional experimental results could improve your opinion of our paper.


 * We would like to thank the reviewer for pointing out these related works. We will reference them in the Related Works section after revision.

* In line 95, $f_{t+1}$ is a function that is sampled from the posterior GP surrogate model, and the posterior GP surrogate model represents our current belief about the objective function because it is calculated (i.e., Equation (1)) using the first $t$ observations of the objective function $f$.

    In line 100, the random features are generated using standard techniques from the classic work of [30], and the same technique has been adopted by the work on FBO [10]. We omitted the details on how to generate the random features due to lack of space, but we will add these details to the paper after revision to make the paper more self-contained. 

    Throughout this paper, we have used a superscript $n$ to indicate that the function is associated with agent $n$ and used a subscript $t$ to indicate that the function is sampled from the GP posterior in iteration $t$. For example, $f^n$ represents the objective function of agent $n$, and $f^n_t$ represents a function that is sampled from the posterior GP surrogate of agent $n$ in iteration $t$. To clarify, the notations $f$ and $f_t$ (without superscripts) are only used to introduce the background on Bayesian optimization, therefore, in the FBO setting (hence in most of the paper), a superscript $n$ is needed to indicate which agent the function is associated with (i.e., $f^n$ and $f^n_t$ are associated with agent $n$).

    We will go through all notations to check for potential confusions and inconsistencies, and revise them accordingly.


This paper proposes an approach to build differential privacy (DP) into federated bayesian optimization, in which users try to find high performing values (e.g. hyper-parameters) for a given scoring function (e.g. accuracy of a trained model) by sampling new values to try from the posterior of a Gaussian process mapping the values to their score. The proposed algorithm adapts an existing federated solution to make it amenable to the typical approach to making iterative procedures DP, imports input space splitting heuristics to improve both utility and privacy, and analyse the worst case utility guarantee of the approach.
  The paper studies and important setup, in which users privately cooperate to solve local hyper-parameter tuning with reduced computational costs. The technical solution is not very novel in the sense that the mapping of existing federated bayesian optimization schemes to popular iterative DP schemes is fairly straight forward, but the results are good. The paper would gain to expanding some of the contributions further.

For the clipping value p. 6, does the bound have to depend on $\varphi_{max}$, or could it be on something like $|| \varphi ||_2$, which should be smaller?

What are the assumptions under which the utility guarantee is derived? It would be nice to state and discuss them in the core of the paper, or at least in the appendix in a more centralized fashion before the proofs.
The utility guarantee also seems to mostly ignore (even degrade) due to the federated approach,  and hence mostly rely on local GP results. This is a bit disappointing, as in practice the federated part is supposed to help. It’d be useful to see an analysis of when the federated learning can improve utility, and under which conditions it happens.
 It would also be nice to at least add the intuition of why the utility guarantee holds despite averaging other contributions in the core of the paper, and a discussion of why this is useful in practice (and when it might not be), as opposed to the typical approach of choosing a contribution at random.

What is the finial outcome for each client? Do they just keep their last posterior, and the federated approach was used exclusively to guide sampling/exploration? If that is indeed the case, it would be useful to show the computational improvement empirically in the evaluation (that is, when does TS reaches the same level of performance as FTS, or at least the DP versions). The evaluation should then give the epsilon/delta values used is different experiments, and this could be contrasted with the computational gains. 

In the evaluation, it would be interesting to see more complex problems (e.g. deep learning hyper-parameter tuning). How will the proposed solution fare under higher dimensional input spaces? Similarly, what data sizes / change in the local loss can be supported before improvements vanish?
 N/A

This paper propose DP-FTS-DE, the first differentially private federated Bayesian optimization protocol. To augment FTS with DP, the paper introduces a centralized server and tweaks the workflow so that the protocol follows a similar pipeline like FL. Hence, the standard process of clipping and adding noise can be used to add DP. The paper also evaluate the proposed protocol both theoretically and empirically. The empirical results show that the information from other clients accelerates convergence.  The work augments FTS with DP by modifying the FTS workflow and applying standard process of clipping and adding noise. It further introduces distributed exploration to accelerate convergence.

The idea of distributed exploration is innovative and works well as shown in the evaluation. The novelty on the privacy side is limited because the standard process is directly applied. The tweak on the BO side to enable the direct application, on the other side, is not trivial but also not very challenging.

The paper is complete, providing both theoretical utility guarantee and empirical evaluation.

The paper is well-written and easy to follow. 

Some minor points:
(1) In the evaluation, the privacy budget is as high as 20. Such high privacy cost does not provide strong privacy guarantee. Is it possible to achieve similar performance with smaller privacy budget?
(2) I am not sure if the comparison in the evaluation is fair. Seems the federated version has access to more data points? How does it compare to centralized version with all the data points? It will be good to provide it as a baseline to compare with.

 Yes, the authors adequately addressed the limitations and potential negative societal impacts.

Meta-review: This paper combines differential privacy with federated Bayesian optimization using the federated Thompson sampling algorithm. We appreciate this research direction, but find that the paper falls below the (high) bar for NeurIPS for a combination of the following reasons.
- The technical novelty of the contribution is limited. The proposed algorithm uses the now standard approach to differentially private federated learning of reducing the problem to iteratively averaging vectors, clipping them, and adding Gaussian noise. The privacy and utility analyses seem to closely mirror prior work.
- Several reviewers were uncertain how to interpret the main utility guarantee (Theorem 1) and its various parameters. This was clarified by the author response, but concerns remain. In particular, it shows limited improvement over the baseline of local learning. Thus the significance of the contribution is not sufficiently clear.
- The clarity of the exposition needs to be improved. The problem setup and motivation for this work are not adequately explained. A high level issue is that the introduction focuses on *algorithms* instead of discussing the *problems* that they are meant to solve.

Decision: accept