SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Hua Wang
Philadelphia, PA 19104
wanghua@wharton.upenn.edu
Sheng Gao
shenggao@wharton.upenn.edu
Huanyu Zhang
Meta Platforms, Inc.
huanyuzhang@meta.com
Weijie J. Su
suw@wharton.upenn.edu
Milan Shen
milanshen@gmail.com
Hyperparameter optimization, also known as hyperparameter tuning, is a widely recognized technique for improving model performance. Regrettably, when training private ML models, many practitioners often overlook the privacy risks associated with hyperparameter optimization, which could potentially expose sensitive information about the underlying dataset. Currently, the sole existing approach to allow privacy-preserving hyperparameter optimization is to uniformly and randomly select hyperparameters for a number of runs, subsequently reporting the best-performing hyperparameter. In contrast, in non-private settings, practitioners commonly utilize "adaptive" hyperparameter optimization methods such as Gaussian process-based optimization, which select the next candidate based on information gathered from previous outputs. This substantial contrast between private and non-private hyperparameter optimization underscores a critical concern. In our paper, we introduce DP-HyPO, a pioneering framework for "adaptive" private hyperparameter optimization, aiming to bridge the gap between private and non-private hyperparameter optimization. To accomplish this, we provide a comprehensive differential privacy analysis of our framework. Furthermore, we empirically demonstrate the effectiveness of DP-HyPO on a diverse set of real-world datasets.
In recent decades, modern deep learning has demonstrated remarkable advancements in various applications. Nonetheless, numerous training tasks involve the utilization of sensitive information pertaining to individuals, giving rise to substantial concerns regarding privacy [29, 6]. To address these concerns, the concept of differential privacy (DP) was introduced by [12, 13]. DP provides a mathematically rigorous framework for quantifying privacy leakage, and it has gained widespread
acceptance as the most reliable approach for formally evaluating the privacy guarantees of machine learning algorithms.
When training deep learning models, the most popular method to ensure privacy is noisy (stochastic) gradient descent (DP-SGD) [4, 35]. DP-SGD typically resembles non-private gradient-based methods; however, it incorporates gradient clipping and noise injection. More specifically, each individual gradient is clipped to ensure a bounded  $\ell_2$  norm. Gaussian noise is then added to the average gradient which is utilized to update the model parameters. These adjustments guarantee a bounded sensitivity of each update, thereby enforcing DP through the introduction of additional noise.
In both non-private and private settings, hyperparameter optimization (HPO) plays a crucial role in achieving optimal model performance. Commonly used methods for HPO include grid search (GS), random search (RS), and Bayesian optimization (BO). GS and RS approaches are typically non-adaptive, as they select the best hyperparameter from a predetermined or randomly selected set. While these methods are straightforward to implement, they can be computationally expensive and inefficient when dealing with large search spaces. As the dimensionality of hyperparameters increases, the number of potential trials may grow exponentially. To address this challenge, adaptive HPO methods such as Bayesian optimization have been introduced [34, 14, 40]. BO leverages a probabilistic model that maps hyperparameters to objective metrics, striking a balance between exploration and exploitation. BO quickly emerged as the default method for complex HPO tasks, offering improved efficiency and effectiveness compared to non-adaptive methods.
While HPO is a well-studied problem, the integration of a DP constraint into HPO has received little attention. Previous works on DP machine learning often neglect to account for the privacy cost associated with HPO [1, 39, 42, 42]. These works either assume that the best parameters are known in advance or rely on a supplementary public dataset that closely resembles the private dataset distribution, which is not feasible in most real-world scenarios.
Only recently have researchers turned to the important concept of honest HPO [28], where the privacy cost during HPO cannot be overlooked. Private HPO poses greater challenges compared to the non-private case for two primary reasons. First, learning with DP-SGD introduces additional hyperparameters (e.g., clipping norm, the noise scale, and stopping time), which hugely adds complexity to the search for optimal hyperparameters. Second, DP-SGD is more sensitive to the selection of hyperparameter combinations, with its performance largely influenced by this choice [28, 10, 31].
To tackle this challenging question, previous studies such as [24, 32] propose running the base algorithm with different hyperparameters a random number of times. They demonstrate that this approach significantly benefits privacy accounting, contrary to the traditional scaling of privacy guarantees with the square root of the number of runs (based on the composition properties from [19]). While these papers make valuable contributions, their approaches only allow for uniformly random subsampling from a finite and pre-fixed set of candidate hyperparameters at each run. As a result, any advanced technique from HPO literature that requires adaptivity is either prohibited or necessitates a considerable privacy cost (polynomially dependent on the number of runs), creating a substantial gap between non-private and private HPO methods.
Given these considerations, a natural question arises: Can private hyperparameter optimization be adaptive, without a huge privacy cost? In this paper, we provide an affirmative answer to this question.
- We introduce the pioneering adaptive private hyperparameter optimization framework, DP-HyPO, which enables practitioners to adapt to previous runs and focus on potentially superior hyperparameters. DP-HyPO permits the flexible use of non-DP adaptive hyperparameter optimization methods, such as Gaussian process, for enhanced efficiency, while avoiding the substantial privacy costs due to composition. In contrast to the non-adaptive methods presented in [32, 24], our adaptive framework, DP-HyPO, effectively bridges the gap between private and non-private hyperparameter optimization. Importantly, our framework not only encompasses the aforementioned non-adaptive methods as special cases, but also seamlessly integrates virtually all conceivable adaptive methods into the framework.
- We provide sharp DP guarantees for the adaptive private hyperparameter optimization. Specifically, when the training procedure is executed multiple times, with each iteration being DP on its own, outputting the best repetition is DP ensured by the composition property. However, applying composition results in excessively loose privacy guarantees. Prior work in [24, 32] presents bounds that are either independent of the number of repetitions or depend logarithmically on it. Nevertheless, these results require that the hyperparameter selection for each iteration follows a uniform sampling distribution. In contrast, DP-HyPO allows arbitrary adaptive sampling distributions based on previous runs. Utilizing the Rényi DP framework, we offer a strict generalization of those uniform results by providing an accurate characterization of the Rényi divergence between the adaptive sampling distributions of neighboring datasets, without any stability assumptions.
- Empirically, we observe that the Gaussian process-based DP-HyPO algorithm outperforms its uniform counterpart across several practical scenarios. Generally, practitioners can integrate any non-private adaptive HPO methods into the DP-HyPO framework, opening up a vast range of adaptive private HPO algorithm possibilities. Furthermore, DP-HyPO grants practitioners the flexibility to determine the privacy budget allocation for adaptivity, empowering them to balance between the adaptivity and privacy loss when confronting various hyperparameter optimization challenges.
2.1 Differential Privacy and Hyperparameter Optimization
Differential Privacy is a mathematically rigorous framework for quantifying privacy leakage. A DP algorithm promises that an adversary with perfect information about the entire private dataset in use – except for a single individual – would find it hard to distinguish between its presence or absence based on the output of the algorithm [12]. Formally, for  $\varepsilon > 0$ , and  $0 \leq \delta < 1$ , we consider a (randomized) algorithm  $M: \mathcal{Z}^n \to \mathcal{V}$  that takes as input a dataset.
Definition 2.1 (Differential privacy). A randomized algorithm  $M$  is  $(\varepsilon, \delta)$ -DP if for any neighboring dataset  $D$ ,  $D' \in \mathcal{Z}^n$  differing by an arbitrary sample, and for any event  $E$ , we have
$$
\mathbb {P} [ M (D) \in E ] \leqslant \mathrm {e} ^ {\varepsilon} \cdot \mathbb {P} [ M \left(D ^ {\prime}\right) \in E ] + \delta .
$$
Here,  $\varepsilon$  and  $\delta$  are privacy parameters that characterize the privacy guarantee of algorithm  $M$ . One of the fundamental properties of DP is composition. When multiple DP algorithms are sequentially composed, the resulting algorithm remains private. The total privacy cost of the composition scales approximately with the square root of the number of compositions [19].
We now formalize the problem of hyperparameter optimization with DP guarantees, which builds upon the finite-candidate framework presented in [24, 32]. Specifically, we consider a set of base DP algorithms  $M_{\lambda}:\mathcal{Z}^{n}\to \mathcal{V}$ , where  $\lambda \in \Lambda$  represents a set of hyperparameters of interest,  $\mathcal{Z}^n$  is the domain of datasets, and  $\mathcal{V}$  denotes the range of the algorithms. This set  $\Lambda$  may be any infinite set, e.g., the cross product of the learning rate  $\eta$  and clipping norm  $R$  in DP-SGD. We require that the set  $\Lambda$  is a measure space with an associated measure  $\mu$ . Common choices for  $\mu$  include the counting measure or Lebesgue measure. We make a mild assumption that  $\mu (\Lambda) < \infty$ .
Based on the previous research [32], we make two simplifying assumptions. First, we assume that there is a total ordering on the range  $\mathcal{V}$ , which allows us to compare two selected models based on their "performance measure", denoted by  $q$ . Second, we assume that, for hyperparameter optimization purposes, we output the trained model, the hyperparameter, and the performance measure. Specifically, for any input dataset  $D$  and hyperparameter  $\lambda$ , the return value of  $M_{\lambda}$  is  $(x,q)\sim M_{\lambda}(D)$ , where  $x$  represents the combination of the model parameters and the hyperparameter  $\lambda$ , and  $q$  is the (noisy) performance measure of the model.
In this section, we focus on related work concerning private HPO, while deferring the discussion on non-private HPO to Appendix F.
Historically, research in DP machine learning has neglected the privacy cost associated with HPO [1, 39, 42]. It is only recently that researchers have begun to consider the honest HPO setting [28], in which the cost is taken into account.
A direct approach to addressing this issue involves composition-based analysis. If each training run of a hyperparameter satisfies DP, the entire HPO procedure also complies with DP through composition across all attempted hyperparameter values. However, the challenge with this method is that the privacy guarantee derived from accounting can be excessively loose, scaling polynomially with the number of runs.
Chaudhuri et al. [7] were the first to enhance the DP bounds for HPO by introducing additional stability assumptions on the learning algorithms. [24] made significant progress in enhancing DP bounds for HPO without relying on any stability properties of the learning algorithms. They proposed a simple procedure where a hyperparameter was randomly selected from a uniform distribution for each training run. This selection process was repeated a random number of times according to a geometric distribution, and the best model obtained from these runs was outputted. They showed that this procedure satisfied  $(3\varepsilon ,0)$  -DP as long as each training run of a hyperparameter was  $(\varepsilon ,0)$  -DP. Building upon this, [32] extended the procedure to accommodate negative binomial or Poisson distributions for the repeated uniform selection. They also offered more precise Renyi DP guarantees for this extended procedure. Furthermore, [8] explored a generalization of the procedure for top-  $k$  selection, considering  $(\varepsilon ,\delta)$  -DP guarantees.
In a related context, [28] explored a setting that appeared superficially similar to ours, as their title mentioned "adaptivity." However, their primary focus was on improving adaptive optimizers such as DP-Adam, which aimed to reduce the necessity of hyperparameter tuning, rather than the adaptive HPO discussed in this paper. Notably, in terms of privacy accounting, their approach only involved composing the privacy cost of each run without proposing any new method.
Another relevant area of research is DP selection, which encompasses well-known methods such as the exponential mechanism [25] and the sparse vector technique [13], along with subsequent studies. However, this line of research always assumes the existence of a low-sensitivity score function for each candidate, which is an unrealistic assumption for hyperparameter optimization.
3 DP-HyPO: General Framework for Private Hyperparameter Optimization
The obvious approach to the problem of differentially private hyperparameter optimization would be to run each base algorithm and simply return the best one. However, running such an algorithm on large hyperparameter space is not feasible due to the privacy cost growing linearly in the worst case.
While [24, 32] have successfully reduced the privacy cost for hyperparameter optimization from linear to constant, there are still two major drawbacks. First, none of the previous methods considers the case when the potential number of hyperparameter candidates is infinite, which is common in most hyperparameter optimization scenarios. In fact, we typically start with a range of hyperparameters that we are interested in, rather than a discrete set of candidates. Furthermore, prior methods are limited to the uniform sampling scheme over the hyperparameter domain  $\Lambda$ . In practice, this setting is unrealistic since we want to "adapt" the selection based on previous results. For instance, one could use Gaussian process to adaptively choose the next hyperparameter for evaluation, based on all the previous outputs. However, no adaptive hyperparameter optimization method has been proposed or analyzed under the DP constraint. In this paper, we bridge this gap by introducing the first DP adaptive hyperparameter optimization framework.
3.1 DP-HyPO Framework
To achieve adaptive hyperparameter optimization with differential privacy, we propose the DP-HyPO framework. Our approach keeps an adaptive sampling distribution  $\pi$  at each iteration that reflects accumulated information.
Let  $Q(D, \pi)$  be the procedure that randomly draws a hyperparameter  $\lambda$  from the distribution  $\pi \in \mathcal{D}(\Lambda)$ , and then returns the output from  $M_{\lambda}(D)$ . We allow the sampling distribution to depend on both the dataset and previous outputs, and we denote as  $\pi^{(j)}$  the sampling distribution at the  $j$ -th
Here,  $\mathcal{D}(\Lambda)$  represents the space of probability densities on  $\Lambda$ .
iteration on dataset  $D$ . Similarly, the sampling distribution at the  $j$ -th iteration on the neighborhood dataset  $D'$  is denoted as  $\pi'^{(j)}$ .
We now present the DP-HyPO framework, denoted as  $\mathcal{A}(D,\pi^{(0)},\mathcal{T},C,c)$ , in Framework 1. The algorithm takes a prior distribution  $\pi^{(0)}\in \mathcal{D}(\Lambda)$  as input, which reflects arbitrary prior knowledge about the hyperparameter space. Another input is the distribution  $\mathcal{T}$  of the total repetitions of training runs. Importantly, we require it to be a random variable rather than a fixed number to preserve privacy. The last two inputs are  $C$  and  $c$ , which are upper and lower bounds of the density of any posterior sampling distributions. A finite  $C$  and a positive  $c$  are required to bound the privacy cost of the entire framework.
Framework 1 DP-HyPO  $\mathcal{A}(D,\pi^{(0)},\mathcal{T},C,c)$
Initialize  $\pi^{(0)}$  , a prior distribution over  $\Lambda$    
Initialize the result set  $A = \{\}$    
Draw  $T\sim \mathcal{T}$    
for  $j = 0$  to  $T - 1$  do  $(x,q)\sim Q(D,\pi^{(j)})$ $A = A\cup \{(x,q)\}$  Update  $\pi^{(j + 1)}$  based on  $A$  according to any adaptive algorithm such that for all  $\lambda \in \Lambda$
$$
c \leq \frac {\pi^ {(j + 1)} (\lambda)}{\pi^ {(0)} (\lambda)} \leq C
$$
Output  $(x,q)$  from  $A$  with the highest  $q$
Note that we intentionally leave the update rule for  $\pi^{(j + 1)}$  unspecified in Framework 1 to reflect the fact that any adaptive update rule that leverages information from previous runs can be used. However, for a non-private adaptive HPO update rule, the requirement of bounded adaptive density  $c\leq \frac{\pi^{(j + 1)}(\lambda)}{\pi^{(0)}(\lambda)}\leq C$  may be easily violated. In Section 3.2, We provide a simple projection technique to privatize any non-private update rules. In Section 4, we provide an instantiation of DP-HyPO using Gaussian process.
We now state our main privacy results for this framework in terms of Rényi Differential Privacy (RDP) [27]. RDP is a privacy measure that is more general than the commonly used  $(\varepsilon, \delta)$ -DP and provides tighter privacy bounds for composition. We defer its exact definition to Definition A.2 in the appendix.
We note that different distributions of the number of selections (iterations),  $\mathcal{T}$ , result in very different privacy guarantees. Here, we showcase the key idea for deriving the privacy guarantee of DP-HyPO framework by considering a special case when  $\mathcal{T}$  follows a truncated negative binomial distribution $^2$ $\mathrm{NegBin}(\theta ,\gamma)$  (the same assumption as in [32]). In fact, as we show in the proof of Theorem 1 in Appendix A, the privacy bounds only depend on  $\mathcal{T}$  directly through its probability generating function, and therefore one can adapt the proof to obtain the corresponding privacy guarantees for other probability families, for example, the Possion distribution considered in [32]. From here and on, unless otherwise specified, we will stick with  $\mathcal{T} = \mathrm{NegBin}(\theta ,\gamma)$  for simplicity. We also assume for simplicity that the prior distribution  $\pi^{(0)}$  is a uniform distribution over  $\Lambda$ . We provide more detailed discussion of handling informed prior other than uniform distribution in Appendix D.
Theorem 1. Suppose that  $T$  follows truncated negative Binomial distribution  $T \sim \mathrm{NegBin}(\theta, \gamma)$ . Let  $\theta \in (-1, \infty)$ ,  $\gamma \in (0, 1)$ , and  $0 < c \leq C$ . Suppose for all  $M_{\lambda}: \mathcal{Z}^n \to \mathcal{Y}$  over  $\lambda \in \Lambda$ , the base algorithms satisfy  $(\alpha, \varepsilon)$ -RDP and  $(\hat{\alpha}, \hat{\varepsilon})$ -RDP for some  $\varepsilon, \hat{\varepsilon} \geq 0$ ,  $\alpha \in (1, \infty)$ , and  $\hat{\alpha} \in [1, \infty)$ . Then the DP-HyPO algorithm  $\mathcal{A}(D, \pi^{(0)}, \mathrm{NegBin}(\theta, \gamma), C, c)$  satisfies  $(\alpha, \varepsilon')$ -RDP where
$$
\varepsilon^ {\prime} = \varepsilon + (1 + \theta) \cdot \left(1 - \frac {1}{\hat {\alpha}}\right) \hat {\varepsilon} + \left(\frac {\alpha}{\alpha - 1} + 1 + \theta\right) \log \frac {C}{c} + \frac {(1 + \theta) \cdot \log (1 / \gamma)}{\hat {\alpha}} + \frac {\log \mathbb {E} [ T ]}{\alpha - 1}.
$$
${}^{2}$  Truncated negative binomial distribution is a direct generalization of the geometric distribution. See Appendix B for its definition.
To prove Theorem 1, one of our main technical contributions is Lemma A.4, which quantifies the Rényi divergence of the sampling distribution at each iteration between the neighboring datasets. We then leverage this crucial result and the probability generating function of  $\mathcal{T}$  to bound the Rényi divergence in the output of  $\mathcal{A}$ . We defer the detailed proof to Appendix A.
Next, we present the case with pure DP guarantees. Recall the fact that  $(\varepsilon, 0)$ -DP is equivalent to  $(\infty, \varepsilon)$ -RDP [27]. When both  $\alpha$  and  $\hat{\alpha}$  tend towards infinity, we easily obtain the following theorem in terms of  $(\varepsilon, 0)$ -DP.
Theorem 2. Suppose that  $T$  follows truncated negative Binomial distribution  $T \sim \mathrm{NegBin}(\theta, \gamma)$ . Let  $\theta \in (-1, \infty)$  and  $\gamma \in (0, 1)$ . If all the base algorithms  $M_{\lambda}$  satisfies  $(\varepsilon, 0)$ -DP, then the DP-HyPO algorithm  $\mathcal{A}(D, \pi^{(0)}, \mathrm{NegBin}(\theta, \gamma), C, c)$  satisfies  $\left((2 + \theta)\left(\varepsilon + \log \frac{C}{c}\right), 0\right)$ -DP.
Theorem 1 and Theorem 2 provide practitioners the freedom to trade off between allocating more DP budget to enhance the base algorithm or to improve adaptivity. In particular, a higher value of  $\frac{C}{c}$  signifies greater adaptivity, while a larger  $\varepsilon$  improves the performance of base algorithms.
3.1.1 Uniform Optimization Method as a Special Case
We present the uniform hyperparameter optimization method [32, 23] in Algorithm 2, which is a special case of our general DP-HyPO Framework with  $C = c = 1$ . Essentially, this algorithm never updates the sampling distribution  $\pi$ .
Algorithm 2 Uniform Hyperparameter Optimization  $\mathcal{U}(D,\theta ,\gamma ,\Lambda)$
Let  $\pi = \mathrm{Unif}(\{1,\dots,|\Lambda|\})$ , and  $A = \{\}$   
Draw  $T \sim \mathrm{NegBin}(\theta, \gamma)$   
for  $j = 0$  to  $T - 1$  do  
 $(x,q) \sim Q(D,\pi)$ $A = A \cup \{(x,q)\}$   
end for  
Output  $(x,q)$  from  $A$  with the highest  $q$
Our results in Theorem 1 and Theorem 2 generalize the main technical results of [32, 24]. Specifically, when  $C = c = 1$  and  $\Lambda$  is a finite discrete set, our Theorem 1 precisely recovers Theorem 2 in [32]. Furthermore, when we set  $\theta = 1$ , the truncated negative binomial distribution reduces to the geometric distribution, and our Theorem 2 recovers Theorem 3.2 in [24].
3.2 Practical Recipe to Privatize HPO Algorithms
In the DP-HyPO framework, we begin with a prior and adaptively update it based on the accumulated information. However, for privacy purposes, we require the density  $\pi^{(j)}$  to be bounded by some constants  $c$  and  $C$ , which is due to the potential privacy leakage when updating  $\pi^{(j)}$  based on the history. It is crucial to note that this distribution  $\pi^{(j)}$  can be significantly different from the distribution  $\pi^{\prime (j)}$  if we were given a different input dataset  $D^{\prime}$ . Therefore, we require the probability mass/density function to satisfy  $\frac{c}{\mu(\Lambda)}\leq \pi^{(j)}(\lambda)\leq \frac{C}{\mu(\Lambda)}$  for all  $\lambda \in \Lambda$  to control the privacy loss due to adaptivity.