# Calibrating the Attack to Sensitivity in Differentially Private Mechanisms
## 5 Kullback-Leibler DP and Chernoff DP for Adversarial Classification
### Chernoff DP for Gaussian Mechanism  
In the classical approach, the best error exponent in hypothesis testing for choosing between two probability distributions is the Kullback-Leibler divergence between these two distributions due to [PERSON]'s lemma [19]. In the Bayesian setting, however, assigning prior probabilities to each of the hypotheses in a binary hypothesis testing problem minimizes the best error exponent when the weighted sum probability of error, i.e., \(\pi=a\alpha+b\beta\) for \(b=1-a\) and \(a\in(0,1)\) which corresponds to the _Chernoff information/divergence_. The Chernoff information between two probability distributions \(f_{0}\) and \(f_{1}\) with prior probabilities \(a\) and \(b\) is defined as  
\[C_{n}(f_{0}||f_{1})=\log\int_{x}f_{0}(x)^{a}f_{1}^{b}(x)dx \tag{78}\]  
The Renyi divergence denoted \(D_{a}(f_{0}||f_{1})\) between two Gaussian distributions with parameters \(\mathcal{N}(\mu_{0},\sigma_{0}^{2})\) and \(\mathcal{N}(\mu_{1},\sigma_{1}^{2})\) is given in [20] by  
\[D_{a}(f_{0}||f_{1})=\ln\frac{\sigma_{1}}{\sigma_{0}}+\frac{1}{2(a-1)}\ln\biggl{(} \frac{\sigma_{1}^{2}}{(\sigma^{2})_{a}^{*}}\biggr{)}+\frac{1}{2}\frac{a(\mu_{ 0}-\mu_{1})^{2}}{(\sigma^{2})_{a}^{*}} \tag{79}\]where \((\sigma^{2})_{a}^{*}=a\sigma_{1}^{2}+b\sigma_{0}^{2}\). Using the following relation between Chernoff information and Renyi divergence \(D_{a}(f_{0}||f_{1})=\frac{1}{1-a}C_{a}(f_{0}||f_{1})\), we obtain the Gaussian univariate Chernoff information with different standard deviations \(\sigma_{i}\) for \(i=0,1\) as follows. \[C(f_{0}||f_{1})=b\ln\frac{\sigma_{1}}{\sigma_{0}}+\frac{1}{2}\ln\frac{\sigma_{ 1}^{2}}{a\sigma_{1}^{2}+b\sigma_{0}^{2}}+\frac{a\cdot b}{2}\frac{(\mu_{0}-\mu_ {1})^{2}}{a\sigma_{1}^{2}+b\sigma_{0}^{2}}.\]  
On the other hand, KL divergence between two Gaussian distributions denoted \(D_{KL}(f_{0}||f_{1})\) is derived as \(\log\left(\frac{\sigma_{1}}{\sigma_{0}}\right)+\frac{1}{2}\frac{\sigma_{0}^{2 }}{\sigma_{1}^{2}}+\frac{(\mu_{1}-\mu_{0})^{2}}{2\sigma_{1}^{2}}-\frac{1}{2}\). The next definition provides an adaptation of Chernoff information to quantify DP guarantee as a stronger alternative to KL-DP of Definition 7 and \((\epsilon,\delta)\)-DP for Gaussian mechanisms. We apply this to our problem setting for adversarial classification under Gaussian mechanisms, where the query output before and after the attack are \(\Sigma_{i}^{n}X_{i}\) and \(\sum_{\tilde{x}}^{n}X_{i}+X_{a}\), respectively. The corresponding distributions are considered as the DP noise with and without the induced value of \(X_{a}\) by the attacker as in our original hypothesis testing problem in (7) in Section 2.1.1. **Definition 8** (Chernoff DP). : _For a randomized mechanism \(P_{Y|X}\) guarantees \(\epsilon-\) Chernoff-DP, if the following inequality holds for all its neighboring datasets \(x\) and \(\tilde{x}\)_  
\[C_{a}(P_{Y|X=x}||P_{Y|X=\tilde{x}})\leq\exp(\epsilon) \tag{80}\]  
_where \(C_{a}(.||. )\) is defined by (78)._  
Ref. [5] (Theorem 1) proves that KL-DP defined in Definition 7 is a stronger privacy metric than \((\epsilon,\delta)\)-DP that is achieved by Gaussian mechanism. Accordingly, the following chain of inequalities are proven to hold for various definitions of DP  
\[\epsilon-\text{DP}\overset{a}{\succeq}\text{KL}-\text{DP}\overset{b}{ \succeq}\text{MI}-\text{DP}\overset{c}{\succeq}\delta-\text{DP}\overset{d}{ \triangleq}(\epsilon,\delta)-\text{DP}\]  
where MI-DP refers to the mutual information DP defined by \(\sup_{i,P_{X^{n}}}I(X_{i};Y|X^{-i})\leq\epsilon\) nats for a dataset \(X^{n}=\{X1,\cdots,X_{n}\}\) with the corresponding output \(Y\) according to the randomized mechanism represented by \(P_{Y|X^{n}}\) where \(X^{-i}\) denotes the dataset entries excluding \(X_{i}\). \(\delta\)-DP represents the case when \(\epsilon=0\) in \((\epsilon,\delta)\)-DP. Chernoff-information-based definition of DP is a _stronger privacy metric_ than KL-DP, and thus \((\epsilon,\delta)\)-DP for the Gaussian mechanism due to prior probabilities. Such a comparison is presented numerically in Section 6.