# Calibrating the Attack to Sensitivity in Differentially Private Mechanisms
## 5 Kullback-Leibler DP and Chernoff DP for Adversarial Classification
### Laplace Mechanisms  
This section is dedicated to the derivation of relative entropy or Kullback-Leibler (KL) divergence between two Laplace distributions and its adaptation to adversarial classification through _KL-DP_. For the described problem and the associated model described in Section 2.1, the neighboring datasets could be imagined as those where the output of the query is \(\sum_{i=1}^{n}X_{i}\) before the attack and \((\sum_{i=1}^{n}X_{i}+X_{a})\) after the attack in both cases of Laplace and Gaussian mechanisms.The corresponding distributions are considered as the DP noise with and without the induced value of \(X_{a}\) by the attacker as in our original hypothesis testing problem in (7). To be consistent with the hypotheses in (7), we set \(P_{Y|X=x}\ Lap(\mu_{0},s/\epsilon)\) and for the neighbor, we have \(\ Lap(\mu_{1},\theta_{8}/\epsilon)\). Hereafter, we derive the relative entropy between \(p_{0}\sim Lap(\mu_{0},b_{0})\) and \(p_{1}\sim Lap(\mu_{1},b_{1})\). \[D(p_{0}||p_{1})=\int p_{0}(z)\log\frac{p_{0}(z)}{p_{1}(z)}dz \tag{66}\] \[=\mathbb{E}_{p_{0}}\Bigg{[}\log\frac{1/2b_{0}\exp\Bigl{\{}-\frac {|z-\mu_{0}|}{b_{0}}\Bigr{\}}}{1/2b_{1}\exp\Bigl{\{}-\frac{|z-\mu_{1}|}{b_{1}} \Bigr{\}}}\Bigg{]}\] (67) \[=\log\biggl{(}\frac{b_{1}}{b_{0}}\biggr{)}-\frac{1}{b_{0}}\mathbb{ E}_{p_{0}}[|z-\mu_{0}|]+\frac{1}{b_{1}}\mathbb{E}_{p_{0}}[|z-\mu_{1}|]\] (68) \[\overset{(a)}{=}\log\biggl{(}\frac{b_{1}}{b_{0}}\biggr{)}-1+ \frac{1}{b_{1}}\mathbb{E}_{p_{0}}[|z-\mu_{1}|] \tag{69}\]  
In step (a), we substituted \(\mathbb{E}_{p_{0}}[|z-\mu_{0}|]\) by \(b_{0}\) since for \(z\sim Lap(\mu,b)\) then \(|z-\mu|\sim Exp(1/b)\) and the corresponding mean for the exponential random variable is the inverse of its parameter. For the last term, \(\frac{1}{b_{1}}\mathbb{E}_{p_{0}}[|z-\mu_{1}|]\), we must consider two different cases due to the absolute value in the exponent of the Laplace distribution. In the following first expansion, the two distributions are centered around \(\mu_{0}\) and \(\mu_{1}\) where \(\mu_{0}<\mu_{1}\). \[\frac{1}{b_{1}}\mathbb{E}_{p_{0}}[|z-\mu_{1}|] =\frac{1}{b_{1}}\int_{p_{0}}|z-\mu_{1}|\frac{1}{2b_{0}}\exp\biggl{\{} -\frac{|z-\mu_{0}|}{b_{0}}\biggr{\}}dz \tag{70}\] \[=\frac{b_{0}}{2b_{1}}\exp\biggl{\{}\frac{\mu_{0}-\mu_{1}}{b_{0}} \biggr{\}}+\frac{\mu_{1}-\mu_{0}}{b_{1}} \tag{71}\]  
Substituting (71) into (69), we finally get the KL divergence between two Laplacians as  
\[D_{I}(p_{0}||p_{1})=\log\biggl{(}\frac{b_{1}}{b_{0}}\biggr{)}-1+\frac{b_{0}}{ b_{1}}\exp\biggl{\{}\frac{\mu_{0}-\mu_{1}}{b_{0}}\biggr{\}}-\frac{\mu_{0}-\mu_{1}}{b_ {1}} \tag{72}\]Simplifying (72) for \(b_{0}=s/\epsilon\) and \(b_{1}=\theta(s/\epsilon)\) and \(\mu_{1}-\mu_{0}=\Delta\mu\) for the hypothesis testing problem defined in (7), we finally get  
\[D_{I}(p_{0}||p_{1})_{\Delta\mu>0}=\log\theta-1+\frac{1}{\theta}\exp\biggl{\{}- \frac{\Delta\mu\epsilon}{s}\biggr{\}}+\frac{\Delta\mu\epsilon}{\theta s} \tag{73}\]  
As for the case of \(\mu_{0}>\mu_{1}\), we have  
\[\frac{1}{b_{1}}\mathbb{E}_{p_{0}}[|z-\mu_{1}|] =\frac{1}{b_{1}}\int_{p_{0}}|z-\mu_{1}|\frac{1}{2b_{0}}\exp\biggl{\{} -\frac{|z-\mu_{0}|}{b_{0}}\biggr{\}}dz \tag{74}\] \[=\frac{b_{0}}{b_{1}}\exp\biggl{\{}\frac{\mu_{1}-\mu_{0}}{b_{0}} \biggr{\}}+\frac{b_{0}}{b_{1}} \tag{75}\]  
Finally, substituting (74) into (69), we get the KL divergence between \(p_{0}\) and \(p_{1}\) for positive \(\mu_{0}\) where \(\mu_{0}>\mu_{1}\) as follows. \[=\mathbb{E}_{p_{0}}\Bigg{[}\log\frac{1/2b_{0}\exp\Bigl{\{}-\frac {|z-\mu_{0}|}{b_{0}}\Bigr{\}}}{1/2b_{1}\exp\Bigl{\{}-\frac{|z-\mu_{1}|}{b_{1}} \Bigr{\}}}\Bigg{]}\]0  
Setting \(\mu_{1}-\mu_{0}=\Delta\mu\), \(b_{0}=s/\epsilon\) and \(b_{1}=\theta(s/\epsilon)\) in (76), \(D_{II}(p_{0}||p_{1})\) yields  
\[=\mathbb{E}_{p_{0}}\Bigg{[}\log\frac{1/2b_{0}\exp\Bigl{\{}-\frac {|z-\mu_{0}|}{b_{0}}\Bigr{\}}}{1/2b_{1}\exp\Bigl{\{}-\frac{|z-\mu_{1}|}{b_{1}} \Bigr{\}}}\Bigg{]}\]1  
**Remark 4**.