SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Lemma 4.1. For a given  $f$  and for  $d_{\mathcal{Y}} \gets \ell_{\alpha}$  and  $d_{\mathcal{X}} \gets \ell_{\beta}$ ,  $\Delta_{LS}(\mathbf{x}) \leq \mathcal{L}(f, \mathcal{N}(\mathbf{x}, R))$ . Proof in Supplementary.
Since local sensitivity is upper bounded by the Lipschitz constant, evaluating the Lipschitz constant suffices as an alternative to evaluating local sensitivity.
Lower bound on testing the validity of  $\Delta_{LS}^{p}$ : The PTR algorithm [11] suggests a proposal on the upper bound  $(\Delta_{LS}^{p})$  of local sensitivity and then finds the distance between the given dataset  $(\mathbf{x})$  and the closest dataset for which the proposed upper bound is not valid. Let  $\gamma(\cdot)$  be a distance query and  $\Delta_{LS}(\mathbf{x})$  be the local sensitivity defined as per the DP framework with respect to  $\mathbf{x}$  such that
$$
\gamma (\mathbf {x}) = \min  _ {\mathbf {x} ^ {\prime} \in \mathcal {X}} \left\{d _ {H} \left(\mathbf {x}, \mathbf {x} ^ {\prime}\right) s. t. \Delta_ {L S} \left(\mathbf {x} ^ {\prime}\right) > \Delta_ {L S} ^ {p} \right\}. \tag {6}
$$
For our privacy definition, the query  $\gamma (\mathbf{x},R)$  can be formulated as follows:
$$
\gamma (\mathbf {x}, R) = \min  _ {\mathbf {x} ^ {\prime} \in \mathcal {X}} \left\{d _ {\mathcal {X}} \left(\mathbf {x}, \mathbf {x} ^ {\prime}\right) s. t. \Delta_ {L S} \left(\mathbf {x} ^ {\prime}, R\right) > \Delta_ {L S} ^ {p} \right\}. \tag {7}
$$
We note that keeping  $d_{\mathcal{X}} = d_H$  and  $R = 1$ , makes the  $\gamma$  query exactly same as defined in the eq 6. Generally, computing  $\gamma(\cdot)$  is intractable due to local sensitivity estimation required for every  $\mathbf{x}' \in \mathcal{X}$  (which depends upon a non-linear neural network in our case). We emphasize that this step is intractable at two levels, first, we require estimating the local sensitivity of a neural network query. Second, we require this local sensitivity over all samples in the data domain. Therefore, we make it tractable by computing a lower bound over  $\gamma(\mathbf{x}, R)$  by designing a function  $\phi(\cdot)$  s.t.  $\phi(\mathbf{x}, R) \leq \gamma(\mathbf{x}, R)$ . Intuitively,  $\phi(\cdot)$  finds the largest possible neighborhood around  $\mathbf{x}$  such that the local Lipschitz constant of the neighborhood is smaller than the proposed local sensitivity. Because the subset of points around  $\mathbf{x}$  whose neighborhood does not violate  $\Delta_{LS}^p$  is half of the size of the original neighborhood in the worst case, we return half of the size of neighborhood as the output. We describe the algorithm procedurally in the supplementary material. More formally,
$$
\phi (\mathbf{x}, R) = \frac {1}{2} \cdot \underset {R ^ {\prime} \geq R} {\arg \max } \{\mathcal {L} (f, \mathcal {N} (\mathbf{x}, R ^ {\prime})) \leq \Delta_ {L S} ^ {p} \}
$$
If there is no solution to the equation above, then we return 0.
Lemma 4.2.  $\phi (\mathbf{x},R)\leq \gamma (\mathbf{x},R)$  . Proof in Supplementary.
Privately testing the lower bound: The next step in the PTR algorithm requires testing if  $\gamma (\mathbf{x})\leq \ln (\frac{1}{\delta} /\epsilon)$ . If the condition is true, then no-answer  $(\bot)$  is released instead of data. Since the  $\gamma$  query depends upon  $\mathbf{x}$ , PTR privatizes it by applying laplace mechanism, i.e.  $\hat{\gamma} (\mathbf{x}) = \gamma (\mathbf{x}) + \mathrm{Lap}(1 / \epsilon)$ . The query has a sensitivity of 1 since the  $\gamma$  could differ at most by 1 for any two neighboring databases. In our framework, we compute  $\phi (\mathbf{x},R)$  to lower bound the value of  $\gamma (\mathbf{x},R)$ . Therefore, we need to privatize the  $\phi$  query. We prove that for distance metrics in  $d_{\chi}$ -privacy, the global sensitivity of the  $\phi (\mathbf{x})$  query is 1.
Lemma 4.3. The query  $\phi(\cdot)$  has a global sensitivity of 1, i.e.  $\forall \mathbf{x}, \mathbf{x}' \in \mathcal{X}$ ,  $d_{abs}(\phi(\mathbf{x}, R), \phi(\mathbf{x}', R)) \leq d_{\mathcal{X}}(\mathbf{x}, \mathbf{x}')$ . Proof in Supplementary.
After computing  $\phi (\mathbf{x},R)$ , we add noise sampled from a laplace distribution, i.e.  $\hat{\phi} (\mathbf{x},R) = \phi (\mathbf{x},R) + \mathrm{Lap}(R / \epsilon)$ . Next, we check if  $\hat{\phi} (\mathbf{x},R)\leq \ln \left(\frac{1}{\delta}\right)\cdot R / \epsilon$ , then we release  $\perp$ , otherwise we release  $\hat{\mathbf{z}} = f(g(\mathbf{x})) + \mathrm{Lap}(\Delta_{LS}^{p} / \epsilon)$ . Next, we prove that the mechanism  $\mathcal{M}_1$  described above satisfies reconstruction-privacy.
Theorem 4.4. Mechanism  $\mathcal{M}_1$  satisfies uniform  $(2\epsilon, \delta/2, R)$ -reconstruction privacy Eq. 2, i.e.  $\forall \mathbf{x}, \mathbf{x}' \in \mathcal{X}$ , s.t.  $d_{\mathcal{X}}(\mathbf{x}, \mathbf{x}') \leq R$
$$
\mathbb {P} (\mathcal {M} (\mathbf {x}) \in S) \leq e ^ {2 \epsilon} \mathbb {P} (\mathcal {M} \left(\mathbf {x} ^ {\prime}\right) \in S) + \frac {\delta}{2} \tag {8}
$$
Proof Sketch: Our proof is similar to the proof for the PTR framework[12] except the peculiarity introduced due to our metric space formulation. First, we show that not releasing the answer  $(\bot)$  satisfies the privacy definition. Next, we divide the proof into two parts, when the proposed bound is incorrect (i.e.  $\Delta_{LS}(\mathbf{x},R) > \Delta_{LS}^p$ ) and when it is correct. Let  $\hat{R}$  be the output of query  $\phi$ .
$$
\begin{array}{l} \begin{array}{c} \mathbb {P} [ \hat {\phi} (\mathbf {x}, R) = \hat {R} ] \\ \mathbb {P} [ \hat {\phi} (\mathbf {x} ^ {\prime}, R) = \hat {R} ] \end{array} = \frac {\exp (- (\frac {| \phi (\mathbf {x} , R) - \hat {R} |}{R} \cdot \epsilon))}{\exp (- (\frac {| \phi (\mathbf {x} ^ {\prime} , R) - \hat {R} |}{R} \cdot \epsilon))} \\ \leq \exp (| \phi (\mathbf {x} ^ {\prime}, R) - \phi (\mathbf {x}, R) | \cdot \frac {\epsilon}{R}) \leq \exp (d _ {\mathcal {X}} (\mathbf {x}, \mathbf {x} ^ {\prime}) \cdot \frac {\epsilon}{R}) \leq \exp (\epsilon) \\ \end{array}
$$
MNIST (ε = 0, 0.10), (ε = ∞, 0.93)	FMNIST (ε = 0, 0.10), (ε = ∞, 0.781)	UTKFace (ε = 0, 0.502), (ε = ∞, 0.732)
Informal	ε = 1	ε = 2	ε = 5	ε = 10	Informal	ε = 1	ε = 2	ε = 5	ε = 10	Informal	ε = 1	ε = 2	ε = 5	ε = 10
Encoder	0.93	0.428	0.673	0.883	0.921	0.779	0.228	0.355	0.605	0.722	0.724	0.617	0.673	0.717	0.721
ARL	0.917	0.329	0.532	0.792	0.882	0.747	0.214	0.319	0.557	0.685	0.71	0.605	0.649	0.691	0.707
C	0.926	0.443	0.684	0.881	0.917	0.781	0.158	0.225	0.422	0.608	0.73	0.623	0.673	0.718	0.724
N	0.923	0.279	0.496	0.816	0.902	0.559	0.136	0.177	0.310	0.462	0.725	0.614	0.667	0.708	0.715
ARL-C	0.896	0.424	0.648	0.839	0.883	0.761	0.196	0.314	0.537	0.682	0.709	0.632	0.684	0.70	0.705
ARL-N	0.88	0.118	0.139	0.21	0.325	0.717	0.294	0.467	0.657	0.705	0.71	0.628	0.674	0.701	0.708
C-N	0.929	0.353	0.574	0.844	0.913	0.774	0.161	0.224	0.411	0.599	0.727	0.616	0.671	0.712	0.722
ARL-C-N	0.921	0.514	0.751	0.891	0.912	0.706	0.371	0.554	0.678	0.695	0.712	0.650	0.690	0.700	0.700
Table 1: Performance comparison for different baselines: Our posthoc framework enables comparison between different obfuscation techniques by fixing the privacy budget  $(\epsilon)$ . First four rows are different approaches to protect against data reconstruction and the remaining rows below are combinations of different approaches. The top row refers to the accuracy corresponding to different datasets under two extremes of epsilons. ARL refers to widely used adversarial representation learning approach for regularizing representation based on a proxy attacker[35, 38, 65, 56]. Contrastive refers to contrastive learning based informally privatizing mechanism introduced in [44].
Therefore, using the post-processing property -  $\mathbb{P}[\mathcal{M}(\mathbf{x}) = \bot] \leq e^{\epsilon} \mathbb{P}[\mathcal{M}(\mathbf{x}') = \bot]$ . Here, the first inequality is due to triangle inequality, the second one is due to Lemma 4.3 and the third inequality follows from  $d_{\mathcal{X}}(\mathbf{x}, \mathbf{x}') \leq R$ . Note that when  $\Delta_{LS}(\mathbf{x}, R) > \Delta_{LS}^{p}$ ,  $\phi(\mathbf{x}, R) = 0$ . Therefore, the probability for the test to release the answer in this case is
$$
\mathbb {P} [ \mathcal {M} (\mathbf {x}) \neq \bot ] = \mathbb {P} [ \phi (\mathbf {x}, R) + \mathrm {L a p} (\frac {R}{\epsilon}) > \log (\frac {1}{\delta}) \cdot \frac {R}{\epsilon} ] = \mathbb {P} [ \mathrm {L a p} (\frac {R}{\epsilon}) > \log (\frac {1}{\delta}) \cdot \frac {R}{\epsilon} ]
$$
Based on the CDF of Laplace distribution,  $\mathbb{P}[\mathcal{M}(\mathbf{x})\neq \bot ] = \frac{\delta}{2}$ . Therefore, if  $\Delta_{LS}(\mathbf{x},R) > \Delta_{LS}^{p}$ , for any  $S\subseteq \mathbb{R}^d\cup \bot$  in the output space of  $\mathcal{M}$
$$
\begin{array}{l} \mathbb {P} [ \mathcal {M} (\mathbf {x}) \in S ] = \mathbb {P} [ \mathcal {M} (\mathbf {x}) \in S \cap \{\bot \} ] + \mathbb {P} [ \mathcal {M} (\mathbf {x}) \in S \cap \{\mathbb {R} ^ {d} \} ] \\ \leq e ^ {\epsilon} \mathbb {P} [ \mathcal {M} (\mathbf {x} ^ {\prime}) \in S \cap \{\bot \} ] + \mathbb {P} [ \mathcal {M} (\mathbf {x}) \neq \bot ] \leq e ^ {\epsilon} \mathbb {P} [ \mathcal {M} (\mathbf {x} ^ {\prime}) \in S ] + \frac {\delta}{2} \\ \end{array}
$$
If  $\Delta_{LS}(\mathbf{x},R)\leq \Delta_{LS}^p$  then the mechanism is a composition of two  $(\epsilon ,\delta ,R)$ -reconstruction private algorithm where the first algorithm  $(\phi (\mathbf{x},R))$  is  $(\epsilon ,\delta /2,R)$ -reconstruction private and the second algorithm is  $(\epsilon ,0,R)$ -reconstruction private. Using composition, the algorithm is  $(2\epsilon ,\delta /2,R)$ -reconstruction private. We describe  $\mathcal{M}_1$  step by step in the algorithm in supplementary. To summarize, we designed the posthoc privacy framework that extends the PTR framework by making it tractable to get  $(\epsilon ,\delta ,R)$ -reconstruction privacy. The exact local Lipschitz constant of the neural network based obfuscator is estimated using mixed-integer programming based optimization developed by [25].
Computational feasibility: Our key idea is to add extra computation on the client side to turn informally private representations to their formal counterpart. This extra computational cost is due to the estimation of the local Lipschitz constant of the obfuscator network. However, the following two factors of our framework make it practically feasible -
1. We compute the local Lipschitz constant (i.e. in a small neighborhood around a given point): Our extension of the propose-test-release framework only requires us to operate in a small local neighborhood instead of estimating the global Lipschitz constant which would be much more computationally expensive.
2. Low number of parameters for obfuscator: Instead of estimating the Lipschitz constant of the whole prediction model, we only require estimation of the obfuscator - a neural network that has a significantly lower number of parameters in comparison to the prediction model.
We experimentally validate both of the above benefits in Sec 5. The fact that the local Lipschitz constant is being computed over the same obfuscator allows room for optimizing performance by caching. Our goal is to demonstrate the feasibility of bridging formal privacy guarantees and CI mechanisms, hence, we did not explore such performance speedups.
Figure 2: Runtime evaluation of local lipschitz computation for different (a) neighborhood radius, (b) input dimensions, and (c) number of layers. While the runtime increases exponentially with dimensions, it plateaus with increase in neighborhood radius. Since the input dimensions are same as embedding dimensions it makes the algorithm favorable to our analysis.
Experimental Setup: We evaluate different aspects of our proposed framework - i) E1: comparison between different adversarial approaches, ii) E2: comparison with local differential privacy (LDP), iii) E3: computational tractability of our proposed framework, and iv) E4: investigating role of ARL in achieving privacy. We use MNIST[34], FMNIST[64] and UTKFace[67] dataset for all experiments. All of them contain samples with high ambient dimensions (MNIST-784, FMNIST-784 and UTKFace-4096). We use a deep CNN based  $\beta$ -VAE[22] for the pre-processing. We use LipMip[25] for computing Lipschitz constant over  $\ell_{\infty}$  norm in the input space and  $\ell_1$  norm in the output space. Our baselines include a simple Encoder that projects the data to smaller dimensions using a neural network. This encoder type approach has been used in the literature as Split Learning [19]. We include widely used ARL techniques [65, 56, 38, 35, 66] and adversarial contrastive learning[44] which we denote as C. We use noisy regularization (denoted by N) to improve classifier performance. We refer the reader to supplementary for a detailed experimental setup, codebase, and hyper-parameters.
E1: Privacy-Utility Trade-off: Since our framework enables comparison between different obfuscation techniques under the same privacy budget, we evaluate test set accuracy as utility in Table 1. Our results indicate that ARL complemented with contrastive and noise regularization helps in attaining overall best performance among all possible combinations.
E2: Comparison between ARL and LDP: While  $\epsilon$ -LDP definition provides a different privacy guarantee than our proposed privacy definition, for reference we compare the performance between LDP and CI and report results in supplementary. We note that for low values of  $\epsilon$ , LDP techniques do not yield any practical utility. This corroborates with the impossibility result of instance encoding [7] and our discussion in Sec 2 about the inapplicability of traditional DP in the context of CI.
E3: Computational feasibility: We report end-to-end runtime evaluation on a CPU-based client and achieve 2 sec/image (MNIST) and 3.5 sec/image (UTKFace). While plenty of room exists for optimizing this runtime, we believe current numbers serve as a reasonable starting point for providing formal privacy in ARL. As discussed in Sec 4, we compare the computation time of the obfuscator across three factors relevant to our setup - i) Dimensionality of the input, ii) Size of the neighborhood, and iii) Number of layers in the Obfuscator. We report the results in Fig. 2.
E4: What role does ARL play in achieving privacy? We investigate the contribution of adversarial training in improving the privacy-utility trade-off. We train obfuscator models with different values of  $\alpha$  (weighing factor) for adversarial training. Our results in Fig 3 indicate that higher weighing of adversarial regularization reduces the local Lipschitz constant, hence reducing the local sensitivity of the neural network. Furthermore, for high values of  $\alpha$ , the change in the local Lipschitz constant reduces significantly for different sizes  $(R)$  of the neighborhood. These two observations can potentially explain that ARL improves reconstruction privacy by reducing the sensitivity of the obfuscator. However, as we observe in Table 1, the classifier can reduce its utility if ARL is not complemented with noisy and contrastive regularization. We believe this finding could be of independent interest to the adversarial defense community.
Figure 3: Local sensitivity comparison for different values of  $\alpha$ : The five bars for each  $\alpha$  represent different neighborhood radii. Increase in the value of  $\alpha$  decreases the local Lipschitz constant (upper bound on local sensitivity) indicating lesser amount of noise to be added for the same level of privacy.
How to select privacy parameter  $R$ ? One of the key difference between  $(\epsilon, \delta, R)$ -neighborhood privacy and  $(\epsilon, \delta)$ -DP is the additional parameter  $R$ . The choice of  $R$  depends upon the neighborhood in which a user wishes to get an  $\epsilon$  level of indistinguishability. We discussed the equivalence of  $R$  and  $k$  in group Differential Privacy in Sec 3.2. To understand the role of  $R$ , we perform reconstruction attacks on privatized encoding obtained from our framework by training an ML model to reconstruct original images. We compare reconstruction results for different values of  $\epsilon$  and  $R$  on four distinct metrics for images and report in Supplementary. To assess the level of indistinguishability, we project the original images into embedding space and sample points from the boundary of neighborhoods of different  $R$ . We observe that as the boundary of the neighborhood increases, the images become perceptually different from the original image. For extremely large radii, the images change significantly enough that their corresponding label may change too. Such visualizations can be used to semantically understand different values of  $R$ .
How to propose  $\Delta_{LS}^p$ ? Our framework requires a proposal on the upper bound of local sensitivity in a private manner. One possible way to obtain  $\Delta_{LS}^p$  is by using the Lipschitz constant of training data samples used in training the obfuscator. We choose  $\Delta_{LS}^p$  by computing the mean  $(\mu)$  and standard deviation  $(\sigma)$  of local sensitivity on the training dataset (assumed to be known under our threat model), then we keep  $\Delta_{LS}^p = \mu + n * \sigma$  where  $n$  allows a trade-off between the likelihood of releasing the samples under PTR and adding extra noise to data. We used  $n = 3$  in our experiments. Since empirically, the value of local sensitivity appears to be following a gaussian, using confidence interval serves as a good proxy. Fig 3 shows that for higher values of  $\alpha$ , the variability in the local Lipschitz constant decreases indicating the validity of the bound would hold for a large number of samples.
Limitations: i) Since we utilize the PTR framework, outlier samples may not get released due to high sensitivity, this is expected since these outlier samples are likely to be misclassified anyway. ii) Lipschitz constant computation is limited to ReLU based DNNs, therefore more sophisticated obfuscator architectures are currently not compatible with our proposed framework and we leave it as part of the future work. iii) Choosing the privacy parameter  $(R)$  could be challenging for certain datasets and might vary based on user preferences. We believe the choice of such a parameter would depend upon the context in which CI is being applied.
Collaborative Inference techniques aim to learn a task-oriented privacy preserving encoding of data. Majority of the works in this area either protect against sensitive attribute leakage [20, 50, 5, 36] or input reconstruction [52, 56, 39, 35, 38]. These techniques usually evaluate their privacy using empirical attacks since the mechanism is learned using gradient based min-max optimization making it infeasible for the worst-case privacy analysis. The problem is exacerbated in the context of input reconstruction because of the lack of formal definition for reconstruction privacy. The goal of our work is to create a framework that makes the existing techniques amenable to formal privacy analysis.
While theoretical analyses [68, 69, 51] of ARL objectives have identified fundamental trade-offs between utility and attribute leakage, they are difficult to formalize as a worst-case privacy guarantee especially for deep neural networks.
Privacy definitions that extend the DP definition to incorporate some of its restrictions [29] include  $d_{\mathcal{X}}$ -Privacy [8], and Pufferfish [30]. Our privacy definition is a specific instantiation of the  $d_{\mathcal{X}}$ -Privacy [8] framework that extends DP to general metric spaces. Our instantiation is focused on reconstruction privacy for individual samples instead of membership inference attacks [13]. Existing works in DP for reconstruction attacks [6, 58] focus on protecting training data from an attacker that has access to model weights (whitebox attacker) or output of model queries (blackbox attacker). In contrast, our work only focuses on the privacy of the data used during the prediction stage and not training data.
Lipschitz constant estimation for neural networks has been used to guarantee network's stability to perturbations. Existing works either provide an upper bound [63, 33, 14], exact Lipschitz constant [25, 24] or Lipschitz constant regularization [53, 23] during the training stage. Some existing works have explored the relationship between adversarial robustness and DP model training [46, 47, 59]. We utilize similar ideas of perturbation stability but for privacy. Shavit and Gjura [54] use Lipschitz neural networks [16] to learn a private mechanism design for summary statistics such as median, however their mechanism design lack privacy guarantee.
Posthoc approach to privacy applies privacy preserving mechanism in a data dependent manner. Smooth sensitivity [41] and PTR [11] reduce the noise magnitude because the local sensitivity is only equal to global sensitivity in the worst case and not average case. Privacy odometer [49], Ex-post privacy loss [37] and Rényi privacy filter [15] track privacy loss as the query is applied over data. Our works build upon the PTR framework in order to give high privacy for less sensitive data. However, as we show in Sec 4, our framework reformulates the PTR algorithm to make it tractable under our setup.
In this work we take a first step towards bridging empirical techniques in ARL and formal privacy mechanisms. The posthoc nature of our framework allows reasoning about privacy without any modification to the obfuscation algorithm, hence, making it easy to integrate for benchmarking existing and future techniques. We introduced a privacy definition that formalizes ARL and designed a corresponding privacy mechanism. An exciting future direction for extending our framework is to integrate NNs for designing more effective privacy mechanisms for other sophisticated queries beyond CI. Data adaptive mechanisms are an exciting area in DP and our improvement to PTR can be potentially applied for other queries where PTR might be intractable currently.
9 Acknowledgements and Disclosure of funding
We thank Mohammad Mohammadi Amiri for his valuable feedback on the early draft. This work is supported by NSF award number 1729931.
[1] Martin Abadi, Andy Chu, Ian Goodfellow, H Brendan McMahan, Ilya Mironov, Kunal Talwar, and Li Zhang. Deep learning with differential privacy. In Proceedings of the 2016 ACM SIGSAC conference on computer and communications security, pages 308-318, 2016.
[2] Miguel E Andrés, Nicolás E Bordenabe, Konstantinos Chatzikokolakis, and Catuscia Palamidessi. Geoindistinguishability: Differential privacy for location-based systems. In Proceedings of the 2013 ACM SIGSAC conference on Computer & communications security, pages 901-914, 2013.
[3] Borja Balle, Giovanni Cherubin, and Jamie Hayes. Reconstructing training data with informed adversaries. In 2022 IEEE Symposium on Security and Privacy (SP), pages 1138-1156. IEEE, 2022.
[4] Yuksel Ozan Basciftci, Ye Wang, and Prakash Ishwar. On privacy-utility tradeoffs for constrained data release mechanisms. In 2016 Information Theory and Applications Workshop (ITA), pages 1–6. IEEE, 2016.