SlowGuess/NeurIPS_2023_OCR huggingface dataset:
To formulate DP for instance encoding, we define an adjacent set  $\mathcal{D}$  and  $\mathcal{D}'$  as two differing inputs. A randomized method  $\mathcal{A}$  is  $(\alpha, \epsilon)$ -Rényi differentially private (RDP) if  $D_{\alpha}(\mathcal{A}(\mathcal{D})||\mathcal{A}(\mathcal{D}')| \leq \epsilon$  for  $D_{\alpha}(P||Q) = \frac{1}{\alpha - 1} \log \mathbb{E}_{x \sim Q}[(\frac{P(x)}{Q(x)})^{\alpha}]$ . As DP provides a different privacy guarantee with dFIL, we use the theorem from Guo et al. [20] to derive an MSE lower bound using DP's privacy metric for an unbiased attacker. Assuming a reconstruction attack  $\hat{\mathbf{x}} = \mathrm{Att}(\mathbf{e})$  that reconstructs  $\mathbf{x}$  from the encoding  $\mathbf{e} = \mathrm{Enc}(\mathbf{x})$ , repurposing the theorem Guo et al. [20] gives:
$$
\mathbb {E} [ | | \hat {\mathbf {x}} - \mathbf {x} | | _ {2} ^ {2} / d ] \geq \frac {\sum_ {i = 1} ^ {d} \operatorname {d i a m} _ {i} (\mathcal {X}) ^ {2} / 4 d}{e ^ {\epsilon} - 1} \tag {6}
$$
for a  $(2,\epsilon)$ -RDP Enc, where  $\mathcal{X}$  is the input data space. We can construct a  $(2,\epsilon)$ -RDP encoder  $\mathrm{Enc}_{RDP}$  from a deterministic encoder  $\mathrm{Enc}_D$  by scaling and clipping the encoding adding Gaussian noise, or  $\mathrm{Enc}_{RDP} = \mathrm{Enc}_D(\mathbf{x}) / \max(1, \frac{||\mathrm{Enc}_D(\mathbf{x})||_2}{C}) + \mathcal{N}(0,\sigma^2)$ , similarly to [1]. The noise to be added is  $\sigma = \frac{(2C)^2}{\epsilon}$  [52]. Equation 6 for DP is comparable to Equation 2 for dFIL, and we use the two equations to compare DP and dFIL parameters. We use Equation 2 because [20] does not discuss the bound against biased attackers.
We evaluate both encoders for split inference using CIFAR-10 dataset and ResNet-18. We split the model after block 4 (split-middle from Section 4.2.1) and did not add any optimizations discussed in Section 4 for simplicity. For the DP-based encoder, we retrain the encoder with scaling and clipping so that the baseline accuracy without noise does not degrade. We ran both models without standardizing the input, which makes  $\mathrm{diam}_i(\mathcal{X}) = 1$  for all  $i$ .
Table 4 compares the test accuracy achieved when targeting the same MSE bound for an unbiased attacker using dFIL and DP, respectively. The result clearly shows that DP degrades the accuracy much more for similar privacy levels (same unbiased MSE bound), becoming impractical very quickly. DP suffers from low utility because DP is agnostic with the input and the model, assuming a
Table 4: Test accuracy when targeting the same MSE bound.
Unbiased MSE bound	1e-5	1e-4	1e-3	1e-2
dFIL-based	93.09%	93.11%	92.52%	87.52%
DP-based	64.64%	56.68%	46.46%	33%
worst-case input and model weights. Our dFIL-based bound uses the information of the input and model weights in its calculation of the bound and can get a tighter bound.
7.6 Attack based on a diffusion model
We additionally designed a powerful, diffusion model-based reconstruction attacker to study the privacy of dFIL against the best-effort attacker, motivated from the fact that recently developed diffusion models [30] are excellent denoisers. During training of a diffusion model, (1) a particularly-designed noise is added to an image, and (2) a DNN is trained to predict and remove the noise [30]. The first part can be thought of as an instance encoder (that is purposely made easy to invert), and we can calculate its dFIL. The second part can be thought of as a reconstruction attacker. As the noising and denoising are specifically designed for the denoising to work well, we expect a mature, pretrained diffusion model to give a very good attack quality. We used DDPM [30] pretrained with CIFAR-10 from Google [67].
Figure 7 shows the result. The first column of each row shows the original image, and other columns show the reconstruction of our DDPM-based attacker with different dFIL. We scale and show dFIL with the same scale as Figure 5, as DDPM works with a different normalized image that produces dFIL at a different scale. Our new attack provided an interesting result: the attack was able to reconstruct conceptually similar images with the original image even when pixel-by-pixel reconstruction was prohibited by high  $1/\mathrm{dFIL}$ . For example, 7th row at  $1/\mathrm{dFIL} = 49.5$  (6th column) successfully reconstructed a white car with a red background, although the reconstruction MSE was high and the design of the reconstructed car was nothing like the original image. The result shows that high-level information of the image (e.g., the color of the car/background, the orientation of the car, etc.) can still be preserved after encoding with a relatively high  $1/\mathrm{dFIL}$ , which is why it is possible to perform downstream training/inference with a privately-encoded data without revealing the original data.
7.7 Per-pixel FIL
Instead of dFIL which is the average of the Fisher information leakage (FIL) for all the features (pixels), we can directly inspect the per-feature FIL to see which features (pixels) leak more information and can be reconstructed more easily. Figure 8 plots the per-pixel FIL from the split-middle setup of ResNet-18 and CIFAR-10 from Section 4.2.2. The result shows that identifying features of an object, such as the contour of a frog (8th column) or the tire of the car (7th column), are more easily leaked, following our intuition.
7.8 Additional figures and tables
Table 5: The reconstruction quality of an input is highly correlated with dFIL. Correct parts in bold.
1/dFIL	Reconstructed text (from split-early)
10-5	it's a charming and often affecting journey.
1	it's cones charmingound
often affecting journey closure
10	grounds yuki cum sign
recklessound fanuche pm stunt
Figure 7: Results from DDPM [30]-based reconstruction attack. High  $1/$  dFIL prevents exact pixel-to-pixel reconstruction, but images that share some high-level features with the original image can be generated unless  $1/$  dFIL is not too high.
Figure 8: Per-pixel FIL from the ResNet-18 + CIFAR-10 split-middle setup in Section 4.2.2. Image best viewed with colors.
(a) 1/dFIL vs. SSIM
(b)  $1 / \mathrm{dFIL}$  vs. reconstructed image quality
Figure 9: Optimizer-based attack with total variation (TV) prior [49] against our split inference system in Section 4. The trend is very similar to Figure 5.
Figure 10: More reconstruction result of Figure 2.
Figure 11: More reconstruction result of Figure 3.
Figure 12: More reconstruction result of Figure 4.
Figure 13: Ten reconstructions with the best and worst SSIM values for various setups. The result is an extension of Figure 5. Images with a simple shape and high color contrast tend to be reconstructed more easily, which matches our intuition. Omitting results from  $1 / \mathrm{dFIL} = 100$ , as no meaningful reconstruction was possible.
Figure 14: Ten images with the lowest and highest dFIL values, for split-middle setup in Figure 5. Images with high dFIL tend to have a simpler shape and a high color contrast, potentially being easier to reconstruct. Individual dFIL value of each samples can potentially be used to detect data that are more leaky.