Title: Bounding the Invertibility of Privacy-preserving Instance Encoding using Fisher Information
Venue: NeurIPS 2023
Accepted: yes
Keywords: privacy

instance encoding

split learning
Abstract: Privacy-preserving instance encoding aims to encode raw data into feature vectors without revealing their privacy-sensitive information. When designed properly, these encodings can be used for downstream ML applications such as training and inference with limited privacy risk. However, the vast majority of existing schemes do not theoretically justify that their encoding is non-invertible, and their privacy-enhancing properties are only validated empirically against a limited set of attacks. In this paper, we propose a theoretically-principled measure for the invertibility of instance encoding based on Fisher information that is broadly applicable to a wide range of popular encoders. We show that dFIL can be used to bound the invertibility of encodings both theoretically and empirically, providing an intuitive interpretation of the privacy of instance encoding.

Bounding the Invertibility of Privacy-Preserving Instance Encoding Using Fisher Information

Abstract

Privacy-preserving instance encoding aims to encode raw data into feature vectors without revealing their privacy-sensitive information. When designed properly, these encodings can be used for downstream ML applications such as training and inference with limited privacy risk. However, the vast majority of existing schemes do not theoretically justify that their encoding is non-invertible, and their privacyenhancing properties are only validated empirically against a limited set of attacks. In this paper, we propose a theoretically-principled measure for the invertibility of instance encoding based on Fisher information that is broadly applicable to a wide range of popular encoders. We show that dFIL can be used to bound the invertibility of encodings both theoretically and empirically, providing an intuitive interpretation of the privacy of instance encoding.

1 Introduction

Machine learning (ML) applications often require access to privacy-sensitive data. A model that predicts diseases with x-ray scans requires access to patient's x-ray images. Next-word prediction models require access to user's text input that can contain sensitive information [23]. To enable ML applications on privacy-sensitive data, instance encoding ([6]; Figure 1, left) aims to encode data in a way such that it is possible to run useful ML tasks-such as model training and inference-on the encoded data while the privacy of the raw data is preserved. The concept is widespread under many different names: private data sharing [16, 17, 48, 5, 34, 32, 37], learnable encryption [35, 73, 71, 70], split learning [63, 55], split inference [40, 11], and vertical federated learning (vFL; [74, 59, 45]) are all collaborative training/inference schemes that operate on (hopefully) privately-encoded user data.

Unfortunately, the vast majority of existing instance encoding schemes rely on heuristics rather than rigorous theoretical arguments to justify their privacy-enhancing properties. Most existing works [35, 73, 64, 65, 45] claimed that their schemes are non-invertible by simply testing them against certain input reconstruction attacks. However, these schemes may not be private under more carefully designed attacks; in fact, many encoding schemes that were initially thought to be private have been shown to be vulnerable over time [6, 7]. While there exist instance encoding schemes that are based on a rigorous theory, e.g. , metric differential privacy (metric-DP; [8, 16, 17, 44]), their theoretical analysis is only applicable to very simple encoders and cannot be applied to more powerful encoders ( e.g. , DNN-based encoders) that are common in practice [73, 64, 65, 45, 5, 34, 32].

Enc

Private

x

Encoding

e

ML training

or inference

10

4

10

2

10 0

10 2

1/dFIL

10

3

10 1

Recon. MSE

Bound-ub

Bound-ours

Attack-ub

Attack-b

To fill the gap, we propose a theoretically-principled and easily applicable framework for quantifying the privacy of instance encoding by bounding its invertibility. Our framework uses (diagonal) Fisher information leakage (dFIL; [18, 22, 20])-an information-theoretic measure of privacy with similar properties to differential privacy (DP; [14, 15]). dFIL can be computed for common privacyenhancing mechanisms and can lower-bound the expected mean squared error (MSE) of an input reconstruction attack. We apply this reasoning to instance encoding and show that dFIL can serve as a good measure for the invertibility of instance encoding, as it can bound the reconstruction MSE against any attacker that tries to reconstruct the private input from the encoding. dFIL can be easily calculated for any encoders that are differentiable and randomized-being applicable to most of the popular encoders with minimal modifications. To the best of our knowledge, our work is the first to theoretically lower-bound the invertibility of instance encoding and use it to design private training/inference systems.

Contributions Our main contributions are as follows:

We adapt the result of prior works [18, 20] for instance encoding to show how dFIL can lower bound the MSE of particular input reconstruction attacks ( i.e. , unbiased attacks) that aim to reconstruct the raw data given the encoding. We show how popular encoders can be modified minimally for dFIL to be applied (Section 3.1).

We extend the result of prior works [18, 20] and show that dFIL can lower bound the MSE of any input reconstruction attack ( e.g. , strong attacks leveraging knowledge of the input prior; Section 3.2). Our extension involves a novel application of the classical van Trees inequality [62] and connecting it to the problem of score matching [57] in distribution estimation.

We evaluate the lower bound using different attacks and encoders, and show that dFIL can be used to interpret the invertibility of instance encoding both theoretically and empirically (Section 3.3).

We show how dFIL can be used as a practical privacy metric and guide the design of privacyenhancing training/inference systems with instance encoding (Section 4-5). We show that it is possible to achieve both high (theoretically-justified) privacy and satisfactory utility.

2 Motivation and background

2.1 Instance encoding

Instance encoding refers to the general concept of encoding raw input x using an encoding function Enc , such that private information contained in x cannot be inferred from its encoding e = Enc( x ) , while e maintains enough utility for the downstream ML tasks. The principle behind the privacyenhancing property of instance encoding is that the function Enc is hard to invert. However, prior works generally justify this claim of non-invertibility based on heuristics rather than rigorous theoretical analysis [64, 65, 45], except for a few cases where the encoder is very simple [16, 17, 44].

Attacks against instance encoding Given an instance encoder Enc , the goal of a reconstruction attack is to recover its input. Formally, given e = Enc( x ) , an attack Att aims to reconstruct x from e : ˆ x = Att( e ) . Such an attack can be carried out in several ways. If Enc is known, ˆ x can be obtained by solving the following optimization [27]: ˆ x = arg min || e -Enc( x 0 ) || 2 2 . This attack can be further

0

x

improved when some prior of the input is known [49, 61]. For instance, images can be regularized with total variation (TV) prior to reduce high-frequency components [49]. Alternatively, if samples from the input distribution can be obtained, a DNN that generates ˆ x from e can be trained [53, 27, 12].

Privacy metrics for instance encoding Measuring how much information an observation leaks about a secret quantity it depends on is a classical question that has long been studied in information theory [38, 3, 13, 33], and instance encoding privacy is directly relevant to the stream of works. Prior works proposed metrics providing a strong notion of privacy ( e.g. , Arimoto's mutual information [3] or Sibson's mutual information [38]). However, it is challenging to apply these metrics to ML and instance encoding which use high-dimensional data [33], because these metrics have to be calculated and optimized over the prior of the dataset, the encodings, and/or their joint distribution, all of which are unknown and hard to accurately model [33].

Differential privacy (DP) [14, 15], one of the most popular frameworks to quantify ML privacy [1], is not suitable for instance encoding [6]. DP guarantees the worst-case indistinguishability of the encoding from two different inputs, which significantly damages the utility [6] (Appendix 7.5). Metric-DP [8]-a weaker notion of DP-is applicable to instance encoding and has been used by several prior works [16, 17, 44, 48, 37]. However, metric-DP is applicable only when the encoder is extremely simple. For example, prior works used pixel-level perturbation ( e.g. , pixelization, blurring [16, 17]) or a single-layer convolution [44] as an encoder. These extremely-simple encoders are in stark contrast with the complex encoders most practitioners use ( e.g. , split learning [64], split inference [65], and vFL [45, 59] all uses DNN-based encoders), and results in a significant utility loss (Section 4.2). Applying metric-DP to an arbitrary DNN-based encoder is NP-hard [66]. Unlike these works that focus on indistinguishability, our work directly quantifies non-invertibility , a weaker but still relevant definition of privacy (Section 3).

Due to the limitations in existing theoretical metrics, the vast majority of works in the field are relying on empirical measures to demonstrate their encoder's privacy-enhancing properties. Some [46, 45] simply test their encoders against a limited set of attacks, while others [64, 65, 51] use heuristical privacy metrics without rigorous theoretical arguments. Using heuristical metrics that are not backed by a theoretical interpretation is dangerous, because a seemingly useful metric can actually severely mis-characterize the privacy of a system [38]. It is of both interest and practical importance to develop a privacy metric that is both theoretically-meaningful and widely applicable.

A concurrent work [72] proposed a notion of PAC privacy that can bound the adversary's attack success rate in theory for instance encoding. PAC privacy bound has not yet been evaluated empirically under a realistic attack and use cases, which remains an interesting future work.

2.2 Fisher information leakage

Fisher information leakage (FIL; [18, 22, 20]) is a measure of leakage through a privacy-enhancing mechanism. Let M be a randomized mechanism on data sample x , and let o ∼ M ( x ) be its output. Suppose the log density function log p ( o ; x ) is differentiable w.r.t. x and satisfies the following regularity condition: E o [ ∇ x log p ( o ; x ) | x ] = 0 . The Fisher information matrix (FIM) I o ( x ) is:

Cramér-Rao bound Fisher information is a compelling privacy metric as it directly relates to the mean squared error (MSE) of a reconstruction adversary through the Cramér-Rao bound [41]. In detail, suppose that ˆ x ( o ) is an unbiased estimate (or reconstruction) of x given the output of the randomized private mechanism o ∼ M ( x ) . Then:

where d is the dimension of x and Tr is the trace of a matrix. Guo et al. [20] defined a scalar summary of the FIM called diagonal Fisher information leakage (dFIL):

hence the MSE of an unbiased reconstruction is lower bounded by the reciprocal of dFIL. dFIL varies with x , allowing it to reflect the fact that certain samples may be more vulnerable to reconstruction.

Limitations Although the Cramér-Rao bound gives a mathematically rigorous interpretation of dFIL, it depends crucially on the unbiasedness assumption, i.e., E o [ˆ x ( o )] = x . In practice, most real-world reconstruction attacks use either implicit or explicit priors about the data distribution and are biased ( e.g. , attacks using TV prior or a DNN). It is unclear how dFIL should be interpreted in these more realistic settings. In Section 3.2, we give an alternative theoretical interpretation based on the van Trees inequality [62], which lower-bounds the MSE of any adversary, biased or unbiased.

3 Quantifying the invertibility of an encoding

Motivated by the lack of theoretically-principled privacy metrics, we propose to adapt the Fisher information leakage framework to quantify the invertibility of instance encoding. We show that many existing encoders can be modified minimally to be interpreted with dFIL and the Cramér-Rao bound. Subsequently, we extend the framework and derive a novel bound for the reconstruction error of arbitrary attacks, by establishing a connection to van Trees inequality and score matching.

Threat model We focus on reconstruction attacks that aim to invert an encoding, i.e. , reconstruct the input x given its encoding e . We assume that the attacker has full knowledge of the encoder Enc except for the source of randomness. We consider both unbiased attacks and biased attacks that can use arbitrary prior knowledge about the data distribution.

Privacy definition We consider Enc to be private if x cannot be reconstructed from the encoding e . We measure the reconstruction error with the mean squared error (MSE), defined as || ˆ x -x || 2 2 /d . Although MSE does not exactly indicate semantic similarity, it is widely applicable and used as a proxy for semantic similarity [69, 43]. Preventing low reconstruction MSE does not necessarily protect against other attacks ( e.g. , property inference [50]), which we leave as future work.

3.1 Fisher information leakage for instance encoding

To adapt the framework of Fisher information to the setting of instance encoding, we consider the encoding function Enc as a privacy-enhancing mechanism ( cf. M in Section 2.2) and use dFIL to measure the invertibility of the encoding e = Enc( x ) . However, many instance encoders do not meet the regularity conditions in Section 2.2, making dFIL ill-defined. For example, popular DNN-based encoders do not produce randomized output, and their log density function log p ( o ; x ) may not be differentiable when operators like ReLU or max pooling are present.

Fortunately, many popular encoders can meet the required conditions with small changes. For example, DNN-based encoders can be modified by (1) replacing any non-smooth functions with smooth functions ( e.g. , tanh or GELU [29] instead of ReLU, average pooling instead of max pooling), and (2) adding noise at the end of the encoder for randomness. Alternative modifications are also possible. In particular, if we add random Gaussian noise to the output of a differentiable, deterministic encoder Enc D ( e.g., DNN): Enc( x ) = Enc D ( x ) + N (0 , σ 2 ) , the FIM of the encoder becomes [22]:

where J Enc D is the Jacobian of Enc D with respect to the input x and can be easily computed using a single backward pass. Then, Equation 2 can be used to bound the reconstruction error, provided the attack is unbiased . Other popular encoders can be modified similarly.

3.2 Bounding the reconstruction of arbitrary attacks

Most realistic reconstruction attacks are biased, and their reconstruction MSE is not lower bounded by Equation 2. Consider an (biased) attacker who knows the mean µ of the input data distribution. If the attacker simply outputs µ as the reconstruction of any input x , the expected MSE will be the variance of the data distribution regardless of dFIL . The above example shows a crucial limitation of the Cramér-Rao bound interpretation of dFIL: it does not take into account any prior information the adversary has about the data distribution, which is abundant in the real world (Section 2.1).

Bayesian interpretation of Fisher information We adopt a Bayesian interpretation of dFIL as the difference between an attacker's prior and posterior estimate of the input x . This is achieved through the classical van Trees inequality [62]. We state the van Trees inequality in Appendix 7.3, and use it to derive our MSE bound for arbitrary attacks below as a corollary; proof is in Appendix 7.4.

Corollary 1. Let π be the input data distribution and let f π ( x ) denote its density function with respect to Lebesgue measure. Suppose that π satisfies the regularity conditions of van Trees inequality (Theorem 2), and let J ( f π ) = E π [ ∇ x log f π ( x ) ∇ x log f π ( x ) ⊤ ] denote the information theorist's Fisher information [2] of π . For a private mechanism M and any reconstruction attack ˆ x ( o )

10

5

10

4

10

3

10

2

10

1

10 0

10 1

10 2

1/dFIL

10

3

10 0

10 3

Recon. MSE

Bound-ub

Bound-ours

Attack-ub

Attack-b

ref

1

10

25

operating on o ∼ M ( x ) :

Implications of Corollary 1 We can readily apply Corollary 1 to instance encoding by replacing M with Enc and o with e as in Section 3.1. Doing so leads to several interesting implications:

Corollary 1 is a population-level bound that takes expectation over x ∼ π . This is necessary because given any fixed sample x , there is always an attack ˆ x ( e ) = x that perfectly reconstructs x without observing the encoding e . Such an attack would fail in expectation over x ∼ π .

The term J ( f π ) captures the prior knowledge about the input. When J ( f π ) = 0 , the attacker has no prior information about x , and Corollary 1 reduces to the unbiased bound in Equation 2. When J ( f π ) is large, the bound becomes small regardless of E π [dFIL( x )] , indicating that the attacker can simply guess with the input prior and achieve a low MSE.

dFIL can be interpreted as capturing how much easier reconstructing the input becomes after observing the encoding ( E π [dFIL( x )] term) as opposed to only having knowledge of the input distribution ( Tr( J ( f π )) /d term). Using such a change in the belief after an observation as a privacy metric is common in information theory [38, 13].

Estimating J ( f π ) The term J ( f π ) captures the prior knowledge of the input and plays a crucial role in Corollary 1. In simple cases where π is a known distribution whose density function follows a tractable form ( e.g. , when the input follows a Gaussian distribution), J ( f π ) can be directly calculated. In such settings, Corollary 1 gives a meaningful theoretical lower bound for the reconstruction MSE. However, most real-world data distributions do not have a tractable form and J ( f π ) must be estimated from data. Fortunately, the ∇ x log f π ( x ) term in J ( f π ) is a well-known quantity called the score function in generative AI literature, and there exists a class of algorithms known as score matching [36, 47, 57] that aim to estimate the score function from the data. We leverage these techniques to estimate J ( f π ) when it cannot be calculated; details are in Appendix 7.1. As J ( f π ) only depends on the dataset and not the encoder, its value can be estimated once for each dataset and shared across the community. The estimation of J ( f π ) is expected to improve with the advancement of generative AI.

Using Corollary 1 in practice When J ( f π ) is known ( e.g. , Gaussian), the bound from Corollary 1 always hold. However, when estimating J ( f π ) from data, the bound can be incorrect due to several reasons, including improper modeling of the score function, not having enough representative samples, or the regularity conditions of the van Trees inequality not being met. The bound can also be loose when tightness conditions of the van Trees inequality do not hold. Even when the bound is not exact, however, Equations 2 and 5 can be interpreted to suggest that increasing 1 / dFIL makes reconstruction harder. Thus, we argue that dFIL still serves as a useful privacy metric that in theory bounds the invertibility of an instance encoding. When not exact, the bound should be viewed more as a guideline for interpreting and setting dFIL in a data-dependent manner.

3.3 Evaluation of the bound

We show that Corollary 1 accurately reflects the reconstruction MSE on both (1) synthetic data with known J ( f π ) , and (2) real world data with estimated J ( f π ) .

10

5

10

4

10

3

10

2

10

1

10 0

10 1

10 2

1/dFIL

10

3

10 0

10 3

Recon. MSE

Bound-ub

Bound-ours

Attack-ub

Attack-b

ref

0.1

1

10

10

5

10

4

10

3

10

2

10

1

10 0

10 1

10 2

1/dFIL

10

3

10 0

10 3

Recon. MSE

Bound-ub

Bound-ours

Attack-ub

Attack-b

ref

0.1

1

10

3.3.1 Synthetic data with known J ( f π )

Evaluation setup We consider a synthetic Gaussian input distribution: x ∼ N (0 , τ 2 I d ) with d = 784 and τ = 0 . 05 . It can be shown that Tr( J ( f π )) /d = 1 /τ 2 , hence a larger τ forces the data to spread out more and reduces the input prior. We use a simple encoder which randomly projects the data to a 10 , 000 -dimensional spaces and then adds Gaussian noise, i.e. , e = Mx + N (0 , σ 2 ) , where M ∈ R 10 , 000 × 784 . We evaluate with a simple encoder to make the attack easier, but note that our bound should hold for more complex encoders ( e.g. , DNN-based) as well.

Attacks We evaluate our bound against two different attacks. An unbiased attack ( Attack-ub ) solves the following optimization: ˆ x ( e ) = arg min x 0 || e -Enc( x 0 ) || 2 2 . The attack is unbiased as the objective is convex, and x is recovered in expectation. A more powerful biased attack ( Attack-b ) adds a regularizer term λ log p τ ( x 0 ) to the above objective, where p τ is the density function of N (0 , τ 2 I d ) . One can show that with a suitable choice of λ , this attack returns the maximum a posteriori estimate of x , which leverages knowledge of the input distribution. Details are in Appendix 7.2.

Result Figure 1 (right) plots the MSE of the two attacks, and the bounds for unbiased (Equation 2) and arbitrary attack (Equation 5). The MSE of Attack-ub (red circle) matches the unbiased attack lower bound ( Bound-ub ; red dashed line), showing the predictive power of Equation 2 against this restricted class of attacks. Under Attack-b (blue triangle), however, Bound-ub breaks. Our new bound from Equation 5 ( Bound-ours ; blue dotted line) reliably holds for both attacks, initially being close to the unbiased bound and converging to guessing only with the input prior (attaining τ 2 ).

3.3.2 Real world data with estimated J ( f π )

Evaluation setup We also evaluated Corollary 1 on MNIST [10] and CIFAR-10 [42]. Here, we estimated J ( f π ) using sliced score matching [57]. As discussed in Appendix 7.1, a moderate amount of randomized smoothing (adding Gaussian noise to the raw input [9]) is necessary to ensure that the score estimation is stable and that regularity conditions of the van Trees inequality are satisfied. We used a simple CNN-based encoder: e = Conv( x ) + N (0 , σ 2 ) .

Attacks We evaluated Attack-ub , which is the same as in Section 3.3.1, and Attack-b , which is a trained DNN that outputs the reconstruction given an encoding [45]. We also evaluated regularizationbased attacks [49, 61] and obtained similar results; we omit those results for brevity.

Result Figures 2 and 3 plot the result with a randomized smoothing noise of N (0 , 0 . 25 2 ) . Again, Bound-ub correctly bounds the MSE achieved by Attack-ub . While Attack-b is not as effective for very low 1 / dFIL , it outperforms Attack-ub for high 1 / dFIL , breaking Bound-ub . In comparison, Corollary 1 estimated using score matching ( Bound-ours ) gives a valid lower bound for both attacks.

Figures 2 and 3 also highlights some of the reconstructions visually. Here, the left-hand side number indicates the target 1 / dFIL and the images are reconstructed using Attack-b . In both figures, it can be seen that dFIL correlates well with the visual quality of reconstructed images, with higher values of 1 / dFIL indicating less faithful reconstructions. See Appendix: Figure 10-11 for more results.

Figure 4 additionally shows the result with a much smaller randomized smoothing noise of N (0 , 0 . 01 2 ) . Unlike previous results, Bound-ours breaks around 1 / dFIL = 10 -3 . We suspect it is due to score matching failing when the data lie on a low-dimensional manifold and the likelihood changes rapidly near the manifold boundary, which can be the case when the smoothing noise is small. Nonetheless, the bound still correlates well with MSE and the visual reconstruction quality. We claim that dFIL still serves as a useful privacy metric, with a theoretically-principled interpretation and a strong empirical correlation to invertibility. More reconstructions are in Appendix: Figure 12.

4 Case study 1: split inference with dFIL

4.1 Private split inference with dFIL

Split inference [40, 4, 65] is a method to run inference of a large DNN that is hosted on the server, without the client disclosing raw input. It is done by running the first few layers of a large DNN on the client device and sending the intermediate activation, instead of raw data, to the server to complete the inference. The client computation can be viewed as instance encoding, where the first few layers on the client device act as an encoder. However, without additional intervention, split inference by itself is not private because the encoding can be inverted [27].

We design a private split inference system by measuring and controlling the invertibility of the encoder with dFIL. Because the encoder of split inference is a DNN, dFIL can be calculated using Equation 4 with minor modifications to the network (Section 3.1).

Optimizations There are several optimizations to improve the model accuracy for the same dFIL: (1) We calculate the amount of noise that needs to be added to the encoding to achieve a target dFIL, and add a similar amount of noise during training. (2) For CNNs, we add a compression layer-a convolution layer that reduces the channel dimension significantly-at the end of the encoder and a corresponding decompression layer at the beginning of the server-side model. Similar heuristics were explored in prior works [11, 45]. (3) We add an SNR regularizer that is designed to maximize the signal-to-noise ratio of the encoding. From Equations 3-4, the noise that needs to be added to √ Tr( J ⊤ ( x ) J Enc ( x ))

D

d

achieve a certain dFIL is

σ

=

∗

dFIL

. Thus, maximizing the signal-to-noise ratio

⊤

(SNR) of the encoding ( e ⊤ e /σ 2 ) is equivalent to minimizing Tr( J Enc D ( x ) J Enc D ( x )) e ⊤ e , which we add to the optimizer during training. These optimizations were selected from comparing multiple heuristics from prior work [60, 28, 46, 65, 45, 11], and result in a notable improvement of test accuracy.

4.2 Evaluation of dFIL-based split inference

4.2.1 Evaluation setup

Models and datasets We used three different models and four different datasets to cover a wide range of applications: ResNet-18 [25] with CIFAR-10/100 [42] for image classification, MLP-based neural collaborative filtering (NCF-MLP) [26] with MovieLens-20M [24] for recommendation, and DistilBert [56] with GLUE-SST2 [68] for sentiment analysis. See Appendix 7.2 for details.

Detailed setups and attacks For ResNet-18, we explored three split inference configurations: splitting early (after the first convolution layer), in the middle (after block 4), and late (after block 6). We evaluated the empirical privacy with a DNN attacker [45] and measured the reconstruction quality with structural similarity index measure (SSIM) [31]. Other popular attacks showed similar trends (see Appendix: Figure 9). NCF-MLP translates a user id (uid) and a movie id (mid) into embeddings

Enc

D

10

5

10

4

10

3

10

2

10

1

10 0

10 1

10 2

1/dFIL

0.0

0.5

1.0

SSIM

early

middle

late

middle early

10

5

late

10

4

10

3

10

2

1/dFIL

10

1

10 0

10 1

10 2

10

5

10

4

10

3

10

2

10

1

10 0

10 1

1/dFIL

0.0

0.5

1.0

Attack acc.

uid top 1

mid top 1

uid top 5

mid top 5

10

5

10

4

10

3

10

2

10

1

10 0

10 1

10 2

1/dFIL

0.0

0.5

1.0

Attack acc.

early

middle

late

with an embedding table and sends them through a DNN to make a prediction. We split the NCFMLP model after the first linear layer of the MLP and tried reconstructing the original uid and mid from the encoding. This is done by first reconstructing the embeddings from the encoding using direct optimization ( ˆ emb ( e ) = arg min emb 0 || e -Enc( emb 0 ) || 2 2 ), and finding the original uid and mid by choosing the closest embedding value in the embedding table: id = arg min || ˆ emb -Emb [ i ] || 2 2 ,

i

where Emb [ i ] is the i -th entry of the embedding table. For DistilBert, we again explored three different splitting configurations: splitting early (right after block 0), in the middle (after block 2), and late (after block 4). We use a similar attack to NCF-NLP to retrieve each word token.

4.2.2 Evaluation results

Privacy Figure 5 shows the attack result for ResNet-18 and CIFAR-10. Setups with lower dFIL lead to lower SSIM (left) and less identifiable images (right), indicating that dFIL strongly correlates with the attack success rate. The figures also show that the privacy leakage estimated by dFIL can be conservative. Some setups show empirically-high privacy even when dFIL indicates otherwise, especially when splitting late. See Appendix: Figure 13 for more results. Figures 6 show the attack result for NCF-MLP (left) and DistilBert (right). Setups with lower dFIL again consistently showed a worse attack success rate. A sample reconstruction for DistilBert is shown in Appendix: Table 5.

Utility Table 1 summarizes the test accuracy of the split inference models, where 1 / dFIL is chosen so that the attacker's reconstruction error is relatively high. For the same 1 / dFIL , our proposed optimizations ( Ours ) improve the accuracy significantly compared to simply adding noise ( No opt. ). In general, reasonable accuracy can be achieved with encoders with relatively low dFIL. Accuracy degrades more when splitting earlier, indicating that using a DNN encoder with many layers can benefit utility. Prior works empirically observed a similar trend where splitting earlier degrades the privacy-utility tradeoff, because the encoding becomes leakier [49]. Our framework supports complex DNN encoders, which is a distinguishing benefit compared to prior theoretical works [16, 17, 44].

5 Case study 2: training with dFIL

5.1 Training on encoded data with dFIL

We consider a scenario where users publish their encoded private data, and a downstream model is trained on the encoded data. We use the first few layers of a pretrained model as the encoder by

freezing the weights and applying the necessary changes in Section 3.1. Then, we use the rest of the model with its last layer modified for the downstream task and finetune it with the encoded data. We found that similar optimizations from split inference benefit this use case as well.

5.2 Evaluation of dFIL-based training

We evaluate the model utility and show that it can reach a reasonable accuracy when trained on encoded data. We omit the privacy evaluation as it is similar to Section 4.2.2.

Evaluation setup We train a ResNet-18 model for CIFAR-10 classification. First, we train the model on one of three different datasets: (1) TinyImageNet [58], (2) CIFAR-100 [42], and (3) held-out 20% of CIFAR-10. Then, layers up to block 4 are frozen and used as the encoder. The CIFAR-10 training set is encoded using the encoder and used to finetune the rest of the model. The setup mimics a scenario where some publicly-available data whose distribution is similar (TinyImageNet, CIFAR-100) or the same (held-out CIFAR-10) with the target data is available and is used for encoder training. Detailed hyperparameters are in Appendix 7.2.

Evaluation result Table 2 summarizes the result. Our design was able to achieve a decent accuracy using encoded data with relatively safe 1 / dFIL values (10-100). The result indicates that model training with privately encoded data is possible. The achieved accuracy was higher when the encoder was trained with data whose distribution is more similar to the downstream task (CIFAR-10). We believe more studies in hyperparameter/architecture search will improve the result.

6 Limitations

We showed that dFIL can theoretically quantify the invertibility of instance encoding by providing a reconstruction error bound against arbitrary attackers. We subsequently showed that dFIL can be used to guide private training/inference systems that uses instance encoding. dFIL has several potential limitations, which needs to be carefully considered before being used:

1. Corollary 1 only bounds the MSE, which might not always correlate well with the semantic quality. To address this, van Trees inequality can be extended to an absolutely continuous function ψ ( x ) to bound E [ || ψ (ˆ x ) -ψ ( x ) || 2 2 /d ] [19], which may be used to extend to metrics other than MSE.

2. Equation 5 provides an average case bound, so some data may experience lower reconstruction MSE than the bound. Similar average case bounds have also been used in prior works [13, 21]. When bounding invertibility, it is unclear how to define the privacy attack game to target worst-case reconstruction bound: for any sample, there exists a trivial attacker that always outputs that fixed sample from the distribution, which gives a worst-case MSE of zero for that sample. Leakier data that might experience lower reconstruction MSE than the bound can be potentially detected and handled by dynamically calculating dFIL for each sample. See Appendix: Figure 14.

3. For data types where MSE is not directly meaningful or the bound is inaccurate, it may not be straightforward to interpret the privacy of an encoding given its dFIL. In such cases, acceptable values of dFIL should be determined for each application through further research. The situation is similar to DP, where it is often not straightforward what privacy parameters ( e.g. , ϵ , δ ) need to be used [39].

4. Systems with the same dFIL may actually have different invertibility, as the bound from dFIL may be conservative. Comparing the privacy of two different systems using dFIL should be done with caution because dFIL is a lower bound rather than an accurate measure of invertibility.

Acknowledgments and Disclosure of Funding

We thank Yang Song for the invaluable discussion regarding the application of sliced score matching to our setup. We also thank Minjae Park and Jisu Kim for the discussion of our bound's implications. Kiwan Maeng was partially funded by the Charles K. Etner Early Career Professorship from Penn State University.

References

Martín Abadi, Andy Chu, Ian J. Goodfellow, H. Brendan McMahan, Ilya Mironov, Kunal Talwar, and Li Zhang. Deep learning with differential privacy. In Edgar R. Weippl, Stefan Katzenbeisser, Christopher Kruegel, Andrew C. Myers, and Shai Halevi, editors, Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security, Vienna, Austria, October 24-28, 2016 , pages 308-318. ACM, 2016.

Efe Aras, Kuan-Yun Lee, Ashwin Pananjady, and Thomas A. Courtade. A family of bayesian cramér-rao bounds, and consequences for log-concave priors. In IEEE International Symposium on Information Theory, ISIT 2019, Paris, France, July 7-12, 2019 , pages 2699-2703. IEEE, 2019.

Shahab Asoodeh, Mario Díaz, Fady Alajaji, and Tamás Linder. Estimation efficiency under privacy constraints. IEEE Trans. Inf. Theory , 65(3):1512-1534, 2019.

Amin Banitalebi-Dehkordi, Naveen Vedula, Jian Pei, Fei Xia, Lanjun Wang, and Yong Zhang. Auto-split: a general framework of collaborative edge-cloud ai. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining , pages 2543-2553, 2021.

Martín Bertrán, Natalia Martínez, Afroditi Papadaki, Qiang Qiu, Miguel R. D. Rodrigues, Galen Reeves, and Guillermo Sapiro. Adversarially learned representations for information obfuscation and inference. In Kamalika Chaudhuri and Ruslan Salakhutdinov, editors, Proceedings of the 36th International Conference on Machine Learning, ICML 2019, 9-15 June 2019, Long Beach, California, USA , volume 97 of Proceedings of Machine Learning Research , pages 614-623. PMLR, 2019.

Nicholas Carlini, Samuel Deng, Sanjam Garg, Somesh Jha, Saeed Mahloujifar, Mohammad Mahmoody, Shuang Song, Abhradeep Thakurta, and Florian Tramèr. An attack on instahide: Is private learning possible with instance encoding? CoRR , abs/2011.05315, 2020.

Nicholas Carlini, Sanjam Garg, Somesh Jha, Saeed Mahloujifar, Mohammad Mahmoody, and Florian Tramèr. Neuracrypt is not private. CoRR , abs/2108.07256, 2021.

Konstantinos Chatzikokolakis, Miguel E. Andrés, Nicolás Emilio Bordenabe, and Catuscia Palamidessi. Broadening the scope of differential privacy using metrics. In Emiliano De Cristofaro and Matthew K. Wright, editors, Privacy Enhancing Technologies - 13th International Symposium, PETS 2013, Bloomington, IN, USA, July 10-12, 2013. Proceedings , volume 7981 of Lecture Notes in Computer Science , pages 82-102. Springer, 2013.

Jeremy M. Cohen, Elan Rosenfeld, and J. Zico Kolter. Certified adversarial robustness via randomized smoothing. In Kamalika Chaudhuri and Ruslan Salakhutdinov, editors, Proceedings of the 36th International Conference on Machine Learning, ICML 2019, 9-15 June 2019, Long Beach, California, USA , volume 97 of Proceedings of Machine Learning Research , pages 1310-1320. PMLR, 2019.

Li Deng. The mnist database of handwritten digit images for machine learning research. IEEE Signal Processing Magazine , 29(6):141-142, 2012.

Xin Dong, Barbara De Salvo, Meng Li, Chiao Liu, Zhongnan Qu, HT Kung, and Ziyun Li. Splitnets: Designing neural architectures for efficient distributed computing on headmounted systems. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 12559-12569, 2022.

Alexey Dosovitskiy and Thomas Brox. Inverting visual representations with convolutional networks. In 2016 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2016, Las Vegas, NV, USA, June 27-30, 2016 , pages 4829-4837. IEEE Computer Society, 2016.

Flávio du Pin Calmon and Nadia Fawaz. Privacy against statistical inference. In 50th Annual Allerton Conference on Communication, Control, and Computing, Allerton 2012, Allerton Park &Retreat Center, Monticello, IL, USA, October 1-5, 2012 , pages 1401-1408. IEEE, 2012.

Cynthia Dwork, Frank McSherry, Kobbi Nissim, and Adam Smith. Calibrating noise to sensitivity in private data analysis. In Theory of cryptography conference , pages 265-284. Springer, 2006.

Cynthia Dwork, Aaron Roth, et al. The algorithmic foundations of differential privacy. Foundations and Trends® in Theoretical Computer Science , 9(3-4):211-407, 2014.

Liyue Fan. Image pixelization with differential privacy. In Florian Kerschbaum and Stefano Paraboschi, editors, Data and Applications Security and Privacy XXXII - 32nd Annual IFIP WG 11.3 Conference, DBSec 2018, Bergamo, Italy, July 16-18, 2018, Proceedings , volume 10980 of Lecture Notes in Computer Science , pages 148-162. Springer, 2018.

Liyue Fan. Differential privacy for image publication. In Theory and Practice of Differential Privacy (TPDP) Workshop , volume 1, page 6, 2019.

Farhad Farokhi and Henrik Sandberg. Fisher information as a measure of privacy: Preserving privacy of households with smart meters using batteries. IEEE Trans. Smart Grid , 9(5):47264734, 2018.

Richard D Gill and Boris Y Levit. Applications of the van trees inequality: a bayesian cramér-rao bound. Bernoulli , pages 59-79, 1995.

Chuan Guo, Brian Karrer, Kamalika Chaudhuri, and Laurens van der Maaten. Bounding training data reconstruction in private (deep) learning. In Kamalika Chaudhuri, Stefanie Jegelka, Le Song, Csaba Szepesvári, Gang Niu, and Sivan Sabato, editors, International Conference on Machine Learning, ICML 2022, 17-23 July 2022, Baltimore, Maryland, USA , volume 162 of Proceedings of Machine Learning Research , pages 8056-8071. PMLR, 2022.

Chuan Guo, Alexandre Sablayrolles, and Maziar Sanjabi. Analyzing privacy leakage in machine learning via multiple hypothesis testing: A lesson from fano. arXiv preprint arXiv:2210.13662 , 2022.

Awni Hannun, Chuan Guo, and Laurens van der Maaten. Measuring data leakage in machinelearning models with fisher information. In Uncertainty in Artificial Intelligence , pages 760-770. PMLR, 2021.

Andrew Hard, Kanishka Rao, Rajiv Mathews, Swaroop Ramaswamy, Françoise Beaufays, Sean Augenstein, Hubert Eichner, Chloé Kiddon, and Daniel Ramage. Federated learning for mobile keyboard prediction. arXiv preprint arXiv:1811.03604 , 2018.

F. Maxwell Harper and Joseph A. Konstan. The movielens datasets: History and context. ACM Trans. Interact. Intell. Syst. , 5(4):19:1-19:19, 2016.

Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In 2016 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2016, Las Vegas, NV, USA, June 27-30, 2016 , pages 770-778. IEEE Computer Society, 2016.

Xiangnan He, Lizi Liao, Hanwang Zhang, Liqiang Nie, Xia Hu, and Tat-Seng Chua. Neural collaborative filtering. In Rick Barrett, Rick Cummings, Eugene Agichtein, and Evgeniy Gabrilovich, editors, Proceedings of the 26th International Conference on World Wide Web, WWW2017, Perth, Australia, April 3-7, 2017 , pages 173-182. ACM, 2017.

Zecheng He, Tianwei Zhang, and Ruby B. Lee. Model inversion attacks against collaborative inference. In David Balenson, editor, Proceedings of the 35th Annual Computer Security Applications Conference, ACSAC 2019, San Juan, PR, USA, December 09-13, 2019 , pages 148-162. ACM, 2019.

Zecheng He, Tianwei Zhang, and Ruby B Lee. Attacking and protecting data privacy in edgecloud collaborative inference systems. IEEE Internet of Things Journal , 8(12):9706-9716, 2020.

Dan Hendrycks and Kevin Gimpel. Gaussian error linear units (gelus). arXiv preprint arXiv:1606.08415 , 2016.

Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. In Hugo Larochelle, Marc'Aurelio Ranzato, Raia Hadsell, Maria-Florina Balcan, and Hsuan-Tien Lin, editors, Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6-12, 2020, virtual , 2020.

Alain Horé and Djemel Ziou. Image quality metrics: PSNR vs. SSIM. In 20th International Conference on Pattern Recognition, ICPR 2010, Istanbul, Turkey, 23-26 August 2010 , pages 2366-2369. IEEE Computer Society, 2010.

Hsiang Hsu, Shahab Asoodeh, and Flávio P. Calmon. Obfuscation via information density estimation. In Silvia Chiappa and Roberto Calandra, editors, The 23rd International Conference on Artificial Intelligence and Statistics, AISTATS 2020, 26-28 August 2020, Online [Palermo, Sicily, Italy] , volume 108 of Proceedings of Machine Learning Research , pages 906-917. PMLR, 2020.

Hsiang Hsu, Natalia Martinez, Martin Bertran, Guillermo Sapiro, and Flavio P Calmon. A survey on statistical, information, and estimation-theoretic views on privacy. IEEE BITS the Information Theory Magazine , 1(1):45-56, 2021.

Chong Huang, Peter Kairouz, Xiao Chen, Lalitha Sankar, and Ram Rajagopal. Generative adversarial privacy. CoRR , abs/1807.05306, 2018.

Yangsibo Huang, Zhao Song, Kai Li, and Sanjeev Arora. Instahide: Instance-hiding schemes for private distributed learning. In Proceedings of the 37th International Conference on Machine Learning, ICML 2020, 13-18 July 2020, Virtual Event , volume 119 of Proceedings of Machine Learning Research , pages 4507-4518. PMLR, 2020.

Aapo Hyvärinen and Peter Dayan. Estimation of non-normalized statistical models by score matching. Journal of Machine Learning Research , 6(4), 2005.

Jacob Imola, Shiva Prasad Kasiviswanathan, Stephen White, Abhinav Aggarwal, and Nathanael Teissier. Balancing utility and scalability in metric differential privacy. In James Cussens and Kun Zhang, editors, Uncertainty in Artificial Intelligence, Proceedings of the Thirty-Eighth Conference on Uncertainty in Artificial Intelligence, UAI 2022, 1-5 August 2022, Eindhoven, The Netherlands , volume 180 of Proceedings of Machine Learning Research , pages 885-894. PMLR, 2022.

Ibrahim Issa, Aaron B. Wagner, and Sudeep Kamath. An operational approach to information leakage. IEEE Trans. Inf. Theory , 66(3):1625-1657, 2020.

Bargav Jayaraman and David Evans. Evaluating differentially private machine learning in practice. In 28th USENIX Security Symposium (USENIX Security 19) , pages 1895-1912, 2019.

Yiping Kang, Johann Hauswald, Cao Gao, Austin Rovinski, Trevor Mudge, Jason Mars, and Lingjia Tang. Neurosurgeon: Collaborative intelligence between the cloud and mobile edge. ACM SIGARCH Computer Architecture News , 45(1):615-629, 2017.

Steven M Kay. Fundamentals of statistical signal processing: estimation theory . Prentice-Hall, Inc., 1993.

Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009.

Matt Kusner, Yu Sun, Nicholas Kolkin, and Kilian Weinberger. From word embeddings to document distances. In International conference on machine learning , pages 957-966. PMLR, 2015.

Mathias Lécuyer, Vaggelis Atlidakis, Roxana Geambasu, Daniel Hsu, and Suman Jana. Certified robustness to adversarial examples with differential privacy. In 2019 IEEE Symposium on Security and Privacy, SP 2019, San Francisco, CA, USA, May 19-23, 2019 , pages 656-672. IEEE, 2019.

Jingtao Li, Adnan Siraj Rakin, Xing Chen, Zhezhi He, Deliang Fan, and Chaitali Chakrabarti. Ressfl: A resistance transfer framework for defending model inversion attack in split federated learning. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2022, New Orleans, LA, USA, June 18-24, 2022 , pages 10184-10192. IEEE, 2022.

Meng Li, Liangzhen Lai, Naveen Suda, Vikas Chandra, and David Z Pan. Privynet: A flexible framework for privacy-preserving deep neural network training. arXiv preprint arXiv:1709.06161 , 2017.

Yingzhen Li and Richard E Turner. Gradient estimators for implicit models. arXiv preprint arXiv:1705.07107 , 2017.

Bo Liu, Ming Ding, Hanyu Xue, Tianqing Zhu, Dayong Ye, Li Song, and Wanlei Zhou. Dp-image: Differential privacy for image data in feature space. CoRR , abs/2103.07073, 2021.

Aravindh Mahendran and Andrea Vedaldi. Understanding deep image representations by inverting them. In IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2015, Boston, MA, USA, June 7-12, 2015 , pages 5188-5196. IEEE Computer Society, 2015.

Luca Melis, Congzheng Song, Emiliano De Cristofaro, and Vitaly Shmatikov. Exploiting unintended feature leakage in collaborative learning. In 2019 IEEE Symposium on Security and Privacy, SP 2019, San Francisco, CA, USA, May 19-23, 2019 , pages 691-706. IEEE, 2019.

Fatemehsadat Mireshghallah, Mohammadkazem Taram, Prakash Ramrakhyani, Ali Jalali, Dean Tullsen, and Hadi Esmaeilzadeh. Shredder: Learning noise distributions to protect inference privacy. In Proceedings of the Twenty-Fifth International Conference on Architectural Support for Programming Languages and Operating Systems , pages 3-18, 2020.

Ilya Mironov. Rényi differential privacy. In 30th IEEE Computer Security Foundations Symposium, CSF 2017, Santa Barbara, CA, USA, August 21-25, 2017 , pages 263-275. IEEE Computer Society, 2017.

Dario Pasquini, Giuseppe Ateniese, and Massimo Bernaschi. Unleashing the tiger: Inference attacks on split learning. In Yongdae Kim, Jong Kim, Giovanni Vigna, and Elaine Shi, editors, CCS '21: 2021 ACM SIGSAC Conference on Computer and Communications Security, Virtual Event, Republic of Korea, November 15 - 19, 2021 , pages 2113-2129. ACM, 2021.

Huy Phan. Pytorch models trained on cifar-10 dataset. https://github.com/huyvnphan/ PyTorch_CIFAR10 , 2013.

Maarten G Poirot, Praneeth Vepakomma, Ken Chang, Jayashree Kalpathy-Cramer, Rajiv Gupta, and Ramesh Raskar. Split learning for collaborative deep learning in healthcare. arXiv preprint arXiv:1912.12115 , 2019.

Victor Sanh, Lysandre Debut, Julien Chaumond, and Thomas Wolf. Distilbert, a distilled version of BERT: smaller, faster, cheaper and lighter. CoRR , abs/1910.01108, 2019.

Yang Song, Sahaj Garg, Jiaxin Shi, and Stefano Ermon. Sliced score matching: A scalable approach to density and score estimation. In Amir Globerson and Ricardo Silva, editors, Proceedings of the Thirty-Fifth Conference on Uncertainty in Artificial Intelligence, UAI 2019, Tel Aviv, Israel, July 22-25, 2019 , volume 115 of Proceedings of Machine Learning Research , pages 574-584. AUAI Press, 2019.

Stanford. [TinyImageNet download link]. http://cs231n.stanford.edu/ tiny-imagenet-200.zip , 2023.

Chandra Thapa, Mahawaga Arachchige Pathum Chamikara, Seyit Camtepe, and Lichao Sun. Splitfed: When federated learning meets split learning. In Thirty-Sixth AAAI Conference on Artificial Intelligence, AAAI 2022, Thirty-Fourth Conference on Innovative Applications of Artificial Intelligence, IAAI 2022, The Twelveth Symposium on Educational Advances in Artificial Intelligence, EAAI 2022 Virtual Event, February 22 - March 1, 2022 , pages 8485-8493. AAAI Press, 2022.

Tom Titcombe, Adam J Hall, Pavlos Papadopoulos, and Daniele Romanini. Practical defences against model inversion attacks for split neural networks. arXiv preprint arXiv:2104.05743 , 2021.

Dmitry Ulyanov, Andrea Vedaldi, and Victor S. Lempitsky. Deep image prior. In 2018 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2018, Salt Lake City, UT, USA, June 18-22, 2018 , pages 9446-9454. Computer Vision Foundation / IEEE Computer Society, 2018.

Harry L Van Trees. Detection, estimation, and modulation theory, part I: detection, estimation, and linear modulation theory . John Wiley & Sons, 2004.

Praneeth Vepakomma, Otkrist Gupta, Tristan Swedish, and Ramesh Raskar. Split learning for health: Distributed deep learning without sharing raw patient data. arXiv preprint arXiv:1812.00564 , 2018.

Praneeth Vepakomma, Abhishek Singh, Otkrist Gupta, and Ramesh Raskar. Nopeek: Information leakage reduction to share activations in distributed deep learning. In 2020 International Conference on Data Mining Workshops (ICDMW) , pages 933-942. IEEE, 2020.

Praneeth Vepakomma, Abhishek Singh, Emily Zhang, Otkrist Gupta, and Ramesh Raskar. Nopeek-infer: Preventing face reconstruction attacks in distributed inference after on-premise training. In 2021 16th IEEE International Conference on Automatic Face and Gesture Recognition (FG 2021) , pages 1-8. IEEE, 2021.

Aladin Virmaux and Kevin Scaman. Lipschitz regularity of deep neural networks: analysis and efficient estimation. In Samy Bengio, Hanna M. Wallach, Hugo Larochelle, Kristen Grauman, Nicolò Cesa-Bianchi, and Roman Garnett, editors, Advances in Neural Information Processing Systems 31: Annual Conference on Neural Information Processing Systems 2018, NeurIPS 2018, December 3-8, 2018, Montréal, Canada , pages 3839-3848, 2018.

Patrick von Platen. google/ddpm-cifar10-32. https://huggingface.co/google/ ddpm-cifar10-32 , 2022.

Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel R. Bowman. GLUE: A multi-task benchmark and analysis platform for natural language understanding. In 7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019 . OpenReview.net, 2019.

Zhou Wang and Alan C Bovik. A universal image quality index. IEEE signal processing letters , 9(3):81-84, 2002.

Liyao Xiang, Hao Zhang, Haotian Ma, Yifan Zhang, Jie Ren, and Quanshi Zhang. Interpretable complex-valued neural networks for privacy protection. In 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020 . OpenReview.net, 2020.

Hanshen Xiao and Srinivas Devadas. Dauntless: Data augmentation and uniform transformation for learning with scalability and security. IACR Cryptol. ePrint Arch. , page 201, 2021.

Hanshen Xiao and Srinivas Devadas. PAC privacy: Automatic privacy measurement and control of data processing. In Helena Handschuh and Anna Lysyanskaya, editors, Advances in Cryptology - CRYPTO 2023 - 43rd Annual International Cryptology Conference, CRYPTO 2023, Santa Barbara, CA, USA, August 20-24, 2023, Proceedings, Part II , volume 14082 of Lecture Notes in Computer Science , pages 611-644. Springer, 2023.

Adam Yala, Homa Esfahanizadeh, Rafael G. L. D'Oliveira, Ken R. Duffy, Manya Ghobadi, Tommi S. Jaakkola, Vinod Vaikuntanathan, Regina Barzilay, and Muriel Médard. Neuracrypt: Hiding private health data via random neural networks for public training. CoRR , abs/2106.02484, 2021.

Qiang Yang, Yang Liu, Tianjian Chen, and Yongxin Tong. Federated machine learning: Concept and applications. ACM Transactions on Intelligent Systems and Technology (TIST) , 10(2):1-19, 2019.

7 Appendix

7.1 Score matching details

We found that using score matching [57] does not work reliably when the data's structure lies on a low-dimensional manifold ( e.g. , natural images). We found that applying randomized smoothing [9], which adds Gaussian noise to the image for robust training, helps stabilize score matching as it smoothens the density function. Randomized smoothing also makes the bound tighter. We observed that adding a reasonable amount of noise ( e.g., standard deviation of 0.25, which was originally used by Cohen et al. [9]) works well in general, but adding only small noise (standard deviation of 0.01) does not. We show both results in Section 3.3.1.

7.2 Hyperparameters

Attacks For attacks in Section 3.3.1, 3.3.2, and 4.2, we used the following hyperparameters. For the optimizer-based attack for Gaussian synthetic input, we used Adam with lr= 10 -3 , and λ =0.1-100 for the regularizer. For the optimizer-based attack for NCF-MLP and DistilBert, we used Adam with lr=0.1. For the DNN-based attack for MNIST and CIFAR-10 (Figure 2, 3, 5), we used a modified DNNfrom Li et al. [45], which uses a series of convolution (Conv) and convolution transpose (ConvT) layers interspersed with leaky ReLU of slope 0.2. All the models were trained for 100 epochs using Adam with lr= 10 -3 . Below summarizes the architecture parameters. For DNN-based attacks in Section 3.3.2, we put a sigmoid at the end. For the attack in Section 4.2, we do not.

Split inference Below are the hyperparameters for the models used in Section 4.2. For ResNet-18, we used an implementation tuned for CIFAR-10 dataset from [54], with ReLU replaced with GELU and max pooling replaced with average pooling. We used the default hyperparameters from the repository except for the following: bs=128, lr=0.1, and weight_decay= 5 × 10 -4 . For NCF-MLP, we used an embedding dimension of 32 and MLP layers of output size [64, 32, 16, 1]. We trained NCF-MLP with Nesterov SGD with momentum=0.9, lr=0.1, and batch size of 128 for a single epoch. We assumed 5-star ratings as click and others as non-click. For DistilBert, we used Adam optimizer with a batch size of 16, lr= 2 × 10 -5 , β 1 =0.9, β 2 =0.999, and ϵ = 10 -8 . We swept the compression layer channel dimension among 2, 4, 8, 16, and the SNR regularizer λ between 10 -3 and 100.

Training Below are the hyperparameters for the models evaluated in Section 5.2. We used the same model and hyperparameters with split inference for training the encoder with the pretraining dataset. Then, we freeze the layers up to block 4 and trained the rest for 10 epochs with CIFAR-10, with lr= 10 -3 and keeping other hyperparameters the same.

Execution Environment All the evaluation was done on a single A100 GPU. The training and evaluation of each model ranged roughly from less than an hour (ResNet-18 with split-early, NCFMLP) to 3-7 hours (ResNet-18 with split-late, DistilBert).

7.3 van Trees inequality

Below, we restate the van Trees Inequality [19], which we use to prove Theorem 5.

Theorem 2 (Multivariate van Trees inequality) . Let ( X , F , P θ : θ ∈ Θ) be a family of distributions on a sample space X dominated by µ . Let p ( x | θ ) denote the density of X ∼ P θ and I x ( θ ) denotes its FIM. Let θ ∈ Θ follows a probability distribution π with a density λ π ( θ ) with respect to Lebesgue measure. Suppose that λ π and p ( x | θ ) are absolutely µ -almost surely continuous and λ π converges to 0 and the endpoints of Θ . Let ψ be an absolutely continuous function of θ with the same output dimension of θ , and ψ n an arbitrary estimator of ψ ( θ ) . Assume regularity conditions from Section 2.2 is met. If we make n observations { x 1 , x 2 , ..., x n } , then:

where div ψ ( θ ) = Σ i ∂ψ i ( θ ) ∂θ i .

7.4 Proof of Corollary 1

Proof. Let ψ be an identity transformation ψ ( θ ) = θ . For the setup in Corollary 1, n = 1 and div( ψ ( θ )) = d , so the multivariate van Trees inequality from Theorem 2 reduces to:

7.5 Comparison with differential privacy

Differential privacy [1] is not well-suited for instance encoding, as we discuss in Section 2.1. We formulate and compare a DP-based instance encoding and compare it with our dFIL-based instance encoding in a split inference setup (Section 4) to show that DP-based instance encoding indeed does not work well.

To formulate DP for instance encoding, we define an adjacent set D and D ′ as two differing inputs. A randomized method A is ( α , ϵ )-Rényi differentially private (RDP) if D α ( A ( D ) ||A ( D ′ )) ≤ ϵ for D α ( P || Q ) = 1 α -1 log E x ∼ Q [( P ( x ) Q ( x ) ) α ] . As DP provides a different privacy guarantee with dFIL, we use the theorem from Guo et al. [20] to derive an MSE lower bound using DP's privacy metric for an unbiased attacker. Assuming a reconstruction attack ˆ x = Att( e ) that reconstructs x from the encoding e = Enc( x ) , repurposing the theorem Guo et al. [20] gives:

for a (2, ϵ )-RDP Enc , where X is the input data space. We can construct a (2, ϵ )-RDP encoder Enc RDP from a deterministic encoder Enc D by scaling and clipping the encoding adding Gaussian noise, or Enc RDP = Enc D ( x ) / max(1 , || Enc D ( x ) || 2 C ) + N (0 , σ 2 ) , similarly to [1]. The noise to be added is σ = (2 C ) 2 ϵ [52]. Equation 6 for DP is comparable to Equation 2 for dFIL, and we use the two equations to compare DP and dFIL parameters. We use Equation 2 because [20] does not discuss the bound against biased attackers.

We evaluate both encoders for split inference using CIFAR-10 dataset and ResNet-18. We split the model after block 4 (split-middle from Section 4.2.1) and did not add any optimizations discussed in Section 4 for simplicity. For the DP-based encoder, we retrain the encoder with scaling and clipping so that the baseline accuracy without noise does not degrade. We ran both models without standardizing the input, which makes diam i ( X ) = 1 for all i .

Table 4 compares the test accuracy achieved when targeting the same MSE bound for an unbiased attacker using dFIL and DP, respectively. The result clearly shows that DP degrades the accuracy much more for similar privacy levels (same unbiased MSE bound), becoming impractical very quickly. DP suffers from low utility because DP is agnostic with the input and the model, assuming a

worst-case input and model weights. Our dFIL-based bound uses the information of the input and model weights in its calculation of the bound and can get a tighter bound.

7.6 Attack based on a diffusion model

We additionally designed a powerful, diffusion model-based reconstruction attacker to study the privacy of dFIL against the best-effort attacker, motivated from the fact that recently developed diffusion models [30] are excellent denoisers. During training of a diffusion model, (1) a particularlydesigned noise is added to an image, and (2) a DNN is trained to predict and remove the noise [30]. The first part can be thought of as an instance encoder (that is purposely made easy to invert), and we can calculate its dFIL. The second part can be thought of as a reconstruction attacker. As the noising and denoising are specifically designed for the denoising to work well, we expect a mature, pretrained diffusion model to give a very good attack quality. We used DDPM [30] pretrained with CIFAR-10 from Google [67].

Figure 7 shows the result. The first column of each row shows the original image, and other columns show the reconstruction of our DDPM-based attacker with different dFIL. We scale and show dFIL with the same scale as Figure 5, as DDPM works with a different normalized image that produces dFIL at a different scale. Our new attack provided an interesting result: the attack was able to reconstruct conceptually similar images with the original image even when pixel-by-pixel reconstruction was prohibited by high 1 / dFIL . For example, 7th row at 1 / dFIL = 49 . 5 (6th column) successfully reconstructed a white car with a red background, although the reconstruction MSE was high and the design of the reconstructed car was nothing like the original image. The result shows that high-level information of the image ( e.g. , the color of the car/background, the orientation of the car, etc .) can still be preserved after encoding with a relatively high 1 / dFIL , which is why it is possible to perform downstream training/inference with a privately-encoded data without revealing the original data.

7.7 Per-pixel FIL

Instead of dFIL which is the average of the Fisher information leakage (FIL) for all the features (pixels), we can directly inspect the per-feature FIL to see which features (pixels) leak more information and can be reconstructed more easily. Figure 8 plots the per-pixel FIL from the split-middle setup of ResNet-18 and CIFAR-10 from Section 4.2.2. The result shows that identifying features of an object, such as the contour of a frog (8th column) or the tire of the car (7th column), are more easily leaked, following our intuition.

7.8 Additional figures and tables

ref

1/dFIL

0.494

1/dFIL

2.19

1/dFIL

6.42

1/dFIL

17.4

1/dFIL

49.5

1/dFIL

159

1/dFIL

606

1/dFIL

2780

1/dFIL

15600

民

Orig.

FIL

Overlaid

1e-05

0.0001

0.001

0.01

0.1

1

10

100

1/dFIL

0.00

0.25

0.50

0.75

1.00

SSIM

split-early

split-middle

split-late

(a) 1/dFIL vs. SSIM

early

middle

1e-05

late

0.0001

0.001

0.01

1/dFIL

0.1

1.0

10.0

100.0

(b) 1/dFIL vs. reconstructed image quality

ref

1e-05

0.0001

0.001

0.01

0.1

1.0

10.0

25.12

ref

1e-05

0.0001

0.001

0.01

0.1

1.0

10.0

100.0

ref

1e-05

0.0001

0.001

0.01

0.1

1.0

10.0

100.0

ref

split-early, 1/dFIL=1.0

recon

ref

split-early, 1/dFIL=10.0

recon

ref

split-middle, 1/dFIL=1.0

recon

ref

split-middle, 1/dFIL=10.0

recon

ref

split-late, 1/dFIL=1.0

recon

ref

split-late, 1/dFIL=10.0

recon

ref

split-early, 1/dFIL=1.0

recon

ref

split-early, 1/dFIL=10.0

recon

ref

split-middle, 1/dFIL=1.0

recon

ref

split-middle, 1/dFIL=10.0

recon

ref

split-late, 1/dFIL=1.0

recon

ref

split-late, 1/dFIL=10.0

recon

(a) Reconstructions with the worst SSIM values.

(b) Reconstructions with the best SSIM values.

Lowest dfil

Highest dfil

?

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nIn this paper the authors propose (diagonal) Fisher information leakage (dFIL) as a theoretically-grounded and practical framework for assessing the privacy guarantees of instance encoding. At its core, dFIL quantifies the potential invertibility of the encoding mapping of an instance encoding scheme. The authors establish, under mild regularity conditions, that the reciprocal of dFIL, plus the information theorist's Fisher information, serves as a lower bound for the mean squared error (MSE) of any input reconstruction attack (Corollary 1). This lower bound is derived from the so-called van Trees' inequality. Furthermore, the authors present extensive numerical experiments that highlight the key features of dFIL as a privacy metric for instance encoding.\n\nSTRENGTHS:\n1. dFIL is a very intuitive privacy metric under unbiased reconstruction attacks. In such cases, the Cramér-Rao bound establishes that the reciprocal of dFIL is a lower bound for the MSE of an adversary attempting to reconstruct the raw data based on the encoding.\n\n2. The limitations of dFIL are discussed thoroughly.\n\n3. The paper is well-written.\n\nWEAKNESSES:\nAlthough dFIL serves as an intuitive privacy metric under unbiased reconstruction attacks, these attacks are uncommon in ML applications. In the case of biased attacks, which are more representative of practical scenarios, the main result (Corollary 1) highlights the need to consider not only dFIL but also the information theorist's Fisher information. Consequently, certain practical and theoretical concerns are shifted to the latter quantity, potentially diminishing the significance of dFIL itself. Furthermore, it is worth mentioning that the paper's technical contribution is minor since the main result is a direct consequence of van Trees' inequality. While this is not a problem in itself, it does not contribute to make the paper stand out.\n\nQUESTIONS:\nOne of the key contributions of this paper is the introduction of dFIL as an intuitive privacy metric for instance encoding. However, it is important to note that, for (general) biased attacks, the information theorist's Fisher information plays an equally, if not more significant role than dFIL. Regrettably, the paper lacks a thorough discussion of the empirical and theoretical properties of the latter quantity, apart from its estimation using score matching. Consequently, it becomes challenging to evaluate the relative importance of dFIL compared to the information theorist's Fisher information. Clarifying these aspects would enhance the overall assessment of dFIL's significance.\n\nAddendum: In their rebuttal, the authors clarified the relative importance of dFIL and the information theorist's Fisher information.\n\nLIMITATIONS:\nThe authors addressed the limitations of the proposed method very well.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2023-06-30"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper discusses the privacy concerns surrounding data encoding methods employed in machine learning (ML) operations. Its primary objective is to present a theoretical framework that quantifies the extent of privacy leakage in an encoding scheme and facilitates the calculation of its invertibility. The authors propose a novel technique that leverages Fisher Information Leakage to compute the lower bound of Mean Squared Error (MSE) for an encoding scheme, considering both unbiased and biased attackers. Finally, The authors evaluate their technique against some simple reconstruction attacks both for training and inference tasks.\n\nSTRENGTHS:\n1. The paper takes a theoretical approach to explain reconstruction attacks. The problem is interesting and the presentation and flow of the paper are good. It was easy to follow and understand.\n2. The authors' approach of initially focusing on unbiased attacks and subsequently extending their scheme to encompass attackers with prior knowledge is a good approach and beneficial in comprehending their contributions. The paper also provides evaluation of their technique considering both unbiased and biased attackers.\n3. The paper is upfront about many of its limitations. This is commendable and helps understand the solution better and paves the way for identifying future research avenues.\n\nWEAKNESSES:\n1. The core contribution of the paper is adopting Fisher Information Leakage and van Trees inequality to come up with a lower bound for MSE. While this is a new idea, it would benefit from more comprehensive construction. Furthermore, the reconstruction attacks examined by the authors to showcase the efficacy of their technique are relatively simplistic. It would be advantageous for the authors to consider incorporating relevant literature from the privacy and security domain and evaluate their approach against more sophisticated and state-of-the-art reconstruction attacks.\n\n2. The paper should provide more rigorous analysis of the privacy and utility trade-off. The paper readily discards Differential Privacy on utility ground, but later adopts noise addition to introduce privacy. Regardless, the privacy guarantee by noise addition and the corresponding utility trade-offs should be analytically explained. While the paper includes an empirical demonstration of the utility trade-off, it falls short of its intended goal of providing a theoretical explanation. \n\n3. The authors should clarify how this lower bound contributes to preventing attacks, developing privacy-preserving encoding techniques, or addressing other important aspects related to data privacy.\n\nQUESTIONS:\nSee weakness.\n\nLIMITATIONS:\nNo", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 2, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-05"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nInstance encoding (and some closely related lines of research) aim at finding ways to encode data examples (in a training set) as $E=Enc(e)$, in such a way that one can train models on the encoded examples $E_1,...E_n$, while $E_i$ does not leak much about $e_i$ to an adversary who inspects it.\n\nPrevious efforts for proposing \"private\" instance encodings have failed due to (1) subsequent attacks, and (2) lack of formal definitions of what privacy means here. In fact, it is known that such mechanisms cannot be DP in a strong sense, while they are also useful for training.\n\nThis paper aims to put forward a new way of arguing about privacy of instance encoding, based on Fisher Information (FI). FI is a useful measure of information revealed in random variables about a related random variable. In particular, knowing FI allows one to lower bound the Mean Squared Error (MSE) of the best reconstruction attack, assuming that the adversary has no prior knowledge about the distribution of the original instance.\n\nMore specifically, using FI, the paper shows how to give an *hardness of inversion* type privacy. The paper shows that both in the (more elementary) case of unbiased attacks (who see the original instances uniformly distributed) as well as more powerful attackers who have some prior information (e.g., know the distribution of the instances), one can lower bound the MSE of the best possible (information theoretic) attack, based on two parameters: FI of the encoding and a second parameter that depends on how flat/concentrated the distribution of the instance x is. Note that the second parameter also inherently shows up as one can approximate x well if it is too concentrated already. \n\nThe paper also discusses how to compute the two parameters of interest, or at least approximate them, in practical settings. \nIn particular, when the encoder is a Neural Net, the paper discusses how to use tools from previous work to approximate them using smooth functions that will be suitable for computing their FI.\n\nThe final product is an \"average case\" notion of privacy that is tailored for inputs coming from (flat enough) distributions.\n\nThe paper also presents experiments to validate its theory.\n\nSTRENGTHS:\nDifferential privacy is the golden standard for privacy at the moment. But it is also a good idea to look for new notions privacy that could be meaningful on their own and useful for certain settings. Hence, I find the overall goal of the paper positive and useful for putting new perspectives on privacy on the table.\n\nI also liked the fact that the paper made an effort to algorithmically compute their notion of privacy by approximating the two parameters of interest using tools from previous work.\n\nWEAKNESSES:\n(also a limitation) Bounding MSE sometimes might not say anything about privacy, when all an adversary wants to do is finding a specific sensitive feature (and not approximating the instance in $\\ell_2$ norm). So, more (context dependent) work is needed for finding whether this notion is actually a good notion of privacy or not.\n\n(also a question) The figures are not clear. For example, in Figure 2, what is the column numbers (Ref, 25, 10, 1) next to MNIST images?\n\n(also a question and limitation) The paper does experiments in which the MSE of certain encoders are evaluated. However, it is not clear to me how *useful* these encodings are *for training models* over the encoded instances. Note that without knowing how useful the encodings are, there is always a trivial fully private encoding: output $\\bot$ all the time.\n\nQUESTIONS:\nsee the questions from the previous box.\n\nis there any hope that your approach can say something about the expected $\\ell_\\infty$ distance between adversary's guess and the true instance (rather than MSE) ?\n\nLIMITATIONS:\nSee the \"weakness\" section for some comments related to limitations as well.\n\nOn the positive side, the paper itself has a a good \"limitations\" section at the end.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-05"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces a new theoretical measure, the \"diagonal Fisher Information Leakage\" (dFIL), for quantifying the privacy leakage in instance encoding mechanisms in machine learning models. The authors construct a framework that balances the trade-off between data privacy and utility in instance encoding.\n\nThe authors start by developing the mathematical foundation of dFIL, leveraging the principle of Cramér-Rao Lower Bound (CRLB). They present a theoretical analysis which demonstrates that dFIL serves as an upper bound on the Mean Squared Error (MSE) of any estimator trying to reconstruct the original input from the encoded instances.\n\nThe paper presents two case studies to showcase the practical application of dFIL. The first one focuses on private inference, where instance encoding is applied to an existing pretrained model. The second case study explores the possibility of training a model on instance-encoded data. The results from these studies confirm that models guided by dFIL can achieve reasonable performance while ensuring a high degree of privacy.\n\nDespite these advancements, the authors acknowledge the limitations of dFIL, such as the MSE bound not always correlating well with the semantic quality of the reconstruction, the average case bound provided by dFIL, and the difficulties of interpreting dFIL values in certain data types where MSE might not be directly meaningful or accurate. Recognizing these limitations, the paper calls for further research to explore and optimize the use of dFIL in designing and analyzing systems that use instance encoding for private inference and training.\n\nSTRENGTHS:\nStrengths:\n\n1. **Originality**: The concept of Diagonal Fisher Information Leakage (dFIL) is novel, offering a fresh perspective on privacy leakage quantification in instance encoding for machine learning models. The authors have done a commendable job in providing a new metric that has potential broad applications in designing and analyzing private machine learning systems.\n\n2. **Quality**: The theoretical foundation of dFIL is well-crafted and robust, using the Cramér-Rao Lower Bound to derive the maximum leakage. This gives dFIL a solid grounding in statistical theory. The authors carefully reason about the limitations of their approach and provide a roadmap for further exploration.\n\n3. **Clarity**: The paper is well-written and structured, making the complex concept of dFIL comprehensible. The authors articulate the theoretical underpinnings and practical implications of dFIL with clarity and precision. The use of case studies adds to the understanding of how dFIL can be applied in practical scenarios.\n\n4. **Significance**: The presented work is of significant value as it paves a way for protecting data privacy in instance encoding mechanisms, a critical issue in today's data-centric machine learning landscape. The demonstrated applications in private inference and training suggest the potential of dFIL to guide the design of machine learning systems that respect privacy constraints while maintaining utility.\n\nWEAKNESSES:\n1. **Limited empirical validation:** Although the theoretical development of dFIL is thorough and robust, the empirical validation seems somewhat limited. The authors conduct case studies on split inference and training with dFIL, which are commendable. However, additional experiments, especially on more diverse datasets and models, could strengthen the validity and generalizability of the dFIL measure. \n\n2. **Absence of comparative analysis:** While the paper introduces dFIL as a novel concept, it doesn't provide a comparative analysis with other similar measures, if any exist. This makes it difficult for readers to evaluate the practical advantages or disadvantages of dFIL in contrast to other measures or approaches. \n\n3. **Limited exploration of potential use-cases:** The authors present two case studies, however, the range of potential applications for dFIL could be broader. Additional use-cases illustrating how dFIL can guide the design of private machine learning systems would provide a more comprehensive view of the practical utility of dFIL. \n\n4. **Interpretability of dFIL:** As the authors acknowledge in the Limitations section, for data types where MSE is not directly meaningful or the bound is inaccurate, interpreting the privacy of an encoding given its dFIL may not be straightforward. More investigation is needed to establish a clear and intuitive interpretation of dFIL values, especially in the context of real-world applications.\n   \n5. **Assumptions and constraints:** The assumptions underlying the derivation of dFIL and its applications need further exploration. For example, how sensitive is dFIL to the assumptions of model linearity, Gaussian noise, and others? The impact of violating these assumptions, and how they can be mitigated, would be valuable areas to explore.\n\nQUESTIONS:\n1. The paper uses mean squared error (MSE) as a metric to quantify the quality of reconstructions. However, as the authors acknowledge, this may not always correlate with the semantic quality of reconstructions. Could you please elaborate more on the choice of this metric and how it might affect the overall performance of the system? And the paper mentions that for certain data types where MSE is not directly meaningful, interpreting the privacy of an encoding given its dFIL might be challenging. Could you provide examples of such data types and explain how dFIL might be adapted to handle them?\n\nLIMITATIONS:\nThe authors have done a commendable job addressing the potential limitations of their work. They detail four specific limitations in section 6:\n\n1. dFIL only bounds the mean squared error (MSE), which may not always correspond well with the semantic quality of reconstructions.\n2. The derived bounds are average-case, meaning that individual data instances could experience lower reconstruction MSE than the bound.\n3. For data types where MSE is not directly meaningful or the bound is inaccurate, interpreting the privacy of an encoding given its dFIL might be difficult.\n4. Systems with the same dFIL may have different invertibility, as the bound from dFIL could be conservative.\n\nThese limitations are articulated clearly, and the authors also suggest possible directions to address these limitations, such as exploring metrics other than MSE and calculating dFIL dynamically for each sample.\n\nIn terms of the broader societal impact, the paper does not directly address this aspect. However, it's understandable, given the theoretical nature of the work. Future research could consider potential misuse of this technology, as well as possible regulatory or ethical issues that might arise from its application, particularly in scenarios where private or sensitive data are involved. Nonetheless, the paper's focus on improving privacy in machine learning is itself an important societal contribution, as it addresses a growing need in an increasingly data-driven world.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-08"}