Learning Private Representations with Focal Entropy
Authors: Tassilo Klein
Moin Nabi
Venue:  (2021)

Abstract: How can we learn a representation with good predictive power while preserving user privacy?
We present an adversarial representation learning method to sanitize sensitive content from the representation in an adversarial fashion.
Specifically, we propose focal entropy - a variant of entropy embedded in an adversarial representation learning setting to leverage privacy sanitization. Focal entropy enforces maximum uncertainty in terms of confusion on the subset of privacy-related similar classes, separated from the dissimilar ones. As such, our proposed sanitization method yields deep sanitization of private features yet is conceptually simple and empirically powerful. We showcase feasibility in terms of classification of facial attributes and identity on the CelebA dataset as well as CIFAR-100. The results suggest that private components can be removed reliably. 

Under review as a conference paper at ICLR 2021
LEARNING PRIVATE REPRESENTATIONS WITH FOCAL ENTROPY
Anonymous authors Paper under double-blind review
ABSTRACT
How learn representation with good predictive power while preserving user privacy? We present an adversarial representation learning method to sanitize sensitive content from the representation in an adversarial fashion. Specifically, we propose focal entropy a variant of entropy embedded in an adversarial repre- sentation learning setting to leverage privacy sanitization. Focal entropy enforces maximum uncertainty in terms of confusion on the subset of privacy-related similar classes, separated from the dissimilar ones. As such, our proposed sanitization method yields deep sanitization of private features yet is conceptually simple and empirically powerful. We showcase fcasibility in terms of classification facial attributes and identity on the CelebA dataset well as CIFAR- 100. The results suggest that private components can be removed reliably.
INTRODUCTION
Lately, the topics of privacy and security are enjoying increased interest in the machine learning community. This can largely be attributed to the success of big data in conjunction with deep learning and the urge create and process ever-larger data sets for mining. However, with the emergence of more and more machine learning services becoming part of our daily lives and making use of our data, special measures must be taken to protect privacy and decrease the risk of privacy creep Narayanan & Shmatikov (2006); Backstrom et al. (2077) Simultaneously, growing privacy concerns entail the risk of becoming a major deterrent in the widespread adoption machine learning and the attainment of their concomitant benefits. Therefore, reliable and accurate privacy-preservee methodologies are needed, which is why the topic lately has enjoyed increased attention the research community.
Several efforts have been made in machine learning to develop algorithms that preserve user privacy while achieving reasonable predictive power. Solutions proposed for privacy in the research com- munity are versatile. A standard approach to address privacy issues in the client-server setup is to anonymize the data of clients. This is often achieved by directly obfuscating the private part(s) the data and/or adding random noise to raw data. Consequently, the noise level controls the trade-off between predictive quality and user privacy (c.g., data-level Differrntial Privacy Dwork (2006)). These approaches associate privacy budget with all operations on the dataset. However, complex training procedures run the risk of exhausting the budget before convergence. A recent solution to such problem has been federated learning McMahan et al. (2016); Geyer et al. (2017). which allows us to collaboratieely train a centralized model while keeping the training data decentralized. The idea behind this strategy is that clients transfer the parameters of the training model in the form of gradient updates to a server instead of the data itself. While such an approach is appealing to train a network with data hosted on different clients. transferring the models between clients and server, and averaging the gradients across the clients generates significant data transmission and extra computations, which considerably prolongs training. Another widely adopted solution is to rely on encoded data representation. Following this notion, instead of transferring the client's data, feature representation is learned on the clients' side and transferred to the server. Unfortunately, the learned features may still contain rich information, which can breach user privacy Osia et al. (2017) 2018 Also, the extracted features can be exploited by an attacker to infer private attributes Salem et al. 2019). Yet, another approach is homomorphic encryption Armknecht et al. (2015). Despite providing strong cryptographic guarantees, in theory, it incurs considerable computational overhead, which still prevents its applicability for SOTA deep learning architectures Srivastava et al. (2019)

---

Under review as a conference paper at ICLR 2021
The recent success of adversarial learning in making the representations fair
Louizos et al. (2013).
unbiased (Madras et al. (2018), and controllably invariant to sensitive attributes Xie et al. [2017) has
led to the increased adoption of Adversarial Representation Learning (ARL) control the private
information encapsulated within the representation Roy & Boddeti (2019): Sadeghi et al. 2019).
In the common ARL formalization of the seeks to extract the desired target attributes while an "adveesary"" seeks to reveal the private attributes. However, the solutions mentioned earlier can only meet its practical promises when the private attributes do not strongly correlate with the target attributes Roy & Boddeti (2019). In this paper, we deal with adversarial privacy-pressere and target attributes are related to each other (e.g.. 'Queen Elizabeth 11.' and "wearing hat', or "Mahatma Gandhi' and 'wearing eyeglasses') to a large extent. The objective of this task is to learn a representation that contains all the information about non-sensitive attributes. At the same time, it omits to encode the sensitive attributes of them. Such representation can be transmitted to the server without concerns regarding the privacy revelation of classifiers having equal and higher capacity than the adversarial proxy used during training. For that, we adopt an ARL procedure and propose to learn representation which maximizes the likelihood of the target information (i.e., attribute predictor) while increasing the uncertainty about the class that each sample belongs to (i.e., class adversary). With that, we intuitively tie the privacy notion the class-level information and sanitize the class-revealing information from the representation in a semantic- fashion.
representation learning. "predictor"
representation learning. In this setting, the sensitive
Specifically, we propose learn the representation using the popular Variational Autoencoders (VAE) Kingma& & Welling (001)), where the latent representation is additionally decomposed into two latent factors: target and residual. Whereas the target part encodes the information for the target task, the residual part identifies and collects the data's private part. In order to sanitize the target representation, we leverage an ARL procedure. There are two general strategies for ARL: the common solution for adversarial optimization is to maximize the loss of the adversary by minimizing the negative log-likelihooo of sensitive variables. However, this is practically sub-optimal from the perspective of preventing information leakage. If the optimization does not reach the equilibrium, the resulting distribution associated with the minimum likelihood solution is subject to leaking the most amount of information. Another solution for adversarial optimization is maximize the adversary's s entropy by enforcing a uniform distribution over the sensitive labels Roy & Boddeti (2019); (Sarhan et al. (020). Such solution provides no information to the adversary. However, it has the risk of weakening the encoder as it partially eliminates the adversary's 's role in the representation learning phase and is provably bound to the adversary's 's optimality. However, fulfilling the necessary optimality conditions impractical. Hence we seek to relax optimality by leveraging a quasi- objective. To this end, we propose to maximize a variant of entropy -focal entropy - for dealing with inter-class uncertainty maximization. Focal entropy enforces the uncertainty to focus on sparse set of similar classes and prevents the vast number of dissimilar classes from overwhelming the uncertainty. Maximization of focal entropy increases the uncertainty in more organic, namely in systematic and semantic-aware fashion. Hence, it is leading to a deeper privacy sanitization during the representation learning phase.
In summary, the main contributions of this paper are three-fold. First, we propose to learn the representations. Second, introduce an ARL setting for this task by adding novel entropy term to the VAE. Third, we demonstrate experimentally that our proposed method leams semantically meaningful privacy-preservin sanitized representation.
2 RELATED WORKS
Much research has been conducted in protecting differential privacy Dwork et al. (2017 Dwork (2006); Ryoo et al.] (2017): Abadi et al. (2016) on data and parameter level by anonymizing raw data directly, incorporating randomized mechanism into the learning process, respectively. Although successful, our method is fundamentally different from them, as we aim to learn a private representation instead of preserving privacy in data or parameter level. While we do not consider their framework here, our method could employ differential privacy during the post-classifier training.
The advantages of learning and transmitting representations instead of data have been investigated recently in many works, see Osia et al. (2017) 2018), and references therein. Nevertheless, such representation is proven contain some privacy revealing information of clients. The recent
2

---

Under review as a conference paper at ICLR 2021
Fffffw))
Enc
Dec
x
Zear
pillker)
Target
Schematic illustration of the proposed approach. Left: The graphical model associated with the minimax game. Right: Our proposed architecture with the two stream network is based on VAE, and augmented with additional predictor loss and (focal) entropy.
Figure I:
success of adversarial learning has led to the increased adoption of this technique for learning representations that preserve sensitive information in different types of data. For instance, Srivvsttaa cct al. (2019) proposed to learn preeeccires representations for automatic speech recognition (ASR). In Yang et al. (2018), representation is learned on the raw student clickstream event data, captured as they watch lecture videos in massive open online courses. In Li et al. (2019), the authors proposed an obfuscator designed to hide privacy-relatee
sensitive information from the features using adversarial training. Similarly, Kim et al.] (2019) is based on adversarial learning, which encodes images obfuscate the patient identity while preserving enough information for medical segmentation task. [Pittaluga et al. (201)) considered formulation based on adversarial optimization between the encoding function and estimators for private tasks. Although our method is also based on ad udversarial learning, we differently facilitate adversarial sanitization using entropy. This leads to a more srivacy-presevvigg representation while maintaining the method complexity.
The most related work to ours is by Roy & Boddeti (2019): Sadeghi et al. (201)), which aims at obtaining a sanitized representation using entropy and adversarial representation learning, respectively. Another related work to our paper proposed by Feutry et al. (2018) aims at learning representations that preserve the relevant part the information while dismissing information about the private labels corresponding to the clients' identity. A key difference compared to this method is that they require labels for the downstream task during representation learning. Recently, Chen et al. (2018) representation learning. Gabbay & Hoshen (2020) propose an approach for disentanglement using shared latent optimization and an asymmetric regularization. Edwards & Storkey (2016) propose sanitize representations utilizing an adversary. [Lio et al. (2019 b) employ an adversary to obtain a sanitized representation, however, also incorporating fairness constraints. [Liu al. (2018) proposed to conventional entropy as an adversary in the for privacy sanitization. To the best of our knowledge, our paper the first work that proposes taking the class similarity into account for the adversary's entropy. Furthermore, fairness as proposed in Creager et al. L [ocatello et al. (2019): Quadrianto et al. (Srhaa [Sarhan al. (220)) is yet another intimately connected notion to privacy. While we do not consider fairness here, our method could also be extended to that problem, leading to exciting future research. Finally, we note that our model is different from the federated learning methodology in McMahan et al. (2016); Geyer et al. (2017), which focuses learning decentralized private model by sharing gradient updates instead of learning representations.
proposed complex method for
context of a vanilla
METHOD
We consider the scenario where we want to learn a dual-objective feature representation. The first objective entails providing high accuracy in terms of target attribute classification. The second objective entails minimizing information leakage w.r.t. sensitive attributes, ideally making classifi- cation impossible. To this end. we learn encoder that is tasked to disentangle the attributes by decomposing the representation into two parts: target and residual partition. The target partition allows classification for the target attributes and is sanitized w.r.t. the sensitive attributes. In contrast,
3

---

Under review as a conference paper at ICLR 2021
the residual partition captures the information w.r.t. the sensitive attribute, subsuming the excess infor- mation. To drive the representation learning, we employ a Variational aato-EEcoder ( Kingmal & Welling (2013). Isolating information about sensitive and non-sensitive attributes to separate subspaces then corresponds to integrating the VAE into a minimax game between the target classifiers and the adversary. Leveraging VAF, in this regard, is beneficial in multiple respects. On the one hand, the VAE backbone allows us to learn better representations in particular when disentangling factors than employing solely a classical supervised approach Le al. (2018); Gyawali et al. (2019). The representation learning objective takes advantage of the modeling flexibility and the large solution space of the VAE. With the objectives primarily orthogonal to each other, they act together rather than conflict. On the other hand, VAE allows for the integration of notion of interpretability Charte) je al. (2000); Ding et al. (2020). Integrating input reconstruction as a light-weight task in minimax game facilitates understanding the sanitization process in a numan-understandable visual way. This is a crucial point yet often overlooked in privacy.
Le
The representation learning problem is formulated game among six players: an encoder E, a decoder D, a target predictor T, a sensitive attribute predictor S, two adversarial classifiers T and S. Whereas the non-ad con-adversarial predictors enforce the utility and the presence of the associated information in the representation, the adversarial predictors inhibit the undesirable information leakage and are directly responsible for disentanglement and sanitization of the representation. What is more, the adversarial classifiers learned during training, they act as surrogate adversaries for unknown oracle post-classifiers. The setup involves observational input æ E X, M target attributes t AT Ai x AM. each having my classes Ar {a ". 0m; J. and sensitive attribute s E As {1, ..... N of N classes.
Background:
The goal yield data representations from an encoder subject to privacy constraints. To learn the representation, VAE is learned jointly along with several predictors. VAEs naturally decompose into two components an encoder and decoder.
The encoder z ~ g( 0g): X + Z, parameterized by 0g produces an embedding z in the la- tent space Z. For the sake of privacy, we employ a modfied VAE encoder. Specifically, we propose an encoder that splits the generated representation into two parts: Star E Z repre- sents the target part and residual part = E Zz. Concatenating the target and the residual parts = [Ztar Free] E 21 x Zg yields again the complete representation. For the following, we assume w.l.o.g. the target and the residual part of being of equal dimensionality. Encoding of the represen- tation entails a stack of shared initial layers that diverge into parallel stacks from the penultimate layer onward into two separate network streams for disentangling the information. To this end, we let Star qtarl (Ztar [x: 6 X 21 denote the encoder of the target representation stream and the encoder of the Zres gres (Zres æe; 0 X + Z2 the residual stream, respectively. Given the two encoding streams (with partially shared parameters), Oger 0g and Ores C Og denoting the parameters of the networks stream. Given the common parameters in the shared stack, 0 nore* o 0. The second component in the VAE, the decoder is given by x ~ p(æ|z; 0D) Z + X, which seeks to reconstruct the observational input, and is parameterized by lD.
Next, we define the predictors for the target and sensitive attributes. First, the target predictor is given as prr(t|/zar fler)) 2, + Ar. Second, the sensitive attribute predictors is given as ps ps(sszres: 6rex)) Zg - As. parameterized by Oear and fres. Last, we have the associated ad- versarial predictors denoted as pr (t) Øresi 0ree): Z2 + Ar and ps Pear) Z1 + As, pa- rameterized by dear and 0res- Note, that the difference between predictors and their adversary is their swapped input source.
Optimization: Learning the representation formalizes as the optimization of multi-player nonzero- sum game given as:
min max A δED(OE,0D) + ar (r (rrg0roor)+
O.. BT,8
Og (S Prea) + Bp (5 0rex) + B3 (g Hear),
with (-I-)) denoting the players, and X, a., 3. E IR weighting scalars. The hyperparameters allow for a trade-off between the utility of target classification and the latent code's 's privacy preservation. The associated process schematic is shown in Fig. [] As the predictors and the adversarial predictors have a competitive relationship. they are optimized differently. Whereas predictors are trained using cross-entropy minimization, the adversarial predictors seek to maximize entropy, which will be
4

---

Under review as a conference paper at ICLR 2021
sm/
05
0.00
0.25
dismier dasmierz
0.00
decmii
0.25
kmite
1m. bur
0.00
0.05
1912002
Figure 2: Illustration of focal entropy and the effect of grouping assuming equal partitioning of probability mass. Center: Visualization of sample configuration; schematic focus regions depicted as circles ranging from narrow (1) to wide (3). Left: Visualization of off-center entropies (similar, dissimilar) for different focus regions scenarios. The more narrow the focus, the more weight "similar" samples have. Conversely, the wider focus range, the more equiprobability is approached. Right: Entropy visualization for focus scenarios.
explained in the forthcoming sections. The latent space should be disentangled at the equilibrium of this minimax game, yielding the representation with the desired privacy sanitization properties.
3.1 ADVERSARIAL REPRESENTATION LEARNING
In the following, we will define the optimization criterion for each player. First, we explain the predictors, followed by the reconstruction, and conclude with the adversarial classifiers' optimization, notably sanitization and herein focal entropy.
Reconstruction:
Leveraging VAE backbone, optimization of ΦED entails minimization of Evi- dence Lower Bound (ELBO). The ELBO is defined
ΦED(OE, 0p) (=((000)))) [log (æ) 0p)] DeL (z|=; 0g) lp (=)),
which decomposes into two parts. The first term corresponds to the reconstruction likelihood measuring the error input reconstruction. The second term corresponds to the prior constraint, where we assume isotropic Gaussian latent prior (z) N (0, I), with I denoting the identity matrix. Furthermore, we assume the posterior approximates to correspond multivariate Gaussians. Given two separate encoding streams, one for the target and the residual part, we yield: Star (Ztar (z; ( N (Mtar, diag(o (tar)) and gres (Eres le; 0T**) N (pres diag(ares)), with E R2th and diag(o)) Raxh the diagonal matrix constructed from vector R
Predictors: The predictors are modeled as conditional distributions. They are trained by:
ΦT (0 Otar) = D (p.(p((t) (prr)plp(t (2ar))))))))))))))))()))))))(( Ozar) (5 frex)) D (p(s)x))lps Zr)sr))): Ores)
where p(t(ar) and (()(:)) denote the ground-truth labels for training input for the target attribute and the sensitive attribute, respectively. (=1(- denotes the Kullback-Leible divergence.
Sanitization: In order to minimize the information leakage across representation partitions, we leverage the maximization of entropy. This enforces the representation be maximally ignorant w.r.t attributes in the domain confusion sense. Specifically, for the adversarial target attribute, we let
(0 0rex) = Dr (pr(t)zr..))144 Ōrex):
where i4 denotes the uniform distribution. Although maximization of entropy is sufficient for non-private attributes to minimize information leakage across representation partitions, postulate that proper sanitization must be conducted w.r.t. to focus classes in a similarity-awan fashion. As such, we aim to sanitize the information of each observation within focus area, i.e., nearest neighbors (NN). For each observation, the NNs share the most commonalities features. As such, discrimination w.r.t. NNs is less likely to be trivially achievable. To this end, training of o (0p, drar) leverages a modification of entropy for deep sanitization.
5

---

Under review as a conference paper at ICLR 2021
3.2 FOCAL ENTROPY
Focal entropy aims at deep sanitization of sensitive information by leveraging a notion of similarity in terms of the sensitive attribute. To this end, it requires partitioning the sensitive attribute into two sets w.r.t. mutual discriminativeness. "similar" classes (A) "dissimilar" classes (B). This partitioning imput specific and is conducted either according to a) label information, or b) using some scoring function obtained using a pre-trained model or "on-the-ffy" during training. Thus, let r(æ: be the scores given the observational input æ, which we will simply denote as for the sake of economy of notation. Then the class predictions given are by the index with maximal score, i.e., p (y) r; 0) = p(r) E arg max r. Then "similar" consists of the set of labels corresponding to the k-largest scores, and "dissimilar" the complement:
ASimilar
84 E AA (k) Vi E (1,..,,))rs,,, 2 rwl
ADissimiaar As ASimilar
(6)
Here '[k] denotes the k-thh largest element of and A denotes the set of & distinct elements in As- Furthermore, we let NA, .B denote to the number classes in and B, where typically NA < NB holds and N NA + N
Given the just defined notion of similarity, focal entropy aims at establishing uniformity in terms of likelihood (w.r.t. the sensitive attribute) within each group with the probability mass divided equally between the two groups (A) and (B). Given NA * NB. this implies reciprocal re- re-weighting of the classes in A and B, respectively.
Assuming NA < NB, this aims at giving proportionally more weight to confusion w.r.. members of similar classes. Analogously, members of dissimilar classes down- weighted accordingly. Consequently, the classifier is forced to be maximally ignorant w.r.t. class properties of the similar class members that share high correlation. The feature representation produced from the encoder should not have any sensitive information in the target part. As this approach implicitly is driving sanitization by focusing on specific targets, we refer this as "focal entropy". Implementation of the focal entropy criterion is essentially equivalent to maximization w.r.t. an off-centered entropy Lallich jt al. (2007) in the special case of normalized uniform probability within each group. That is, in contrast to conventional entropy that takes its maximal value when the distribution of the class variable uniform (++ ..., +) = U. focal-entropy takes its maximum non-centered. See Fig. 2 for the schematic illustration of the proposed concept.
Specifically, focal entropy seeks to have the entropy peak T RRN, with T o U. Assuming equally divided probability mass, we yield defined as:
NB TA....NA N + NB NA IN..
NA
Σ T =1
N + NA
NB
Further, let I(.) denote the entropy w.r.t. a vector p = (pi, .... pN) E R of probabilities with Σ, P = defined as h((p) = ブラー Pj log In order to achieve entropy achieving its maximum at T, the class probabilities p have to be transformed. To that end, each E [0, is mapped to corresponding Tj. according to:
Pj NT;
N(, +1 Pj Jj N(1 T3)
iFOSP,ST
if Spspss Sp.SI.
1.
(8)
In order fulfill the properties of an entropy, the πy have to be normalized subsequently. This is achieved according to πÿ (ΣN mT) = कj. Thus we yield the off-centered entropy 77.) of probabilities p defined as rppp) h((T)) = E Ty log wy. Consequently, adversarial sanitization seeks maximize the off-centered entropy w.r.t. private attribute on Star according to:
$3(0g. 0rar) n)(o )
with 0 E R denoting the softmax vector of the adversarial predictor ps Star; Hear). Stor
Maximizing the off-centered entropy is analogous to minimizing the Kullback-Leibler divergence w.r.t. T, such that we yield:
(0F Orar) DKI. (ūr Ztar )[;; Bear).
(10)

---

Under review as a conference paper at ICLR 2021
CIFAR-100
Krizhevsky
2009)
CIFAR-100 Krizhevsky
20099
Method
Tar.
Adv.
Method
Tar.
Adv.
Random Chance
1.0
0.01
Random Chance
1.0
0.01
Roy & Boddeti (2019)
0.71
0.16
Ours (entropy)
0.70
0.16
Sadeghi al. (20T9)
0.80
0.16
Ours (F. entropy)
0.82
0.16
Ours
0.82
0.16
CelebA Guo et al. (2016)
CelebA Guo al. (2016)
Method
Tar.
Adv.
Method
Tar.
Adv.
Random Chance
1.0
< 0,001
Random Chance
1.0
< 0.001
Ours (entropy)
0.90
0.061
Szabo et al. (2018)
0.09
Ours (F. entropy)
0.90
< 0.01
Harsh Jha et al. (2018)
0.14
Denton Bırodkar
(2017)
0.03
Table 2: Ablation study (Focal entropy) on
Gabbay & Hoshen
2020
< 0.01
CIFAR-100 and CelebA.
Bouchacourt al.
2018)
0.88
0.178
CelebA Guo et al. (2016)
Liu ct al. [al...2001 (2015)
0.873
Method
Tar.
Adv.
Zhang et al. (2014)
0.854
Random Chance
1.0
< 0.001
Liu al. 2018)
0.878
Ours (normal cls.)
0.90
0.007
Ours
0.90
0.01
Ours (strong cls.)
0.90
0.009
Table I: Results on CelebA and CIFAR-100.
Table 3: Probing analysis on CelebA
should be noted that focal entropy contrasts with conventional entropy in terms of promoting hub formation. Here hub relates to the adversarial mapping of multiple sensitive "similar" entities, i.e., IDs for CelebA, to the same "target". Essentially, entropy drives for equiprobability of classes and suppresses the formation of hubs, whereas focal entropy enforces the collapse of "similar" sensitive entities on the same hub. Hence, entropy induces a bijective mapping due to its invariance to random relabeling, whereas focal entropy induces surjective mapping w.r.t. hubs. It should be noted that surjective mappings give rise to a loss of information as the input is not guaranteed to be recovered through inversion Nielsen al. 2020).
4 EXPERIMENTS
Setup: We experimentally validate the proposed method using two datasets, which exhibit different characteristics. First, the algorithm is tested on CIFAR- MAAAR-100/Krihhess/ee (2009) dataset. It consists of 100 classes, grouped into 20 superclasses. The classes are (theoretically) entirely mutually exclusive, such that each item associated with precisely one fine-grained category belonging to one coarse superclass. In the privacy setting, the encoder is forced to learn representation that is superclass aware while not revealing any information about the fine-gra mne-grained classes. Second, the CelebA dataset of celebrity face images Guo et al. (2016). This dataset contains large number of identities (people) with multiple observations of each. The "in-the-wild" nature of face images offers richer testhed for our method as both identities, and contingent factors are significant sources of variation. Furthermore, the class structure does not permit a well-defined separation between public and private spaces.
Implementation Details: We employ Stochastic Gradient Descent (SGD) with momentum 0.9 and weight-decay of le-4 and a batch size of 200, with a learning rate of 0.01. Furthermore, we employ leamingg rate decay at every 30th epoch by factor of 0.1. On CelebA, to learn target and adversary classifiers smoothly, we perform pre-training without adversary objective for 100 epochs. This allows us to learn stable representation crucial for gentle modification, i.e., disentangling the information of the sensitive and the target variable. Subsequently, the model is trained for another 100 epochs, allowing for a slow "burn-in" of adversary objectives such it cannot undone. On CIFAR- 100, the model is trained for 3000 epochs. As for the trade-off parameters of Eq. [ we employ the following
scheme. CIFAR: equal weights for target loss and adversary: ag = ar 1, 3r = 33 = 1, NN size of 5. CelebA: weighting according to number of objectives or For reconstructing weighting, we always assume = 1.0. For grouping in focal entropy, we assume the A; NN size of k = 16.
0g 1.0, 31 40 Bs 1.0.
7

---

Under review as a conference paper at ICLR 2021
0.8
PLAR.
Como SARL
Target Accuracy
0.6
+
No Pireacy
DDR
0.4
0.2
10
10 Adversarial Accuracy
(a) t-SNE embeddings
(b) Trade-off curve
Figure Left: t-SNE embedding of latent representation for subset of 200 IDs. Left t-SNE:
embedding of the target part Stor. Righ t-SNE: embedding of residual part Zres. I-SNE is unable to reveal any sort of regular structure Star w.r.t. private attribute. Right: Trade-Off curve between target accuracy and adversarial accuracy on CIFAR-100.
4.1 PRIVACY SANITIZATION ANALYSTS
We evaluated the privacy sanitization in terms of target and adversarial accuracy two benchmarks:
the first testhed, we adopt the "simulated" privacy problem proposed by Roy Boddeti (2019) designed on the CIFAR-100 dataset. We treat the coarse (superclass) and fine (class) labels as the target and sensitive attribute, respectively. The task is to learn the superclasses' features while not revealing the information about the underlying classes. Here note that "ideally" we desire a predictor accuracy of 100%, an adversary accuracy of 1% (random chance for 100 classes). In Tab. we report the accuracy achieved by the attribute predictor and adversary. From these results, we observe that with our proposed method, the representation achieves the best target accuracy while being comparable on adversary accuracy compared the SOTA sanitization methods Roy & Boddet (2019); Sadeghi er al.) (201)). Using conventional entropy for adversary instead of our proposed focal entropy results significant performance drop (from 82% to 70% in terms of target accuracy at comparable adv. accuracy). In Fig. (3b] we report the characteristics of the proposed approach in terms of trade-off curves that portray the correlation between privacy and target utility. We compared with the most relevant works to us: MaxEnt-ARL and ML-ARL from Roy & Boddeti (2019).
CIFAR-100:
(SdegSh al. (2019), Orthogonal Disentangled Representations (ODR) from Sarhan et al. (2020). and vanilla baseline (No Privacy) without sanitization. As can be seen, the higher the correlation to privacy, the higher appears the loss of accuracy at high level of sensitive accuracy. Furthermore, the proposed approach features significantly better utility trade-off in the high sarget-accuracy domain. Additionally, to assess the effect of our proposed focal entropy in terms of the adversarial component, comparing it with conventional entropy. We report the results in Tab.
CelebA: As the second testbed, we adopt the CelebA dataset, which has a richer structure to be treat the
utilized for privacy while being less structured in terms of similarity. Specifically, attribute labels as target and celebrity identity (i.e., ID) as the source of sensitive information. The task learn classify the attributes while revealing the information about the identities. We note that ideally, we desire lower adversary accuracy (< 0.001) I compared to the previous case, as the number of ID classes is an order of magnitude higher. In Tab. [[] we report the turset/advvrsary accuracy. We observe that the representation we learn achieves higher target accuracy and lower adversary accuracy than the strong baseline MI-VAE [Bocchcourtt et al. (2018). Using standard entropy instead of our proposed focal entropy in CelebA shows considerable higher privacy leakage (namely, 0.061 and < 0.01) see Tab. 2 Moreover, to analyze the sanitization of privacy in the latent representation, we visualize the t-SNE of the target and residual parts of the representation in Fig. Ba For visualization, subset of 200 IDs was chosen randomly from the test set. As can be seen, the private class ussociations appear random and cannot be recovered from the target part. This contrasts sharply with the residual part, where common IDs form clusters, giving rise to a proper margin, confirming quantitative results in terms of target/adversary accuracy. Additionally,
8

---

Under review as a conference paper at ICLR 2021
Figure 4: Visualization of adversary ID re-mapping graph CelebA for entropy to focal entropy with different k-NNs on Star. Nodes correspond to IDs, edges to ad adversarial re-mapping of an ID to facilitate adversary confusion. Node size/brightness scales with number of associations (the bigger/brighter, the more IDs are mapped to specific node). From left to right increasing & for focal entropy: l =1 standard entropy), 16 and 64.
performed probing experiment to assess the classifier capacity's dependency regarding the learned representation's 's privacy leakage potential. Specifically, we determined the adversarial accuracy on the learned representation w.r.t the classifier's strength. The st rong probing classifier has approximately double the proxy classifier's capacity that used for training see Tab. 3
4.2
HUB ANALYSIS
This section provides an analysis of how the application of focal entropy, with its integration of the notion of k K-NN, promotes the formation of "hubs". By varying the neighborhood size focal entropy manifests itself between two extremes: i) Choosing a small k, focal entropy in the limit (k 1) approaches conventional entropy no hubs are formed remapping is a "one-to- one" correspondence. This can be attributed to conventional entropy being invariant to random relabeling, inducing a bijective mapping, giving rise to information preservation. This prone to sub-optimal solutions since the forced equalization of classes by disregarding semantic similarity of classes makes the model susceptible to find shortcuts such as label swapping. In this regard, target accuracy oscillations co-occurring with degenerate ID-remappings such uniformity across all labels or temporary collapse to very few hubs were observed. ii) Choosing a large k: focal entropy promotes the formation of a single dominant hub, which is also referred to the classical hubness problem. This phenomenon is related to the convergence of pairwise similarities between elements to constant the space's 's dimensionality increases Radovanović et al. (2010) (collapsing on single hub/trivial solution). The single hub then manifests itself as the favored result of queries Dinu et al. (2014) - giving rise to trivial solutions rather than deep sanitization. Choosing non-exin non-extreme k leads to the formation of numerous equisized hubs, inducing a surjective mapping. As the formation of multi-hubs coincides with collapse, less information is required to establish mapping. Hence, this characteristic gives rise to information removal and is thus imperative for proper sanitization. Figure depicts the graphs induced by remapping for four different neighborhood configurations. Analyzing the nodes' average degree in the corresponding graphs, we observe continuous decrease with growing neighborhood size k. Specifically, starting with entropy and increasing the increasing for focal entropy, yield the average degrees of: 13.91, 7.51, 7.2, and 3.0.
5 CONCLUSION
In this paper, an adversarial representation learning method proposed to deal with a setting where the target and sensitive attributes are related to each other to a large extent. Training the representation entails decomposition into a target and a residual part. Here, the target part is shareable without privacy infringement and facilitates applicability for target task. In contrast, the residual part subsumes all the private non-shareable information. Our proposed adversarial learning method employs focal entropy for deep sanitization of privacy from the representation. Subsequently, confirm that our proposed method learns sanitized data representations using a manageable level of supervision. Future work will entail adaptive and "on-the-fly" similarity grouping during training.
experiments

---

Under review as a conference paper at ICLR 2021
REFERENCES
Martin Abadi, Andy Chu, lan Goodfellow, H Brendan McMahan, llya Mironov, Kunal Talwar, and Li Zhang. Deep learning with differential privacy. In Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security, pp. 1088-11.440 ACM, 2016.
Frederik Armknecht, Colin Boyd, Christopher Carr, Kristian Gjøsteen, Angela Jäschke, Christian A. Reuter, and Martin Strand. A guide to fully homomorphic encryption. IACR Cryptology ePrint Archive, 2015:1192, 2015, URL http: /ttp//dbpp... uni triitrriiee .e de/ aaapier.ddee///oooooaaa/////aa Tacr2215. htm TattttmmAAAArre wwrmetrrrmnniniee BCGJR815
Lars Backstrom, Cynthia Dwork, and Jon Kleinberg. Wherefore art thou r3579x?: anonymized social networks, hidden patterns, and structural steganography. In Proceedings of the 16th international conference on World Wide Web, pp. 181-190. ACM, 2007.
Diane Bouchacourt, Ryota Tomioka, and Sebastian Nowozin. Multi-level variational autoencoder:
Learning disentangled representations from grouped observations. In Thirty-Second AAAI Confer- ence on Artificial Intelligence, 2018.
David Charte, Francisco Charte, María J. del Jesus, and Francisco Herrera. An analysis on the use of autoencoders for representation learning: Fundamentals, learning task case studies, explainability
and challenges. Newrocompurting. 404:93 107, 2020. ISSN 0925-2312. doi: https://doi. 10t.:..001//...ccoooo.... art.icle/p11//092223122
URL http::///// http://wwww.iee.e..dddeee rect. com/ science/
Jiawei Chen, Janusz Konrad, and Prakash Ishwar. Vgan-based image representation learning for facial expression recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 1570-1579 1579, 2018.
Elliot Creager, David Madras, Jörn-Henrik Jacobsen, Marissa A Weis, Kevin Swersky, Toniann Pitassi, and Richard Zemel. Flexibly fair representation learning by disentanglement. arXiv preprint 190.:0996600589 2019.
Emily L Denton and vighnesh Birodkar. Unsupervised learning of disentangled representations from video. In I. Guyon, U. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett (eds.), Advances in Neural Information Processing Systems 30, pp. 4414-4423, Curran Associates, Inc., 2017.
Zheng Ding, Yifan Xu, Weijian Xu, Gaurav Parmar, Yang Yang, Max Welling, and Zhuowen Tu. Guided variational autoencoder for disentanglement learning. In Proceedings of the IEEE/CVF Conference Computer Vision and Patlern Recognition (CVPR), June 2020.
Georgiana Dinu, Angeliki Lazaridou, and Marco Baroni. Improving zero-shot learning by mitigating the hubness problem, 2014.
Cynthia Dwork. Differential privacy. In 33rd International Colloquium on Automata, Languages and Programming, part II (ICALP 2006), volume 4052 of Lecture Notes in Computer Science, pp- 1-12. Springer Verlag, July 2006. ISBN 3-540-35907-9.
Cynthia Dwork, Adam Smith, Thomas Steinke, and Jonathan Ullman. Exposed! a survey of attacks on private data. Annual Review of Statistics and Its Application, 4:61-84, 2017.
Harrison Edwards and Amos Storkey. Censoring representations with adversary. In International Conference in Learning Representations (ICLR2016), 2 2016.
Clément Feutry, Pablo Piantanida, Yoshua Bengio, and Pierre Duhamel. Learning anonymized representations with adversarial neural networks. arXiv preprint arXiv:1802.09386 2018
Aviy Gabbay and Yedid Hoshen. Demystifying inter-class disentanglement. In International Conference on Learning Representations (ICLR), 2020.
Robin Geyer, Tassilo Klein, and Moin Nabi. Differentially private federated learning: A client level perspective. arXiv preprint 172.0777 2017.
10

---

Under review as a conference paper at ICLR 2021
Yandong Guo, Lei Zhang, Yuxiao Hu, Xiaodong He, and Jianfeng Gao. Ms-celeb- Im: A dataset and benchmark for large-scale face recognition. In European Conference on Computer Vision, pp- 87-102. Springer, 2016.
Prashnna Kumar Gyawali, Zhiyuan Li, Cameron Knight, Sandesh Ghimire, B. Milan Horácek, John L. Sapp, and Linwei Wang. Improving disentangled representation learning with the beta bernoulli process. CoRR, abs/1909.01883 2019. URL
00..09.01833 01839
Ananya Harsh Jha, Saket Anand, Maneesh Singh, and VSR Veeravasarapu. Disentangling factors of variation with cycle-consistent variational auto-encoders. In The European Conference on Computer Vision (ECCV), September 2018.
Bach Ngoc Kim, Christian Desrosiers, Jose Dolz, and Pierre-Mare Jodoin. Privacy-net: An adversarial approach for identity-obfuseatted segmentation. arXiv preprint arXiv: (00,://99 2019.
Diederik P Kingma and Max Welling. Auto-encoding variational bayes, 2013. cite arxiv: 1312.61 14.
Alex Krizhevsky. Learning multiple layers of features from tiny images. Technical report, 2009.
Stéphane Lallich, Philippe Lenca, and Benoît Vaillant. Construction of an off-centered entropy for supervised learning. In ASMDA 2007 XIlth International Symposium on Applied Stochastic Models and Data Analysis, May 29 June 1, Chania, Crete, Greece, pp. Crete, Greece, May 2007. C. Skiadas. URL BtLLhtts::// haahaaaaaa surchiivs--ovvertes
Lei Le, Andrew Patterson, and Martha White. Supervised autoencoders: Improving generalization performance with unsupervised regularizers. In S. Bengio, H. Wal- lach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, and R. Garnett (eds.), Advances
in Neural Information Processing Systems, volume 31, pp. 107-117. Curran Asso- ciates, Inc., 2018 URL htttps://proceeeett wewpingg...uurrre 2a38a4a 01233314493500 6c44 35
neurips wwtppppp/p/1.1c..c. /ttper////1///11ee.
Ang Li, Jiayi Guo, Huanrui Yang, and Yiran Chen. Deepobfuscator: Adversarial training framework for preeeccirree image classification. arXiv preprint arXiv: 04126, 2019.
Jiachun Liao, Chong Huang, Peter Kairouz, and Lalitha Sankar. Learning generative adversarial representations (GAP) under faireess and censoring constraints. CoRR, abs/1910.00411, 2019a.
00411
Jiachun Liao, Chong Huang, Peter Kairouz, and Lalitha Sankar. Learning generative adversarial representations (gap) under faimesss and censoring constraints. arXiv preprint 1,0.041910.00411, 2019b.
Yu Liu, Fangyin Wei, Jing Shao, Lu Sheng, Junjie Yan, and Xiaogang Wang. Exploring disentangled feature representation beyond face identification. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2080-2089, 2018
Ziwei Liu, Ping Luo, Xiaogang Wang, and Xiaoou Tang. Deep learning face attributes in the wild. In Proceedings of the IEEE international conference on computer vision, pp. 3730-3738, 2015.
Francesco Locatello, Gabriele Abbati, Thomas Rainforth, Stefan Bauer, Bernhard Schölkopf. and Olivier Bachem. On the faimess of disentangled representations. Advances in Neural Information Processing Systems, pp. 1461 1-14624, 2019.
Christos Louizos, Kevin Swersky, Yujia Li, Max Welling, and Richard Zemel. The variational fair autoencoder. arXiv preprint arXiv:1511. 000300, 2015.
David Madras, Elliot Creager, Toniann Pitassi, and Richard Zemel. Learning adversarially fair and transferable representations. arXiv preprint arXiv:/802 06309, 2018.
H Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, et al. Commumication--fficiinn learning of deep networks from decentralized data. arXiv preprint arXiv:1602.05622 2016.
Arvind Narayanan and Vitaly Shmatikov. How to break anonymity of the netflix prize dataset. arXiv preprint es/0610105, 2006.
11

---

Under review as a conference paper at ICLR 2021
Didrik Nielsen, Priyank Jaini, Emiel Hoogeboom, Ole Winther, and Max Welling. Survae flows: Surjections to bridge the gap between vaes and flows, 2020.
Seyed Ali Osia, Ali Shahin Shamsabadi, Ali Taheri, Kleomenis Katevas, Hamid R Rabice, Nicholas D Lane, and Hamed Haddadi. Privacy-peeervvig deep inference for rich user data on the cloud. arXiv preprini arXiv: /7 710.01727, 2017.
Seyed Ali Osia, Ali Taheri, Ali Shahin Shamsabadi, Minos Katevas, Hamed Haddadi, and Hamid RR Rabiee. Deep private-feature extraction. IEEE Transactions on Knowledge and Data Engineering. 2018.
Francesco Pittaluga, Sanjeev Koppal, and Ayan Chakrabarti. Learning privacy preserving encodings through adversarial training. In 2019 IEEE Winter Conference on Applications of Computer Vision (WACV), pp. 791-799. IEEE, 2019.
Novi Quadrianto. Viktoriaa Sharmanska, and Oliver Thomas. Discovering fair representations in the data domain. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 8227-8236, 2019.
Miloš Radovanović, Alexandros Nanopoulos, and Mirjana Ivanović. Hubs space: Popular nearest neighbors in high digeediimesionnll data. Journal of Machine Learning Research, 11(86):2487-2531 2010. /Rtt:t http:/rro....pppp///////aaaaaaaa
html
Proteek Chandan Roy and Vishnu Naresh Boddeti. Mitigating information leakage in image repre- sentations: A maximum entropy approach. In Proceedings of the IEEF Conference on Computer Vision and Pattern Recognition, 2019.
Michael S Ryoo. Brandon Rothrock, Charles Fleming, and Hyun Jong Yang. human activity recognition from extreme low resolution. In Thirty-First AAAI Conference on Artificial Intelligence, 2017.
Bashir Sadeghi, Runyi Yu, and Vishnu Boddeti. On the global optima of kernelized adversarial representation learning. In The IEEE International Conference on Computer Vision (ICCV), October 2019.
Ahmed Salem, Apratim Bhattacharya, Michael Backes, Mario Fritz, and Yang Zhang. Updates-leak: Data set inference and reconstruction attacks in online learning. arXiv preprint 11X1v:1904.01067, 2019.
Mhd Hasan Sarhan, Nassir Navab, Abouzar Eslami, and Shadi Albarqouni. Fairness by learning orthogonal disentangled representations. European Conference Computer Vision (ECCV), 2020.
Brij Mohan Lal Srivastava, Aurélien Bellet, Marc Tommasi, and Emmanuel Vincent. Privacy- preserving adversarial representation learning in asr: Reality or illusion? Proc. Interspeech 2019, pp. 3700-3704, 2019.
Attila Szabo, Qiyang Hu, Tiziano Portenier, Matthias Zwicker, and Paolo Favaro. Challenges in disentangling independent factors of variation, 2018.
Qizhe Xie, Zihang Dai, Yulun Du, Eduard Hovy, and Graham Neubig, Controllable invariance through adversarial feature learning. In Advances in Neural Information Processing Systems, pp. 585-596, 2017.
Tsung-Yen Yen Yang, Christopher Brinton, Prateek Mittal, Mung Chiang, and Andrew Lan. Learn- ing informative and private representations via generative adversarial networks. In 2018 IEEE International Conference on Big Data (Big Data), pp. 1534-1543. IEEE, 2018.
Ning Zhang, Manohar Paluri, Mare' Aurelio Ranzato, Trevor Darrell, and Lubomir Bourdev. Panda: Pose aligned networks for deep attribute modeling. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 1637-1644, 2014.
12

---

Under review as a conference paper at ICLR 2021
LEARNING PRIVATE REPRESENTATIONS WITH FOCAL ENTROPY (APPENDIX)
A Anonymous authors Paper under double-blind review
In the following sections, we add additional details omitied in the main paper due space restrictions. In Sec. [ present ablation study different components of the loss function, underlining the importance of each term. In Sec. 2 we analyze the impact of varying the neighborhood size of k-NN in focal entropy the adversarial accuracy with experiments conducted on the CclebA dataset. In Sec. 3 the sanitization convergence behavior of different classifiers involved in the adversarial minimax game is analyzed. In Sec. 4 we analyze the effect of the classifier-strenggh on the privacy leakage and dependence on training time. In Sec. we present more visualizations around the concept of hub formation and the connection to focal entropy. It contains a visualization of hub forming identities and zoom-in visualization of the adversarial remapping of the identities in CelebA Next, we generated reconstructions on CelebA dataset samples generated in Sec. 6 In Sec. 7 we present detailed results of the accuracy and privacy trade-off on all the attributes in CelebA dataset. Finally, architectural details are presented in Sec. g
1 ABLATION ANALYSIS ON LOSS COMPONENTS
To assess the contribution of each component of our objective function, we evaluated each module's performance separately, gradually adding components: reconstruction loss, target classification loss, adversary loss. For that, we report the ablation study of the loss components CelebA in Tab. 1 Furthermore, Fig. 1a shows the dependency between adversarial accuracy and the number of training epochs. As can be seen, beyond 1000 epochs, the adversarial accuracy drops below 0.01, ncaring chance level.
0.00
0.08
0.000
0.08
0.15
0.055
0.62
0.00
-0.055
200
800
1000
1200
1400
0.00
200
400
800
1000
1200
1800
Epachs
(a) Adversarial classifier training time depen- dency
(b) Normal and strong adversarial classifier training time dependency
Figure Left: Relationship between adversarial accuracy and the number training epochs on CelebA. The translucent. band corresponds to 50% confidence minimum and maximum adversarial accuracy, respectively. Right: Relationship between adversarial accuracy for strong (red) and normal classifier (blue) w.r.t. the number of training epochs CelebA. The translucent band corresponds to 50% confidence interval. Dashed lines correspond to minimum and maximum adversarial accuracy, respectively.

---

Under review as a conference paper at ICLR 2021
2 NEIGHBORHOOD SIZE
Focal entropy entails notion similarity tied the integration of N-NN. Here, we study the effect of varying & on local entropy and the associaled the adversary accuracy. See Fig. 2 for a visualization of this relationship on the CelebA dataset. As can be seen, the adversary accuracy has oscillatory behavior with various local minima, reaching optimum around 16 This can be attributed to the superpositioning of different hubs, each exhibiting a different similarity pattern. Analysis of hub formation is explained in Sec. 4.2 in the main paper and Fig. in the main paper.
0.150
0.125
Adversarial Accuracy
0.100
0.075
0.050
0.025
0.000
-0.025
Neighborhood Size
Figure 2: Relationship between adversary accuracy and N-NN size on CelebA dataset. Translucent band corresponds to 50% confidence interval. Dashed lines correspond to minimum, and maximum adversary accuracy, respectively.
SANITIZATION CONVERGENCE BEHAVIOUR
This section explores the behavior of standard entropy and the proposed local entropy for sanitization. Fig. 3 depicts the classification performance during the training of different classifiers involved in the minimax optimization scheme: target classifier accuracy, adversarial sensitive attribute accuracy, and sensitive attribute accuracy. As can be seen, employing standard entropy for sanitization results in re-occurring patterns of oscillations. This be attributed to degenerato/ /trivial solutions and "shorteuts". In contrast to that, focal entropy shows relatively smooth convergence behavior. More details on the behavior can be found in Sec. 4.2 in the main paper.
Targek Accuray
Adverardal Accursty
Senstibe
017-
Figure 3: Sanitization convergence behavior of standard entropy and focal entropy CIFAR- 100 for different classifiers: Left: Target accuracy, Center: Adversarial accuracy, Right: Sensitive attribute accuracy
2

---

Under review as a conference paper at ICLR 2021
4 PROBING ANALYSIS WITH STRONG CLASSIFIER
This section provides more detail on assessing the classifier strength in terms of privacy kage and the dependence on training time. We thereby largely follow the protocol of Harsh Jha ett al. (2018); Sadeghi et al. (2019). Specifically, we employed stronger post-classifer (Tab. 3 in the main paper) compared to the one used for learning the representation. The stronger post-classifier is endowed with additional layer stack (see Tab. [6c for architectural details), trained for 100 epochs. The results in Tab. of the main paper suggest no significant changes in target and adversarial accuracy. Figure [b extends these results, depicting the relationship w.r.t. the number epochs. As can be seen, the difference between the strong and normal classifier is largely constant, independent of the epoch.
CelebA Guo et al. (2016)
Method
Target Acc.
Adversarial Acc.
Upper-bound / Random Chance
1.0
< 0,001
Our Method [only Rec. loss]
0.88
Our Method (Rec. + Tor, loss)
0.91
0.751
Our Mothod[full] (Rec. + Tar. + Adv. loss)
0.90
0.01
Table Ablation analysis for loss components on CelebA dataset.
Figure Visualization of CelebA identities of adversary classification network. The network
(green) corresponds the ncarest neighborhood size k = 5.
3

---

Under review as a conference paper at ICLR 2021
5 EXTENDED HUB ANALYSIS
This section provides a visually more detailed analysis of how the application of focal entropy promotes the formation of "hubs" (explained in Sec. 4.2 in the main paper). To study that, analyzed the identity remapping of IDs the CelebA dataset. Eipploynng focal entropy results in surjective ID confusion pattern by taking similar IDs into account for privacy sanitization.
5.1 VISUALIZATION OF HUB FACES:
To study hubs' semantics, we visualize the CelebA identities of the network corresponding to focal entropy with k-nearest neighborhood size & = 5. See Fig. 4 for the visualization of the hub faces. As can be seen, the hubs exhibit rich diversity in facial properties.
5.2 ADVERSARIAL IDENTITY MAPPING:
Figure Visualization of the remapping of IDs in CelebA due to adversarial representation learning. Source IDs (left) are remapped to new target IDs (right). Pictures on the left are samples that get mapped to a hub; separation with bar indicates different target hub. Pictures the right are the visualization of hub identities. Visualization contains a subset of 150 IDs, with targets getting at least four associations. The first number at each node indicates the ID, the second the number of images per ID. Node splicing indicates the remapping of a single ID) to multiple adversarial targets.
Figure 5 is zoom-in version of a graph as shown in Fig. the main paper, with -earest neighborhood size k = 5. This visualization provides more in-depth view of how the adversarial process leads to a remapping of identities. In order to avoid visual clutter, subset of identities and largets was chosen. As can be seen, instead of being collapse of facial stereotype, each hub is associated with a diverse looking set of identities, giving rise to the deep sanitization of the representation.
6 QUALITATIVE RESULTS
Figure 7 shows different reconstructions of additional CelebA identities (equal male and female) at different privacy levels. Each column is two different samples from CelebA (one male and female), and from top to bottom, the privacy disclosure is decreasing for each. can be noticed that visualizations from residual latent part and target latent parts confirm the sanitization visually.
4

---

Under review as a conference paper at ICLR 2021
6.9-
phayy
-63
Menty Hiplly Apea
Devin
Hott Demtions
New
the
Pert
oeong deoiee
Heby Aldee
Mt
Privacy Correlatoo
Figure Attribute-level Privacy Analysis: The normalized A-Accuracy and privacy trade-off'
on CelebA dataset. See Tab. 2 for detailed results.
ATTRIBUTE-LEVEL PRIVACY ANALYSIS
In Tab. 2 report numerous statistics CelebA attributes. Specifically, analyze the attribute classification accuracy behavior w.r.t. varying level of privacy of attributes. To our knowledge, this the first exploration of attribute-level privacy-preseriigg representation learning algorithms on the Celeb-A datasct. The rich correlation structure amongst all attributes makes this a challenging privacy dataset; i.c., for non-private attributes and low accuracy for private ones. Therefore representative subsets of the original altributes were chosen for a more in-depth analysis as indicated by the first column in Tab. 2 The subsets were chosen on the grounds of different values of correlation with IDs, reflecting different privacy levels each and, as such, exhibiting "relatively unbiasedness" in terms of frequency within the dataset. Each of the three subsets (low, intermediate, and high privacy, resp.) consists of 5 attributes. Data statistics reported comprise normalized accuracy (w.r.t. prior) of the classifiers trained the target representation Star, the A-accuracy for each attribute corresponding to the differenco of the accuracy between
difficult achieve high accuracy
classifiers the one trained on Star; and the one trained Zres- The last but one column corresponds to the correlation between the level of "privateness" and each attribute. As it can be seen in the table, the more unrelated variable is the identity, the higher gains in accuracy manifest. Finally, the last column corresponds to the frequency (prior) of the attribute in the dataset. For a visualization of the privacy-attribute correlation, see Fig. D As can be seen, the higher the correlation to privacy, the higher appears the loss accuracy. For example, privacy-revealing attributes such as gender and facial attributes strongly correlate, therefore exhibiting significant loss in accuracy. In contrast, attributes related to temporal facial features such as gestures feature low privacy correlation and therefore achieve strong accuracy. Conversely, the more unrelated variable to the identity, the higher gains in accuracy manifest.
8 ARCHITECTURAL DETAILS
We describe the architectures of each part of our model. Table shows the architectures of the VAF, i.e., the encoder and the decoder. It should be noted that the last two layers of the encoder in Tab. 3 arise from layer splitting to accommodate for partitioning target. and residual representations. This is highlighted with dashed lines. Furthermore, we provide the architectures of classifiers in Tab. 6, Architectures for target and adversarial classifiers are identical.

---

Under review as a conference paper at ICLR 2021
CIFAR- 100 Krizhevsky (2009)
Group
Attribute
Target Accuracy
A-Acuracyy
Correlation
Prior
Low
Mouth Slightly Open
0.92
0.187
0.222
0.483
Low
Smiling
0.912
0.16
0.239
0.482
Low
High
Cheekbones
0.861
0.119
0.267
0.455
Low
Wavy Hair
0.801
0.0248
0.326
0.32
Low
Oval Face
0.738
0.00962
0.332
0.284
Attractive
0.789
-0.0144
0.335
0.513
Pointy Nose
0.752
-0.0248
0.351
0.277
Straight Hair
0.791
-0.0437
0.356
0.208
Bags Under Eyes
0.835
0.0151
0.358
0.205
Brown Hair
0.849
0.00913
0.361
0.205
Arched Eyebrows
0,826
-0.006
0.369
0.267
Wearing Earrings
0.865
0.0569
0.371
0.189
Big Nose
0.814
-0.0635
0.396
0.235
Narrow _Eyes
0.89
0.0404
0.402
0.115
Bangs
0.949
0.0819
0.405
0.152
Bushy Eyebrows
0.893
-0.0505
0.419
0.142
Blond Hair
0.936
-0.0159
0.429
0.148
Big Lips
0.78
-0.0721
0.434
0.241
Intermediate
Black Hair
0.87
-0.00142
0.370
0.239
Intermediate
Heavy _Mkeup
0.89
0.0131
0.380
0.387
Intermediate
Wearing_Neckkee Necklace
0.882
0.00404
0.403
0.123
Intermediate
Wearing Lipstick
0.906
-0.00617
0.415
0.472
Intermodiate
No Lo__eeard
0.935
-0.00149
0.435
0.835
Clock Shadow
0.913
-0.0467
0.435
0.111
Wearing Nocktic
0.941
0.0509
0.437
0.0727
Receding Hairline Blurry
0.931
-0.0212
0.439
0.0798
0.956
0.125
0.443
0.0509
Rosy Cheeks
0.95
0.0581
0.448
0.0657
Eyeglasses
0.987
0.0752
0.457
0,0651
Chubby
0.952
-0.0341
0.458
0.0576
Wcaring Hat
0.982
0.0657
0.459
0.0485
Double Chin
0.958
-0.0421
0.46
0.0467
Pale Skin
0.967
0.108
0.461
0.0429
Goatee
0.955
-0.0446
0.461
0.0628
Young
0.866
-0.0211
0.462
0.774
High
Sideburns
0.957
-0.0531
0.463
0.0565
High
Gray Hair
0.971
-0.0344
0.471
0.0419
High
Mustache
0.965
-0.0413
0.474
0.0415
High
Bald
0.984
-0.0332
0.486
0.0224
High
Male
0.958
-0.0262
0.494
0.417
Table 2: CelebA attribute-level privacy analysis. The group label indicates whether, and for which group attribute was selected for visualization of Fig. in the main paper. Accuracy on the target and residual representation part, respectively. Prior is the attribute frequency bias dataset split.

---

Under review as a conference paper at ICLR 2021
OTTO
Figure Visualization CelebA data and reconstructions at different privacy levels. (From top to bottom, privacy revelation decreasing).

---

Under review as a conference paper at ICLR 2021
Layer
Output
Parameters
Input: 128 x 128 x 3
Layer
Output
Parameters
Conv-2d
64 x 64
x
st.
Inpul: 1024
BatchNorm
Linear
32768
LeakyReLU
negative slope: 0.01
BatchNorm
Conv-2d
32 x 32
128 x [ x 3), st. 2
LeakyReLU
negative slope: 0.01
BatchNorm
DeConv-2d
16 x 16
256 x [3 x 3/. st.
LeakyReLU
negative slope: 0.01
BatchNorm
Conv-2d
16
16
256 x 3 x 3), st.
LeakyRel.U
negative slope:
0.01
BatchNorm
Conv-2d
32 x 32
128 x 3 x 3/ st.
LeakyReLU
negative slope: 0.01
BatchNorm
Conv-2d
8 8x8
512 x 3 x 3), st.
LeakyReLU
negative slope:
0.01
BatchNorm
Conv-2d
64 x 64
64 x 3 x 3/ st. 2
LeakyReLU
negative slope: 0.01
BatchNorm
Linear
I x 4096
LeakyReLU
negative slope: 0.01 128 x 128
Linear
1 x 512
Conv-2d
3 x 3 x 3], st.
Linear
I Tx 4096
Tanh
Linear
1 x 512
Table 4: Decoder
Table 3: Encoder
Table 5: Architectural details of VAE components. Parameters for convolutions correspond to: number kernels kernel size J, and stride. Dashed lines in the encoder denote the two separate streams.
Layer Output size
Params
Layer
Output size
Params
Linear
256
Linear 256
BatchNorm
BatchNorm
PReLU
PReLU
Dropout drop-rate: 0.2
Dropout drop-rate: 0.5
Linear
128
Linear
128
BatchNorm
BatchNorm
PReLU
PReLU
Linear
#classes
Linear
#IDs or #attributes X
(n) Classifier CIFAR-100 100
(b) Normal classifier on ColehA
Layer
Output size
Params
Linear
256
BatchNorm
PReLU
Dropout drop-rate: 0.5
Linear 128
BatchNorm
PReLU
Dropout drop-rate: 0.5
Linear 256
BatchNorm
PReLU
Linear
#
(c) Strong ID classifier on CelebA
Table 6: Architectural details of the used classifiers
8

---

Under review as a conference paper at ICLR 2021
REFERENCES
Yandong Guo, Lei Zhang, Yuxiao Hu, Xiaodong He, and Jianfeng Gao. Ms-celeb-1m: A dataset and benchmark for large-scale face recognition. In Europeon Conference Computer Vision, pp. 87 102. Springer, 2016.
Ananya Harsh Jha, Saket Anand, Maneesh Singh, and VSR Veeravasarapu. Disentangling factors of variation with cycle-consistent variational auto-encoders. In The European Conference on Computer Vision (ECCV), September 2018.
Alex Krizhevsky. Learning multiple layers of features from tiny images. Technical report, 2009.
Bashir Sadeghi, Runyi Yu, and Vishnu Boddeti. On the global optima of kernelized adversarial representation learning. In Proceedings of the IEEE International Conference on Compuler Vision, pp. 7971-7979, 2019.


Reviews:
[{"review_id": "k26Ujfge5lR", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "## Summary\n- The paper tackles the problem of adversarial representation learning i.e., to learn a low-dimensional representation of the image that encodes target-relevant non-sensitive information, but not (often correlated) sensitive information.\n- The approach follows a VAE-based adversarial framework, with the core novelty being the use of maximizing focal entropy  (i.e., entropy among a set of similar attributes) over sensitive attributes in the representation.\n- Evaluation performed on CIFAR100 and CelebA indicates that the approach outperforms similar baselines under certain conditions.\n\n---\n\n## Strengths\n\n**1. Focal entropy**\n- I appreciate the insight exploited by the authors -- to encourage representations with high entropy over a small set of correlated sensitive attributes\n\n**2. Evaluation**\n- I found the evaluation quite thorough. The paper compares with many recent baselines, demonstrates performances curves and additional analysis to explain the model's behaviour.\n\n---\n\n## Concerns\n\n### Major Concerns\n\n**1. Objective**\n- There are a few things unclear to me in the objective (Eq. 1-6) and would appreciate if the authors clarified them.\n- (a) I don't get the motivation for the two pairs of the classifiers $(T, S)$ and adversarial $(\\tilde{T}, \\tilde{S})$ that tries to simultaneously minimize/maximize both the target $p(a|z)$ and sensitive attributes $p(y|z)$.\n- (b) Especially, I don't understand the reasoning behind having an adversarial loss on the target attribute -- shouldn't this be maximized in all cases?\n- (c) I am also confused with the sanitization term (Eq. 6). In particular of why the sensitive attribute classifier $\\tilde{S}$ is a function of the target attribute classifier $\\tilde{\\theta}_{tar}$?\n- (d) The term $\\phi_{\\tilde{T}}$ appears to be undefined to complement Eq. 6.\n- (e) It would also be nice if the authors extended Eq. 6 (which maximizes standard entropy) with the final objective to maximize focal entropy.\n- (f) Is the VAE term (Eq. 5) necessary (esp. reconstruction)? After all, this appears as a secondary objective opposed to the primary objective of learning a representation $z$ which minimizes information leakage.\n\n**2. Writing - Sec. 3**\n- My issues in understanding can be partly attributed to the writing in Sec. 3, which I found difficult to follow for a few reasons.\n- (a) Some terms seem to be overloaded e.g., two notations for encoder $q(x; \\theta_E)$ and $E(x; \\theta_E)$. I was also thrown off by many ways the components of the model/objective are conveyed: Losses $\\phi_T$, parameters $\\theta_T$, player $T$, adv. player $\\hat{T}$, etc. I recommend finetuning the second paragraph on page 4.\n- (b) While the architecture in Fig. 1 is designed to accompany the text, I find it somewhat incomplete and unclear. For one, there are six players/blocks in the proposed architecture. However, only the encoder and decoder are shown in the figure. Furthermore, while there are five loss terms, only four of them are shown in the figure. I am also not sure if the blue/red backgrounds in the residual and target streams code something in particular. Perhaps one solution is to split the figure into two: one for the architecture and another for information flow?\n\n**3. Results**\n- While the focal entropy makes sense to me (i.e., by increasing entropy over a small set of similar classes), I wonder if the results confidently back up that is it indeed better than related baselines.\n- (a) The improvements seem somewhat marginal e.g., an improvement of 2% accuracy (Table 1) over prior work on both CIFAR100 and CelebA.\n- (b) But what I find more revealing of the performance is the trade-off curve in Fig. 3b (thanks for the presenting this!). It appears that the proposed focal entropy approach (blue curve) outperforms Kernel-SARL (red curve) only for in a small operating range of high target accuracy. Overall it appears that MaxEnt-ARL and Kernel-SRL offers better trade-offs.\n\n\n### Minor Concerns\n\n**4. Decoder/Reconstruction**\n- Since the task is to partly perform reconstruction as well, I find missing evaluation on how good the reconstructed samples are.\n\n**5. Same equilibrium issues as before?**\n- The introduction (second paragraph, p2) remarks that the adversarial min-max formulation of the problem has an issue that there is significant leakage if the optimization does not reach equilibrium.\n- Wouldn't this be an issue in the proposed work as well?\n\n### Nitpicks\n\n**6. Some nitpicks**\n- Please label the axes in Fig. 2.\n- Not sure what this sentence means \"increases the uncertainty in a more organic fashion\" - please rephrase\n- Typo in citation \"Radovanovi263 et al. (2010)\"", "extra_scores": {}, "extra_text": {}}, {"review_id": "j5yB8TANtJq", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Summary: This paper gives a method in the class of learning representations which have some information censored. In particular, the authors propose a setup where there are many “private” classes, and some classes are more similar than others – this maps (I think) onto the privacy setting, where each class is like one individual. They give a modification of an entropy loss, focal entropy, which is conducive to this type of learning, and show in experiments that this method can be successful.\n\nI recommend reject due to a lack of clarity in the formulation, motivation, and writing of this paper. I think the idea has promise, the experiments are fine, and if correctly communicated can be a successful paper, but as it stands I found the paper pretty confusing and I’m not really sure what the method is for.\n\nStrong points: \n-\tThe problem as I understand it is interesting and the work is well-situated in the privacy literature. \n-\tFocal entropy seems like a good idea and as far as I know is novel as a loss. \n-\tExperiments on CIFAR and CelebA demonstrate some good behaviours from this method, mostly beating baselines\n-\tExperiments are mostly good, decently thorough\n\nWeak points and Clarifications:\n-\tThe main weak point of this paper is the exposition and clarity – I find that the problem setup is not explained particularly well. Especially as someone who is more familiar with the fair representation learning literature, I get confused when the authors refer to a sensitive attribute, or private part of the data, in this setting – it seems to be a different notion than I am used to and it is never clearly defined. If I squint I can see how it maps onto privacy nicely but I would prefer if the authors make that clear.\n-\tThe introduction could use a rewrite – it doesn’t set up the main points of the paper particularly clearly and leaves the reader a little confused. Consider stating more clearly off the top what the problem statement is, and move much of the content of p1 to related work\n-\tThe motivation of “highly overlapping information” is pretty imprecise. Having read the paper I can kind of see what you’re getting at but it’s not clear – I think you mean instead information hierarchy? Sub and super classes\n-\tAs far as I can tell the exact equation for focal entropy is never actually given? Let me know if I’m wrong\n-\tFigure 3b – you mentioned the better tradeoff in the high accuracy domain, but not what appears to be the worse tradeoff in the low accuracy domain. Is there anything I’m missing about understanding this figure?\n\nOther feedback:\n-\tP2 “the solutions mentioned earlier can only meet its practical promises …” – need a citation for this or explanation\n-\tP2: explain these suboptimalities more, I’m not sure exactly what you’re referring to\n-\tP2: you bring up a “vast number of dissimilar classes” without explaining why that’s relevant – in much work we deal with binary attributes/classes\n-\tEq 4: looks a lot like standard fair representation learning, so need to explain more clearly in motivation why it’s different\n-\t“Although maximization of entropy is sufficient for nonprivate attributes to minimize information leakage across representation partitions, we postulate that proper sanitization must be conducted w.r.t to focus classes in a similarity-aware fashion” – this sentence is very important to the paper, and I don’t quite understand any of it. If you work hard on this sentence (why do you postulate that? What is a focus class? Why would we ever consider similarity-awareness and what is that?) you’ll go a long way to clarifying your intro\n-\tTop of 3.2 – need to clarify what s is\n-\tEq 7 – this makes it look like Y_similar is a set of tuples but I think you mean set of elements of Y?\n-\t“would like the entropy peak shifted such that uniformity wrt similar classes gets dominant” – this is another idea which is not being clearly communicated right now and seems central to this work\n-\tThe “reweighting vector” is not introduced – not clear what we will do with it. You never actually show how this is included in the entropy calculation\n-\tWould like to see some sort of statistic on hubs, along with these pictures\n", "extra_scores": {}, "extra_text": {}}, {"review_id": "J94qMQPxRSW", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "########################################################################## Summary: The paper studies how to learn private representations that only captures the non-sensitive attributes of the dataset. They propose an adversarial representation learning method that employs VAEs. Specifically, the architecture in the VAE contains 6 players with 2 as adversarial classifiers. They introduce focal entropy as the objective function instead of entropy for adversarial classifiers to achieve deep sanitization. They empirically evaluate the method by reporting the target task accuracy and attribute inference accuracy on two datasets.\n\n########################################################################## Reasons for score: I like the paper largely. It provides a nice practical method for sanitization, although it is hard to give theoretical privacy guarantees of VAEs. The method has shown to be a good defense for individual attribute inference attacks. My main concern is the experimental results are only on two datasets with one task/sensitive attribute setting. As an empirical/methodology paper, I would expect more empirical results. \n\nAlso, I have a question regarding the sanitization section. It's not clear if focal entropy should be used in both $\\tilde{T}$ and $\\tilde{S}$. The paper mentioned \"training of $\\tilde{T}$ leverages a modification of entropy ...\" However, I think it should be $\\tilde{S}$ or both. It seems that Equation (6) should be negative KL divergence since we want to force the distribution to be close to the uniform distribution. \n\n########################################################################## Minor Comments:\n\n1. In Equation (6), the bracket should be after $\\theta$.\n\n2. I don't think $\\pi$ is clearly mentioned before Equation (9). The notation should be more precise.\n\n 3. I'm guessing it can also protect the dataset-level (proprietary) attribute inference. I'm interested in seeing the results, but it is not required for this submission.", "extra_scores": {}, "extra_text": {}}, {"review_id": "2fLzVfVuS7h", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "\n### Summary\nThis paper presented a method for learning private representations. This method is based on adversarial representation learning and the main technique contribution comes from focal entropy. The experimental results show that focal entropy can improve the accuracy of the target predictor without increasing adversarial accuracy.\n\n### Pros\n1. Focal entropy is effective in reducing the information leakage of the learned representation while improves the target accuracy over the state-of-the-art.\n2. This paper covers a wide range of experiments.\n\n### Cons\n1. Some technique details are not presented clearly. I have listed some of them here:\n   a. How to divide the representation $z$ encoded by the encoder into two sub-representations, namely, target and residual representations.\n   b. There is a high-level description of focal entropy instead of equations.\n2. Some notations are very confusing. For example, in the optimization goal (Equation (2)), the adversarial loss $\\phi_{tilde{S}}$ is related to the parameter $\\theta_{res}$. However, in the following description, this loss is related to $\\theta_{tar}$(Equation (6))\n\n### Comments\n1. The optimization objective involves many individual terms. The experiments should further provide results to show how the trade-off parameters $\\beta$ controls privacy leakage and target accuracy.\n2. This work is done in an adversarial training manner, can leakage reduction be achieved in a differentially private training manner, i.e., training the encoder using dp-sgd?", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper focuses on a notion of privacy in learning representations. 

One of the primary concerns of the reviewers was clarity of the writing and results. Numerous concerns are mentioned in the reviews, and also more engagement with the fairness literature was desired. One reviewer felt that some of the claims in the paper were unsubstantiated, for example: understanding the sanitization process in a human-understandable visual way", "integration of a notion of interpretability". It was felt that the changes required were more than could be expected for a camera ready version. The authors are recommended to revise the paper with a particular eye for clarity to a new reader.

The notion and measurement of privacy was also considered to be somewhat shaky. It is understood that the nature of privacy considered in this paper is different from differential privacy. That said, the latter is a rigorous definition, and the one in this paper seems to be rather empirical in nature. There are no formal guarantees in terms of privacy preservation, and it is not clear whether the representations could leak information when evaluated with a different network. As privacy is a mission-critical property, some justification of why the heuristic measurement of privacy is acceptable.

As a side note, the authors should consider using the \citep command for parenthetical citations in the text.
Decision: Reject