C ENSORING R EPRESENTATIONS WITH M ULTIPLEA DVERSARIES OVER R ANDOM S UBSPACES
Anonymous authors
Paper under double-blind review
A BSTRACT
In practice, often there are explicit constraints on the representations that are acceptable in a machine learning application in the real world; for example, representations of data must not contain identifying information so as to avoid privacy issues.
This paper improves the performance of the recently proposed adversarial feature learning approach (AFL) for incorporating such explicit constrains, by introducing the concept of the vulnerableness of the adversary.
In AFL, the censoring representation is done by training the networks to deceive the adversary that try to predict the sensitive information from the network, and the success of the AFL relies on the choice of the adversary.
This motivate the use of high capacity networks as an adversary for improving the performance; however, this approach does not work well in practice as reported in this paper.
Instead of the capacity of networks, this paper proposes to consider the vulnerableness in design of the adversary, i.e., the adversary should be designed not to be easily fooled.
We also propose a method multiple adversaries over random subspaces (MARS) that instantiate the concept, and provides empirical validations on the efﬁcacy of the proposed method compared to the various baselines, indicating the importance of the proposed concept.
This is signiﬁcant because it gives new implications about designing the adversary, which is important to improve the performance of AFL framework.
I NTRODUCTION
Since its invention over 10 years ago (Hinton et al., 2006), deep neural networks (DNN) have shown signiﬁcant performance improvements in various ﬁelds.
When we apply DNN or more generally machine learning techniques to real-world data, one of the key challenges is how to systematically incorporate the desired constraints into the learned representations in a controllable manner.
For example, when these techniques are applied to the data that contain a lot of user information (such as images with user name (Edwards & Storkey, 2016) or data of wearables (Iwasawa et al., 2017)), the desired representations should not contain user information that may result in privacy issues.
Moreover, for legal and ethical reasons, machine learning algorithms have to make fair decisions, which do not rely on sensitive variables such as gender, age, or race (Louizos et al., 2016; Edwards & Storkey, 2016).
This requires removal of information related to speciﬁc factors (such as user ID, race etc.)
from the representation; this is called censoring representations in this paper.
One of the recently proposed approaches for censoring representation is adversarial feature learning (AFL) (Edwards & Storkey, 2016; Iwasawa et al., 2017; Xie et al., 2017), which employs the adversarial training framework to constrain the representations (Figure 1-a).
Speciﬁcally, AFL considers an adversarial classiﬁer who attempts to predict sensitive variables from the representations of a
DNN, and simultaneously trains the DNN to deceive the classiﬁer.
By alternatively or jointly (using gradient reversal layer proposed by Ganin & Lempitsky (2015)) training the adversary and DNN in such a manner, AFL ensures that there is little or no information about the sensitive variables in the representations.
Although some previous studies report signiﬁcant performance improvements of the AFL in the context of censoring representations, the success of the AFL depends on the choice of the adversarial classiﬁer.
In other words, AFL can only eliminate the information that the adversary can correctly predict.
For example, if we use a logistic regression as the adversarial classiﬁer, AFL can only
S
~
S L ~
S 1 ~
X … …
E
R …
M Encoder
Classifier
D
Discriminator
L (Y, R)
V (S, R) (a) AFL with an adversary
S
~
S L ~
S 1 ~
X … …
E
R…
M
MARS
L (Y, R)
V (S, R 1 ) D
V (S, R 2 ) D
R…
[Figure]
eliminate the information that is linearly separated in the representation spaces and cannot eliminate any non-linear dependency.
This motivates the use of high-capacity networks as an adversary, such as DNNs for removing complex information more correctly; however, this approach does not work well in practice as reported in ( Iwasawa et al., 2017) and this paper.
This implies that only the capacity of the adversary is not a determinant factor of the success of adversarial feature learning; it is still unclear what is required for improving the quality of AFL.
In this paper, we propose to consider the vulnerableness in the design of the adversary, and verify the importance of vulnerableness by proposing multiple-adversaries over random subspace (MARS)
that instantiate the concept.
The term vulnerableness indicates how the adversarial classiﬁer is easy to be fooled when updating the target networks (encoder in Figure 1-a) to deceive the adversary.
We assume that the AFL will fail if an adversary is too easy to be fooled, even if it have enough capacity for predicting the sensitive variables.
The proposed method, MARS, is designed to be less vulnerable, by incorporating multiple adversaries where each adversary tries to predict sensitive variables from a different subset of the representations.
This makes adversary have diverse view and therefore less vulnerable to the update of the encoder as the encoder need to deceive a set of diverse adversaries.
In this paper, we validates the importance of vulnerableness for improving the performance of AFL, by empirically showing that (1) MARS archives better performance compared to baselines (that uses a single adversary and multiple adversaries over the entire representation spaces), and (2) MARS improve the vulnerableness compared to the baseline.
The primary contributions of this paper are as follows:
(cid:15)
This is the ﬁrst study that introduces the concept of vulnerableness in AFL, and empirically validates the importance of the concept by showing the superior performance of the proposed method that instantiate the concept of vulnerableness.
This is signiﬁcant because it gives implications about designing the adversary in AFL, which is important in practice.
(cid:15)
The proposed method achieved signiﬁcant performance gain in the task of censoring representations.
The empirical validation using three user-anonymization tasks shows that the proposed method allows the learning of signiﬁcantly more anonymized representations with negligible performance degradation.
Speciﬁcally, the probability of correctly predicting the user ID from learned representations is 0 : 124 and 0 : 112 points better on average than that of a single adversary and multiple adversaries over entire representation spaces, respectively.
P ROBLEM D EFINITION AND R ELATED W ORKS 2. P ROBLEM D EFINITION : C ENSORING R EPRESENTATIONS
Censoring representation means obtaining unbiased features.
Here, unbiased features are features that are less affected by S , where S is a random variable that we want to remove from the data for some reason.
One typical reason is related to fairness or privacy, which requires the output of neural networks not to be affected by unfair information or not contain user information.
It should be noted that poor design of the censoring procedure signiﬁcantly reduces the utility of data.
For example, the output of random mapping f rand apparently has no information about S , but it also gives no information about target Y .
Alternatively, as a more realistic example, a neural network with limited capacity possibly acquires less information about S , but it may also result in poorer performance.
Therefore, the primary goal of censoring representation is to obtain an encoder
E that reduces information about S , while maintaining information about Y .
Formally, the task can be written as a joint optimization problem of the loss:
[Equation]
where X indicates the input random variable, E is an encoder that transforms X to representation R , (cid:21) is the weighting parameter, and V and L are loss functions that represent how much information about S and Y is present, respectively.
Note that S can be any form of variable such as binary variable, categorical variable, or continuous variable.
In this paper, we especially consider a special variant of censoring representation tasks, where we learn E with deep neural networks and S is the user ID (anonymization tasks).
2. R ELATED W ORKS : A DVERSARIAL F EATURE L EARNING
A recently proposed approach for the censoring representation task is adversarial feature learning (AFL).
In AFL, V ( E ( X ) ; S ) is measured by an external neural network D , i.e., if the external networks can accurately predict S from R , AFL regards that R has too much information about S and if it is difﬁcult to accurately predict S , AFL regards that R has little or no information about S .
The external network D is called discriminator or adversary in this context.
The information is used to update the weights of the encoder E so that the updated representations have less information about S .
Formally, AFL solves the joint optimization problems:
[Equation]
where M : R !
Y is the label classiﬁer, D : R !
S is the adversarial classiﬁer (see Figure 1), and q M and q D are conditional distributions parameterized by M and D , respectively.
The optimization procedure is called adversarial training because E and D are in an adversarial relationship ( E tries to minimize of log-likelihood q D and D tries to maximize it).
To the best of the authors knowledge, the adversarial training framework was ﬁrst introduced by (Schmidhuber, 1992), and later re-invented by (Goodfellow et al., 2014) in the context of an imagegeneration task.
In the context of censoring representations, Edwards & Storkey (2016) ﬁrst proposed the use of adversarial training to remove sensitive information from the representations.
They show its efﬁcacy for fair classiﬁcation tasks ( S is binary) compared to an LFR (Learned Fair Representation) proposed by Zemel et al.
(2013) that regularizes the l 1 distance between the distributions for data with different S .
Iwasawa et al.
ﬁrst applied the adversarial training to learn anonymized representations on the data of wearables, where S is categorical.
More recently, Xie et al.
(2017) ﬁrst introduced the notion of adversarial feature learning and showed superior performances compared to variational fair auto-encoder (Louizos et al., 2016).
Although AFL already shows state-of-the-art performance to learn unbiased representations, how to improve its performance is still a challenge, which is tackled in this paper.
This work is motivated by previous studies conducted by Iwasawa et al.
They reported that simply using highcapacity networks as an adversary (such as deep neural networks) sometimes fails to improve the performance; on the other hand, the leaned representations are still highly biased if the adversary does not have enough capacity.
Our work can be regarded as introduction of the novel design consideration (i.e., vulnerableness) of adversary for improving the quality of AFL and validation of it by proposing a method that instantiate the concept, which is not done in the previous works.
It is worth mentioning that the concept of multiple adversaries itself is already proposed and veriﬁed in the context of the image generation with adversarial training (Durugkar et al., 2017), and this paper does not insist that using multiple adversaries is the primal novelty.
From the methodological perspective, the proposed method (MARS) can be seen as an extension of the concept of multiple adversaries by introducing the concept of diversity, and we verify that this extension is important in the context of censoring representations.
Though the primary focus of this paper is limited to the context of adversarial feature learning, we further discuss the applicability of the proposed method on the other application of adversarial training, such as image/text generation (Goodfellow et al., 2014; Zhang et al., 2017) or domain adaptation(Ganin & Lempitsky, 2015); this is done at the end of this paper.
M ULTIPLE -A DVERSARIES OVER R ANDOM S UBSPACES
The proposed method, multiple-adversaries over random subspaces (MARS) , considers multiple adversaries where each adversary is in charge of different subsets of features.
The development of
MARS is motivated by the assumption that the adversary should not be vulnerable and it could be achieved by enforcing the adversary to have diverse view.
Figure 1 shows the comparison of overall architectures of the proposed method between AFL with an adversary, and with multiple adversaries over random subspaces (i.e., proposed method).
Since the primary difference between naive AFL and MARS is how to build the adversary (training adversaries) and how to use the information of adversary (training encoders), the explanation is focused on the part.
3. T RAINING A DVERSARIES
There are many possible options to select a subspace; however, this paper speciﬁcally considers the case where each subspace is randomly selected.
Suppose that n is the number of dimensions of representation R 2 R n , and K is the number of adversaries.
In MARS, each adversary D k is trained to predict S over a randomly selected subset of features R k , whose dimension is m k and m k < n .
Each adversary is trained to maximize the expected log-likelihood.
Formally, the optimization problem for D k is as follows
[Equation]
(1) where Sub k is a function that return the subset of R , which is ﬁxed before the training.
Speciﬁcally, each Sub k determine whether to remove the i -th dimension of R with the probability of (cid:11) .
The (cid:18) D k is the parameter of D k .
3. T RAINING E NCODER
The adversaries are then integrated and a single prediction is made to train the encoder.
Formally, the probability distribution of q D ( s = s i
j h
= E ( x )) is parameterized by an ensembled adversary
D :
[Equation]
which means, the predictions of adversaries D k are integrated by averaging the predictions.
Note that integration operator F is not essential to be the averaging, it may be the max operator or the soften version of the max operator proposed in ( Durugkar et al., 2017).
The encoder is trained for minimizing the log-likelihood of Eq.2 and negative log-likelihood q M ( y j h ) .
Formally,
[Equation]
where (cid:18) E and (cid:18) M are the parameter of E and M respectively.
The overall algorithm is shown in
Algorithm 1.
Algorithm
Require: parameter (cid:21) , (cid:11)
Initialize Sub k for all k 2 1 (cid:1) (cid:1) (cid:1) K based on (cid:11)
Initialize neural networks f E; M; D 1 ; (cid:1) (cid:1) (cid:1) D K g while training() do
Update weights of D k for all k 2 1 (cid:1) (cid:1) (cid:1) K (eq. 1)
Update weights of E and M (eq. 3) end while
E XPERIMENTS 4. D ATASETS
We used two datasets to demonstrate the efﬁcacy of MARS.
Both datasets are related to human activity recognition using the data of wearables, where privacy issues have recently been pointed out by (Iwasawa et al., 2017).
In both datasets, the task is to learn anonymized representations ( R that does not contain user-identiﬁable information), while maintaining classiﬁcation performance.
The opportunity recognition dataset (Sagha et al., 2011)
consists of sensory data regarding human activity in a breakfast scenario, and the task is to predict the activity performed by a user.
A wide variety of body-worn, object-based, and ambient sensors are used (see Figure 1 in Sagha et al.
(2011) for more details).
Each record consists of 113 real-value sensory readings, excluding time information.
We considered two recognition tasks: gesture recognition (Opp-G) and locomotion recognition (Opp-L).
Opp-G requires the recognition of 18 class activities 1 , while Opp-L requires the recognition of 4 class locomotion-activities, i.e., stand, walk, sit, and lie.
Given a sampling frequency of 30 Hz, the sliding window procedure with 30 frames and a 50% overlap produced 57,790 samples.
We used data of subjects 2–4 as training/validation sets (90% for training and 10% for validation) and that of subject 1 as test sets.
The USC-HAD dataset (Zhang & Sawchuk, 2012) is a relatively new benchmark dataset, which contains a relatively large number of subjects (14 subjects: 7 males and 7 females).
There areactivity classes corresponding to the most basic and common activities of people’s daily lives 2 .
MotionNode, which is a 6-DOF inertial measurement unit specially designed for human motion sensing applications, is used to record the outputs from accelerometers that record 6 real sensory values.
The sliding window procedure, using 30 frames and a 50% overlap, produced 172, samples.
We used data of subjects 1–10 as training/validation sets (90% for training and 10% for validation) and that of subjects 11–14 as test sets.
4. E XPERIMENTAL S ETTING
In all experiments, we parameterized the encoder E by convolutional neural networks (CNN) with three convolution-ReLU-pooling repeats followed by one fully connected layer and M by a logistic regression, in accordance with a previous study (see Figure 2 in Iwasawa et al.
(2017) for more details).
Every component of the network was trained with the Adam algorithm (learning rate is set to 0.0001) ( Kingma & Ba, 2015) (150 epochs).
In the optimization, we used the annealing heuristics to weighting parameter (cid:21) in accordance with (Iwasawa et al., 2017); we linearly increased (cid:21) during 15 epochs to 135 epochs of training.
To demonstrate the efﬁcacy of the proposed method, we compare the following four methods and its variants:
(1) None: w/o adversary (correspond to normal CNN), (2) Adv: w/ a single adversary, (3) MA:
w/ multiple adversaries where each adversary tries to predict from the entire space, and (4)
MARS: w/ multiple adversaries where each adversary tries to predict from a different subspace of the entire space.
Each adversary is parametrized by multi-layer perceptron (MLP) with 800 hidden units.
If we need to express the type of adversary, we denote it by a sufﬁx.
For example, Adv-LR means an adversary is parametrized by a logistic regression and MA-LR means that each adversary is parametrized by a logistic regression.
The hyper-parameter (cid:21) is set to 1.0, 1.0, 0.1 for Opp-G, OppL, and USC-HAD respectively.
Without mentioning otherwise, we set the number of adversaries K as 10 for MA and MARS, and (cid:11) for determine size of the subset as 0 : 8 .
Following previous works, we evaluated the level of anonymization by training a classiﬁer f eva that tries to predict S over learned representations.
This means that we regarded that a learned representation is anonymized well if we could not build an accurate classiﬁer over the representations.
Unless mentioned otherwise, we use a multi-layer perceptron with 800 hidden units as f eva .
We later discuss how the choice of classiﬁer for f eva and D affects the results by performing empirical evaluations.
[Figure]
4. E XPERIMENTAL R ESULTS
The results for the three tasks are shown in Figure 2.
The ﬁrst row shows the performance when predicting S (lower is better), and the second row shows the performance when predicting y (higher is better).
Each column corresponds to different tasks.
The results show that the proposed method,
MARS, has the best performance regarding the anonymization level while maintaining the classiﬁcation performance.
Speciﬁcally, in terms of the average of all three tasks, MARS’s accuracy on S is 0.743 compared to 0.917, 0.908, 0.867 and 0.855 achieved by None, Adv-LR, Adv and MA respectively.
Although the classiﬁcation performance drops slightly for USC-HAD compared to None (from 0.537 to 0.496), it is still better than MA (0.478) that is the 2nd best baseline in accuracy on
S ; there is no performance degradation for the other two tasks.
Table 1 lists the user classiﬁcation accuracy when different adversarial classiﬁers D and evaluator f eva are used.
In addition to the representations shown in Figure 2, we compared a logistic repression (Adv-LR), DNNs with 400-200 hidden units, ensemble of DNNs (MA-DNN), and ensemble of
DNNs over random subspaces (MARS-DNN) for E .
Adv 2 and Adv 5 correspond to the case where we train an adversary with MLP 2 iterations or 5 iterations against one iteration for training encoder
E .
Adv 0 : 1 and MA 0 : 1 correspond to the results of smaller weighting parameter (cid:21) (1/10 of the default).
For evaluator f eva , we tested LR, multi-layer perceptron with 50 hidden units (MLP 1 ), and multi-layer perceptron with 800 hidden units (MLP 2 ).
The best performance in each condition (each combination of datasets and f eva ) is highlighted in underline, and the best performance without signiﬁcantly harming Y -accuracy (speciﬁcally, 5% or less degradation against None) is highlighted in bold.
The asterisk denotes that the method droped the performance on Y more than 5%.
We can make the following observations.
(1) In general, the representations learned with adversary/adversaries tend to more anonymized; however, a stronger evaluator still accurately predicts
S .
For almost all conditions, the proposed methods (MARS or MARS-DNN) show the best performance.
Note that although MV-DNN sometimes show better performances than MARS, it signiﬁcantly reduces classiﬁcation accuracy for Y .
Especially, in the case when the evaluator is
DNN, only the proposed method can give meaningful performance gain without any performance degradation.
(3) Just increasing the iterations of training adversary doesn’t pay off, as shown in the poor or no performance improvements of Adv 2 or
Adv 5 against Adv
In addition to the quantitative evaluation, we also give qualitative evaluation by visualizing the learned representations using t-SNE (Maaten & Hinton, 2008) (Figure 3).
Speciﬁcally, we randomly sample 1,000 examples from the validation sets, and reduce the dimension of data by using principal component analysis over the output of the E to speed up the computations and suppress some noise.
The number of components is selected such that the amount of variance that needs to be explained is greater than 95% .
The selected number of components is mentioned on top of each
[Table]
[Figure]
#Component=
[Figure]
ﬁgure.
Each color (and marker) denotes different users ( j S j =3).
The results show that the representations learned with multiple adversaries (MA and MARS) provide qualitatively more plausive results; we can see that the data for different users lie in almost similar places in the representations of MA or MARS; however, the representations of None, Adv-LR, and Adv are partially clustered based on the users.
Both MA and MARS give representations that are less inﬂuenced by the users; however, the representations of MARS seem to lie in more low-dimensional manifold spaces visually and require signiﬁcantly small number of components to represent the learned information, which possibly suggests that MARS eliminates unimportant information to a greater extent.
Figures 4-a and b compares the performance between different K and (cid:11) .
Note that (cid:11) = 0 : 0 means the method using all feature spaces, which is actually MA.
The results shows that accuracy on predicting S is more sensitive to K than that on predicting Y , as shown in Figure 4-a and b. Specifically, we can see the parameter sensitivity curve is almost similar shape (reverse bell curve) among (cid:11) , though the peak is different depending on (cid:11) .
Figures 4-c and d shows the performance of D and the log-likelihood of each discriminator q D k during training.
In Figure 4-d, each line represent the loss of D k before the update of encoder, and dotted lines represent the change of the loss by the update.
The shaded area represents the minimum and maximum loss, which means that larger shaded area indicates the higher variety of the D k .
The results demonstrate that: (1) Higher (cid:11) tends to gives the higher accuracy of D , (2) The inﬂuences of
2 4 6 8 10 12 14 #Adversary 0.
A cc u r a c y o n S 0.00.
(a) Accuracy on s 2 4 6 8 10 12 14 #Adversary 0.
A cc u r a c y o n Y 0.00.
(b) Accuracy on y 0 20 40 60 80 100 120 140 #Epch 0.
A cc u r a c y o f D 0.00.
(c) Accuracy of D 0 20 40 60 80 100 120 140 #Epch 0.
− l o g q D k ( s | h k ) 0.
(d) (cid:0) log q D k
the update to each adversary is more varied when (cid:11)
= 0.8 compared to (cid:11) = 0.0, which implies the adversaries of higher (cid:11) have more diversity and it make the D less vulnerable against the update.
D ISCUSSION AND C ONCLUSION
This study proposed MARS, which incorporates multiple adversaries where each adversary has a different role, and conducted empirical validations on the efﬁcacy of the proposed method for obtaining censoring representations, speciﬁcally user-anonymization for the data of wearables (Iwasawa et al., 2017).
Figure 2 shows that MARS gives anonymized features compared to Adv (which relies on a single adversary) and MA (which relies on multiple adversaries over the entire spaces), while maintaining the utility of data.
Table 1 compares more variants of each method, and shows the efﬁcacy of the proposed method against many evaluators.
In addition to the quantitative evaluations,
Figure 3 qualitatively shows that the proposed method provides well anonymized representations.
Figure 4-d shows that the each adversary in MARS have diverse role, resulting MARS more robust to the update of E as a whole.
All these results show that the proposed method is more effective in removing the inﬂuence of a speciﬁc factor (user in experiments) compared to the previous methods; this proves the underlying assumption of this study, i.e., the adversary needs to be less vulnerable, and this can be achieved by incorporating random subspace methods.
One of the reason why MARS works well is that the adversary is designed to have diverse-views by incorporating random subspace methods, resulting the encoder need to be stronger to deceive the adversary.
It is worth mentioning that the capacity or accuracy of the adversary is not the only a deﬁnitive factor that determine the success of the adversarial feature learning, as shown by the superior performance of MARS over MA that have 1 1 (cid:0) (cid:11) times larger capacity of MARS.
As mentioned in the related work section, such knowledge is important to design the adversary in practice, and is not mentioned in prior studies of adversarial feature learning.
Although this paper focused on the case where the subsets are randomly selected and ﬁxed, this might not be necessary.
One of the possible extensions is to determine subsets with more sophisticated ways (e.g., by performing clustering or soft-clustering on the representation spaces after few training iterations), or to learn how to select the subset itself by adding the criterion regarding diversity of adversaries.
Also, it might be possible to realize the diversity of adversaries by methods other than subspace selection.
One possible way is to constrain weights of two adversaries so that they are orthogonal view, which is used in semi-supervised learning using co-training (Saito et al., 2017), or it might be worth a try to adding different noises for each adversary.
It might be worth mentioning about applicability and implications of MARS for other applications of adversarial training, such as image generation.
From the perspective of the applicability, the
MARS itself does not rely on any domain-speciﬁc settings and is therefore general enough for many applications based on adversarial training.
For example, we can build multiple-adversaries upon the subset of feature spaces (maybe not on the image spaces).
This makes discriminator have diverseview, so it might be useful for preventing mode collapse that is one of the well-known problems in image-generation with adversarial training.
In the context of image-generation, Generative Multi
Adversarial Networks proposed by Durugkar et al. (2017), which also use multiple adversaries, shows that multiple adversary is useful for generating better images, and for avoiding mode collapse.
It might be interesting to see if enhancing the diversity of discriminators by preparing asymmetric adversaries as with this paper helps to generate better image or to avoid mode collapse better.