A Generalized and Distributable Generative Model for Private Representation Learning
Abstract
We study the problem of learning data representations that are private yet informative, i.e., providing information about intended “ally” targets while obfuscating sensitive “adversary” attributes.
We propose a novel framework, Exclusion-Inclusion
Generative Adversarial Network (EIGAN), that generalizes adversarial private representation learning (PRL) approaches to generate data encodings that account for multiple (possibly overlapping) ally and adversary targets.
Preserving privacy is even more difﬁcult when the data is collected across multiple distributed nodes, which for privacy reasons may not wish to share their data even for PRL training.
Thus, learning such data representations at each node in a distributed manner (i.e., without transmitting source data) is of particular importance.
This motivates us to develop D-EIGAN, the ﬁrst distributed PRL method, based on fractional parameter sharing that promotes differentially private parameter sharing and also accounts for communication resource limitations.
We theoretically analyze the behavior of adversaries under the optimal EIGAN and D-EIGAN encoders and consider the impact of dependencies among ally and adversary tasks on the encoder performance.
Our experiments on real-world and synthetic datasets demonstrate the advantages of EIGAN encodings in terms of accuracy, robustness, and scalability; in particular, we show that EIGAN outperforms the previous state-of-the-art by a signiﬁcant accuracy margin (47% improvement).
The experiments further reveal that D-EIGAN’s performance is consistent with EIGAN under different node data distributions and is resilient to communication constraints.
Introduction
Training machine learning (ML) models frequently requires sharing data among multiple parties, e.g., cloud services aggregating data from multiple users to learn a global model.
Such sharing naturally raises privacy concerns in terms of exposing sensitive attributes in datasets.
Even when data is anonymized at the source, de-anonymization attacks are possible when coupled with auxiliary datasets, as was famously shown in the Netﬂix challenge [1].
A widely used technique for obfuscating sensitive attributes in data is differential privacy (DP) [2], a context-agnostic technique.
DP introduces noise (e.g., Laplace) into a dataset to provide membership security.
However, noise injection can impact ML training/inference signiﬁcantly.
This makes such context-agnostic privacy techniques unsuitable in scenarios where only a few attributes need to be obfuscated.
For example, upon sharing patient vitals for preventive healthcare [3], both privacy (e.g., gender, ethnicity anonymization) and predictivity (e.g., accurate diagnosis) are important.
These drawbacks of context-agnostic privacy measures motivate private representation learning (PRL) [4], which exploits existing knowledge of sensitive attributes in a dataset.
PRL considers privacy and predictivity as joint (and possibly competing) objectives, and learns a transformation on the data that balances the goals of (i) obfuscating sensitive attributes of interest to an “adversary” (adv.)
while (ii) preserving predictivity on intended targets for an “ally”.
Conventionally, the literature on PRL assumes the existence of a single sensitive attribute and a central dataset [4, 5, 6, 7].
However, most real-world datasets have multiple sensitive attributes and are collected across multiple distributed nodes.
Healthcare records, for example, are (i) spread across hospitals in different regions, (ii) consist of potentially multiple sensitive attributes, such as mental health, gender, and ethnicity that prevent sharing of source data, and (iii) governed by regulations that vary from one region to another, e.g., while GDPR (in Europe) considers racial/ethnic origin as sensitive information, HIPAA (in USA) does not.
These challenges call for a generalized and distributed PRL methodology that takes into account multiple sensitive attributes, trains on data distributed across nodes, and learns representations that incorporate the privacy/predictivity goals of each node.
Communication-efﬁciency is also a key objective in distributed learning, particularly when it is being deployed in network settings where nodes are restricted to communicate over limited-bandwidth links [8], e.g., remote health analytics across user devices at the network edge.
In this paper, we propose a novel
PRL architecture called ExclusionInclusion Generative Adversarial
Network (EIGAN) , which addresses the aforementioned challenges.
EIGAN is a generalized gametheoretic PRL technique designed to generate encodings “inclusive” of signals that are of utility to a set of allies, while “exclusive” of signals that can be used by adversaries to recover sensitive attributes.
Further, to address the privacy vulnerabilities of pooling raw data, we develop
D-EIGAN (for Distributed-EIGAN), where multiple EIGAN nodes train encoders on their local datasets and synchronize their model parameters periodically, as depicted in Fig.
D-EIGAN implements distributed training without noticeable model degradation compared to the centralized EIGAN, while accounting for realistic factors of communication constraints and non-i.i.d data distributions across nodes.
Related work.
Recent works in PRL [4, 5, 6] have proposed centralized architectures that jointly maximize the loss in predicting sensitive attributes while minimizing the loss of target task prediction.
Speciﬁcally, [4] proposed a three-network encoder-ally-adversary architecture and showed that the achievable tradeoff between the two objectives is better than that provided by DP.
In [5], the problem was formulated as a non-zero-sum game between the three networks to minimize information leakage in encoded image representations.
[6] experimentally outperform [5, 9, 10] using a minimax optimization among three networks, and derive its closed-form solution when the networks are linear maps.
We demonstrate that EIGAN converges to the optimal performance obtained by these closed form solutions.
EIGAN also has computational advantages over [6] as it does not depend on matrix inversions.
Other PRL works take an information-theoretic approach.
[7] view PRL as minimization of the utility lost in the learned representation, subject to an upper bound on mutual information between the output representation and the sensitive attribute.
Similarly, [11] formulate the minimax problem in terms of
KL-divergence.
EIGAN, on the other hand, considers a log-loss PRL formulation, which promotes interpretability and training stability over multiple objectives (discussed in Sec. 2.1).
Furthermore, our experiments show that EIGAN signiﬁcantly outperforms the state-of-the-art [7] in the single ally/adversary case.
Distinct from all prior work in PRL, we consider multiple sensitive attributes and distributed learning.
It is worth mentioning two other directions in adversarial learning related to PRL.
One addresses privacy-preservation through synthetic data generation [12], which differs from EIGAN’s goal of learning a data transformation.
The other is fair representation learning [13, 14], which seeks representations that are intrinsically fair in the sense of promoting demographic parity on a single attribute [15].
Contributions.
Our contributions are summarized below:
We introduce EIGAN (Sec. 2.1), generalizing PRL to account for multiple potentially overlapping target and sensitive attributes.
We prove that EIGAN’s encoder utility is maximized if the adversary outputs follow a uniform distribution, and consider the effect of correlations between ally and adversary objectives (Prop. 1).
To the best of our knowledge, D-EIGAN (Sec. 2.2) is the ﬁrst technique for distributed training of PRL models.
We show that when the nodes engaged in the training possess i.i.d.
datasets, the objective of D-EIGAN exhibits similar properties to EIGAN (Prop. 3).
Our experiments (Sec. 3) reveal that EIGAN signiﬁcantly outperforms the state-of-the-art in PRL (Tab. 1, Fig. 4) and is robust to the choice of adversary architectures (Tab. 2).
We also demonstrate that D-EIGAN matches the performance of EIGAN even as the number of nodes increases (Fig. 7), and is robust even when nodes have different objectives (Fig.
We further show the resilience of D-EIGAN to non-i.i.d data distributions across nodes, and under communication restrictions that require partial parameter sharing and delayed model aggregations in the system (Fig. 9).
EIGAN Formulation and Model Learning
Overview.
Our PRL methodology consists of two phases: training and testing.
In the training phase,
EIGAN – knowing the sensitive/target labels of interest to adversary/ally on the train dataset – aims to learn the encoder by simulating allies and adversaries.
Each of the allies, adversaries, and encoder independently maximize their own utilities by updating their local model parameters.
The selﬁsh maximization by each player naturally leads to the minimax optimization in (2).
In the testing phase, the test data undergoes a transformation through the trained encoder.
The transformed data is used for conventional training and inference by the actual allies and adversaries on their respective tasks of interest.
In Sec. 2.1, we present the EIGAN formulation for centralized model training, and derive properties of the solution.
Then, we extend it to the distributed learning case, D-EIGAN, in Sec. 2.2.
Refer to
App.
A for the proofs of the propositions.
2.1 EIGAN: Centralized Model Architecture
We ﬁrst consider a system consisting of n allies, indexed by A 1 , ..., A n ; and m adversaries, indexed by V 1 , ..., V m .
Ally A
i is characterized by model parameters θ A i and a set of target attributes/labels
Y A i following distribution Y Ai .
A i aims to associate each input sample with its corresponding target attribute in Y A i .
Similarly, adversary V j parameterized by θ V j wishes to associate input samples with a set of (known) sensitive attributes/labels Y V
j following distribution Y V j .
The goal of EIGAN is to learn an encoder E parameterized by θ E that maximizes the performance of
A 1 , ..., A n while minimizing the performance of V 1 , ..., V m .
The encoder uses a centrally-located dataset X consisting of N samples, where each sample is represented as a d -dimensional feature vector x j ∈ R d , j = 1 , ..., N .
We let E ( x ; θ E ) denote the output of the encoder for a data sample x realized via the parameters θ E .
E ( x ; θ E ) : R d → R l is in general a non-linear differentiable function (e.g., a neural network), where l is the dimension of the representation output by the encoder, and typically l ≤ d .
For a sample x ∈ X , the encoded representation E ( x ; θ E ) is what the allies
A 1 , ..., A n and adversaries V 1 , ..., V m are provided with for their tasks, as depicted in Fig.
1(a).
We quantify the utility functions of the allies and adversaries as:
[Equation]
[Table] where p Ai ( Y | E ( X ; θ E )) and p Vj ( Y | E ( X ; θ E )) denote the probabilities of successful inference of target labels Y
∼ Y
A i and sensitive labels Y ∼ Y V j for ally
A i and adversary V j , respectively, over the outputs that the encoder E provides for the dataset X .
This leads to our minimax game among three types of players, in which two (the encoder and allies) are colluding against the third (the adversary).
Speciﬁcally we formulate the optimization problem:
[Equation]
[Equation]
where
[Equation]
Here, α Ai , α Vj > 0 denote normalized importance parameters placed on each objective such that [Equation]
= [Equation]
+ [Equation] mj
[-=
-]=1 α V j = 1 .
Similar to the encoder, we assume that the ally and adversary are non-linear, differentiable functions.
The encoder in (2) seeks to maximize the achievable utility of the allies while minimizing those of the adversaries, operating in conjunction with the allies in the inner max layer of (2).
The adversaries then operate on the encoder result in the outer min layer, where each adversary V j aims to maximize its utility u Vj by updating θ Vj , as it cannot access other ally/adversary’s parameters directly.
It is worth noting that, similar to the formulation based on mutual information in [7], our analysis on the expected posterior distribution of the predictions in EIGAN map directly to interpretable metrics such as accuracy [16] and generalization error [17], instead of the worst case guarantees provided by context-agnostic privacy frameworks such as DP.
Intuitively, the encoder will attempt to diminish the adversary predictions to a random guess, i.e., to a uniform distribution over its target labels.
However, this may be difﬁcult to achieve when the interests of the allies and adversaries are related, which makes the weights α A i , α V j important to the minimax solution in (2) formalized in the proposition below:
Proposition 1.
Let O denote the set of all ( i, j ) pairs of allies
A i and adversaries V j for which
Y A i ∩ Y V j (cid:54) = ∅ , i.e., overlapping interests.
Given a ﬁxed encoder E in EIGAN architecture, if
O = ∅ , the overall score in (2) is maximized when the adversaries’ output predictions follow a uniform distribution.
On the other hand, if O (cid:54) = ∅ , then for each overlapping label, the architecture proposed by (2) considers the utility of the attributes that have the higher importance weight, i.e., A i
if α A i > α V j and V j if α A i < α V j .
Prop. 1 shows that given an encoded representation, if the allies and adversaries possess nonoverlapping interests, then a uniform prediction distribution among the sensitive parameters of interest to the adversaries is adopted by the optimal solution.
This resembles the initial GAN result in [18] obtained for a two-network architecture.
In App.
D.3, we consider an experiment with such overlapping interests and equal importance weights, and ﬁnd that EIGAN is unable to balance the objectives.
In practice, coincidental overlaps between ally and adversary interests would be relatively rare, but could nonetheless occur.
In such cases, EIGAN must balance predictivity and privacy, which leads to different ally and adversary outputs described in Prop. 1.
We further analyze EIGAN’s characteristics when there is a linear relationship between the target distribution of an ally and an adversary (see
Prop. 2 in App.
A.2).
Model training.
We train the encoder and the allies/adversaries in EIGAN by alternately updating their parameters using stochastic gradient descent (SGD) to minimize their log-loss functions.
For the encoder, we deﬁne the log-loss L E for a single training instance as a weighted combination of the predictive capability of the allies and adversaries as
[Equation]
where (cid:104) .
(cid:105) denotes inner product, and log is applied element-wise.
y A i and y V j are the binary vector representations of the true class labels for ally A i and adversary V j , respectively, while ˆ y A
i and ˆ y V
i are the vectors of soft predictions (i.e., probabilities) for each class.
Here, we have made the simpliﬁcations α Ai = α/n ∀ i and α Vj =
(1 − α )
/m
∀ j , where α ∈ (0 , 1) is tuned to emphasize either predictivity (higher α ) or privacy (lower α ).
It can be seen that the minimization of loss L E is equivalent to the maximization of utility deﬁned by (3).
In each epoch, we average L E over a minibatch of size J to obtain an estimate of (1), and update θ E based on the gradient.
Then, we update the θ A i and θ V i according to (4).
For the full algorithm, see Alg.
A1 in App.
B.
Alternative objectives to (4) can be found in PRL literature.
In particular, recent works [19, 11, 7] formulate the adversarial loss using KL divergence.
Our choice of log-loss over KL-divergence is based on the facts that: (i) KL divergence fails to give meaningful value under disjoint distributions [20], and thus is less interpretable, and (ii) log-loss is more stable under varying priors during training, which is important in the EIGAN scenario since training data is affected by mini-batches, non-i.i.d distributions, and the evolving representations.
Our experimental results in Tab.
1 validate our choice of formulation in terms of obtaining improvements over the state-of-the-art that uses KL divergence (discussed further in Sec. 3.1).
Visual demonstration.
Fig.
2 is a visual demonstration of EIGAN’s trained representation on a synthetic dataset.
There are two allies A 1 and A 2 which are each interested in separating data points along one of the horizontal axes, and an adversary V that is interested in separation along the vertical axis.
We see in (b) that the EIGAN encoding collapses the data along the vertical axis while retaining separability in the other two dimensions.
Other illustrations are given in App.
C.
2.2 D-EIGAN: Distributed Adversarial Learning
The distributed setting for EIGAN (D-EIGAN) is depicted in Fig. 1(b).
There are K nodes in the system, denoted E (1) , ..., E ( K ) , and a parameter server for model synchronization.
Each node E ( k ) has a set of allies, denoted A ( k ) 1 , ..., A ( k ) n ( k ) with target label sets Y A ( k )
= { Y A ( k ) 1 , ..., Y A ( k ) n ( k ) } , a set of adversaries, denoted V ( k ) 1 , ..., V ( k ) m
( k ) with target sets Y V ( k ) =
{ Y V ( k ) 1 , ..., Y V ( k ) m ( k ) } , and a subset
X k ⊂ X of N k datapoints from the overall dataset X of N samples.
These local datasets are in general non-overlapping, and may differ in size.
While the speciﬁc allies and adversaries may differ at each node, the goal is to train encoder models that maximize all allies’ and minimizes all adversaries’ performances, so that the encodings are meaningful throughout the system.
Since sharing the raw datasets could potentially leak sensitive information, each node E ( k ) will train its own local encoder
E ( k ) ( x ; θ E ( k ) ) , and the server in Fig. 1(b) will periodically aggregate the locally-trained models.
The utility function for node E ( k ) is deﬁned as
[Equation]
where θ A ( k ) = (cid:8) θ
A ( k ) i (cid:9) n ( k ) i
=1 and θ V ( k ) =
(cid:8) θ V ( k ) j (cid:9) m ( k ) j =1 denote the sets of ally and adversary parameters at node E ( k ) , and u A ( k ) i , u V ( k ) j denote the utility functions of A ( k ) i ,
V ( k ) j deﬁned analo-
V j
14: Download initial server 15: for number do 16:
| chosen parameter server: θ
E q θ E
Q 17:
Sample
X
Update k
← θ E k
− η E · ∇ θ E
Update ally/ parameters: θ
A ( k ) i ← θ A ( k ) i − η A · ∇ θ A ( k ) i
L A ( k ) i , θ V ( k )
i ← θ V ( k ) i − η V · ∇ θ V ( k ) i
L V ( k ) i 20: φ · | θ E ( k ) | parameters: ˜ θ E [Table]
for gously to (1).
α A ( k ) i , α V ( k ) j > 0 denote the normalized importance parameters for node E ( k ) , where
[Equation]
= 1 .
This leads to the following minimax game for the distributed case:
[Equation]
where S V = { θ V ( k ) }
Kk =1 , S E = { θ E ( k ) }
Kk =1 , S A
=
{ θ A ( k ) } Kk =1 .
The constraint in (6) ensures that the optimal encoder is the same across all nodes, even though each node may have different allies and adversaries.
In this way, an encoded datapoint E ( k ) ( x ; θ E ) at node k could be transferred to another node k (cid:48) and applied to a task A ( k (cid:48) )
i privately, e.g., for anonymized user data sharing during single sign-ons.
Distributed model training.
While solving (6) in a distributed manner, we learn both a global model (encoder) and individual (personalized) local models (allies and adversaries) [21], unlike standard
Federated Learning (FL).
Our algorithm consists of two iterative steps.
The ﬁrst is local update : each E ( k ) conducts a series of δ SGD iterations.
For each minibatch in SGD, training proceeds as in the centralized case, with the encoder, allies’, and adversaries’ parameters updated via SGD to minimize the log-losses L ( k ) E ,
L ( k ) A i , and L ( k ) V j deﬁned as in (4) but in this case for each node.
The second step is global aggregation , in which each E ( k ) uploads its locally-trained encoder to the parameter server to construct a global version, after every δ SGD iterations.
We introduce a sparsiﬁcation technique here in which each node selects a fraction φ of its parameters at random to upload for each aggregation.
Letting Q k be the
[Figure]
indices chosen by E ( k ) , then the vector recovered at the server is ˜ θ
E ( k ) , where ˜ θ E ( k ) ( q ) =
θ E ( k ) ( q ) if q ∈ Q k and 0 otherwise.
With this, the global aggregation becomes the weighted average θ E = (cid:80)
k N k N ˜ θ E ( k ) .
Then, the server also selects a fraction φ of indices at random to synchronize each node k with on the downlink.
Letting Q be these indices, each node k sets θ E ( k ) ( q ) =
θ E ( q ) if q ∈ Q and makes no change to the q th parameter otherwise.
The pseudo-code of the training procedure is given in Alg.
The synchronization frequency δ and sparsiﬁcation factor φ are directly related to the amount of data transferred through the system: as δ increases, uplink transfers to the server occur less frequently; as φ decreases, each uplink/downlink transmission requires fewer communication resources.
This is an important consideration in networking applications where the nodes communicate over a resourceconstrained channel [22].
Fractional parameter sharing, similar to pruning (both choose a subset of parameters), mimics the additive-noise DP mechanism [23] on model weights, reducing associated leakage [24] to any untrusted entity with access to the system.
We study the effect of δ and φ on
D-EIGAN performance in Sec. 3.2.
In D-EIGAN, the allies and adversaries may differ at each node, and each node trains an individual local encoder.
Since the encoder parameters are globally synchronized, however, the local encoder implicitly trains using global union of allies/adversaries across nodes.
In the case that the nodes have same objectives and i.i.d.
datasets, we show that D-EIGAN yields the same properties as Prop. 1:
Proposition 3.
Given a set of ﬁxed encoders in the D-EIGAN architecture, if all the nodes have the same number of allies and adversaries with the same sets of target labels Y A ( k ) =
Y A ( k (cid:48) ) and
Y V ( k ) =
Y V ( k (cid:48) ) , 1 ≤ k, k (cid:48) ≤ K , then Prop. 1 holds for all the allies and adversaries belonging to different nodes if the local datasets at each node are i.i.d.
When the nodes have different objectives, we can show that the importance of each objective is proportional to the number of nodes implementing it; see Prop. 4 in App.
A.4.
Experimental Evaluation and Discussion
We now turn to an experimental evaluation of our methodology.
We analyze EIGAN’s convergence characteristics and compare its performance with relevant baselines in Sec. 3.1, and evaluate DEIGAN compared to the centralized case and as the system characteristics change in Sec. 3.2.
Datasets.
We consider four datasets: MNIST [25], MIMIC-III [26], Adult [27], and FaceScrub [28].
MNIST consists of 60,000 handwritten digits with labels 0-9.
MIMIC has medical information for hospital admissions with attributes, such as patient vitals and medication; we obtain a dataset consisting of 58,976 admission data points by joining multiple tables on patient IDs.
Adult consists of 45,223 records extracted from the 1994 census data.
Facescrub is a dataset comprising over 22, images of celebrities with identity and gender labels.
In MIMIC, we consider survival (2-class) as the ally objective, and gender (2-class) and race (3-class) as adversary objectives.
In the FaceScrub dataset, as in [7], the ally objective is user identity (200class), and the adversary objective is gender (2-class).
In MNIST, we consider whether a digit is even or odd (2-class) as the ally objective, and the label of the digit (10-class) as the adversary objective.
In Adult, as in [6], the ally objective is an annual income classiﬁcation (more or less than 50K) and the adversary objective is gender.
We also generate synthetic Gaussian datasets to analyze the effect of ally/adversary class overlap in some experiments.
Implementation.
We employ fully connected networks (FCNs) for the encoder, allies, and adversaries in the experiments on MIMIC and the synthetic datasets.
The FCN encoder uses ReLU activation for the hidden layers and tanh activation for the ﬁnal fully-connected layer, whereas the ally and adversaries use sigmoid activation in the ﬁnal layer.
We use dropout and L2-regularization to prevent network overﬁtting.
For FaceScrub, we employ U-Net [29] for the encoder and Xception-Net [30] for the ally/adversary, as in [7].
For Adult, we employ linear FCN as in [6].
Unless otherwise stated, we set α = 0 .
5 (i e., equal privacy/predictivity importance).
We train to minimize cross-entropy loss over 70/30 training/test splits on a system with 8GB GPU memory and 64 GB
RAM.
Baselines.
We consider six baselines: principal component analysis (PCA), autoencoders [31], differential privacy (DP), and the methods in [6], [7].
Autoencoders and PCA preserve information content and do not have explicit privacy objectives; they are expected to give encoded data that has good predictivity.
PCA chooses the number of components retaining 99% of the variance, and we train the autoencoder to transform data to the same dimensional space as PCA.
As discussed in
Sec. 1, DP is widely used for context-agnostic privacy.
For DP, we employ the Laplace mechanism [4].
[7] is the most recent state-of-the-art in adversarial PRL; in this case, we use their open-source implementation and compare on the setting described in their paper.
We also compare against the closed form optimal solution of [6] for linear maps on their Adult dataset use case.
All of our code and trained models are available at https://github.com/shams-sam/ PrivacyGANs .
For each experiments, we report log-loss and/or accuracy from the testing step of PRL as discussed in Sec. 2.
3.1 Centralized EIGAN
Performance comparison.
We ﬁrst compare the ally and adversary losses over training epochs between EIGAN, autoencoder, PCA, and DP on the MIMIC dataset in Fig.
Note that the recent baselines [6, 7] cannot handle multiple adversary objectives.
It is observed in (a) that EIGAN is able to match the adversary losses of DP, while in (b) the EIGAN ally loss matches that of PCA and autoencoder while outperforming DP by a signiﬁcant margin.
Thus, EIGAN is capable of achieving private representations while simultaneously maintaining the predictivity of the encoded representations.
Next, we compare EIGAN with [6] and [7] on the Adult and Facescrub dataset settings considered in these works, respectively.
Note that the linearity requirement in [6] impedes its usage on non-linear models like the U-Net and Xception-Net employed for Facescrub by [7].
For comparison, we adjust α in (4) to equalize the resulting adversary performances between the models.
Tab. 1 gives the results:
EIGAN matches the performance of [6]’s optimal closed-form solution on Adult.
On the Facescrub dataset, it displays a 47% improvement in the ally’s task of identity recognition when compared to [7].
This validates our choice of optimization using cross-entropy loss in (4) for PRL over the technique of optimization using KL divergence that is common in recent PRL literature [7, 19, 11].
Robustness of learned representation.
We next consider the robustness of EIGAN’s learned representation to ally and adversary architectures that deviate from the one used for training.
Tab.shows the performance of varying architectures (ResNet [32], ResNext [33], etc.)
for allies and adversaries applied to the data encoded using EIGAN trained with ResNet152 adversary on MNIST.
We see that the representations learned by EIGAN are able to obfuscate adversary targets from the other networks.
Adversary accuracy remains signiﬁcantly below the performance on the unencoded data, validating the robustness to differences between simulated and actual adversaries.
Varying ally/adversary overlap.
Next, we consider the effect of class overlap for the ally/adversary objectives on model performance.
To do this, we generate a 2D dataset consisting of four Gaussians with means at ( x, y ) =
(1 , 1) , (1 , 2) , (2 , 1) , (2 , 2) , each corresponding to one class.
The variance of these Gaussian-distributed classes is adjusted to achieve varying degrees of overlap.
Fig.
7(c) shows an instance of this dataset: the ally is interested in differentiating color, while the adversary wants to differentiate shape.
Fig.
4 shows the effect of the ally label variance on the resulting accuracies for
EIGAN, the method in [7], and the unencoded data.
As the ally variance increases, we observe that (a) the accuracy of the adversary for EIGAN remains consistently lower than that of the others, while (b) the accuracy on the ally objective for EIGAN remains higher than that of [7] and is comparable to the unencoded case.
Similar results are observed when changing the adversary variance (see App.
D.1).
Varying system dimensions.
We also considered the impact of the encoding dimension l and the number of allies/adversaries on EIGAN’s performance using MIMIC.
We summarize our key ﬁndings here: (i) We observe (in Fig. 5) that the ﬁnal testing loss obtained by an adversary (ally) under varying number of allies (adversaries) stays reasonably constant.
(ii)
We ﬁnd (in Fig. 6) that as encoding dimension l is increased, the training requires fewer epochs to converge, and is able to achieve a lower testing loss, even as l exceeds the input dimension.
Thus, EIGAN encodings are robust to the number of objectives that are included in the system.
3.2 Distributed EIGAN (D-EIGAN)
Varying number of nodes.
For the distributed case, we ﬁrst study the effect of increasing the number of training nodes K .
We use synthetic Gaussian data and generate non-i.i.d.
data distributions across the nodes by increasing the variance of the Gaussians at each subsequent node k (Fig. 7(c) shows the distribution for k = 3 ).
Fig.
7(a)&(b) show the resulting ally and adversary accuracies obtained when trained on D-EIGAN, on (centralized) EIGAN, and on the unencoded data.
As K increases, the ally performance degrades in each case, due to the higher variance for each class exhibited in the overall dataset X .
Overall, we see that D-EIGAN matches the performance of the centrally-trained
EIGAN in both metrics, which shows that distributed learning can yield a comparable solution when all parameters ( φ = 1 ) are synchronized frequently ( δ = 1 ).
See App.
E.1 for similar observations on i.i.d.
data across nodes.
Varying objectives across nodes.
Next, we study the effect of varying ally and adversary objectives across nodes.
For this, we consider the MIMIC dataset and allocate the dataset across K = 10 nodes randomly so that each has a different distribution of patient data.
In Fig.
8, we show the accuracies achieved by D-EIGAN on the one ally and two adversary objectives for two cases: (a) when each node has all three objectives, and (b) when each node has the ally objective, but half have one adversary objective and half have the other.
The EIGAN performance on the full dataset is included for comparison.
The dataset is non-i.i.d distributed, i.e., non-uniformly sampled across nodes.
We see that D-EIGAN in (a) only has a slight improvement over (b) in the case of the gender adversary, which indicates that D-EIGAN is robust to varying node objectives, even though the aggregation period has increased ( δ = 2 ) and the fraction of parameters shared has decreased ( φ = 0 . 8 ) from Fig.
The implication of this is that once a data sample is encoded at a node via D-EIGAN, it can be transferred to another node with different objectives and securely applied to ally tasks there, e.g., referring to the healthcare use case in Sec.
1, if a patient moves to a different hospital with different health regulations.
Similar conclusions are drawn when the data is i.i.d across nodes (see App. E.2).
Varying synchronization parameters.
Finally, we consider the impact of the aggregation period δ and the sparsiﬁcation factor φ on D-EIGAN.
This has implications for the communication resources between the nodes and the server required for training, as discussed in Sec. 2.2.
For this experiment, we use the setting from the experiment in Fig. 8(a), i.e., with non-i.i.d data and all nodes having all three objectives.
In Fig.
9, we show the performance of D-EIGAN as (a) δ increases and (b) φ increases (EIGAN shown for comparison).
In (a), we see that D-EIGAN is robust to the number of training epochs between aggregations, implying that it can be increased to limit the frequency of transmissions to and from the server.
In (b), we similarly observe generally robust performance as the fraction of sharing changes, though surprisingly, the performance noticeably decreases once φ reaches 1 and all are shared.
A similar effect was observed by [34], that in the case of distributed model training over non-i.i.d datasets, sparsiﬁcation actually can enhance performance because it minimizes the effect of data bias at each node on the global model.
Indeed, in the i.i.d case, we do not observe this effect (see App. E.3).
Thus, we conclude that D-EIGAN is well suited for communication-constrained environments.
Conclusion
We developed the ﬁrst methodology for generalized and distributable PRL.
EIGAN accounts for the presence of multiple allies and adversaries with potentially overlapping objectives, and D-EIGAN addresses privacy concerns and resource constraints in scenarios with decentralized data.
We proved that for an optimal encoding, the adversary’s output from EIGAN follows a uniform distribution, and that dependencies between ally and adversary interests requires careful balancing of objectives in encoder optimization.
Our experiments showed that EIGAN outperforms six baselines in jointly optimizing predictivity and privacy on different datasets and system settings.
They also showed that
D-EIGAN achieves comparable performance to EIGAN with different numbers of training nodes and as the training parameters vary to account for communication constraints. Acknowledgements
S.S. Azam, C. Brinton, and S. Bagchi were supported by the Northrop Grumman Cybersecurity
Research Consortium.
A Propositions and Proofs A.1 Proof of Proposition
Suppose ˆ Y A i = p
Ai ( Y | E ( X )) and ˆ Y V
j = p
Vj ( Y | E ( X )) , where p Ai ( Y | E ( X )) and
p Vj ( Y | E ( X )) denote the posterior probabilities of successful inference of target labels Y
∼ Y
A i and sensitive labels Y ∼ Y V j for ally
A i and adversary V j , respectively, given the outputs encoder E provides for the dataset X .
Then, the utilities in (1) can be expressed as
[Equation]
where 1 ≤ i ≤ n and 1 ≤ j ≤ m .
Let H Q
=
H
( P, Q ) denote the cross-entropy of Q with respect to
P deﬁned as H Q =
H ( P, Q ) =
E x ∼ P [ − log Q ] , then (7) can be re-stated as:
[Equation]
The maximization of ally utilities u
A i and minimization of adversary utilities
u V j ∀ i, j in the optimization objective (3) can be re-written as minimization of its negative given by,
[Equation]
Through (9), it can be observed that the minimization occurs when entropy of allies (cid:80) ni
=1 α A i H
A i is minimized while that of adversaries (cid:80) mj
=
α V j H V
j is maximized.
Using the deﬁnition of entropy, each of the allies and adversaries has a global optimum and can be optimized separately if their labels are non-overlapping.
Note that ally and adversary entropies are non-negative, and given a ﬁxed encoder E , the sum of ally entropies is minimized when individual entropies are minimized.
For each ally, individual entropy H A i is minimized when ˆ Y A i takes the value of 1 ∀ i as every ally label is then predicted correctly.
Similarly for adversaries, each individual entropy H V j is maximized when ˆ Y V j = 1 / | Y V
j | is the uniform distribution.
Thus, it can be seen that, at the optimal solution, the adversaries’ output follows a uniform distribution, as it minimizes the overall entropy in (9), or equivalently maximizes the utility in (3).
Given that ( A i , V j ) ∈ O is the set of all ( i, j ) pairs of allies
A i and adversaries V j for which
Y A i ∩ Y V j (cid:54)
= ∅ , the ally and adversary objectives in (9) are overlapping if O (cid:54) = ∅ .
Given that the encoder is ﬁxed, for allies/adversaries not included in O , the associated utilities can be independently optimized.
We are thus left with the maximization of the following:
[Equation]
For the k th element in O , ( A i ( k ) , V j ( k ) ) , we have Y A
i ( k ) ( c ) =
Y V j ( k ) ( c ) ∀ c ∈ C k ∀ k ; , where C k is the set of indices of elements in Y A
i ( k ) ∩ Y V j ( k ) (cid:54)
= ∅ .
Separating the indices c for which the ally/adversary try to predict the same label (i.e. u A i ( c )
=
u V j ( c ) ), we can express (10) as follows:
[Equation]
k (cid:32) (cid:88) c ∈C k ( α A i − α V j ) u
A i ( c ) (cid:124) (cid:123)(cid:122) (cid:125) utility w.r.t.
overlapping labels, U O + + (cid:88) c/ ∈C k α A i
u A i ( c ) − α V j u V j ( c ) (cid:124) (cid:123)(cid:122) (cid:125) utility w.r.t.
non-overlapping labels ,U O− (cid:33) .
The utilities in (11) reward only one of the two discriminators ( A i ,
V j ) ∈
O predicting on overlapping label c ∈ C if α A i (cid:54) =
α V j .
If α A i = α V j for ( A i , V j ) ∈
O , then U O + = 0 , and no optimization occurs w.r.t.
the overlapping labels in Y ∼ Y
A i .
A.2 Proposition
Proposition 2.
Assume that the number of labels of interest is the same among all the allies and adversaries.
For any adversary V j , the distribution of its prediction over its set of labels of interest does not follow a uniform distribution if sufﬁcient weight is given to the ally utilities (i.e., α A i , ∀ A i , is sufﬁciently large) and the distribution of prediction of one ally A i , can be deﬁned as a linear combination of the distribution of predictions of V j and that of other allies/adversaries.
Proof.
Without loss of generality, consider a system with one ally network with a scalar output ˆ Y A and m adversary networks with scalar outputs ˆ Y V j for 1 ≤ j ≤ m .
The true distribution of each predicted output is Y A for the ally and Y V j for the adversaries, and Y A and Y V j are the actual labels drawn from those distributions respectively.
The true values and predictions between that of the ally and the adversaries have the relation, Y A
= (cid:80) mj
=1 w j Y V j , and ˆ Y A
=
(cid:80) mj
=1 w j ˆ Y V j where w j is scaling weight.
The cross entropy of the entire system is given by U
= α A Y A log( ˆ Y A ) − (cid:80)
[Equation] j Y V j log( ˆ Y V j ) .
Optimizing for the output of a speciﬁc adversary V n , we obtain:
[Equation]
Notably, ˆ Y V n only returns a non-uniform distribution when α V n Y V n < α A Y A .
If the weight α A is not large enough to maintain the inequality, the value of ˆ Y V n cannot be obtained via (12) and will have a uniform distribution.
If α V n Y V n = α A Y A , then the cross entropy U = 0 and no optimization occurs.
A.3 Proof of Proposition
Given that the global encoder is the average of the local encoders in the federated learning procedure for a single synchronization across K local nodes, the maximization of the expectation in (6) can be described as the maximization of ally utilities and minimization of adversary utilities given by:
[Equation]
j = α V ( k ) j u V ( k ) j   .
In (13), A ( k ) i and V ( k ) i refer to the i th ally or adversary of the k th local node.
Since data at each node is i.i.d, the distributions Y are the same at each node, and thus each node has the same objective function.
Using the result of Prop. 1 and assuming that A ( k 1 ) i , V ( k 1 )
j = A ( k 2 ) i , V ( k 2 )
j ∀ i, j, k 1 , k 2 (i.e., the ally and adversary labels are same across all nodes), the output of the adversaries at each node follow a uniform distribution.
The ally and adversary objectives in (13) are overlapping if O (cid:54) =
∅ given that ( A i , V j ) ∈ O is the set of all A i , V j pairs for which Y A
i =
Y V j .
Since each of the local nodes have the same overlapping ally/adversary labels with potentially different weights α A ( k ) i and α V ( k ) j , their utilities can be expressed using entropy as in (8).
The ﬁnal optimization of the distributed system can be expressed as the minimization of following:
[Equation]
The entropy values given in (14) reward only one of the two discriminators predicting label Y A i if (cid:80)
[Equation]
= [Equation] k ) i , these two networks have no contribution to U O , and no optimization occurs..
A.4 Proposition
Proposition 4.
If the allies and adversaries located at the K nodes of D-EIGAN have nonoverlapping target sets, i.e., Y A ( k ) (cid:54) =
Y A ( k (cid:48) ) and Y V ( k ) (cid:54) =
Y V ( k (cid:48) ) , 1 ≤ k, k (cid:48) ≤ K , then individual encoders under D-EIGAN consider the union of these local allies, (cid:83)
Kk =1 Y
A ( k ) , and adversaries , (cid:83)
Kk =1 Y V ( k ) for optimization as a result of the global aggregation step.
The weights α A ( k ) i and α V ( k ) i associated with the allies/adversaries are scaled by the ratio of the number of nodes that implement them locally to the total number of nodes.
Proof.
Without loss of generality, consider a two network D-EIGAN.
Let node 1 have 2 allies and 1 adversary with objectives:
Y A c , Y A 1 , and Y V 1 , and node 2 have 2 allies and 1 adversary with objectives:
Y A c , Y A 2 and Y V 2 .
Here, objective Y A c is common among them, while the rest are different.
Utilities of individual nodes can be calculated using (3):
[Equation]
Under federated training, the equivalent loss function that is optimized by the D-EIGAN can be calculated using (6):
[Equation]
which shows that the overall objective under D-EIGAN considers all the objectives, but the associated weights are lower for non-common allies/adversaries.
In contrast to a D-EIGAN where all allies and adversaries are common across nodes, the difference is the weights associated with objectives.
B Pseudocode of EIGAN
− 1 | J | (cid:80) j ∈ J ( L V i )
j 14:
Update local allies/adversaries parameters: θ A i ← θ A i − η A · ∇ θ Ai L A
i , θ V i ← θ V
i − η V · ∇ θ Vi L V i 15: end for
C Additional Visualizations
D Additional EIGAN Experiments D.1 Comparison with the Method in [7]
D.2 Comparison on MIMIC-III
D.3 Comparison on MNIST
E Additional
Experiments E.1 Varying Number of Nodes
E.2 Varying Objectives across Nodes
E.3 Varying Synchronization Parameters