CaPC Learning: Confidential and Private Collaborative Learning
Authors: Christopher A. Choquette-Choo
Natalie Dullerud
Adam Dziedzic
Yunxiang Zhang
Somesh Jha
Nicolas Papernot
Xiao Wang
Venue: ICLR 2021 Poster (2021)
Keywords: machine learning
deep learning
privacy
confidentiality
security
homomorphic encryption
mpc
differential privacy

Abstract: Machine learning benefits from large training datasets, which may not always be possible to collect by any single entity, especially when using privacy-sensitive data. In many contexts, such as healthcare and finance, separate parties may wish to collaborate and learn from each other's data but are prevented from doing so due to privacy regulations. Some regulations prevent explicit sharing of data between parties by joining datasets in a central location (confidentiality). Others also limit implicit sharing of data, e.g., through model predictions (privacy). There is currently no method that enables machine learning in such a setting, where both confidentiality and privacy need to be preserved, to prevent both explicit and implicit sharing of data. Federated learning only provides confidentiality, not privacy, since gradients shared still contain private information. Differentially private learning assumes unreasonably large datasets. Furthermore, both of these learning paradigms produce a central model whose architecture was previously agreed upon by all parties rather than enabling collaborative learning where each party learns and improves their own local model. We introduce Confidential and Private Collaborative (CaPC) learning, the first method provably achieving both confidentiality and privacy in a collaborative setting. We leverage secure multi-party computation (MPC), homomorphic encryption (HE), and other techniques in combination with privately aggregated teacher models. We demonstrate how CaPC allows participants to collaborate without having to explicitly join their training sets or train a central model. Each party is able to improve the accuracy and fairness of their model, even in settings where each party has a model that performs well on their own dataset or when datasets are not IID and model architectures are heterogeneous across parties. 

Published as a conference paper at ICLR 2021
CAPC LEARNING: CONFIDENTIAL AND PRIVATE COLLABORATIVE LEARNING
Christopher A. Choquette-Choc; Natalie Dullerud; Adam Dziedzic* University of Toronto and Vector Institute [christepher. choquette choo, natal ie .dd.llllluuu @ma utorontoo.ca ady@vectorine ast itute.
Yunxiang Zhang"
Somesh Jha University of Misconsin-Madison and XaiPient jha@cs. wisc. edu
The Chinese University of Hong Kong yunxiang. zhang@ie.... cuhk. edu.hk
Nicolas Papernot
Xiao Wang
University of Toronto and Vector Institute nicolas, eat
Northwestern University wangxiaočcs wwrthwestern.een
ABSTRACT
Machine learning benefits from large training datasets, which may not always be possible to collect by any single entity, especially when using privacy-sensitive data. In many contexts, such healthcare and finance, separate parties may wish to collaborate and learn from each other's data but are prevented from doing due to privacy regulations. Some regulations prevent explicit sharing of data between parties by joining datasets in a central location (confidentiality), Others also limit implicit sharing of data, e.g., through model predictions (privacy). There is currently no method that enables machine learning in such a setting, where both confidentiality and privacy need be preserved, prevent both explicit and implicit sharing of data. Federated learning only provides confidentiality, not privacy, since gradients shared still contain private information. Differentially private learning assumes unreasonably large datasets. Furthermore, both of these learning paradigms produce central model whose architecture was previously agreed upon by all parties rather than enabling collaborative learning where each party learns and improves their own local model. We introduce Confidential and Private Collaborative (CaPC) learning, the first method provably achieving both confidentiality and privacy in a collaborative setting. We leverage secure multi- party computation (MPC), homomorphic encryption (HE), and other techniques in combination with privately aggregated teacher models. We demonstrate how CaPC allows participants collaborate without having to explicitly join their training or train a central model. Each party is able to improve the accuracy and fairness of their model, even in settings where each party has model that performs well on their own dataset or when datasets are not IID and model architectures are heterogeneous across parties!!
INTRODUCTION
The predictions of machine learning (ML) systems often reveal private information contained in their training data (Shokri et al.) 2017 Carlini al. 2019) test inputs. Because of these limitations, legislation increasingly regulates the use of personal data (Mantelero) 2013). The relevant ethical
"Equal contributions, authors ordered alphabetically. Work done while the author at Vector Institute.
I Equal contributions, authors ordered alphabetically.
Code available http:///ggta..oom//eeecaabbbbabr

---

Published as a conference paper at ICLR 2021
Qserying Party
Decrypt
encrypted
1a
1b
Endfr
Send
Garbled crcult: argmax of logit
One mnteorrrammma(ff 7J+
Label
query Enc(g)
1c
Answeriigg
Ger bied crruitt angman sum shares
M.
Cutput logits
orgenezl
[PWaise())
Content Service Provider
P
Enclal
M,
Output
Ingits
Enctr.)
Add Laplacian/ /
Goussion
Nosse())
voctor
Figure I: Confidential and Private Collaborative (CaPC) Learning Protocol:
la Querying party P., sends
encrypted query g to each answering party P., 1 * 1,. Each Pi engages a secure 2-party computation protocol to evaluate Enc(g) on M, and outputs encrypted logits Enc(r.). 1b Each answering party, P., generates a random vector A; and sends Enc(r: - +t) the querying party, Pi,, who decrypts get r: P; Ic Each answering party P. runs Yao's garbled circuit protocol (Y,) with querying party P.. to get s; for P., and 8; for P: s.t. S, + &; is the one-hot encoding of argmax of logits. 2 Each answering party sends &; to the privacy guardian (PG). The PG sums A, from each P; and adds Laplacian or Gaussian noise DP. The querying party sums 8; from each Y, computation. argmax of querying party and PG's noisy share. The label is oulput the querying party.
3 The PG and the querying party run Yao's garbled circuit Y, to obtain
concerns prompted researchers to invent ML algorithms that protect the privacy of training data and confidentiality of test inputs (Abadi et al. 2016 Konečný et al., 2016 Juvekar et al. 2018).
Yet, these algorithms require a large dataset stored either in single location or distributed amongst billions of participants. This is the case for example with federated learning (McMahan et al., (017). Prior algorithms also assume that all parties are collectively training a single model with a fixed architecture. These requirements are often too restrictive in practice. For instance, a hospital may to improve medical diagnosis for patient using data and models from other hospitals. In this case, the data stored in multiple locations, and there are only a few parties collaborating. Further, each party may also want to train models with different architectures that best serve their own priorities.
We propose strategy that lets fewer heterogeneous parties learn from each other collaboratively, enabling each party to improve their own local models while protecting the confidentiality and privacy of their data. We call this Confidential and Private Collaborative (CaPC) learning.
Our strategy improves on confidential inference (Boemer 2020) and PATE, the private aggregation of teacher ensembles (Papernot al., 2017). Through structured applications of these two techniques, we design a strategy for inference that enables participants to operate an ensemble of heterogeneous models, i.e. the teachers, without having to explicitly join each party's 's data teacher model at single location. This also gives each party control at inference, because inference requires the agreement and participation of each party. In addition, our strategy provides measurable confidentiality and privacy guarantees, which we formally prove. We use the running example of network of hospitals to illustrate our approach. The hospitals participating in CaPC protocol need guarantees on both confidentiality (i.e., data from hospital can only be read by said hospital) and privacy (i.e., hospital can infer private information about other hospitals' data by observing their predictions).
First, one hospital queries all the other parties over homomorphic encryption (HE), asking them to label an encrypted input using their own teacher models. This can prevent the other hospitals from reading the input (Boemer et al.) 2019). improvement hospitals to provide a prediction to the querying hospital without sharing their teacher models.
PATE, and allows the answering
The answering hospitals use multi-party computation (MPC) to compute an aggregated label, and add noise during the aggregation to obtain differential privacy guarantees (Dwork et al.) 2014). This is achieved by a privacy guardian (PG), which then relays the aggregated label the querying hospital. The PG only needs to be tmmi-trusted: we operate under the honest-bue sireess-but-cuee aur-icuio assumption. The use of MPC ensures that the PG cannot decipher each teacher model's individual prediction, and the noise added via noisy argmax mechanism gives differential privacy even when there are few participants.
2

---

Published as a conference paper at ICLR 2021
This is significant advantage over prior decentralized approaches like federated learning, which require billions of participants to achieve differential privacy, because the sensitivity of the histogram used in our aggregation is lower than that of the gradients aggregated in federated learning. Unlike our approach, prior efforts involving few participants thus had to prioritize model utility over privacy and only guarantee confidentiality (Sheller al. 2020
Finally, the querying hospital can learn from this confidential and private label to improve their local model. Since the shared information is label rather than gradient, as used by federated learning, CaPC participants do not need to share a common model architecture; in fact, their architectures can vary throughout the participation in the protocol. This favors model development to a degree which is not possible in prior efforts such as federated learning.
We show how participants can instantiate various forms of active and online learning with the labels returned by our protocol: each party participating in the CaPC protocol may (a) identify deficiencies of its model throughout deployment and (b) finetune the model with labels obtained by interacting with other parties. Intuitively, we achieve the analog of doctor querying colleagues for second opinion on a difficult diagnostic, without having to reveal the patient's medical condition. This protocol leads to improvements in both the accuracy and fairness (when there is a skew in the data distribution of each participating hospital) of model predictions for each of the CaPC participants.
To summarize, our contributions are the following:
We introduce CaPC learning: a confidential and private collaborative learning platform that provides both confidentiality and privacy while remaining agnostic to MI techniques.
Through a structured application of homomorphic encryption, secure MPC, and private aggregation, we design a protocol for CaPC. We use two-party deep learning inference and design an implementation of the noisy argmax mechanism with garbled circuits.
Our experiments on SVHN and CIFAR 10 demonstrate that CaPC enables participants to collaborate and improve the utility of their models, even in the heterogeneous setting where the architectures of their local models differ, and when there are only a few participants.
Further, when the distribution of data drifts across participating parties, show that CaPC significantly improves fairness metrics because querying parties benefit from knowledge learned by other parties different data distributions, which is distilled in their predictions.
We release the source code for reproducing all our experiments.
BACKGROUND
Before introducing CaPC, we first go over elements of cryptography and differential privacy that are required to understand it. Detailed treatment of these topics can be found in Appendices A and B
2.1
CRYPTOGRAM APHIC PRELIMINARIES FOR CONFIDENTIALITY
The main cryptographic tool used in CaPC secure multi multi-party computation (MPC) Yao 1986).
MPC allows set of distrusting parties to jointly evaluate function their input without revealing anything beyond the output. In general, most practical MPC protocols can be classified into two cate- gories: 1) generic MPC protocols that can compute any function with the above security goal (Malkhi iet al. 2004); and 2) specialized MPC protocols that can be used to compute only selected functions (e.g.. private set intersection (Pinkas et al. 2020). secure machine learning (Mohassel & Zhang 2017)) Although specialized MPC protocols are less general, they are often more efficient in execu- tion time. Protocols in both categories use similar cryptographic building blocks, including (fully) homomorphic encryption (Gentry) 2009), secret sharing (Shamir 1979). oblivious transfer (Rabin) 2005), garbled circuits (Yao) 1986). To understand our protocol, it is not necessary to know all details about these cryptographic building blocks and thus describe them in Appendix A. Our work uses these cryptographic preliminaries for secure computation at prediction time, unlike recent approaches, which explore new methods to achieving confidentiality training time (Huang et al. b).
The cryptographic protocol designed in this paper uses specialized MPC protocol for securely evaluating a private ML model on private data, and a generic two-party computation protocol to compute argmax in different forms. For the generic two-party computation, use classical Yao's
3

---

Published as a conference paper at ICLR 2021
garbled-cireuit protocol that can compute any function in Boolean circuit. For secure classification of neural networks, our protocol design is flexible to work with most existing protocols (Boemer
et al. 2020; 2019 Gilad-Bachrace et al. 2016; Mishra et al. 2020 Most existing protocols are different in how they handle linear layers (e.g. convolution) and non-linear layers (e.g. ReI.U). For instance, one can perform all computations using a fully homomorphic encryption scheme resulting in low communication but very high computation, or using classical MPC techniques with more communication but less computation. Other works (Juvekar et al. 2018) use hybrid of both and thus enjoy further improvement in performance (Mishra et al. 2020). We discuss it in more details in Appendix A.2
2.2 DIFFERENTIAL PRIVACY
Differential privacy is the established framework for measuring the privacy leakage of a randomized algorithm (Dwork et al. 2006). In the context of machine learning, it requires the training algorithm to produce statistically indistinguishable outputs any pair of datasets that only differ by one data point. This implies that an adversary observing the outputs of the training algorithm (e.g.. the model's parameters, or its predictions) can improve its guess most by a bounded probability when inferring properties of the training data points. Formally, we have the following definition.
Definition 1 (Differential Privacy). A randomized mechanism M with domain 'D and range R. satisfies (e, -)dfffeereeti privacy if for any subset S C R. and any adjacent datasets d, d' E 'D, i.e.
||d d'll 1, the following inequality holds:
Pr [M(()) E E) S (PPr(MMM) E ES
(1)
In our work, we obtain differential privacy by post-processing the outputs of an ensemble of models with the noisy argmax mechanism Dwork et al. (2014) (for more details on differential privacy, please refer to Appendix [B). à la PATE (Papernot et al. (017). We apply the improved analysis of PATE (Papernot al. 2018 to compute the privacy guarantees obtained (i.e., a bound on a). Our technique differs from PATE in that each of the teacher models is trained by different parties whereas PATE assumes centralized learning setting where all of the training and inference is performed by single party. Note that our technique is used at inference time, which differs from recent works in differential privacy that compare neuron pruning during training with mechanisms satisfying differential privacy (Huang et al. 2020c). We use cryptography to securely decentralize computations.
3
THE CAPC PROTOCOL
We now introduce our protocol for achieving both confidentiality and privacy in collaborative (CaPC) learning. To do so, we formalize and generalize our example of collaborating hospitals from Section
3.1 PROBLEM DESCRIPTION
small number of parties (Prbee1.8) each holding a private dataset D, = {(zz yo or Je and capable of fitting a predictive model M, to it, wish to improve the utility of their individual models via collaboration. Due to the private nature of the datasets in question, they cannot directly share data or by- sy-products of data (e.g.. model weights) with each other. Instead, they will collaborate by querying each other for labels of the inputs about which they are uncertain. In the active learning paradigm, one party Pr. poses queries in the form of data samples x and all the other parties {P, (Pi)iyi. together provide answers in the form of predicted labels ÿ. Each model [MM.)eej.KK can exploited in both the querying phase and the answering phase, with the querying party alternating between different participants (Pi (ee in the protocol.
To obtain the strong confidentiality and privacy guarantees that we described, we require semi-trusted third party called the privacy guardian (PG). We assume that the PG does not collude with any party and that the adversary can corrupt any subset of parties {P.)ien.c]- When more than one party gets corrupted, this has no impact on the confidentiality guarantee, but the privacy budget obtained c will degrade by a factor proportional to C' because the sensitivity of the aggregation mechanism increases (see Section 3.3). We work in the honest-but-curinn but-curious setting, commonly adopted assumption in cryptography which requires the adversary to follow the protocol description correctly but will try to infer information from the protocol transcript.
Threat Model.
4

---

Published as a conference paper at ICLR 2021
3.2 CAPC PROTOCOL DESCRIPTION
Our protocol introduces a novel formulation of the private aggregation of teachers, which implements two-party confidential inference and secret sharing to improve upon the work Papernot al. (2017) and guarantee confidentiality. Recall that the querying party Pt. initiates the protocol by sending an encrypted imput x to all answering parties P., i & i. We use sk and pk to denote the secret and public keys owned by party P.,. The proposed protocol consists of the following steps:
1. For cach i # i., P. (with model parameters M; as its input) and Ps. (with x, sks pk as its input) secure two-party protocol. As the outcome, P. obtains 8; and P.. obtains 8, such that 8 + 8; = One (arg max(r:)) where r; are the predicted logits.
This step could be achieved by the following:
a) P., and P. run a secure two-party ML classification protocol such that P., learns nothing while P. learns Encp4 (r:), (rr), where r; are the predicted logits.
b) P. generates random vector Å; performs the following computation the encrypted data Encp+(rr) Encp. (fA) = Encp: f;). and sends the encrypted difference to Pi.. who decrypts and obtains (r:: f::)
c) Pj (with r; as input) and P.. (with T; - r. as input) engage in Yao's two-party garbled- circuit protocol to obtain vector 8, for Pa, and vector &; for P., such that 8, + s; OneHot (arg max(r:)).
2. Pt sends & to the PG. The PG computes á Σizi, 8, + (eN, where DPNoise() is element-wise Laplacian Gaussian noise whose variance is calibrated to obtain desired differ- ential privacy guarantee E; whereas P:, computes 8 = =Etyi. 8,.
3. The PG and P, engage in Yao's two-party garbled- circuit protocol for computing the argmax: Pi. gets arg max( & + s) and the PG gets nothing.
Next, we elaborate on the confidentiality and privacy guarantees achieved by CaPC.
3.3 CONFIDENTIALITY AND DIFFERENTIAL PRIVACY GUARANTEES
Confidentiality Analysis. We prove in Appendix E that the above protocol reveals nothing to Pa or the PG and only reveals the final noisy results to P... The protocol is secure against semi-honest adversary corrupting any subset of parties. Intuitively, the proof can be easily derived based on the security of the underlying components, including two-party classification protocol, secret sharing, and Yao's garbled circuit protocol. As discussed in Section [.1 and Appendix [p.] for secret sharing of unbounded integers, need to make sure the random padding is picked from domain much larger than the maximum possible value being shared. Given the above, a corrupted Pi, cannot learn anything about M,, of the honest party due to the confidentiality guarantee of the secure classification protocol; similarly, the confidentiality of x against corrupted PP; is also protected. Intermediate values all secretly shared (and only recovered within garbled circuits) so they are visible any party.
Differential Privacy Analysis. Here, any potential privacy leakage in terms of differential privacy is incurred by the answering parties {Palezs, for their datasets { (D: [Dihess.. because these parties share the predictions of their models. Before sharing these predictions to P.., we follow the PATE protocol: we compute the histogram of label counts y. then add Laplacian or Gaussian noise using a sensitivity of 1, and finally return the argmax of Üo to P... Since PP. only sees this noisily aggregated label, both the data-dependent and data-independent differential privacy analysis of PATE apply to Pt. (Papernot et al. 2017 2018). Thus, when there are enough parties with high consensus, we can obtain tighter bound on the privacy budget as the true plurality will more likely be returned (refer to Appendix B for more details on how this is achieved in PATE). This setup assumes that only one answering party can be corrupted. If instead C' parties are corrupted, the sensitivity of the noisy aggregation mechanism will scaled by C and the privacy guarantee will deteriorate. There no privacy leakage to the PG; it does not receive any part of the predictions from (P.)
EXPERIMENTS
CaPC aims to improve the model utility of collaborating parties by providing them with new labelled data for training their respective local models. Since designed the CaPC. protocol with techniques
5

---

Published as a conference paper at ICLR 2021
for confidentiality (i.e., confidential inference and secret sharing) and differential privacy (i.e., private aggregation), our experiments consider the following three major dimensions:
1. How well does collaboration improve the model utility of all participating parties?
2. What requirements are there to achieve privacy and how can these be relaxed under different circumstances? What is the trade-off' between the privacy and utility provided by CaPC?
3. What is the resulting computational cost for ensuring confidentiality?
4.1 IMPLEMENTATION
We use the HEEtransformer library with MPC (MP2ML) by Boemer (2020) step la of our protocol for confidential two-party deep learning inference. To make our protocol flexible to any private inference library, not just those that return the label predicted by the model (HE-transforme) only
returns logits), we incorporate steps lb and Ic of the protocol outside of the private inference library. The EMP toolkit (Wang et al. 2016) for generic two-party computation is used to compute the operations including argmez and sum via the garbled circuits. To secret share the encrypted values, we first convert them into integers over a prime field according to the CKKS parameters, and then perform secret sharing on that domain to obtain perfect secret sharing. We use the single largest logit value for each M; obtained on its training set D; in plain text to calculate the necessary noise.
4.2 EVALUATION SETUP
Collaboration. We use the following for experiments unless otherwise noted. We uniformly sample from the training set in use] without replacement, to create disjoint partitions, D;, of equal size and identical data distribution for each party. We select K = 50 and K = 250 as the number of parties for CIFARIO 10 and SVHN, respectively (the number is larger for SVHN because we have more data). We select Q 3 querying parties, P... and similarly divide part of the test set into Q separate private pools for each P., to select queries, until their privacy budget of E is reached (using Gaussian noise with a= 40 SVHN and 7 on CIFAR10). We are left with 1, 000 and 16, 032 evaluation data points from the test set of CIFARIO and SVHN respectively. We fix c <= 2 and 20 for SVHN and CIFARIO, respectively (which leads evaluation set. Querying models are retrained on their DD plus the newly labelled data; the difference in accuracies is their accuracy improvement.
550 queries per party), and report accuracy on the
We use shallower variants of VGG, namely VGG-5 and VGG-7 for CIFARIO and SVHN, respec- tively, to accommodate the small size of each party's private dataset. We instantiate VGG-7 with convolutional layers and one final fully-connected layer, thus there are 7 functional layers overall, Similarly, VGG-5 has 4 convolutional layers followed by a fully connected layer. The ResNet- 10 architecture starts with a single convolutional layer. followed by 4 basic blocks with 2 convolutional layers in each block, and ends with a fully-connected layer, giving 10 functional layers in total. The ResNet-8 architecture that we use excludes the last basic block and increases the number of neurons in the last (fully-connected) layer. We present more details on architectures in Appendix F22
We first train local models for all parties using their non-overlapping private datasets. Next, run the CaPC protocol to generate query-answer pairs for each querying party. Finally, we retrain the local model of each querying party using the combination of their original private dataset and the newly obtained query-answer pairs. We report the mean accuracy and class-specific accuracy averaged over 5 runs for all retrained models, where each uses a different random seed.
Heterogeneity and Data Skew. Where noted, our heterogeneous experiments (recall that this is a newly applicable setting that CaPC enables) use VGG-7, ResNet-8 and ResNet- 10 architectures for 3 parties, each. One model of each architecture is used for each of Q = querying parties. Our data skew experiments use 80% less data samples for the classes "horse". 'ship', and 'truck' on CIFARI0 and 90% less data for the classes 1 and 2 on SVHN. In turn, unfair ML algorithms perform worse on these specific classes, leading to worse balanced accuracy (see Appendix D). We adopt balanced accuracy instead of other fairness metrics because the datasets we use have no sensitive attributes, making them inapplicable. We employ margin, entropy, and greedy k-center active learning strategies
K
For the SVHN dataset, we combine its original training set and extra to get larger training set.

---

Published as a conference paper at ICLR 2021
(described in Appendix C to encourage ML algorithms to sample more queries from regimes that have been underrepresented and to improve their fairness performance.
4.3 COLLABORATION ANALYSIS
We first investigate the benefits of collaboration for improving each party's model performance in several different settings, namely: homogeneous and heterogeneous model architectures across querying and answering parties, and uniform and non-uniform data sampling for training data. From these experiments, we observe: increased accuracy in both homogeneous settings and heterogeneous settings to all model architectures (Section 4.3.I) and improved balanced accuracy when there is data skew between parties, i.e., non-uniform private data (Section 4.3.2)
4.3.1 UNIFORMLY SAMPLED PRIVATE DATA
The first setting we consider is uniform distribution of data amongst the parties- is no data drift among parties. Our set up for the uniform data distribution experiments is detailed in Section 4.2 We evaluate the per-class and overall accuracy before and after CaPC in both homogeneous and heterogeneous settings on the CIFARIO and SVHN datasets.
In Figure 2 we see there is a consistent increase in accuracy for each class and overall in terms of mean accuracy across all parties on the test sets. We observe these improvements in both the homogeneous and heterogeneous settings for both datasets tested. As demonstrated in Figure 2 there a greater climb in mean accuracy for the heterogeneous setting than the homogeneous setting on SVHN. Figures 5 6 and 7 provide breakdown of the benefits obtained by each querying party. We can see from these figures that all querying parties observe an increase in overall accuracy in heterogeneous and homogeneous settings with both datasets; additionally, the jump in accuracy is largely constant between different model architectures. In only 6.67% of all cases were any class-specific accuracies degraded, but they still showed a net increase in overall model accuracy.
50
Slage
96
96
Before Calh After CsPC
90
50
85
Actunity
80
60
76
75
:0
70
60
855
55
frog homeasop truc
Classes
CF AR10. Homogensois
SVHN Homagenezus
SVHN, Heterogenenuus
Figure 2: Using CaPC to improve model performance. Dashed lines represent mean accuracy. With homogeneous models, we observe a mean increase of 4.09 and of 1.92 percentage points on CIFAR10 10 and SVHN, respectively, and an increase of 2.64 with heterogeneous models; each party still sees improvements despite differing model architectures (see Figure 7 in Appendix F).
4.3.2 NON-UNIFORMLY SAMPLED PRIVATE DATA
In this section, we focus our analysis on two types of data skew between parties: varying size of data per class and total size of data provided; the setup is described in Section 4.2] To analyze data skew. we explore the balanced accuracy (which measures mean recall per-class basis, see Appendix D) We use balanced accuracy in order to investigate aggregate fairness gains offered by CaPC. Random sampling from non-uniform distributions leads to certain pitfalls: c.g., underrepresented classes not specifically targeted sampling. Thus, we additionally utilize active learning techniques, namely entropy, margin, and greedy-k-cente (see Definitions 6 g in Appendix C and analyze balanced accuracy with each strategy.
In Figure 3] skew between the private data of participating parties. Even random sampling can drastically improve balanced accuracy. Leveraging active learning techniques, can achieve additional benefits in
that CaPC has significant impact on the balanced accuracy when there is data
7

---

Published as a conference paper at ICLR 2021
balanced accuracy. In particular, observe that entropy and margin sampling achieves the greatest improvement over random sampling in per-class accuracy for the less represented classes 'horse', "ship', and 'truck' CIFARIO and classes and 2 SVHN. These enhancements by the underlying mechanisms of margin and entropy sampling because the less-represented classes have higher margin/entropy: the queries per class for each method shown in Figure 9 Through these experiments, we show that in data skew settings, the CaPC protocol can significantly improve the fair performance models measured by balanced accuracy), especially when combined with active learning techniques. Note that we see similar trends with (normal) accuracy as well.
be explained
80
90
70
Accuracy
Accuraoy
50
40
Sampling Method
Sampling Method
Before CaPC
Margin
Before CaPC
Margin
30
Random
Cenner-
Random
Greedy-k- Centte
Entropy
Entropy
20
plane car
dog frog ship
Classes
Classes
Figure 3: Using CaPC with active learning to improve balanced accuracy under non-uniform data distribution. Dashed lines are balanced accuracy (BA). We observe that all sampling strategies significantly improve BA and the best active learning scheme can improve BA by total of 9.94 percentage-point (an additional 0.8 percentage points over Random sampling) on CIFARIO (left) and total of 5.67 percentage-pone (an additional 0.38) on SVHN (right).
4.4 PRIVACY VERSUS UTILITY
We now study the trade-off between privacy and utility of our obtained models. Recall that we add Gaussian (or Laplacian) noise to the aggregate of predicted labels of all parties. Under the uniform setting, we choose the standard deviation a by performing a (random) grid search and choosing highest noise before significant loss in accuracy is observed. In doing so, each query uses minimal e while maximizing utility. Figure [I in Appendix showwws a sample plot for 250 models. For more details how is calculated, please refer to Appendix
As we increase the number of parties, we can issue more queries for a given privacy budget (E) which leads to higher accuracy gain. In Figure 4 report the accuracy gain achieved using CaPC with various numbers of parties, K. With a fixed total dataset size, increasing the number of parties decreases their training data size, leading worse performing models. These models the largest benefit from CaPC but, importantly, we always see net improvement across all values of K.
90
Number of parties
89
150
200
250
300
400
88
Accuracy (96)
87
Number of parties:
Accuracy 0.62
1.45
2.39
3.07
3.87
gain (%)
86
150
Best
3.50
3.32
2.60
2.40
1.91
85
200
84
250
300
Figure Accuracy gain for balanced SVHN using CaPC versus number of parties and pri- vacy budget, E. With more parties, we can achieve a higher accuracy gain at a smaller bound on e.
83
82
0.0
0.5
1.0
1.5
2.0
2.5
3.0
3.5
Privacy budget (E)
8

---

Published as a conference paper at ICLR 2021
4.5 COMPUTATIONAL COSTS OF CONFIDENTIALITY
The incorporation of confidentiality CaPC increases computational costs. We segment the analysis of computational overhead of CaPC into three parts corresponding to sequential steps in the protocol: (I) inference, (2) secret sharing between each querying and answering party, and (3) secret sharing between the querying party and the PG. Each of these steps is analyzed in terms of the wall-clock time (in seconds). We use the default encryption setting in trE-transformer and vary the modulus range, N. which denotes the max value of given plain text number increase the maximum security level possible. -r-ttranfforme only supports inference on CPUs and is used in step (1).
Step (1) with neural network inference using MPC incurs the highest CPU and network costs (see Table and Figure 13 in Appendix [+] Even the base level of security increases computational cost by 100X, and high security levels see increases up to 1000X, in comparison the non-encrypted inference on CPU. Compared to step (I), the rest of the CaPC protocol incurs a negligible overhead to perform secret sharing. Overall, CaPC incurs only low additional cost over the underlying MP2ML framework, as shown in Figure 13 which enables applicability and scalability as these tools progress.
5 DISCUSSION AND CONCLUSIONS
CaPC is a secure and private protocol that protects both the confidentiality of test data and the privacy of training data, which are desired in applications like healthcare and finance. Our framework facili- tates collaborative learning using heterogeneous model architectures and separate private datasets, even if the number of parties involved is small. It offers notable advantages over recent methods for learning with multiple participants, such as federated learning, which assumes training of a single fixed model architecture. CaPC does not assume homogencous model architecture and allows parties to separately and collaboratively train different models optimized for their own purposes. Federated learning also requires a large number of parties while CaPC provides gains in accuracy with significantly fewer participants, even in contexts where each party already has a model with high accuracy. Notably, CaPC incurs low overhead on top of underlying tools used for secure neural network inference.
Through our experiments, we also demonstrate that CaPC facilitates collaborative learning even when there exists non i.i.d (highly skewed) private data among parties. Our experiments show that CaPC improves on the fair performance of participating querying models as indicated by improvements in the balanced accuracy, a common fairmess metric. Further, we observe significant increase in per-class accuracy on less-represented classes on all datasets tested. Notably, CaPC is easily configured to leverage active learning techniques to achieve additional faimess improvement gains or to learn from other heterogeneous models trained with fairness techniques, e.g., with synthetic minority oversampling (Chawla et al. 2002). In future work, we look to analyzing the fairness implications of CaPC in contexts where there is discrimination over a private datasct's sensitive attributes, not just class labels. In these cases, other fairness metries like equalized odds and equal opportunity (see Appendix D) can be explored.
We note some limitations of the proposed protocol. HE-transformer does not prevent leaking certain aspects of the model architecture, such as the type of non-linear activation functions and presence of MaxPooling layers. CaPC improves upon existing methods in terms of the necessary number of parties; however, it would be favorable to see this number decreased under 50 for better flexibility and applicability in practice.
In the face of this last limitation, when there are few physical parties, we can generate larger number of virtual parties for CaPC, where each physical party subdivides their private dataset into disjoint partitions and trains multiple local models. This would allow CaPC to tolerate more noise injected during aggregation and provide better privacy guarantees. Note that each physical party could select queries using dedicated strong model instead of the weak models used for answering queries in CaPC. This setting is desirable in cases where separate models are required within a single physical party, for example, in multi-national organization with per-country models.

---

Published as a conference paper at ICLR 2021
ACKNOWLEDGMENTS
We would like to acknowledge our sponsors, who support our research with financial and in-kind contributions: Microsoft, Intel, CIFAR through the Canada CIFAR AI Chair and catalyst programs, NFRF through an Exploration grant, and NSERC COHESA Strategic Alliance. Resources used in preparing this research were provided, in part, by the Province of Ontario, the Government of Canada pttprinsttuute.aaapateeett
through CIFAR, and companies sponsoring the Vector Institute Finally, we would like to thank members of CleverHans Lab for their feedback, especially: Tejumade Afonja, Varun Chandrasekaran, Stephan Rabanser, and Jonas Guan.
REFERENCES
Martin Abadi, Andy Chu, Ian Goodfellow, H Brendan McMAhan, Ilya Mironov, Kunal Talwar, and Li Zhang. Deep learning with differential privacy. In Proceedings of the 2016 ACM SIGSAC Conference Computer and Communications Security, pp. 308 318, 2016.
Fabian Boemer. he-transformer. 2020. [Online; accessed ---eepeembe--2220.
Fabian Boemer, Yixing Lao, Rosario Cammarota, and Casimir Wierzynski. Ngraph-he: A graph compiler for deep learning on homomorphically encrypted data. In Proceedings of the 16th ACM International Conference on Computing Frontiers, CF 11, pp. 3-13, New York, NY, USA, 2019. Association for Computing Machinery.
Fabian Boemer, Rosario Cammarota, Daniel Demmler, Thomas Schneider, and Hossein Yalame. MP2ML: a prixed-protocol machine learning framework for private inference. In Melanie Volkamer and Christian Wressnegger (eds.), ARES 2020: The 15th International Conference on Availability, Reliability and Security, Virtual Event, Ireland, August 25-28, 2020, pp. 14:1-14:10. ACM, 2020.
Zvika Brakerski, Craig Gentry, and Vinod Vaikuntanathan. (leveled) fully homomorphic encryption without bootstrapping. ACM Transactions on Computation Theory (TOCT), 6(3):1-36, 2014.
Nicholas Carlini, Chang Liu, Úlfar Erlingsson, Jernej Kos, and Dawn Song. The secret sharer: Evaluating and testing unintended memorization in neural networks, In 28th {USENIX) Security Symposium (UUEENIX} Security 19), pp. 267-284, 2019.
Nitesh V Chawla, Kevin W Bowyer, Lawrence O Hall, and W Philip Kegelmeyer. Smote: synthetic minority over-sampling technique. Journal of artificial intelligence research, 16:321-357, 2002.
Jung Hee Cheon, Andrey Kim, Miran Kim, and Yongsoo Song. Homomorphic encryption for arithmetic of approximate numbers. In International Conference on the Theory and Application of Cryptology and Information Security, pp. 409-437. Springer, 2017.
Cynthia Dwork, Frank McSherry, Kobbi Nissim, and Adam Smith. Calibrating noise to sensitivity in private data analysis. In Theory of cryptography conference, pp. 265-284, Springer, 2006.
Cynthia Dwork, Guy N Rothblum, and Salil Vadhan. Boosting and differential privacy. In 2010 IEEE 51st Annual Symposium on Foundations of Computer Science, pp. 51-60. JEEE, 2010.
Cynthia Dwork, Aaron Roth, et al. The algorithmic foundations of differential privacy. Foundations and Trends in Theoretical Computer Science, 9(3-4):21 1-407, 2014.
David Evans, Yan Huang, Jonathan Katz, and Lior Malka. Efficient identification. Proceedings of the 17th conference Network and Distributed System Security Symposium, NDSS, volume 68, 2011.
biometric
Reza Zanjirami Farahani and Masoud Hekmatfar. Facility location: concepts, models, algorithms and case studies. Springer, 2009.
Craig Gentry. fully homomorphic encryption scheme, volume 20. Stanford university Stanford, 2009.
10

---

Published as a conference paper at ICLR 2021
Ran
Nathan Dowlin, Kim Laine, Kristin Lauter, Michael Nachrig, and John Wernsing. Cryptonets: Applying neural networks to encrypted data with high throughput and accuracy. In International Conference Machine Learning, pp- 201-210, 2016.
Sebastien Godard. sar (sysstat). 2020. [Online; accessed 10-September-2220].
Moritz Hardt, Eric Price, and Nati Srebro. Equality of opportunity in supervised learning. In Advances in neural information processing systems, pp. 3315-3323, 2016.
Yangsibo Huang, Zhao Song, Danqi Chen, Kai Li, and Sanjeev Arora. Texthide: Tackling data privacy in language understanding tasks. arXiv preprini 2010.06053, 2020a.
Yangsibo Huang, Zhao Song. Kai Li, and Sanjeev Arora. Instahide: Instance-hiding schemes for private distributed learning. arXiv preprint 2010.02772, 2020b.
Yangsibo Huang, Yushan Su, Sachin Ravi, Zhao Song. Sanjeev Arora, and Kai Li. Privacy-preserving learning via deep net pruning. arXiv preprint 2003.01876, 2020c.
Yuval Ishai, Joe Kilian, Kobbi Nissim, and Erez Petrank. Extending oblivious transfers efficiently. In Annual International Cryptology Conference, pp. 145-161. Springer, 2003.
Chiraag Juvekar, Vinod Vaikuntanathan, and Anantha Chandrakasan. Gazelle: A low latency framework for secure neural network inference. In 27th USENIX Security Symposium (USENIX Security 18), pp. 1651-1669, 2018.
Jakub Konečný, H Brendan McMahan, Felix X Yu, Peter Richtárik, Ananda Theertha Suresh, and Dave Bacon. Federated learning: Strategies for improving communication efficiency. arXiv 2016.
preprint 16100.v6600.000
David D. Lewis and William A. Gale. A sequential algorithm for training text classifiers. In Pro- ceedings of the 17th Annual International ACM-SIGIR Conference on Research and Developmens in Information Retrieval. Dublin, Ireland, 3-6 July 1994 (Special Issue of the SIGIR Forum), pp. 3-12, 1994.
Dahlia Malkhi, Noam Nisan, Benny Pinkas, and Yaron Sella. Fairplay-a secure two-party computa- tion system. In Proceedings of the 13th Conference on USENIX Security Symposium Volume 13, SSYM 04, pp. 20 USA, 2004. USENIX Association.
Alessandro Mantelero. The eu proposal for a general data protection regulation and the roots of the 'right to be forgotten'. Computer Law & Security Review, 29(3):229-25) 2013.
H Brendan McMahan, Daniel Ramage, Kunal Talwar, and Li Zhang. Learning differentially private recurrent language models. arXiv preprint arXiv:1710.0696 2017.
Ilya Mironov. Rényi differential privacy. In 2017 IEEE 30th Computer Security Foundations Symposium (CSF), pp. 263-275, IEEE, 2017.
Pratyush Mishra, Ryan Lehmkuhl, Akshayaram Srinivasan, Wenting Zheng, and Raluca Ada Popa. Delphi: A cryptographic inference service for neural networks. In 29th USENIX Security Sym- posium (USENIX Security 20), pp. 2505-2522. USENIX Association, August 2020. ISBN 978-1-939133-17-5.
Payman Mohassel and Yupeng Zhang. Secureml: A system for scalable privacy-preservine machine learning. 2017 IEEE Symposium on Security and Privacy (SP), pp. 19-38. IEEE, 2017.
Nicolas Papernot, Martín Abadi, Úlfar Erlingsson, Ian J. Goodfellow, and Kunal Talwar. Semi- supervised knowledge transfer for deep learning from private training data. In 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Confer- ence Track Proceedings. 2017.
Nicolas Papernot, Shuang Song. Ilya Mironov, Ananth Raghunathan, Kunal Talwar, and Úlfar Erlingsson. Scalable private learning with PATE. In 6th International Conference Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30 May 3, 2018, Conference Track Proceedings, 2018
11

---

Published as a conference paper at ICLR 2021
Benny Pinkas, Mike Rosulek, Ni Trieu, and Avishay Yanai. Psi from paxos: Fast, malicious private set intersection. In Anmual International Conference on the Theory and Applications of Cryptographic Techniques, pp. 739-767. Springer, 2020.
Michael O. Rabin. How to exchange secrets with oblivious transfer. Cryptology ePrint Archive, Report 2005/187, 2005.
iacr. ora.0o0511//00 200.
Tobias Scheffer, Christian Decomain, and Stefan Wrobel. Active hidden markov models for informa- tion extraction. In International Symposium on Intelligent Data Analysis, pp. 309-318. Springer, 2001.
Ozan Sener and Silvio Savarese. Active learning for convolutional neural networks: A core-set approach. arXiv preprint arXiv: 1tt.::770.0000 778800489, 2017.
Burr Settles. Active learning literature survey. Computer Sciences Technical Report 1648, University of Aacconsin-Madisee 2009.
Adi Shamir. How to share secret. Communications of the ACM, 20MM,2)(1)))226611 1979.
Claude Shannon. mathematical theory of communication. Bell system technical journal, 27(3):
379 423, 1948.
Micah J. Sheller, Brandon Edwards, G. Anthony Reina, Jason Martin, Sarthak Pati, Aikaterini Kotrotsou, Mikhail Milchenko, Weilin Xu, Daniel Marcus, Rivka Colen, and Spyridon Bakas. Federated learning in medicine: facilitating multi-institutional collaborations without sharing patient data. Scientific Reports, 10(1):12598, 2020. doi: 10. 0033///1198-000---111 https: //d::i//ooo.oo org/l0...o.oooooooooooo.. 1038//41598-000000 http:///.....1131111199-22222220 69250-1
URL
Reza Shokri, Mareo Stronati, Congzheng Song, and Vitaly Shmatikov. Membership inference attacks against machine learning models. In 2017 IEEE Symposium on Security and Privacy (SP), pp. 3-18. IEEE, 2017.
Xiao Wang, Alex J. Malozemoff, and Jonathan Katz. EMP-toolkit: Efficient MultiParty computation toolkit.
com wwtpmm/empotooiiieeeeeoooooooeoe 2016.
Andrew Chi-Chih Yao. How to generate and exchange secrets (extended) ). In 27th Annual Symposium on Foundations of Computer Science, pp. 162-167, Toronto, Ontario, Canada, October 27-29, 1986. IEEE Computer Society Press.
A MORE BACKGROUND ON CRYPTOGRAPHY
A.1 CRYPTOGRAPHIC BUILDING BLOCKS
Homomorphic encryption. Homomorphic encryption defines an encryption scheme such that the encryption and decryption functions are homomorphic between plaintext and ciphertext spaces, Although it is known that fully homomorphic encryption can be constructed based on lattice- based assumptions, most applications only require weaker version with bounded number of multiplications on each ciphertext. Schemes with this constraint are much more practical, including for example, BGV (Brakerski al. 20144, CKKS (Cheon et al. 2017). etc.
Secret sharing. Secret sharing denotes scheme in which datum, the secret, is shared amongst group of parties by dividing the secret into parts such that each party only has one part, or 'share' of the secret. The secret can only be recovered if a certain number of parties conspire combine their shares. It is easy to construct secret sharing modulo a positive integer. If the application does not allow modular operation, one still achieve statistically secure secret sharing by using random shares that are much larger than the secret being shared (Evans et al.) 2011).
Oblivious transfer. Oblivious transfer involves two parties: the sending party and the receiving party. The sending party has two pieces of information, 8o and 81, and the receiver wants to receive Sy. where b E {0, 1}. such that the sending party cannot learn 5 and the receiving party cannot learn s-b- In general, oblivious transfer requires public-key key operations, however, it is possible execute large number of oblivious transfers with only a very small number of kublic-key operations based on oblivious transfer extension (Ishai et al. 2003
12

---

Published as a conference paper at ICLR 2021
Garbled circuits. In Yao's garbled circuit protocol for two-party computation, each of the two parties assumes a role, that of garbler or that of evaluator. The function on which to compute each of the two parties' inputs is described (termed labels) representing O and 1 in the Boolean circuit describing f and replaces the binary values with the generated labels for each wire in the circuit. At each gate in the circuit, which can be viewed as a truth table, the garbler uses the labels of each possible combination of inputs to encrypt the corresponding outputs, and permutes the rows of the truth table. The garbler then uses the generated labels for O and I to encode their own input data and sends these labels and the garbled Boolean eircuit to the evaluator. The evaluator now converts their binary input data to the corresponding labels through 1-2 oblivious transfer protocol with the garbler. After receiving the labels for their input, the evaluator evaluates the garbled circuit by trying to decrypt each row in the permutable truth tables at each gate using the input labels; only one row will be decryptable at each gate, which is the output label for the outgoing wire from the gate. The evaluator eventually finishes evaluating the garbled circuit and obtains the label for the output of the function computed on the garbler's 's and the evaluator's 's input. The garbler then must provide the true value for the output label so that both parties can get the output.
Boolean circuit. The garbler randomly generates aliases
A.2 PROTECTING CONFIDENTIALITY USING MPC
Neural networks present a challenge to secure multi-party computation protocols due to their unique structure and exploitative combination of linear computations and non-linear activation functions. Cryptographic inference with neural networks can be considered in two party computation case in which one party has confidential input for which they wish to obtain output from a model and the other party stores the model; in many cases the party storing the model also wishes that the model remains secure.
Confidential learning and inference with neural networks typically uses homomorphic encryption (HE) or secure multi-party computation (MPC) methods. Many libraries support pure HE MPC protocols for secure inference of neural networks; a comprehensive list can be viewed in (Boemer ct al. 2020). Notably, libraries such as nGraph-HE (Boemer et al., 2019)) and CryptoNets (Gilad- Bachrach et al., 2016) provide pure homomorphic encryption solutions to secure neural network inference. nGraph-HE. an extension of graph compiler nGraph, allows secure inference of DNNs through linear computations at each layer using CKKS homomorphic encryption scheme (Cheon jt al. 2017 Boemer et al., 2019). CryptoNets similarly permit confidential neural network inference using another leveled homomorphic encryption scheme, YASHE'
et al., 2016).
On the other hand, several libraries employing primarily MPC methods in secure NN inference frameworks rely on ABY, tool providing support for common non-polynomial activation functions in NNs through use of both Yao's 's GC and GMW.
In DL contexts, while pure homomorphic encryption methods maintain model security, their failure to support common nol-polyloiial activation functions leads to leaking of pre-activation values (feature maps at hidden layers). Tools that use solely MPC protocols avoid leaking pre-activation values as they can guarantee data confidentiality on non-polynomial activation functions but may compromise the security of the model architecture by leaking activation functions or model structure.
Recent works on secure NN inference propose hybrid protocols that combine homomorphic encryp- tion schemes, and MPC methods to build frameworks that try to reduce leakages common in pure HE and MPC protocols, Among recent works that use hybrid protocols and do not rely on trusted third parties are Gazelle (Juvekar al. 2018), Delphi (Mishra et al. 2020). and MP2ML (Boemer et al. 2020).
Gazelle. Delphi and MP2ML largely support non- nol-poollomial activation functions encountered in convolutional neural networks, such as maximum pooling and rectified linear unit (ReLU) operations. Gazelle introduced several improvements over previous methods for secure NN inference primarily relating to latency and confidentiality. In particular, Gazelle framework provides homomorphic en- cryption libraries with low latency implementations of algorithms for single instruction multiple data (SIMD) operations, ciphertext permutation, and homomorphic matrix and convolutional operations, pertinent to convolutional neural networks. Gazelle utilizes kernel methods to evaluate homomorphic operations for linear components of networks, garbled circuits to compute non- non-linear activation functions confidentially and additive secret sharing to quickly switch between these cryptographic protocols. Delphi builds on Gazelle, optimizing computation of both linear and non-linear com-
13

---

Published as a conference paper at ICLR 2021
putations in CNNs by secret sharing model weights in the pre-processing stage speed up linear computations later, and approximating certain activation functions such as ReLU with polynomials. MP2ML employs nGraph-HE for homomorphic encryption and ABY framework for evaluation of non-linear functions using garbled circuits.
B MORE BACKGROUND ON DIFFERENTIAL PRIVACY
One of the compelling properties of differential privacy is that it permits the analysis and control of cumulative privacy cost over multiple consecutive computations. For instance, strong composition theorem (Dwork al. 2010) gives tight estimate of the privacy cost associated with a sequence of adaptive mechanisms M,
Theorem (Strong Composition), For EnrE, 8, 2 O, the class of (s, -)differrnttilly private mecha- nisms satisfies (s', kō + 8 ddifffrrrltti privacy under k-fold adaptive composition for:
= 2kV2kloog log(0/0))) + ke(e" -1
(2)
To facilitate the evaluation of privacy leakage resulted by randomized mechanism M, it is helpful to explicitly define its corresponding privacy loss CM and privacy loss random variable CM. Particularly, the fact that M is (e, 5) diiifeerentially private equivalent a certain tail bound on CM-
Definition 2 (Privacy Loss). Given a pair of adjacent datasets d, d' D and an auxiliary impul aur, the privacy loss cM of a randomized mechanism M evaluated at an outcome E R is defined as:
Pr Pr[M(aux, d) = o) co aux, d, d') A log Pr[M(aur, d') = o]
For an outcome O E R. sampled M(d), Cm(aux, d, d') takes the value c aux, d, d')-
Based on the definition of privacy loss, Abadi al. (Abadi et al.) 2016) introduced the moments accountant to track higher-order moments of privacy loss random variable and achieved even tighter privacy bounds for k-fold adaptive mechanisms.
Definition 3 (Moments Accountant). Given any adjacent datasets d, d' E D and any auxiliary input auz, the moments accountant of a randomized mechanism M is defined as:
am(A)
2
max O.M (A ouxx, d, d') aaz,d,
(4)
where (M aux, d, d & log (aux d, d' d')) is obtained by taking the logarithu of the privacy loss random variable.
As a natural relaxation the conventional (e, -differrreel privacy, Rényi differential privacy (RDP) (Mironoy 2017) provides more convenient and accurate approach to estimating privacy loss under heterogeneous composition.
Definition 4 (Rényi Divergence). For two probability distributions P and defined over RR the Rényi divergence of order between them is defined as:
(x(P A-
log E E [(P(=)/Q(=))) = - log E [(P())/Q(z)>>-1] (5)
Definition 5 (Rényi Differential Privacy). randomized mechanism M is said to satisfy EéRényi differential privacy of order X, or (A,E)-RDP for short, for any adjacent datasets d, d' E D:
Pr[M(d)= Pr[M(d') (1
Dx(M(d) M(d')
A log Miad
SE
(6)
Theorem (From RDP to DP). If randomized mechanism M guarantees (), RDDP then also satisfies (e + log(1/8) A -)ddifferettial privacy for any d E 10.
Building upon the moments accountant and RDP techniques, Private Aggregation of Teacher En- sembles (PATE) (Papernot et al.] 2017) provides a flexible approach to training machine learning models with strong privacy guarantees. Precisely, rather than directly learning from labeled private
14

---

Published as a conference paper at ICLR 2021
data, the model that gets released instead learns from unlabeled public data by querying teacher ensemble for predicted labels. Models in the ensemble are themselves trained on disjoint partitions of the private dataset, while privacy guarantees are enabled by applying the Laplace mechanism to the ensemble's s aggregated label counts. Coupled with data-dependent privacy analysis, PATE achieves tighter estimate of the privacy loss associated with label queries, especially when the consensus among teacher models is strong. Given this motivation, the follow-up work of PATE (Papernot et al., 2018) further improves the privacy bound both by leveraging a more concentrated noise distribution to strengthen consensus and by rejecting queries that lack consensus.
C
MORE BACKGROUND ON ACTIVE LEARNING
Active learning, sometimes referred to as query learning, exploits the intuition that machine learning algorithms will be able to learn more efficiently if they can actively select the data from which they learn. For certain supervised learning tasks, this insight is of particularly important implications, as labeled data rarely exists in abundance and data labeling can be very demanding (Settles) (2tt).s
In order to pick queries that will most likely contribute to model learning, various pool sampling methods have been proposed to estimate the informativeness of unlabeled samples. Uncertainty-basee based approaches (Lewis & Gale 1994), such as margin sampling and entropy sampling, typically achieve a satisfactory trade-off between sample utility and computational efficiency. We also explore core-set approach to active learning using greedy-k-center sampling (Sener & Savarese 20177)
Definition 6 (Margin Sampling (Scheffer al., (200T)). Given an unlabeled dataset d and clas- sification model with conditional label distribution Po r),, margin sampling outputs the most informative sample:
x²
= arg min (o ()) - Po(gal a))
=Ed
where d1 and ú2 stand for the most and second most probable labels for x, according to the model. Definition 7 (Entropy Sampling). Using the setting and notations in Definition O margin sampling can be generalized by using entropy (Shannon 1948) as an uncertainty measure as follows:
Σ Polys |x) log Po 1z)
arg max
sEd
where y; ranges over all possible labels.
Definition 8 (Greedy-K-cente Sampling). We aim to solve the k-center problem defined by Farahani & Hekmatfar (2009), which is, infuitively, the problem of picking k center points that minimize the largest distance between a data point and its nearest center. Formally, this goal is defined as
min max min Axx,,x)) S:SSUDikk
i JESUD
where 'D is the current training set and S is our new chosen center points. This definition can can be solved greedily as shown in (Sener & Savarese 2017)
D MORE BACKGROUND ON FAIRNESS
Due to the imbalance in sample quantity and learning complexity, machine learning models may have disparate predictive performance over different classes or demographic groups, resulting in unfair treatment of certain population. To better capture this phenomenon and introduce tractable countermeasures, various fairness-related criteria have been proposed, including balanced accuracy, demographic parity, equalized odds (Hardt et al. 2016)) etc.
Definition 9 (Balanced Accuracy). Balanced accuracy captures model utility in terms of both accuracy and fairness. It is defined as the average of recall scores obtained on all classes.
Among the criteria that aim to alleviate discrimination against certain protected attributes, equalized odds and equal opportunity Hardt et al. (2016) are of particular research interests.
Definition 10 (Equalized Odds). A machine learning model is said to guarantee equalized odds with respect to protected attribuee and ground truth label Y if its prediction Y and A are conditionally independent given Y. In the case of binary random variables A, Y, Y, this is equivalent to:
Pr v = 1|A 11A=00Y- = =Pr = 1|A 1, = 3) y BE {0,
(10)
15

---

Published as a conference paper at ICLR 2021
To put it another way, equalized odds requires the model to have equal true positive rates and equal false positive rates across the two demographic groups A = and
Definition 11 (Equal Opportunity). Equal opportunity is relaxation of equalized odds that requires non-diicciminaaion only within a specific outcome group, often referred as the advantaged group. Using previous notations, the binary case with advantaged group = is equivalent to:
Pr
0,
8 1, -1
(11)
E PROOF OF CONFIDENTIALITY
Here we prove that our protocol described in the main body does not reveal anything except the final noised result to P... In can be proven in the standard real-world ideal-worls paradigm, where the ideal functionality takes inputs from all parties and sends the final results to P... We use to denote the set of corrupted parties. Below, we describe the simulator (namely S). The simulator strategy depends on if i, is corrupted.
If i. E A, our simulator works as below:
1.a) The simulator simulates what honest parties would do.
1.b) For each & A. S sends fresh encryption of random Tę to P...
1.c) For each & A, sends random s; to Pt. on be half of the 2PC functionality between P; and P...
2-3 S sends the output of the whole computation P,, on behalf of the 2PC functionality between PG and P..
If i. & A. our simulator works as below:
If I.a) behalf of P...
é A. for each i E A, S computes a fresh encryption of zero and sends it to P; on
1.b) The simulator simulates what honest parties would do.
1.c) For each i A. S sends random 8; to P, on behalf of the 2PC functionality between P, and P...
2-3 The simulator simulates what honest parties would do.
Assuming that the underlying encryption scheme is CPA secure and that 2PC protocols used in step 1, 2 and 3 are secure with respect to standard definitions (i.c., reveals nothing beyond the outputs), our simulation itself is perfect.
F DETAILS ON EXPERIMENTAL SETUP
F.1 MNIST AND FASHION-MNIST
We use the same setup as for CIFARIO and SVHN datasets with the following adjustments. We select K 250 the default number of parties. For the imbalanced classes select classes and 2 for MNIST as well as Trouser and Pullover for Fashion-MNIST We use the Gaussian noise with Æ = 40 (similarly SVHN). We are left with 000 evaluation data points from the (similarly to CIFAR10). We fix the default value of 2.35 for MNIST and 3.89 for Fashion-MNIST. We use variant of the LeNet architecture.
F.2 DETAILS ON ARCHITECTURES
To train the private models on subsets of datasets, we downsize the standard architectures, such as VGG-16 18. Below is the detailed list of layers in each of the architectures used (generated using torchsummary). The diagram for ResNet- also includes skip connections and convolutional layers for adjusting the sizes of feature maps.
16

---

Published as a conference paper at ICLR 2021
VGG-7 for SVHN:
Layer type
Output Shape
Param
Conv2d-1
[-1, 64, 32, 32]
728
BatchNorm2d-2
[-1, , 64, 32, 32]
128
RelU-3
[-1, 64, 32, 32]
0
MaxPco12d-4
[-1, 64, 16, 16]
0
Conv2d-5
[-1, 128, 16, 16]
73, 728
B
[-1, 128, 16, 161
256
ReLU-7
[-1, 128, 16, 16]
MaxPool2d-8
[-1, 128, 8, 8]
0
Conv2d-9
[ [-1, 256, 8, 8]
294, 912
BatchNorm2d-10
[-1, 256, 8, 8]
512
ReLU-11
[-1, 256, 8, 8]
0
Conv2d-12
[-1, 256, 8, 8]
589, 824
BatchNerm2d-13
[-1, 256, 8, 8]
512
ReLU-14
1-,, 256, 8, 8]
0
MaxPcol2d-15
1-1, 256, 4, 4]
Conv2d-16
1 1-1, 512, 4, 41 [-1, 512, 4, 4]
1, 179,
HatchNerm2d-11
1, 024
RelUU18
[ [-1, 512, 4, 4]
0
Conv2d-19
[-1, 512, 4, 4]
2, 2559,29
BatchNorm2d-20
-1, 512, 4, 41
024
ReLU-21
[-1, 512, 4, 4]
Linear-22
[-1, 10]
5, 130
Total params: 4, ,07,007770 722
Params sizc 17.20
ResNet-100
Layer type
Output Shape
Param
Conv2d-1
[-1, 64, 32, 32]
1, 728
BatchNorm2d-2
[-1, 64, 32, 32]
128
Conv2d-3
[-1, 64, 32, 32]
36, 864
BatchNorm2d-4
[-1, 64, 32, 32]
128
Conv2d-5
[-1, 64, 32, 32]
36, 864
BatchNerm2d-6
[-1, 64, 32, 32]
128
BasicBlock-7
[-1, 64, 32, 32]
0
Conv2d-8
[-1, 128, 16, 16]
73, 728
BatchNorm2d-9
[-1, 128, 16, 16
256
Conv2d-10
|-1, 128, 16, 16
147, 456
BatchNorm2d-11
[-1, 128, 16, 161
256
Conv2d-12
[-1, 128, 16, 16]
192
BatchNorm2d-13
[-1, 128, 16, 16]
256
BasicBlock-14
[-1, 128 16, 16]
0
Conv2d-15
[-1, 256, 8, 8]
294, 912
BatchNozm2d-16
[-1, 256, 8, 8]
512
Conv2d-17
[-1, 256, 8, 8]
589, 824
BatchNerm2d-18
[-,, 256, 8, 8]
512
Conv2d-19
[-1, 256, 8, 8]
32, 768
BatchNorm2d-20
[ [-1, 256, 8, 8] [ [-1, 256, 8, 8]
512
BasicBlock-21
0
Conv2d-22
[-1, 512, 4, 4]
1, 1799,, 648
HatchNorm22-23
[-1, 512, 4, 4]
1, 024
17

---

Published as a conference paper at ICLR 2021
Conv2d-24
[-1, 512, 4, 4]
3,95, 296
BatchNorm2d-25
[-1, 512, 4, 4]
0,4
Conv2d-26
[-1, 512, 4, 41
131, 072
BatchNerm2d-27
[-1, 512, 4, 4]
0,4
BasicBlock-28
[-1, 512, 4, 4]
0
Linear-29
[-1, 10]
130
Total params: : 4, 903, 242
Params size MB: 70
LeNet style architecture for MN IST:
Layer type
Output Shape
Conv2d-1
[-1, 20, 24, 24]
520
MaxPool2d-2
Conv2d-3
[-1, 50, 8, 8]
25, 050
MaxPool2d-4
Linear-5
[-1, 500]
400, 500
ReLU-6
Linear-7
[-1, 10]
5, 010
Total params: 431, 080
Trainable params: 431,080
Non-trainable params:
Input size MB: 0.00
Porward/backward pass size MB: 0.12
Params size MB: 1.64
Estimated Total Size MB: 1.76
18

---

Published as a conference paper at ICLR 2021
G ADDITIONAL EXPERIMENTS AND FIGURES
90
Before CaPC Mean
90
90
After CaPC Mean
80
Before CaPC
80
80
After CaPC
Accuracy
80
60
60
50
50
50
40
40
40
plane
car
bird
Gal
deer
dog
frog
horse ship
truck
plana
car
hird
Gaf
doer
dog
frog
horse
ship
truck
plane
car
bird
Cal
deer
dog
frog
horse
ship
truck
Classes
Classes
Classes
CIFAR10, Homogeneous, QP1
CIFAR10, Homogeneaus, QP2
CIFAR10, Homogeneous, QP3
Figure 5: Using CaPC to improve each party's model performance on the CIFAR10 dataset.
We observe that each separate querying party (QP) sees per-class and overall accuracy bonus using CaPC.
95
95
95
90
85
85
85
Accuracy
75
75
75
Before CaPC Mean
After CaPC Mean
70
CaPC
CaPC
65
65
6
8
0
2
4
6
7
8
2 3 4
5
7
9
Classes
Classes
Classes
SVHN, Homogeneous, QP1
SVHN, Homogeneous, QP2
SVHN, Homogeneous, QP3
Figure 6: Using CaPC to improve each party's model performance on the SVHN dataset. We observe that all querying parties (QPs) see net increase overall, with nearly every class seeing improved performance.
19

---

Published as a conference paper at ICLR 2021
95
95
95
85
85
85
80
Apcuracy
75
75
75
70
70
Before CaPC Mean
85
After CaPC Mean
60
Before CaPC
60
80
After CaPC
0
1
2
3
5
6
8
0
2
4
6
7
8
9
0
1
2
3 4
5
7
9
Classes
Classes
Classes
SVHN, Model: VGG-7
SVHN, Model: ResNet-8
SVHN, Model: Resnet-10
Figure Using CaPC improve each party's heterogeneous model performance on the SVHN dataset. Each querying party adopts different model architecture (1 of 3) and of all answering parties adopt each model architecture. All model architectures see benefits from using CaPC.
100
100
85
80
80
Accurapy
Accuracy
85
Before CaPC Mean
80
Before CaPC Meon
After CaPC Mean
Mean
Bafora CaPC
75
Before CaPC
After CaPC
After CaPC
70
T-shirlToop
Trouser
Dress
Coat
Sandal
Shirt
Sneaker
Bag
Ankle Boot
3
7
Classes
Fashion-MNIST. Homogeneous
MNIST, Homogeneous
Classes
Figure& 8: Using CaPC to improve model performance on balanced MNIST on Fashion-MNISTT.
Dashed lines represent mean accuracy. We observe a mean increase of 4.5% for MNIST 2.35) and 2.9% for Dashion-MNIST (c 3.89).
Method
Forward Pass (Step la)
CPU, 8192 CPU, p = 16384
14.22 0.11
29.46 + 2.34 57.26 0.39
CPU, P =32768
GPU, no encryption CPU, no encryption
3.15 0.22
0.152 + 0.0082
QP-AP (Steps 1b and 1c) QP-PG (Steps 2 and
0.12 + 0.0058
0.030 0.0045
Table 1: Wall-clock time (sec) of various encryption methods with batch size of 1. We vary the modulus range, P. which denotes the max value of a given plain text number. Note that the GPU slower than the CPU because of the mini-batch with single data item and data transfer overhead to and from the GPU. We use the CryptoNet-Rede ReeN model provided by HE-tren -E-ttaanfformer (Boemer 2020).
20

---

Published as a conference paper at ICLR 2021
17.5
Random
35
Random
Entropy
Entropy
7
Margin
30
Margin
Number Quenñs
Gready-k-Center
Number Queries
Greedy-k-
12.5
25
10.0
20
7.5
10
2.5
0.0
plane
car
bird
deer dog
horse ship
truck
Classes
Classes
CIFAR10, Homogeneous
SVHN, Homogeneous
Figure Using active learning to improve CaPC fairness. classes are sampled more frequently than in random strategy.
We observe that underrepresented
100
100
90
80
Acouracy
Accuracy
60
60
40
50
Sampling Method
Sampling Method
Before CaPC
Margin
Before
CaPC
Margin
40
Random
Greedy- k- Center
Random
Greedy- CeCtter
Entropy
Entropy
30
3
8
T-shir/Top
Trouser
Pullover
Coat
Sandal
Shirt
Sneaker
Ankie Boot
Classes
Classes
Figure 10: Using CaPC with active learning to improve balanced accuracy under non-uniform
data distribution. Dashed lines are balanced accuracy (BA). We observe that all sampling strategies significantly improve BA and the best active learning scheme can improve BA by total of 10.10 percentage-poing (an additional 0.45 percentage points over Random sampling) on MNIST (left) and total of 10.94 percentage- points (an additional 2.48) on Fashion MNNIST (right).
21

---

Published as a conference paper at ICLR 2021
Vole Gap Between Top-2 Chosen Classess
Accuracy
80.0
90-
Accuracy
Gap
Gap
Chosen
45.0
Betwwenn Top-2 Chosen
80
Chosen
-75
70
-70
Annurany,
Accurany,
50
665
40.0
80
60
40
-65
50
35,0
-50
30
32.55
40
-45
0
10
40
100
150
200
Noise Standard Deviation,
Noise Standard Deviatio,
CIFAR10, Homogeneous
SVHN, Homogeneous
Gap Bellween Top-2 Chosen Classes
Vole Gap Belweeen Top-2 Chosen Classes
80
Accuraty
Gap
200
Gap
Accuracy
180
Chasan
Chasan
-80
70
160
Accuracy,
70
Accuracy.
140
60
140
50
120
120
40
40
100
100
50 100
150
50
100
150
200
Noise Standard Deviation, 0 MNIST, Hamogeneous
Noise Standard Deviation o Fashion-MNIST, Homageneous
Figure 11: Tuning the amount of noise (a) in CaPC. We tune the amount of Gaussian noise that should injected the noisy argmax mechanism by varying the standard deviation. We choose the highest noise: 7 for CIFARIO, having a significant impact on the model accuracy, allowing a minimal privacy budget expenditure while maximizing utility. We train 50 models for CIFARIO and 250 models for SVHN, MNIST, and Fashion-MNIST..
40 for SVHN, MNIST, and Fashion -NIST, without
94
Number of parties
150
200
250
300
400
92
Accuracy (%
Number of parties:
Accuracy
4.11
3.33
4.50
4.69
8.39
gain (%)
88
150
Best 4.50
2.50
2.35
2.00
1.63
200
250
300
Figure 12: Accuracy gain for balanced MNIST using CaPC versus number of parties and pri- vacy budget, With more parties, we can achieve a higher accuracy gain at a smaller bound on E.
400
82
3
Privacy budget (e)
22

---

Published as a conference paper at ICLR 2021
100
CPU
100
CPU
Utilization (%6)
80
Utilization (96)
80
NET
NET
60
60
-MEM
MEM
40
40
16
20
20
0
0
15 Time step (sec)
10
20
25
5
10
15
20
25
30
35
Time step (sec)
MP2ML (HE-transformer for nGraph).
CaPC (built on top of the MP2ML framework).
Measuring the CPU, Network (NET), and Memory (MEM) usage over time CaPC. We use the CryptoNet-Reet ReLU model provided HE-transformet (Boemer 2020 and (oo dard (020) (System Activity Report) to perform this nicro-analysis We label the steps according to the CaPC protocol shown in Figure The network usage reaches its peaks during execution of
Figure 13:
ReLU and then MaxPool, where the intermediate feature maps have to be exchanged between the querying and answering parties for the computation via garbled circuits.
23


Reviews:
[{"review_id": "yGDQx-JWW3a", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper works on the problem of collaborative learning while preserving both confidentiality and privacy of the data points. It combines techniques from secure multi-party computation and differential privacy for the same, and improves on confidential inference and PATE in the process. The new technique is called CaPC. Finally, it states empirical results as evidence for the improved accuracy.\n\nWeakness:\n1. The evaluation is done on just two datasets. So, it is a little hard to judge whether the techniques would generalise or not.\n2. The writing of the paper itself is not that great because it is difficult to understand the low level details of the experiments.\n3. They talk very little about improving on the fairness guarantees.\n\nStrengths:\n1. Their techniques enable collaborative learning even in settings where the local architectures of different parties are different.\n2. The algorithms they provide improve on fairness.\n3. Their empirical results are better than the previously known methods.\n\nEvaluation: I believe the combination of secure multi-party computation and differential privacy is not totally new, but since it yields decent results, I would say that the paper deserves a chance to be accepted.\n\n", "extra_scores": {}, "extra_text": {}}, {"review_id": "wgveyYLPxk", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "\n\nThis work motivated by healthcare and finance where separate parties may wish to collaborate and learn from each other's data but are prevented from doing so due to privacy regulations. This paper propose Confidential and Private Collaborative (CaPC) learning, the first method provably achieving both confidentially and privacy in a collaborative setting. This work also discussed about fairness. I liked this part, since it seems very cool. However, I'm not convinced by the method in this work is better than InstaHide (I could be wrong).\n\n\n\n\nMinor comments\nIn Section 2.1, this paper should be discussed, since it proposed a way to ``encrypt'' the images/texts. \n\nInstaHide: instance-hiding schemes for private distributed learning\nhttps://arxiv.org/abs/2010.02772\nICML 2020\nYangsibo Huang, Zhao Song, Kai Li, Sanjeev Arora.\n\n\nTextHide: Tackling Data Privacy in Language Understanding Tasks\nhttps://arxiv.org/abs/2010.06053\nEMNLP 2020\nYangsibo Huang, Zhao Song, Danqi Chen, Kai Li, Sanjeev Arora\n\nIn Section B, it lists many theorems/definition about differential privacy. In Section C, it list many backgrounds about sampling. In Section D., it list many definitions on Fairness. I don't quite see the point of having them in appendix, since none of them got mentioned in Appendix E, which is the proof of the main theory result in this paper.\n\nThis paper is closely related differential privacy. I think this paper should also be mentioned somewhere.\n\nPrivacy-preserving Learning via Deep Net Pruning\nhttps://arxiv.org/abs/2003.01876\nYangsibo Huang, Yushan Su, Sachin Ravi, Zhao Song, Sanjeev Arora, Kai Li.\n", "extra_scores": {}, "extra_text": {}}, {"review_id": "kBZmdWip1j9", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Summary:\nThe authors combine several cryptographic techniques to create a federated systems that allows several entities to run classification against all the model held be the participants without revealing information in the process. In particular, the sample to be classified is not revealed to any other party, and differential privacy is used to protect the training data that was used to train the models. A central semi-honest coordinator is used to aggregate the results and add the differential privacy without learning any private information.\n\nPros:\nThe strength of this works lie in combining relevant techniques and to show experimentally that the resulting system does improve over using a local model both when the training is distributed evenly or in skewed manner while taking privacy considerations into account.\n\nCons:\n- From a cryptographic point of view, the combination of techniques is somewhat expectable.\n- I'm wondering about the low statistical security (${2^-23}$). This seems to be related to the usage of (unbounded) integer secret sharing. Would it be possible to use secret sharing modulo an integer, in which case the security could be perfect?\n- I think it would be easier to follow if steps 1-3 were combined in the description because they all take between the same pairs of parties. The exact techniques used don't seem to matter as long as the output secret sharing is the desired result, namely the one-hot vector.\n- I find the term collaborative learning somewhat overblown because the proposed protocol only runs classification collaboratively.\n\nOverall:\nDespite the points above, I'm in favor of acceptance because the paper seems to improve on previous work, and because it is written very well.\n\nMinor issues:\n3.3: leakeage\n4.1: odd juxtaposition in the formatting of \"arg max\" and \"sum\"\nFigure 3: very hard to read in black-and-white\n", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: This work describes a system for collaborative learning in which several agents holding data want to improve their models by asking other agents to label their points. The system preserves confidentiality of queries using MPC and also throws in differentially private aggregation of labels (taken from the PATE framework). It provides expriments showing computational feasibility of the system. The techniques use active learning to improve the models.

Overall the ingredients are fairly standard but are put together in a new (to the best of my , admittedly limited, knowledge of this area). This seems like a solid attempt to explore approaches for learning in a federated setting with strong limitations on data sharing.
Decision: Accept (Poster)