Original: Secure Multiparty Computation for Synthetic Data Generation from Distributed Data
Revised: Secure Multiparty Computation for Synthetic Data Generation from Distributed Data

Original: Anonymous Author(s)

Original: AfﬁliationAddress email

Original: Abstract
Revised: Abstract

Original: Legal and ethical restrictions on accessing relevant data inhibit data science researchin critical domains such as health, ﬁnance, and education.
Revised: Legal and ethical restrictions on accessing relevant data inhibit data science research in critical domains such as health, ﬁnance, and education.

Original: Synthetic data generationalgorithms with privacy guarantees are emerging as a paradigm to break this datalogjam.
Revised: Synthetic data generation algorithms with privacy guarantees are emerging as a paradigm to break this data logjam.

Original: Existing approaches, however, assume that the data holders supply theirraw data to a trusted curator, who uses it as fuel for synthetic data generation.
Revised: Existing approaches, however, assume that the data holders supply their raw data to a trusted curator, who uses it as fuel for synthetic data generation.

Original: This severely limits the applicability, as much of the valuable data in the worldis locked up in silos, controlled by entities who cannot show their data to eachother or a central aggregator without raising privacy concerns.
Revised: This severely limits the applicability, as much of the valuable data in the world is locked up in silos, controlled by entities who cannot show their data to each other or a central aggregator without raising privacy concerns.

Original: To overcome thisroadblock, we propose the ﬁrst solution in which data holders only share encrypteddata for differentially private synthetic data generation.
Revised: To overcome this roadblock, we propose the ﬁrst solution in which data holders only share encrypted data for differentially private synthetic data generation.

Original: Data holders send shares toservers who perform Secure Multiparty Computation (MPC) computations whilethe original data stays encrypted.
Revised: Data holders send shares to servers who perform Secure Multiparty Computation (MPC) computations while the original data stays encrypted.

Original: We instantiate this idea in an MPC protocolfor the Multiplicative Weights with Exponential Mechanism (MWEM) algorithmto generate synthetic data based on real data originating from many data holderswithout reliance on a single point of failure.
Revised: We instantiate this idea in an MPC protocol for the Multiplicative Weights with Exponential Mechanism (MWEM) algorithm to generate synthetic data based on real data originating from many data holders without reliance on a single point of failure.

Original:  Introduction
Revised:  Introduction

Original: We live in an era of abundant data, where enormous amounts of personal data are collected dailyvia smartphones, social media, smartwatches, medical devices, among many other services.
Revised: We live in an era of abundant data, where enormous amounts of personal data are collected daily via smartphones, social media, smartwatches, medical devices, among many other services.

Original: Thesedatasets have helped researchers and industry understand our behavior better on both individual andcollective levels and have also allowed important research studies in many disciplines, includinghealth, education and economy.
Revised: These datasets have helped researchers and industry understand our behavior better on both individual and collective levels, and have also allowed important research studies in many disciplines, including health, education and economy.

Original: At the same time, we see an increase in privacy regulations globally.
Revised: At the same time, we see an increase in privacy regulations globally.

Original: Following the introduction of the GDPR, 1 more than 60 jurisdictions around the world have proposedpostmodern data privacy protection laws.
Revised: Following the introduction of the GDPR, 1 more than 60 jurisdictions around the world have proposed postmodern data privacy protection laws.

Original: By 2024, 75% of the world’s population will have itspersonal information covered under modern privacy regulations [30].
Revised: By 2024, 75% of the world’s population will have its personal information covered under modern privacy regulations [30].

Original: While privacy regulations areof extreme importance from an ethics perspective, they can potentially result in data stored in silos,compromising data usage and data sharing, and stalling research.
Revised: While privacy regulations are of extreme importance from an ethics perspective, they can potentially result in data stored in silos, compromising data usage and data sharing, and stalling research.

Original: Synthetic data generation is emerging as a paradigm to break this data logjam.
Revised: Synthetic data generation is emerging as a paradigm to break this data logjam.

Original: While data synthesis is arguably best known as a means to create training examples for data hungry deep learning models [27], it is increasingly acknowledged and proposed as a privacy-enhancing technology (PET) 
Revised: While data synthesis is arguably best known as a means to create training examples for data hungry deep learning models [27], it is increasingly acknowledged and proposed as a privacy-enhancing technology (PET) [18, 22, 32, 33, 35, 36].

Original: When done well, synthetic data has the same distribution or characteristics as the underlying, real data, but, crucially, without replicating personal information.
Revised: When done well, synthetic data has the same distribution or characteristics as the underlying, real data, but, crucially, without replicating personal information.

Original: The latter is often formalized through the notion of Differential Privacy (DP) [14], which intuitively means thatthe synthetic data should not reveal speciﬁcs about individual records in the underlying, real data
Revised: The latter is often formalized through the notion of Differential Privacy (DP) [14], which intuitively means that the synthetic data should not reveal speciﬁcs about individual records in the underlying, real data.

Original: What the existing approaches for generating synthetic data and publication of data with DP guaranteeshave in common, is that they all assume that the original, real data, exists with one data holder, or, ifthe data originates from different data holders, that the latter are able to send their data to a centralaggregator who in turn will use it as input for synthetic data generation or DP publication algorithms.
Revised: What the existing approaches for generating synthetic data and publication of data with DP guarantees have in common, is that they all assume that the original, real data, exists with one data holder, or, if the data originates from different data holders, that the latter are able to send their data to a central aggregator who in turn will use it as input for synthetic data generation or DP publication algorithms.

Original: Much of the valuable data in the world however is under the control of entities (companies, banks,hospitals, biomedical research institutes etc.)
Revised: Much of the valuable data in the world however is under the control of entities (companies, banks, hospitals, biomedical research institutes etc.)

Original: who cannot show their data to each other or to a centralaggregator without raising privacy concerns.
Revised: who cannot show their data to each other or to a central aggregator without raising privacy concerns.

Original: This is the bottleneck that we address, namely how togenerate synthetic data based on the combined data from multiple data holders that no one is allowedto see .
Revised: This is the bottleneck that we address, namely how to generate synthetic data based on the combined data from multiple data holders that no one is allowed to see .

Original: This includes data that is horizontally distributed, such as healthcare data across differenthospitals, or ﬁnancial data held by different banks, as well as data that is vertically distributed, suchas advertising data where the publishers hold the input features while advertisers have the label, andmany more.
Revised: This includes data that is horizontally distributed, such as healthcare data across different hospitals, or ﬁnancial data held by different banks, as well as data that is vertically distributed, such as advertising data where the publishers hold the input features while advertisers have the label, and many more.

Original: Finally, in addition to the cross-silo scenarios described above, our proposed solutionmakes a scenario practical where millions of users could provide their data to produce a syntheticdataset in a way that private, individual data would never be exposed in plaintext (i.e. without beingencrypted) to any entity – practically implementing a "synthetic data as a service" model.
Revised: Finally, in addition to the cross-silo scenarios described above, our proposed solution makes a scenario practical where millions of users could provide their data to produce a synthetic dataset in a way that private, individual data would never be exposed in plaintext (i.e. without being encrypted) to any entity – practically implementing a "synthetic data as a service" model.

Original: The contributions of this paper are the following: (1) We introduce a framework for syntheticdata generation from distributed databases that utilizes Secure Multiparty Computation (MPC) [10]protocols that are run by two or more computing parties to emulate a trusted curator.
Revised: The contributions of this paper are the following: (1) We introduce a framework for synthetic data generation from distributed databases that utilizes Secure Multiparty Computation (MPC) [10] protocols that are run by two or more computing parties to emulate a trusted curator.

Original: This simulationenables the generation of synthetic data from training data held by multiple data holders, withoutrequiring these data holders to disclose their data to anyone in an unencrypted manner.
Revised: This simulation enables the generation of synthetic data from training data held by multiple data holders, without requiring these data holders to disclose their data to anyone in an unencrypted manner.

Original: (2) We modifythe Multiplicative Weights with Exponential Mechanism (MWEM), to generate synthetic data with
Revised: (2) We modify the Multiplicative Weights with Exponential Mechanism (MWEM), to generate synthetic data with

Original: DP guarantees, based on real data originating from many data holders, and without reliance on asingle point of failure.
Revised: DP guarantees, based on real data originating from many data holders, and without reliance on a single point of failure.

Original: (3) We propose an MPC protocol for secure sampling from distributed datausing the exponential mechanism.
Revised: (3) We propose an MPC protocol for secure sampling from distributed data using the exponential mechanism.

Original: Novelty w.r.t.
Revised: Novelty w.r.t.

Original: existing work.
Revised: existing work.

Original: Previous proposals for differentially private synthetic data generationfrom distributed databases use federated learning (FL) for training the data synthesizer [4, 37, 38].
Revised: Previous proposals for differentially private synthetic data generation from distributed databases use federated learning (FL) for training the data synthesizer [4, 37, 38].

Original: In these methods, each data holder sends model weights (without privacy protection) to a trustedaggregator, who computes the average of model weights and adds Laplacian noise.
Revised: In these methods, each data holder sends model weights (without privacy protection) to a trusted aggregator, who computes the average of model weights and adds Laplacian noise.

Original: Our proposalremoves the need for data holders to disclose model parameters, and the need to rely on a single pointof failure, by emulating the trusted aggregator with MPC.
Revised: Our proposal removes the need for data holders to disclose model parameters, and the need to rely on a single point of failure, by emulating the trusted aggregator with MPC.

Original: Additionally, previous works utilizing FLto train data synthesizers only account for horizontally partitioned data.
Revised: Additionally, previous works utilizing FL to train data synthesizers only account for horizontally partitioned data.

Original: While MPC has emerged as a paradigm for privacy-preserving training of ML models over distributeddata (e.g. [1, 2, 13, 16, 26, 34]) and privacy-preserving inference with trained ML models (e.g. [12,15 , 21, 25, 29]), and it has been proposed for secure computation of histograms (e.g. [5]), the idea ofusing MPC for privacy-preserving generation of synthetic data, as we propose here, is novel and apractical and secure technological solution.
Revised: While MPC has emerged as a paradigm for privacy-preserving training of ML models over distributed data (e.g. [1, 2, 13, 16, 26, 34]) and privacy-preserving inference with trained ML models (e.g. [12, 15, 21, 25, 29]), and it has been proposed for secure computation of histograms (e.g. [5]), the idea of using MPC for privacy-preserving generation of synthetic data, as we propose here, is novel and a practical and secure technological solution.

Original:  Preliminaries
Revised:  Preliminaries

Original: Differential Privacy [14].
Revised: Differential Privacy [14].

Original: A randomized algorithm M , which takes inputs from an input space Dand outputs values from an output space O , provides (cid:15) -Differential
Revised: A randomized algorithm M , which takes inputs from an input space D and outputs values from an output space O , provides (cid:15) -Differential

Original: Privacyif for all subsets S⊆ O ,and for all neighboring databases D and D (cid:48) ∈ ( D ) (i.e., D and D (cid:48) differ in at most one entry), Pr [ [Equation]
Revised: Privacy all subsets S

Original: D

Original: [Equation]

Original: Pr [ M

Original: [Equation]

Original: D (cid:48)

Original: [Equation]

Original: ∈ S ]

Original: The DP concept quantiﬁes privacy loss while providing an understandable notion of privacy: outputsof a DP algorithm for datasets that vary by a single entry are indistinguishable, bounded by theprivacy parameter (cid:15) .

Original: In this paper we use the well known exponential and Laplace mechanisms forcreating (cid:15) -DP

Original: algorithms (see App.

Original: A).

Original: Multiplicative Weights with Exponential Mechanism Algorithm [17].

Original: The MWEM algorithmtakes as input a and a set of linear queries Q (e.g queries).

Original: The algorithmaims to produce a distribution A over D such that the answers to the queries q in Q when run over

Original: A are similar to when run over D , i.e. the difference between q ( A ) and q ( D ) should be small.

Original: Thisis achieved by repeatedly sampling a query for which the difference is still large (line

Original: Alg. 1),

Original: Input: Dataset D over a universe D , set of linear queries Q , number of iterations T , and privacy parameter (cid:15) > 0 .
Revised: Input: Dataset D over a universe D , set of linear queries Q , number of iterations T , and privacy parameter (cid:15) > 0 .

Original: Let n denote | D | , the number of records in D .
Revised: Let n denote | D | , the number of records in D .

Original: Let A 0 denote n times the uniform distribution over D .
Revised: Let A 0 denote n times the uniform distribution over D .

Original: 1 for i ∈ { 1 , . . .
Revised: 1 for i ∈ { 1 , . . .

Original: , T } do
Revised: , T } do

Original: Exponential Mechanism: sample a query q i ∈ Q using the Exponential Mechanism parametrized with epsilon value (cid:15)/ 2 T and the score function: s i ( D, q ) =
Revised: Exponential Mechanism: sample a query q i ∈ Q using the Exponential Mechanism parametrized with epsilon value (cid:15)/ 2 T and the score function: s i ( D, q ) =

Original: | q ( A i − 1 ) − q ( D ) |
Revised: | q ( A i − 1 ) − q ( D ) |

Original: Laplace Mechanism: Let measurement m i =
Revised: Laplace Mechanism: Let measurement m i =

Original: q i ( D ) + Lap (2 T/(cid:15) )
Revised: q i ( D ) + Lap (2 T/(cid:15) )

Original: Multiplicative Weights: Let A i be n times the distribution whose entries satisfy
Revised: Multiplicative Weights: Let A i be n times the distribution whose entries satisfy

Original: [Equation]
Revised: [Equation]

Original: Output: A =
Revised: Output: A =

Original: avg i<T A
Revised: avg i< T A

Revised: i and for all neighboring databases D and D (cid:48) ∈

Revised: ( D ) (i.e., D and D (cid:48) differ in at most one entry),

Original: [Equation]
Revised: [Equation]

Revised: The DP concept quantiﬁes privacy loss while providing an understandable notion of privacy: outputs of a DP algorithm for datasets that vary by a single entry are indistinguishable, bounded by the privacy parameter (cid:15) .

Revised: In this paper we use the well known exponential and Laplace mechanisms for creating (cid:15) -DP algorithms (see App.

Original: and updating the weight that A places on each record x with the Multiplicative Weights update ruleto better approximate the distribution of D w.r.t.
Revised: A). We can relax this deﬁnition by adding an additive term δ to it: Pr [ M ( D ) ∈ S ] ≤ e (cid:15) · Pr [ M ( D (cid:48) ) ∈ S ] + δ .

Revised: This relaxed notion is called approximate differential privacy or ( (cid:15), δ ) − DP.

Revised: Multiplicative Weights with Exponential Mechanism Algorithm [17].

Revised: The MWEM algorithm takes as input a dataset D ⊆ D and a set of linear queries Q (e.g. counting queries).

Revised: The algorithm aims to produce a distribution

Revised: A over D such that the answers to the queries q in Q when run over

Revised: A are similar to when run over D , i.e. the difference between q ( A ) and q ( D ) should be small.

Revised: This is achieved by repeatedly sampling a query for which the difference is still large (line 2 in Alg. 1), and updating the weight that A places on each record x with the Multiplicative Weights update rule to better approximate the distribution of D w.r.t.

Original: q (line 4).
Revised: q (line 4).

Original: Furthermore, MWEM satisﬁes (cid:15) -DP byleveraging the exponential mechanism for query selection, and the Laplace mechanism to perturb thequery results.
Revised: Furthermore, MWEM satisﬁes (cid:15) -DP by leveraging the exponential mechanism for query selection, and the Laplace mechanism to perturb the query results.

Original: Secure Multiparty Computation (MPC).
Revised: Secure Multiparty Computation (MPC).

Original: MPC protocols enable a set of parties to jointly computethe output of a function over the private inputs of each party, without requiring any of the parties todisclose their own private inputs [9].
Revised: MPC protocols enable a set of parties to jointly compute the output of a function over the private inputs of each party, without requiring any of the parties to disclose their own private inputs [9].

Original: MPC protocols are designed to prevent and detect attacks byan adversary corrupting one or more parties to learn private information or to cause the result of thecomputation to be incorrect.
Revised: MPC protocols are designed to prevent and detect attacks by an adversary corrupting one or more parties to learn private information or to cause the result of the computation to be incorrect.

Original: The adversary  can have different levels of adversarial power.
Revised: The adversary – which we assume to be static – can have different levels of adversarial power.

Original: In thesemi-honest model, even corrupted parties follow the instructions of the protocol, but the adversaryattempts to learn private information from the internal state of the corrupted parties and the messagesthat they receive.
Revised: In the semi-honest model, even corrupted parties follow the instructions of the protocol, but the adversary attempts to learn private information from the internal state of the corrupted parties and the messages that they receive.

Original: MPC protocols that are secure against semi-honest or “passive” adversaries preventsuch leakage of information.
Revised: MPC protocols that are secure against semihonest or “passive” adversaries prevent such leakage of information.

Original: In the malicious adversarial model, the corrupted parties can arbitrarilydeviate from the protocol speciﬁcation.
Revised: In the malicious adversarial model, the corrupted parties can arbitrarily deviate from the protocol speciﬁcation.

Original: Providing security in the presence of malicious or “active”adversaries, i.e. ensuring that no such adversarial attack can succeed, comes at a higher computationalcost than in the passive case.
Revised: Providing security in the presence of malicious or “active” adversaries, i.e. ensuring that no such adversarial attack can succeed, comes at a higher computational cost than in the passive case.

Original: The protocols that we propose are sufﬁciently generic to be used insettings with passive or active adversaries.
Revised: The protocols that we propose are sufﬁciently generic to be used in settings with passive or active adversaries.

Original: This is achieved by changing the underlying MPC schemeto align with the desired security setting.
Revised: This is achieved by changing the underlying MPC scheme to align with the desired security setting.

Original: We consider an honest-majority 3-party computing settingout of which at most one party can be corrupted (3PC) [3, 11], an honest-majority 4-party computingsetting with one corruption (4PC) [11], and a dishonest-majority 2-party computation setting whereeach party can only trust itself (2PC) [8].
Revised: We consider an honest-majority 3-party computing setting out of which at most one party can be corrupted (3PC) [3, 11], an honest-majority 4-party computing setting with one corruption (4PC) [11], and a dishonest-majority 2-party computation setting where each party can only trust itself (2PC) [8].

Original: In all these MPC schemes, data is encrypted by splittingit into secret shares, which are distributed to a set of computing parties that run MPC protocolsand perform computations over these secret shares, in our case to generate synthetic data.
Revised: In all these MPC schemes, data is encrypted by splitting it into secret shares, which are distributed to a set of computing parties that run MPC protocols and perform computations over these secret shares, in our case to generate synthetic data.

Original: As allcomputations are done over encrypted values, the servers do not learn the values of the inputs norof intermediate results, i.e. MPC provides input privacy .
Revised: As all computations are done over encrypted values, the servers do not learn the values of the inputs nor of intermediate results, i.e. MPC provides input privacy .

Original: Below we propose an MPC protocol togenerate synthetic data under DP guarantees, i.e. providing  output privacy as well, so that the outputof the synthetic data generation process can be published.
Revised: Below we propose an MPC protocol to generate synthetic data under DP guarantees, i.e. providing [Table] output privacy as well, so that the output of the synthetic data generation process can be published.

Original: We refer to App.
Revised: We refer to App.

Original: B for more details on the MPC primitives used.
Revised: B for more details on the MPC primitives used.

Original:  Method
Revised:  Method

Original: We address the scenario where, instead of residing with one entity, the dataset D that we wish to giveas input to the MWEM algorithm is distributed among multiple data holders who cannot disclosetheir data to anyone in an unencrypted manner.
Revised: We address the scenario where, instead of residing with one entity, the dataset D that we wish to give as input to the MWEM algorithm is distributed among multiple data holders who cannot disclose their data to anyone in an unencrypted manner.

Original: We distinguish between the data holders who holdthe data sets, and the computing parties who run the MPC protocols for synthetic data generation andnoise addition.
Revised: We distinguish between the data holders who hold the data sets, and the computing parties who run the MPC protocols for synthetic data generation and noise addition.

Original: Our solution works in scenarios in which each data holder (e.g. hospital or bank) isalso a computing party, as well as in scenarios where the data holders outsource the computations tountrusted servers (computing parties)  data holders  secret shares of their data to  set ofparties loss of generality the computing parties havesecret shares of [[ D of D which they can use to compute a secret-sharing of the result
Revised: Our solution works in scenarios in which each data holder (e.g. hospital or bank) is also a computing party, as well as in scenarios where the data holders outsource the computations to untrusted servers (computing parties) instead. The data holders send secret shares of their data to a set of computing parties.

Original: [Table] [Table] Secret [Table]
Revised: Without loss of generality, we assume that the computing parties have secret shares of [[ D ]] of D , which they can use to compute a secret-sharing of the query result [[ q ( D )]] for each q in Q using primitive MPC protocols for addition and multiplication (see App. B).

Original: The execution of the overall MWEM algorithm can be coordinated by one of the  data holdersor any other entity interested in generating the synthetic data.
Revised: The execution of the overall MWEM algorithm can be coordinated by one of the computing parties or data holders or any other entity interested in generating the synthetic data.

Revised: Such a setup will not require the data holders to be online after they have secret shared the query answers based on their private data implementing data generation-as-a-service model which is independent of the number of data holders.

Original: Indeed, there are only two crucialsteps in Alg.
Revised: Indeed, there are only two crucial steps in Alg.

Original:  that rely directly on the encrypted data [[ D ]] , or rather [[ q ( D )]] , hence requiring MPC computations involving all computing parties: (1) the query selection in line 2; and (2) themeasurement in line 3.
Revised: 1 that rely directly on the encrypted data [[ D ]] , or rather [[ q ( D )]] , hence requiring MPC computations involving all computing parties: (1) the query selection in line 2; and (2) the measurement in line 3.

Original: Note that the output of the computations in line 2 and line 3 is protectedwith DP guarantees.
Revised: Note that the output of the computations in line 2 and line 3 is protected with DP guarantees.

Original: In other words, if we let the computing parties run MPC protocols for thecomputations and the DP mechanisms, then they can publicly reveal the selected query (line 2)and the perturbed query result (line 3), which can subsequently be used for further computations.
Revised: In other words, if we let the computing parties run MPC protocols for the computations and the DP mechanisms, then they can publicly reveal the selected query (line 2) and the perturbed query result (line 3), which can subsequently be used for further computations.

Original: Furthermore, there is no need to encrypt the synthetic data distribution A , as it is not based on anyinformation from D that is not already protected with DP.
Revised: Furthermore, there is no need to encrypt the synthetic data distribution A , as it is not based on any information from D that is not already protected with DP.

Original: This is a welcome observation becauseit means that query evaluations need to be done only once over encrypted data, namely to compute[[ q ( D )]] , and any further query evaluations on new versions of A can be done in-the-clear, i.e. withoutthe need for encryption.
Revised: This is a welcome observation because it means that query evaluations need to be done only once over encrypted data, [Table] [Table] [Table]

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: namely to compute [[ q ( D )]] , and any further query evaluations on new versions of A can be done in-the-clear, i.e. without the need for encryption.

Original: Description of π QEM .
Revised: Description of π QEM .

Original: For the secure query sampling on line 2 of Alg.
Revised: For the secure query sampling on line 2 of Alg.

Original: 1, we propose MPC-protocolπ QEM (see Prot. 2) which is called with privacy budget (cid:15) (cid:48) =
Revised: 1, we propose MPC-protocol π QEM (see Prot. 2) which is called with privacy budget (cid:15) (cid:48) =

Original: (cid:15)/ (2 T ) .
Revised: (cid:15)/ (2 T ) .

Original: π QEM consists of two parts:on lines 1–9 the parties compute secret shares of the probability distribution over the queries, whileon lines 10–23 the parties subsequently sample a query q k from that distribution.
Revised: π QEM consists of two parts: on lines 1–9 the parties compute secret shares of the probability distribution over the queries, while on lines 10–23 the parties subsequently sample a query q k from that distribution.

Original: Pseudocode for acorresponding algorithm in-the-clear, i.e. without regards for privacy, is given in Alg. 4 in App.
Revised: Pseudocode for a corresponding algorithm in-the-clear, i.e. without regards for privacy, is given in Alg. 4 in App.

Original: A,while the MPC primitives used in Alg. 1 are described in App.
Revised: A, while the MPC primitives used in Alg. 1 are described in App.

Original: B. The number and the kind ofoperations to construct the probability distribution and to compute the threshold (lines 1–9 in Alg. 4)are deterministic in the sense that they do not depend on the value of the data, hence their MPCcounterpart in Prot.
Revised: B. The number and the kind of operations to construct the probability distribution and to compute the threshold (lines 1–9 in Alg. 4) are deterministic in the sense that they do not depend on the value of the data, hence their MPC counterpart in Prot.

Original: 2 is relatively straightforward.
Revised: 2 is relatively straightforward.

Original: The implementation of the counterpart of thefor-loop that starts on line 11 in Alg. 4 requires more care, as exiting the for-loop prematurely couldallow an adversary to infer the value of the returned index from the runtime.
Revised: The implementation of the counterpart of the for-loop that starts on line 11 in Alg. 4 requires more care, as exiting the for-loop prematurely could allow an adversary to infer the value of the returned index from the runtime.

Original: The code in line 18-23in Prot.
Revised: The code in line 18in Prot.

Original: 2 is written to prevent such side-channel attacks.
Revised: 2 is written to prevent such side-channel attacks.

Original: To understand this part of the code, notethat we have a list c [1 ..N ] of non-decreasing values, i.e. the cumulative probability sums, and we – orrather the computing parties – have to ﬁnd the ﬁrst index i in c [1 ..N ] for which c
Revised: To understand this part of the code, note that we have a list c [1 ..N ] of non-decreasing values, i.e. the cumulative probability sums, and we – or rather the computing parties – have to ﬁnd the ﬁrst index i in c [1 ..N ] for which c

Original: [ i ] > t.
Revised: [ i ] > t.

Original: In a mockexample with N = 10 , and assuming that the ﬁrst such c [ i ] value is at position 7, the tests on line 20will generate the results 0,0,0,0,0,0,1,1,1,1.On line 21, these results are accumulated in s , which [Table] RANDOM protocol
Revised: In a mock example with N = 10 , and assuming that the ﬁrst such c [ i ] value is at position 7, the tests on linewill generate the results 0,0,0,0,0,0,1,1,1,1.

Original: [Table]

Original: RANDOM

Original: [Table]

Original: [Table]

Original: [Table] [Table] [Table] eventually becomes 4, and the desired index is computed as N − ( s − 1)
Revised: On line 21, these results are accumulated in s , which eventually becomes 4, and the desired index is computed as N − ( s − 1)

Original: = 10 − 3 = 7 .
Revised: = 10 − 3 = 7 .

Original: Lines 22–23take care of the edge case when c [ i ] ≤ t for all i (i.e. s is 0 ).
Revised: Lines 22– take care of the edge case when c [ i ] ≤ t for all i (i.e. s is 0 ).

Original: We protect the value of s by employing
Revised: We protect the value of s by employing

Original: MPC primitives for multiplication to simulate a conditional statement.
Revised: MPC primitives for multiplication to simulate a conditional statement.

Original: Description of π LAP .
Revised: Description of π LAP .

Original: For the measurement computed in line 3 of Alg. 1, we design π LAP (see Prot.3) to securely sample noise from from the Laplacian distribution and add to the secret sharing ofq i ( D ) .
Revised: For the measurement computed in line 3 of Alg. 1, we design π LAP (see Prot. 3) to securely sample noise from from the Laplacian distribution and add to the secret sharing of q i ( D ) .

Original: The noise is sampled as b · ln x · c where b = 2 T/(cid:15) is the privacy budget, x is a random valuedrawn from the uniform distribution in [0,1] and c is a random value selected from {− 1 , 1 } .
Revised: The noise is sampled as b · ln x · c where b = 2 T/(cid:15) is the privacy budget, x is a random value drawn from the uniform distribution in [0,1] and c is a random value selected from {− 1 , 1 } .

Original: On lines1–2, the parties straightforwardly compute x and its natural log.
Revised: On lines 1–2, the parties straightforwardly compute x and its natural log.

Original: To compute c, the parties, on line 3,generate secret shares of a random bit [[ r ]] , i.e. a value ∈ { 0 , 1 } is chosen, where each value has achance of 50% to be chosen.
Revised: To compute c, the parties, on line 3, generate secret shares of a random bit [[ r ]] , i.e. a value ∈ { 0 , 1 } is chosen, where each value has a chance of 50% to be chosen.

Original: On line 4, the parties transform r to a value ∈ {− 1 , 1 } using the logic c= 2 · r − 1 .
Revised: On line 4, the parties transform r to a value ∈ {− 1 , 1 } using the logic c = 2 · r − 1 .

Original: Line 5 is straightforward where the parties compute secret shares of measurement [[ m i ]]for the query q i , which is then made public for further computations in Alg. 
Revised: Line 5 is straightforward where the parties compute secret shares of measurement [[ m i ]] for the query q i , which is then made public for further computations in Alg. Remark.

Revised: We provide a proof-of-concept of our proposed framework through the MWEM algorithm, a well-known algorithm for releasing data for statistical analysis.

Revised: However, the proposed general framework can be adapted to other synthetic data generation techniques.

Original:  Experiments
Revised:  Experiments

Original: We evaluate MPC-MWEM against a centralized version of MWEM [17] using two publicly availabledatasets, namely the Car and Adult datasets, which have been featured in previous DP syntheticdataset generation analyses [17, 31].
Revised: We evaluate MPC-MWEM against a centralized version of MWEM [17] using two publicly available datasets, namely the Car and Adult datasets, which have been featured in previous DP synthetic dataset generation analyses [17, 31].

Original: In all the results below, centralized refers to the setting inwhich all data holders disclose their data to a central, trusted curator who runs the MWEM algorithmover all the data combined, while distributed refers to the setting in which the data holders secretshare their data with computing parties who run MPC protocols.
Revised: In all the results below, centralized refers to the setting in which all data holders disclose their data to a central, trusted curator who runs the MWEM algorithm over all the data combined, while distributed refers to the setting in which the data holders secret share their data with computing parties who run MPC protocols.

Original: The distributed setting protects theprivacy of the inputs, while the centralized setting does not.
Revised: The distributed setting protects the privacy of the inputs, while the centralized setting does not.

Original: The results for the centralized settingare obtained with an implementation of MWEM in SmartNoise [31].
Revised: The results for the centralized setting are obtained with an implementation of MWEM in SmartNoise [31].

Original: For the distributed setting, weimplemented our MPC protocols π QEM and π LAP in the MPC framework MP-SPDZ [19].
Revised: For the distributed setting, we implemented our MPC protocols π QEM and π LAP in the MPC framework MP-SPDZ [19].

Original: Experimental Settings.
Revised: Experimental Settings.

Original: We empirically validate the utility of the produced synthetic data andmeasure performance by training logistic regression (LR) models using synthetic data and testing themodels on real data, as in [31].
Revised: We empirically validate the utility of the produced synthetic data and measure performance by training logistic regression (LR) models using synthetic data and testing the models on real data, as in [31].

Original: We evaluate model performance using AUC-ROC.
Revised: We evaluate model performance using AUC-ROC.

Original: We compare theperformance of models trained on synthetic data generated in the centralized mode, and synthetic datagenerated in the distributed mode using MPC where the data is split horizontally across data holders.
Revised: We compare the performance of models trained on synthetic data generated in the centralized mode, and synthetic data generated in the distributed mode using MPC where the data is split horizontally across data holders.

Original: We repeat the comparison process for different privacy parameter values.
Revised: We repeat the comparison process for different privacy parameter values.

Original: We measure runtimes ofour method for different numbers of MWEM iterations T and compare with the centralized setting,while keeping other parameters constant.
Revised: We measure runtimes of our method for different numbers of MWEM iterations T and compare with the centralized setting, while keeping other parameters constant.

Original: 4 We use a maximum number of iterations of 1000 and otherdefault parameters of LR available in Scikit-learn [28] to train the models. 
Revised: 4 We use a maximum number of iterations of 1000 and other default parameters of LR available in Scikit-learn [28] to train the models. [Figure]

Original: Quantitative Analysis of Utility.
Revised: Quantitative Analysis of Utility.

Original: In Fig.
Revised: In Fig.

Original: 1, we investigate the trade-off between the privacy parameter (cid:15) and utility of models trained with synthetic data generated by the two different modes(centralized and distributed).
Revised: 1, we investigate the trade-off between the privacy parameter (cid:15) and utility of models trained with synthetic data generated by the two different modes (centralized and distributed).

Original: The models perform similarly in terms of AUC-ROC, for the differentvalues of (cid:15) .
Revised: The models perform similarly in terms of AUC-ROC, for the different values of (cid:15) .

Original: Additionally, the trends are also consistent for both datasets.
Revised: Additionally, the trends are also consistent for both datasets.

Original: In the experiments usingthe Car dataset we see a upward trend for both modes, whereas for the adult dataset we see a smallspike for (cid:15)
Revised: In the experiments using the Car dataset we see a upward trend for both modes, whereas for the adult dataset we see a small spike for (cid:15)

Original: = 1 for both modes.
Revised: = 1 for both modes.

Original: Based on similar trendlines for both settings, we conclude that MPCemulates the centralized mode of operation.
Revised: Based on similar trendlines for both settings, we conclude that MPC emulates the centralized mode of operation.

Original: The small differences observed in the plots are a resultof the noise introduced by the DP mechanisms.
Revised: The small differences observed in the plots are a result of the noise introduced by the DP mechanisms.

Original: The results in Fig.
Revised: The results in Fig.

Original: 1 are averaged over 10 runs.
Revised: 1 are averaged over 10 runs.

Original: Quantitative Analysis of Iterations and Runtime.
Revised: Quantitative Analysis of Iterations and Runtime.

Original: We measure runtime for different values of thenumber of iterations T , which is a hyperparameter of MWEM.Previous works have demonstrated [Figure] [Table]
Revised: We measure runtime for different values of the number of iterations T , which is a hyperparameter of MWEM.

Original: [Figure]

Original: [Table] [Figure] [Table] the trade-off between the number of iterations and quality of the synthetic data [17].
Revised: Previous works have demonstrated the trade-off between the number of iterations and quality of the synthetic data [17].

Original: Tab. 1 showsthe runtime for different choices of T averaged over 3 runs for the centralized setting and for thedistributed setting with 2, 3, and 4 computing parties.
Revised: Tab. 1 shows the runtime for different choices of T averaged over 3 runs for the centralized setting and for the distributed setting with 2, 3, and 4 computing parties.

Original: All MPC based computations were done inring Z q with q = 2 64 .
Revised: All MPC based computations were done in ring Z q with q = 2 64 .

Original: As observed, the runtimes increase with T .
Revised: As observed, the runtimes increase with T .

Original: We note that the runtimes furtherdepend on the dimensions of the datasets and the number of queries, as shown in [17].
Revised: We note that the runtimes further depend on the dimensions of the datasets and the number of queries, as shown in [17].

Original: The increasedruntimes for the distributed setting when compared to their corresponding centralized setting are dueto the runtimes of the MPC protocols.
Revised: The increased runtimes for the distributed setting when compared to their corresponding centralized setting are due to the runtimes of the MPC protocols.

Original: For example, in a 3PC passive security setting for | T | = 1 , eachcall to π QEM adds ∼ 0.74 secs for | Q | = 400 and π LAP adds ∼ 0.006 secs to the synthetic generationprocess.
Revised: For example, in a 3PC passive security setting for | T | = 1 , each call to π QEM adds ∼ 0.74 secs for | Q | = 400 and π LAP adds ∼ 0.006 secs to the synthetic generation process.

Original: The differences in runtime observed across different security settings are in line with existingliterature [11].
Revised: The differences in runtime observed across different security settings are in line with existing literature [11].

Original: All the experiments were run on Azure D8ads_v5 8 vCPUs, 32Gib RAM.
Revised: All the experiments were run on Azure D8ads_v5 8 vCPUs, 32Gib RAM.

Revised: The runtimes of our proposed method depend only on the computing parties and the threat model.

Revised: Our proposed MPC protocols can be made scalable and efﬁcient by replacing the appropriate and efﬁcient underlying MPC schemes.

Original: We note that our approach works for any partitioning of data – horizontal, vertical or mixed – withappropriate MPC protocols for any required preprocessing steps.
Revised: We note that our approach works for any partitioning of data – horizontal, vertical or mixed – with appropriate MPC protocols for any required preprocessing steps.

Original: We will present the details, furtheranalysis and experiments in the full version.
Revised: We will present the details, further analysis and experiments in the full version.

Revised: [Table]

Original:  Conclusion and Future Work
Revised:  Conclusion and Future Work

Original: In this paper we introduced and started the study of a novel approach for generating differentiallyprivate synthetic data from distributed databases based on MPC.
Revised: In this paper we introduced and started the study of a novel approach for generating differentially private synthetic data from distributed databases based on MPC.

Original: Our experiments show that utilizing
Revised: Our experiments show that utilizing

Original: MPC to emulate a central authority produces synthetic datasets with utility at par with data producedin a centralized fashion.
Revised: MPC to emulate a central authority produces synthetic datasets with utility at par with data produced in a centralized fashion.

Original: While the simplicity of MWEM makes it attractive to many applications andto adapting it to our framework, an important direction for future work is the development of efﬁcient
Revised: While the simplicity of MWEM makes it attractive to many applications and to adapting it to our framework, an important direction for future work is the development of efﬁcient

Original: MPC protocols for more recently introduced synthetic data generation techniques such as DPGANand DP-CGAN [33, 36], drawing inspiration from recent MPC protocols for training of deep neuralnetworks [20].
Revised: MPC protocols for more recently introduced synthetic data generation techniques such as DPGAN and DP-CGAN [33, 36], drawing inspiration from recent MPC protocols for training of deep neural networks [20].

Original: A Differential Privacy Mechanisms
Revised: A Differential Privacy Mechanisms

Original: The exponential mechanism provides the ability of making differentially private selections of the bestalternative in a discrete set K , where “best” is based on a scoring function.
Revised: The exponential mechanism provides the ability of making differentially private selections of the best alternative in a discrete set K , where “best” is based on a scoring function.

Original: Deﬁnition 1 Exponential Mechanism [23].
Revised: Deﬁnition 1 Exponential Mechanism [23].

Original: Let s : D × K → R be a quality scoring functionwhere s ( D, k ) denotes the quality of result k on dataset D and K is the set of possible results.
Revised: Let s : D × K → R be a quality scoring function where s ( D, k ) denotes the quality of result k on dataset D and K is the set of possible results.

Original: Theexponential mechanism E selects k from R such that the probability that a particular k is selected isproportional to exp( (cid:15) · s ( D, k ) / 2) .
Revised: The exponential mechanism E selects k from R such that the probability that a particular k is selected is proportional to exp( (cid:15) · s ( D, k ) / 2) .

Original: In other words, the exponential mechanism samples k from thedistribution satisfying
Revised: In other words, the exponential mechanism samples k from the distribution satisfying

Original: [Equation]
Revised: [Equation]

Original: To guarantee (cid:15) -differential privacy, the scoring function s is required to satisfy a stability property,where for each result k
Revised: To guarantee (cid:15) -differential privacy, the scoring function s is required to satisfy a stability property, where for each result k

Original: the difference | s ( D, k ) − s ( D (cid:48) , k ) | is at most the number of records thatwould have to be added or removed to change D to D (cid:48) .
Revised: the difference | s ( D, k ) − s ( D (cid:48) , k ) | is at most the number of records that would have to be added or removed to change D to D (cid:48) .

Original: In the MWEM algorithm, the set of “results” to be selected from at each iteration is the set of queries
Revised: In the MWEM algorithm, the set of “results” to be selected from at each iteration is the set of queries

Original: Q = { q 1 , q 2 , . .
Revised: Q = { q 1 , q 2 , . .

Original: . , q N } , and the value of the scoring function for query q i is s ( D, q i ) =
Revised: . , q N } , and the value of the scoring function for query q i is s ( D, q i ) =

Original: | q ( A ) − q i
Revised: | q ( A ) − q i

Original: ( D ) | ,i.e. the difference in the answer for query
Revised: ( D ) | , i.e. the difference in the answer for query

Original: q i when run over the approximate data A vs. when runover the real data D .
Revised: q i when run over the approximate data A vs. when run over the real data D .

Original: Alg.
Revised: Alg.

Original: 4 provides pseudocode for the exponential mechanism for query selectionin-the-clear, i.e. assuming that the data set D has been disclosed in its entirety to a central aggregator[17, 31].
Revised: 4 provides pseudocode for the exponential mechanism for query selection in-the-clear, i.e. assuming that the data set D has been disclosed in its entirety to a central aggregator [17, 31].

Original: Lines 1–6 generate the probability distribution over the set of results (queries) as per Eq.
Revised: Lines 1–6 generate the probability distribution over the set of results (queries) as per Eq.

Original: (1),while lines 8–13 sample a result (query).
Revised: (1), while lines 8–13 sample a result (query).

Original: , number of linear queries N , and privacy parameter (cid:15) (cid:48) .

Original: 1 // Compute the probability distribution over the set of queries 2 for i ← 1 to N do err [ i ] = 0 .
Revised: 1 // Compute the probability distribution over the set of queries 2 for i ← 1 to N do err [ i ] = 0 .

Original: 5 · (cid:15) (cid:48) · abs( q i ( A )
Revised: 5 · (cid:15) (cid:48) · abs( q i ( A )

Original: − q i ( D ) )
Revised: − q i ( D ) )

Original: //Note :
Revised: //Note :

Original: s ( D, q i ) =
Revised: s ( D, q i ) =

Original: | q i ( A ) − q i ( D ) | 4 max_err = max( err ) 5 for i ← 1 to N do err
Revised: | q i ( A ) − q i ( D ) | 4 max_err = max( err ) 5 for i ← 1 to N do err

Original: [ i ] = exp( err [ i ] − max_err)
Revised: [ i ] = exp( err [ i ] − max_err)

Original: Sample the query 8 e_s
Revised: Sample the query 8 e_s

Original: = (cid:80) Ni =

Revised: = (cid:80) Ni =

Original: ( err [ i ]) 9 r = random value drawn from uniform distribution in [0 , 1] 10 c =11 for i ← 1 to N do c
Revised: ( err [ i ]) 9 r = random value drawn from uniform distribution in [0 , 1] 10 c =11 for i ← 1 to N do c

Original: = c + err [ i ] if c > r · e_s then
Revised: = c + err [ i ] if c > r · e_s then

Original: Output: return i
Revised: Output: return i

Original: Output: return N
Revised: Output: return N

Original: Deﬁnition 2 l 1 -sensitivity.
Revised: Deﬁnition 2 l 1 -sensitivity.

Original: The l 1 -sensitivty of a function f : D → R is:
Revised: The l 1 -sensitivty of a function f : D → R is:

Original: [Equation]
Revised: [Equation]

Original: where D and D (cid:48) are neighboring databases.
Revised: where D and D (cid:48) are neighboring databases.

Original: We note that, for a linear query f , ∆ f = 1 .
Revised: We note that, for a linear query f , ∆ f = 1 .

Original: The Laplace distribution with 0 mean and scale λ , denoted by Lap ( λ ) , has a probability densityfunction Lap ( x | λ ) =
Revised: The Laplace distribution with 0 mean and scale λ , denoted by Lap ( λ ) , has a probability density function Lap ( x | λ ) =

Original: 12 λ e − xλ .
Revised: 12 λ e − xλ .

Original: It can be used to obtain an (cid:15)
Revised: It can be used to obtain an (cid:15)

Original: -differentially private algorithm to answernumeric queries [14].
Revised: -differentially private algorithm to answer numeric queries [14].

Original: Deﬁnition 3 Laplace Mechanism.
Revised: Deﬁnition 3 Laplace Mechanism.

Original: Let f : D
Revised: Let f : D

Original: → R be a numeric query.
Revised: → R be a numeric query.

Original: The Laplace mechanism isdeﬁned as:
Revised: The Laplace mechanism is deﬁned as:

Original: [Equation]
Revised: [Equation]

Original: where η is drawn from the Laplace distribution Lap ( ∆ f(cid:15) ) .
Revised: where η is drawn from the Laplace distribution Lap ( ∆ f(cid:15) ) .

Original: B Secure Multiparty Computation Primitives
Revised: B Secure Multiparty Computation Primitives

Original: The protocols in Sec.
Revised: The protocols in Sec.

Original: 3 are sufﬁciently generic to be used in dishonest-majority as well as honestmajority settings, with passive or active adversaries.
Revised: 3 are sufﬁciently generic to be used in dishonest-majority as well as honestmajority settings, with passive or active adversaries.

Original: This is achieved by changing the underlying
Revised: This is achieved by changing the underlying

Original: MPC scheme to align with the desired security setting.
Revised: MPC scheme to align with the desired security setting.

Original: In the MPC schemes used in Tab. 1, allcomputations are done on integers modulo q , i.e., in a ring Z q = { 0 , 1 , . . .
Revised: In the MPC schemes used in Tab. 1, all computations are done on integers modulo q , i.e., in a ring Z q = { 0 , 1 , . . .

Original: , q − 1 } , with q a power of2.
Revised: , q − 1 } , with q a power of 2.

Original: As is common in MPC, any input real values from the data holders are converted to integers usinga ﬁxed-point representation [7].
Revised: As is common in MPC, any input real values from the data holders are converted to integers using a ﬁxed-point representation [7].

Original: Below we give a high level description of the 3PC schemes used in
Revised: Below we give a high level description of the 3PC schemes used in

Original: Tab. 1.
Revised: Tab. 1.

Original: For more details and a description of the other MPC schemes, we refer to the papers about2PC [8], 3PC [3, 11], and 4PC [11].
Revised: For more details and a description of the other MPC schemes, we refer to the papers about 2PC [8], 3PC [3, 11], and 4PC [11].

Original: Replicated sharing (3PC).
Revised: Replicated sharing (3PC).

Original: In a replicated secret sharing scheme with 3 servers (3PC), a value xin Z q is secret shared among servers (parties) S 1 , S 2 , and S 3 by picking uniformly random sharesx 1 , x 2 , x 3 ∈ Z q such that x 1 + x 2 + x 3 =
Revised: In a replicated secret sharing scheme with 3 servers (3PC), a value x in Z q is secret shared among servers (parties) S 1 , S 2 , and S 3 by picking uniformly random shares x 1 , x 2 , x 3 ∈ Z q such that x 1 + x 2 + x 3 =

Original: x
Revised: x

Original: mod q , and distributing ( x 1 , x 2 ) to S 1 , ( x 2 , x 3 ) to S 2 ,and ( x 3 , x 1 ) to S 3 .
Revised: mod q , and distributing ( x 1 , x 2 ) to S 1 , ( x 2 , x 3 ) to S 2 , and ( x 3 , x 1 ) to S 3 .

Original: Note that no single server can obtain any information about x given its shares.
Revised: Note that no single server can obtain any information about x given its shares.

Original: We use [[ x ]] as a shorthand for a secret sharing of x .
Revised: We use [[ x ]] as a shorthand for a secret sharing of x .

Original: Passive security (3PC).
Revised: Passive security (3PC).

Original: The 3 servers can perform the following operations through carrying outlocal computations on their own shares: addition of a constant, addition of secret shared values,and multiplication by a constant.
Revised: The 3 servers can perform the following operations through carrying out local computations on their own shares: addition of a constant, addition of secret shared values, and multiplication by a constant.

Original: For multiplying secret shared values [[ x ]] and [[ y ]] , we have thatx · y = ( x 1 + x 2 + x 3 )( y 1 + y 2 + y 3 ) , and so S 1 computes z 1 = x 1 · y 1 + x 1 · y 2 + x 2 · y 1 , S 2computes z 2 = x 2 · y 2 + x 2 · y 3 + x 3 · y 2 and S 3 computes z 3 = x 3 · y 3 + x 3 · y 1 + x 1 · y 3 .
Revised: For multiplying secret shared values [[ x ]] and [[ y ]] , we have that x · y = ( x 1 + x 2 + x 3 )( y 1 + y 2 + y 3 ) , and so S 1 computes z 1 = x 1 · y 1 + x 1 · y 2 + x 2 · y 1 , Scomputes z 2 = x 2 · y 2 + x 2 · y 3 + x 3 · y 2 and S 3 computes z 3 = x 3 · y 3 + x 3 · y 1 + x 1 · y 3 .

Original: Next,the servers obtain an additive secret sharing of 0 by picking uniformly random u 1 , u 2 , u 3 such thatu 1 + u 2 + u 3 = 0 , which can be locally done with computational security by using pseudorandomfunctions, and S i locally computes v i =
Revised: Next, the servers obtain an additive secret sharing of 0 by picking uniformly random u 1 , u 2 , u 3 such that u 1 + u 2 + u 3 = 0 , which can be locally done with computational security by using pseudorandom functions, and S i locally computes v i =

Original: z i + u i .
Revised: z i + u i .

Original: Finally, S 1 sends v 1 to S 3 , S 2 sends v 2 to S 1 , and S 3sends v 3 to S 2 , enabling the servers S 1 , S 2 and S 3 to get the replicated secret shares ( v 1 , v 2 ) , ( v 2 , v 3 ) ,and ( v 3 , v 1 ) , respectively, of the value v = x · y .
Revised: Finally, S 1 sends v 1 to S 3 , S 2 sends v 2 to S 1 , and Ssends v 3 to S 2 , enabling the servers S 1 , S 2 and S 3 to get the replicated secret shares ( v 1 , v 2 ) , ( v 2 , v 3 ) , and ( v 3 , v 1 ) , respectively, of the value v = x · y .

Original: This protocol only requires each server to send asingle ring element to one other server, and no expensive public-key encryption operations (such ashomomorphic encryption or oblivious transfer) are required.
Revised: This protocol only requires each server to send a single ring element to one other server, and no expensive public-key encryption operations (such as homomorphic encryption or oblivious transfer) are required.

Original: This MPC scheme was introduced by
Revised: This MPC scheme was introduced by

Original: Araki et al.
Revised: Araki et al.

Original: Active security (3PC).
Revised: Active security (3PC).

Original: In the case of malicious adversaries, the servers are prevented from deviatingfrom the protocol and gain knowledge from another party through the use of information-theoreticmessage authentication codes (MACs).
Revised: In the case of malicious adversaries, the servers are prevented from deviating from the protocol and gain knowledge from another party through the use of information-theoretic message authentication codes (MACs).

Original: For every secret share, an authentication message is also sentto authenticate that each share has not been tampered in each communication between parties.
Revised: For every secret share, an authentication message is also sent to authenticate that each share has not been tampered in each communication between parties.

Original: Inaddition to computations over secret shares of the data, the servers also need to update the MACsappropriately, and the operations are more involved than in the passive security setting.
Revised: In addition to computations over secret shares of the data, the servers also need to update the MACs appropriately, and the operations are more involved than in the passive security setting.

Original: For eachmultiplication of secret shared values, the total amount of communication between the parties isgreater than in the passive case.
Revised: For each multiplication of secret shared values, the total amount of communication between the parties is greater than in the passive case.

Original: We use the MPC scheme SPDZ-wiseReplicated2k recentlyproposed by Dalskov et al.
Revised: We use the MPC scheme SPDZ-wiseReplicated2k recently proposed by Dalskov et al.

Original: [11] that is available in MP-SPDZ [19].
Revised: [11] that is available in MP-SPDZ [19].

Original: MPC primitives.
Revised: MPC primitives.

Original: The MPC schemes listed above provide a mechanism for the servers to performcryptographic primitives through the use of secret shares, namely addition of a constant, multiplicationby a constant, and addition of secret shared values, and multiplication of secret shared values (denotedas π MUL ).
Revised: The MPC schemes listed above provide a mechanism for the servers to perform cryptographic primitives through the use of secret shares, namely addition of a constant, multiplication by a constant, and addition of secret shared values, and multiplication of secret shared values (denoted as π MUL ).

Original: Building on these cryptographic primitives, MPC protocols for other operations have beendeveloped in the literature.
Revised: Building on these cryptographic primitives, MPC protocols for other operations have been developed in the literature.

Original: We use [19]:
Revised: We use [19]

Original:  • Secure random number generation from uniform distribution π GR − RANDOM :
Revised: : • Secure random number generation from uniform distribution π GR − RANDOM :

Original: In π GR − RANDOM , eachparty generates l random bits, where l is the fractional precision of the power 2 ring representation ofreal numbers, and then the parties deﬁne the bitwise XOR of these l bits as the binary representationof the random number jointly generated.• Secure random bit generation π GR − RNDM − BIT : In π GR − RNDM − BIT , each party generates the secretshare of a single random bit, such that the generated bit is either 0 or 1 with a probability of 0.5.• Secure equality test π EQ : At the start of this protocol, the parties have secret sharings [[ x ]] ; at theend if x = 0 , then they have a secret share of 1 , else a secret sharing of 0 .•
Revised: In π GR − RANDOM , each party generates l random bits, where l is the fractional precision of the power 2 ring representation of real numbers, and then the parties deﬁne the bitwise XOR of these l bits as the binary representation of the random number jointly generated. • Secure random bit generation π GR − RNDM − BIT : In π GR − RNDM − BIT , each party generates the secret share of a single random bit, such that the generated bit is either 0 or 1 with a probability of 0.5. • Secure equality test π EQ : At the start of this protocol, the parties have secret sharings [[ x ]] ; at the end if x = 0 , then they have a secret share of 1 , else a secret sharing of 0 .

Original:  Secure less than test π LT : At the start of this protocol, the parties have secret sharings [[ x ]] and [[ y ]]of integers x and y ; at the end of the protocol they have a secret sharing of 1 if x < y , and a secretsharing of 0 otherwise.•
Revised: • Secure less than test π LT : At the start of this protocol, the parties have secret sharings [[ x ]] and [[ y ]] of integers x and y ; at the end of the protocol they have a secret sharing of 1 if x < y , and a secret sharing of 0 otherwise.

Original:  Secure greater than test π GT :
Revised: • Secure greater than test π GT :

Original: At the start of this protocol, the parties have secret sharings [[ x ]] and[[ y ]] of integers x and y ; at the end of the protocol they have a secret sharing of 1 if x > y , and asecret sharing of 0 otherwise.
Revised: At the start of this protocol, the parties have secret sharings [[ x ]] and [[ y ]] of integers x and y ; at the end of the protocol they have a secret sharing of 1 if x > y , and a secret sharing of 0 otherwise.

Original: • Other primitives : We use secure maximum protocol ( π MAX ), secure exponential protocol ( π EXP )and secure logarithm protocol ( π LN ) as the building blocks for our protocols.
Revised: • Other primitives : We use secure maximum protocol ( π MAX ), secure exponential protocol ( π EXP ) and secure logarithm protocol ( π LN ) as the building blocks for our protocols.

Original: π LN uses the polynomial expansion for computing logarithm and π EXP in turn uses the π LN to compute exponential.π MAX inherently uses the π GT repeatedly over a list by employing variant of Divide-n-Conquerapproach.
Revised: π LN uses the polynomial expansion for computing logarithm and π EXP in turn uses the π LN to compute exponential. π MAX inherently uses the π GT repeatedly over a list by employing variant of Divide-n-Conquer approach.

Original: At the start of all of these primitives, parties hold the secret sharings [[ x ]] and at the endof the protocol they hold the secret shares of the corresponding computed values.
Revised: At the start of all of these primitives, parties hold the secret sharings [[ x ]] and at the end of the protocol they hold the secret shares of the corresponding computed values.

Original: MPC protocols can be mathematically proven to guarantee privacy and correctness.
Revised: MPC protocols can be mathematically proven to guarantee privacy and correctness.

Original: We follow theuniversal composition theorem that allows modular design where the protocols remain secure even ifcomposed with other or the same MPC protocols [6].
Revised: We follow the universal composition theorem that allows modular design where the protocols remain secure even if composed with other or the same MPC protocols [6].

Original: Implementing DP in MPC.
Revised: Implementing DP in MPC.

Original: Keeping in mind the dangers of implementing DP with ﬂoating point arithmetic [24] we stick with the best practice of using ﬁxed-point and integer arithmetic as recommended by, for example, OpenDP 5 .
Revised: Keeping in mind the dangers of implementing DP with ﬂoating point arithmetic [24], we stick with the best practice of using ﬁxed-point and integer arithmetic as recommended by, for example, OpenDP 5 .

Original: We implement all our DP mechanism using their discrete representations and use 32 bit precision to ensure correctness.
Revised: We implement all our DP mechanism using their discrete representations and use 32 bit precision to ensure correctness.

Revised: We also remark that ﬁnite precision issues can also impact the exponential mechanism.

Revised: If the utility of one of the classes in the exponential mechanism collapses to zero after the mapping into ﬁnite precision, pure DP becomes impossible to achieve.

Revised: We can deal with such situation by using approximate DP for an appropriate value of δ .

Revised: Such situation did not happen with the data sets used in our experiments.