SecureFedYJ: a safe feature Gaussianization protocol for Federated Learning
Authors: Tanguy Marchand
Boris Muzellec
Constance Béguier
Jean Du Terrail
Mathieu Andreux
Venue: NeurIPS 2022 Accept (2022)

Abstract: The Yeo-Johnson (YJ) transformation is a standard parametrized per-feature unidimensional transformation often used to Gaussianize features in machine learning. In this paper, we investigate the problem of applying the YJ transformation in a cross-silo Federated Learning setting under privacy constraints. For the first time, we prove that the YJ negative log-likelihood is in fact convex, which allows us to optimize it with exponential search. We numerically show that the resulting algorithm is more stable than the state-of-the-art approach based on the Brent minimization method. Building on this simple algorithm and Secure Multiparty Computation routines, we propose SECUREFEDYJ, a federated algorithm that performs a pooled-equivalent YJ transformation without leaking more information than the final fitted parameters do. Quantitative experiments on real data demonstrate that, in addition to being secure, our approach reliably normalizes features across silos as well as if data were pooled, making it a viable approach for safe federated feature Gaussianization.

SecureFedYJ:: a safe feature Gaussianization protocol for Federated Learning
Tanguy Marchand Owkin Inc., New York, USA. tanguy. caarchandwwin..coo
Boris Muzellec Owkin Inc., New York, USA. boris boris. www.mmlecloooii....mmmmmmm com
Constance Beguier*
Jean Ogier du Terrail Owkin Inc., New York, USA. j jean. jean.du-terrai www.errrlroooinncoooo com
Mathieu Andreux
Owkin Inc., New York, USA. mathieu. andreux@owkin com
Abstract
The Yeo-Johnson (YJ) transformation is a standard parametrized per-feature unidi- mensional transformation often used Gaussianize features in machine learning. In this paper, we investigate the problem of applying the YJ transformation in a cross-silo Federated Learning setting under privacy constraints. For the first time, we prove that the YJ negative log-likelihood is in fact convex, which allows us optimize it with exponential search. We numerically show that the resulting algorithm is more stable than the state-of-the-arr approach based on the Brent minimization method. Building on this simple algorithm and Secure Multiparty Computation routines, we propose SECUREFEDYJ, a federated algorithm that performs a pooled-equivalent YJ transformation without leaking more information than the final fitted parameters do. Quantitative experiments on real data demon- strate that, addition being secure, our approach reliably normalizes features across silos well as if data were pooled, making it viable approach for safe federated feature Gaussianization.
Introduction
approach that was recently proposed to train machine learning (ML) models across multiple data holders, or clients, without centralizing data points, notably for privacy reasons. While many FL applications have been proposed, main settings have emerged [ cross-device FL, involving large number of small edge devices, and cross-silo FL, dealing with smaller number of clients, with larger computational capabilities. Due the sensitivity and relative local scarcity of medical data, healthcare is promising application of cross-silo FL 40 to train biomedical MI. model between different hospitals as all the datasets were pooled in central server. In this paper, we focus on the cross-silo silo setting.
Federated Learning (FL)
55 32
The constraints of cross-silo FL Although cross-silo FL resembles standard distributed learning, it faces least two important distinct challenges: privacy and heterogeneity. Due to data sensitivity, clients might impose stringent security and privacy constraints on FL. collaborations. This arises in coopetitive FL projects, where models are jointly trained industrial competitors' datasets (55). as well as medical FL applications, where conservative data regulations might apply. In this setting, using standard FL algorithms such FEDAVG 32 might not provide enough privacy guarantees, as privacy attacks such as data reconstruction can be carried out based on the clients' gradients [56] 54
"Contribution done while at Owkin, Inc.
36th Conference on Neural Information Processing Systems (NcurIPS 2022).

---

Various protocols based on Secure Multiparty Computation (SMC) (see Section 2 for more details), such as Secure Aggregation E can mitigate this shortcoming by disclosing only the sum of the gradients from all clients to the server, without disclosing each gradient individually.
An additional constraint is that data might present statistical heterogeneity across clients, i.e. the local clients' data distributions may not be identical. In the case of medical applications, such heterogeneity may be caused e.g. by environmental variations or differences in the material that was used for acquisition - 47 21 While different ways of adapting federated training algorithmss have these solutions do not address data
been proposed to automatically tackle heterogeneity [28 20 2
harmonization and normalization prior to FL training.
Preprocessing in ML Data preprocessing is a crucial step in many ML applications, leading to important performance gains, Among others, common preprocessing methods include data whitening, principal component analysis (PCA) [22] or zero component analysis 22 20 46]. However, linear normalization methods might not suffice when the original data distribution highly non-Gaussian. For tabular and time series data, popular approach to Gaussianize the marginal distributions is to apply feature- wise non-linea transformations. Two commonly-un Box-Cox [ transformation and its extension, the Yeo-Johnson (YJ) transformation [52]. Both have
parametric methods are the
been used in multiple applications, such as climate and weather forecast (53 50
economics [k]
and genomic studies 7 58 9.
Problem and contributions In this paper, we investigate the problem of data normalization in the cross-silo FL setting, by exploring how to apply the YJ transformation to distributed dataset. This problem arises frequently in medical cross-silo FL. e.g. when trying to jointly train models on genetic data (see e.g. [1 (7)). Due to data heterogeneity, client: indeed, there is no guarantee that transformation parameters fitted on a single client would be relevant for other clients' data. Hence, it is necessary to fit normalization methods on the full federated dataset. Moreover, this setting, data privacy is of paramount importance, and therefore FL protocols should be carefully designed. Our main contributions this problem
single client can act reference
follows:
1. We prove that the negative YJ log-likelihood is convex (Section B which is novel result, to the best our knowledge.
2. Building this property, we introduce ExPYJ, method to fit the YJ transformation based on exponential search (Section 3). We numerically show that this method is more stable than standard approaches for fitting the YJ transformation based on the Brent minimization method [6].
3. We propose SECUREFEDYJ (Section 4) secure way extend EXPYJ in the cross-silo FL setting using SMC. We show that SECUREFEDYJ does not leak any information on the datasets apart from what is leaked by the parameters minimizing the YJ negative log- likelihood (Section 3 same results as the pooled-equivalent EXPYJ, regardless of how the data is split across the clients. We check this property in numerical experiments (Section $ The core ideas behind the resulting algorithm, SECUREFEDYJ, are summarised in Figure
and Proposition I. By construction, SECUREFEDY provides the
Finally, we illustrate contributions in numerical applications on synthetic and genomic data in
Section 5
Background
The Yeo-Johnson transformation The YJ transformation 1 was introduced in order to Gaus- sianize data that can be either positive or negative. It was proposed as a generalization of the Box-Cox transformation [3], that only applies to non-negative data. The YJ transformation consists in applying to each feature a monotonic function V (AA parametrized by a scalar X, independently of the other features. Thus, there are as many X as there are features. For real number x, ( .r) is defined as:
(1 + 1)a 11/2, 11/2,
if 2 0, 0,
In(x + 1),
if 0,x 0, if < 0,
V (A x
(I)
-I(-x + 1)2-> 11/(2 A)),
In(- + 1),
if < 1, 2

---

0.4
1=00
Yeo-Johnson
0.2
0.1
0.0
0.0
-2
4/A,
(a) V (A, .) for various A
(b) The YJ transformation applied skew-normal distribution
Figure 1: The Yeo-Johnson transformation applies 1-D univariate transform to Gaussianize data.
Figure la shows the shape of the YJ function for various values of X.
The Yco-Johnson likelihood Let us consider real-valued samples (=,) and let us apply the YJ transformation (X, these samples Gaussianize their distribution. The log-likelihood that { (1, comes from a Gaussian with mean μ and variance o² is given by (derivation details provided in Appendix A.I:
n
log CrJ(Cc, 1)) Log((TTo))
(A), - + - Σ sgn(x;) log(l=;|+ +1).
20
For given X, the log-likelihood is maximized for E A) p.)²² Once we replace and o² by Ma and o², becomes:
a - ((A, and of
n log (A) = log log(o + (A- 1) E sgn(z;) log(1z;| + +1 log(2
"
(2)
see (52] Maximizing the YJ log-likelihood is therefore a I-dimensional problem for each feature.
Once the optimal A, is found, the transformed data V (()) is usually renormalized by subtracting its empirical mean μ4, and dividing by the square root its empirical variance o². Figure Ib shows an example of the YJ transformation applied to skew-norms distribution. Note that in typical application, the triplet (A., pa, o.)) is fitted on the training data only, and is then used to Gaussianize the test dataset during inference.
Minimization methods in dimension 1 As seen above, fitting a YJ transformation can be reduced to ID optimization problem. To tackle this problem, introduce two standard ID minimization methods: (i) Brent minimization
[6] and (ii) exponential search [
Brent minimization 6 (not to be confused with the Brent-Dekker method, see 6, chapters 3 and 4) based on golden section search and successive
widely used method for ID optimization. parabolic interpolations, and does not require evaluating any derivatives. This algorithm is guaranteed to converge to a local minimum with superlinear convergence of order at least 1.3247. Standard implementations of the YJ transformation, in particular the scikit-learn implementation [36], based on the Brent minimization method to minimize the negative log-likelihood provided by Eq.
Exponential search G is dichotomic algorithm designed for unbounded search spaces. The idea is first find bounds, and then to perform a classic binary search within these bounds. This algorithm can be used to find the minimum of convex differentiable functions with linear convergence, as explained in Appendix B In this work, we build on exponential search to propose a federated version of the YJ transform, for two main reasons: (i) it is more numerically stable than Brent minimization, as shown in Section and Figure 2 (ii), may conveniently be adapted federated setting, as shown in Section and (iii,, this latter federated adaptation offers strong privacy garantees, as shown by Proposition
Secure Multiparty Computation As illustrated by various privacy gradient attacks 15
54
sensitive information on the clients" datasets can be leaked the central server during FL training. One way to mitigate this risk is to use Secure Multiparty Computation (SMC) protocols to hide

---

individual contributions to the server. SMC enables one to evaluate functions with inputs distributed across different users without revealing intermediate results and is often based on secret sharing. SMC protocols tailored for MI. use-cases have been recently proposed 212
33 49
These protocols are either designed to enhance the privacy of FI. trainings, or to perform secure inference, i.e. to enable the evaluation of model trained privately on server without revealing the data nor the model.
A popular FL algorithm relying on SMC is Secure Aggregation (SA) Schematically, in SA each client adds a random mask their model update before sending the central server. These masks have been tailored in such a way that they all together sum to zero. Therefore, the central server cannot see the individual updates of the clients, but it can recover the sum of these updates by adding all the masked quantities sent from them.
More generally, SMC routine schematically works as follows (we refer Appendix D for further details). Let us consider the setting where K parties k = 1,..., K want to compute 9 = (K) denote private inputs. Each party knows (*) and is not willing to share it. During the first step, secret sharing, each party splits its private input (*) into K secret shares 11 ....,,,,,, 1..,,
fh(") ..... h (x) for a known function f. where
(k) and sends the shares hy to the party k'. These <
secret shares are constructed in such a way that (i) knowing (k) does not provide any information on the value of h (8), and (ii) (R) can be reconstructed from the vector (b1 ...., . (k)) For simplicity, we denote [h [ = (h(*), . (k))
hxx the vector of share secrets. In second step, the computation, each party k' computes the quantity denoted gk" using the secret shares they know along with intermediate quantities exchanged with the other parties. The way to compute Ik' depends on ʃ and on the SMC protocol that is used, and is chosen so that g = f(h
((()) can be reconstructed from (91..... (2.)...))..... Said otherwise, 9k' are secret shares of g: [g] = (g1,... (g .K)) Finally, during the reveal step, each party & reveals Ile to all other parties, and each party can reconstruct from (91....9k) 9
Threat model In this work, we consider an honest-but-curiout setting 35 the server will deviate from the agreed protocol, but each party can potentially try infer as much information as possible using data they see during the protocol. This setting is relevant for cross-silo FL, where participants are often large institutions whose reputation could be ternished by a more malicious behaviour.
Neither the clients nor
3 A novel method to optimize the Yeo-Johnson log-likelihood: EXPYJ
In this section, we leverage the convex- ity of the negative log-likelihood of the YJ transformation (see Proposition 3.Ip to propose
Algorithm 1 EXPYJ
Input: data 2;, total data size n, number of steps fmax
method to find the opti- mal X, using exponential search. While this method only offers linear conver- gence, compared to the super-linear con- vergence of Brent minimization method, we demonstrate two of its advantages: (i) is more numerically stable, and (ii) it is easily amenable to an FL setting with strong privacy guarantees. The method proposed in this section is based on the following result.
Initialize Ar-o +- O, A +- co, Ai=o + -00
Compute S,p
for / = to /max
for g E (V -), (A, 2, 1,
Compute S,
end for A = =gn nS - 2Sy 2S, Seea 22.
Nr, X t E Ae)
end for
X + Atmax
Compute n4_ Sv n and 0² = Syz/nn /n - Output: The fitted triplet (Na, Me, o?!)
Proposition 3.1. The negative log- is
likehood X + - log CYJ(A) strictly convex.
The proof of Proposition 3.T builds upon the work of (26 which shows that the negative log-likelihood of the Box-Cox transformation [5] is convex. The complete proof is deferred to Appendix C
4

---

The exponential YJ algorithm The pseudo-code of the proposed algorithm is presented in Al- gorithm []] and relies on the exponential search presented in Algorithm 2 (cf Appendix B for more details on exponential search). An illustration of EXPYJ is shown in Figure 6 in Appendix B Due to the strict convexity of the negative log-likelihood of the YJ transformation, we may perform the exponential search described in Section 2 and Appendix B To do so, it is enough to obtain the sign of the derivative. Let Øλ (A, )²² = ( (Ax,)))AVV (A, and (o(x) = sgn(x) + log([ær) + 1). Further,
for g {* 0₂ (A, -), (A, -)2 0x (A, .). sp), let us define S. def Σ g(z)). The derivative
of the log-likelihood is available in closed form (see Appendix A.33 A.3)
nu Sav 2(S Sav )/n Sp.
0x log LyJ
2 S4a - S
Notice that Sya - S.yt /n can be expressed as a variance, hence is non-negative. We may therefore obtain sgn [Ox log CyJ] while avoiding performing division by computing
sgn [0x log LYJ] = sgn [n.S 25 Sow (S s4/n]]
(3)
Avoiding this division is erucial make the overall procedure more numerical stable, as explained below, and eases the use of SMC routines.
Median
25000
cg
.... Max
Brent
Exp()
o²
20000
10-1
.oge
15000
10
10000
10-4
5000
40
10
Number iterations Meaad
Figure 2: Comparison of EXPYJ and scikit-learn. Left: For each of the 106 features (see Ap- pendix E. we compute the relative difference dA = |AxxPY] - Aaxl/|Askl and plot its median, maximum and 25%-75% and 10%-90% percentiles across the 106 features. Right: Negative log- likehood of the transformation for the mean area of the cell of each sample of the Breast Cancer dataset. Full orange bars correspond to values of À for which the likelihood computed using scikit- learn returns oo as o ({x,)) is equal to 0 up to float-64 machine precision. Dotted lines correspond to the A. found using Brent minimization or EXPYJ with one client.
We check the accuracy of ExPYJ on the datasets presented in Appendix E. In particular, compare the results provided by EXPYJ with the outputs of the scikit- learn algorithm based on Brent minimization.
Accuracy of EXPYJ
Algorithm EXPUPDATE
Input: X, A+ E {-1,1}
if A then
x - À - (X+ 2) if x+ <
For 2 of the 108 features present the datasets, the seikit- learn implementation leads to numerical instabilities discussed hereafter. Therefore, we focus our comparison on the 106 remaining features, that we aggregated regardless of the dataset. reuurtts the relative difference ÅA between the results obtained by EXPYJ and by the scikit-learn implementation as a function of the number iteration mmx (as defined in Algorithm T. These results show that this relative difference is of order less than 10 when fmax = 40.
else X t max 22, 1)
else
A+ -
À A (A)+))/ else X + min (2), -1)
m22,
end if Output: Updated X, X+,X
Numerical stability of EXPYJ Our experiments demonstrate that EXPYJ is numerically more stable than Brent minimization. Indeed, for some values of and some datasets {arz), the transforma- tion ( concentrates all data points in a small interval such that the values of (A, (2+3) are all
5

---

equal up to machine precision. In that case, the log- likelihood is not well-defined and the term log o takes the value -o, which prevents Brent minimization from converging. This phenomenon does not affect the EXPYJ routine as we do not compute directly the sign of 0, = 0xa 2 i / Σ, (xx))
but rather the sign of o% 0xC - oē, ((z)), see Eq.
Figure 2 iillustrates this in the case of a feature of the Breast Cancer Dataset. The X, returned by the Brent minimization method of scikit-lear is - 14.53 while the minimizer of the negative log- logllikelhhood found by the EXPYJ is -0.21. particular, Figure shows the values of the negative log- og-likelihood as a function of À computed using 64-bit float precision. The orange vertical full bands correspond to values for which o2 is zero within the machine precision, resulting to a negative log-likelihood of oo. This instability happens for 2 of the 108 features used in numerical experiments, where blindly applying the Brent-based YJ transformation leads to all data points collapsing to zero, while EXPYJ succeeds in transforming the data distributions to more Gaussian-like ones. Appendix E.4 shows that this issue also arises in other real-life datasets.
4 Applying the Yeo-Johnson transformation in FL
So far, only considered the centralized setting, where data is accessible from single server. Yet, as mentioned in Sections D [ many real-world situations require working with heterogeneous data split between different centers, and to take privacy constraints into account. When the data is split across centers h = 1,...,K K and the function to optimize is separable, i.e. of the form F(A) = Exm (A) where each f& can be computed from data present in the center & exclusively, Federated Learning techniques were recently proposed. In short, they consist in repeatedly performing a few rounds of local optimization in each center, before aggregating local parameters in the server. We refer to 23 for an overview of recent. advances in FI.. In our case, however, the YJ negative log-likelihood
is not separable, due to the log-variance term. Indeed, turning the variance into a separable term would require sharing the global YJ mean μr to all centers at each iteration. Compared to the method we propose in this section, this would lead to more privacy leakage.
We now introduce SECUREFEDYJ, secure federated algorithm that builds upon EXPYJ to apply YJ transformations. This algorithm satisfies the two following properties: (i) it is yields the same results as if the data were freely accessible from a single server, and (ii) it leaks as little information as possible about the underlying datasets, as shown by Proposition 4.1 Finally, converges in limited number of iterations, thanks to the linear convergence of the underlying exponential search.
i.c.
SECURFFEDYJ SECUREFEDYJ is federated adaptation of EXPYJ presented in Section 3 to find the best parameters (A,, o²) of the YJ transformation when training datasets are split across different clients. It relies on SMC to ensure that only the final triplet (X., Me, o?) fitted on the training datasets is revealed, without leaking any other information apart from the overall total number of training samples n. Indeed, at each intermediate step, only the sign of 0₂ log LyJ is revealed, and the mean and variance of the transformed data is only revealed at the last step. The pseudo-code of the resulting algorithm is presented in Algorithm Bl and relies the exponential search presented in Algorithm 2 A functional representation of SECUREFEDYJ is displayed Figure 7
In Algorithm 3 we label the clients by k = 1...., K and each client h holds data {IK.; i
1....,na}, nn.. We suppose that the total number of samples n Σ Tig is public and shared to all clients. For a given function g, we denote Sp.g the sum Sp.g Σn g(xr,.)) As introduced in Section 2 we use double brackets [-]] to indicate an SMC secret shared across the clients (see
def
Appendix for more details).
Privacy leakage In Proposition we show that Algorithm Jnly reveals information already
contained in the fitted triplet (Ax,, o.). In comparison, turning the YJ negative ligeeiiid Q into a log-separable function before using off-the-sher FL methods would require sharing μ and its gradient and centrally computing o² for intermediate values of X at each iteration. This could
potentially lead to uncontrolled privacy leakage.
Proposition 4.1. The fitted parameter X, contains all the information revealed during the inter- mediate steps of SECUREFEDYJ. More precisely, there exists deterministic function F such

---

Algorithm 3 SECUREFEDY!
Input:
Data {I J. total data size number of steps max
Notations: [|]] indicates SMC secret shared across clients. Any operation such as S(I-D: II- ...)) where f can be the sum, product, or the sign, designs an SMC routine across the
[:]
clients as described in Appendix [D.5]
Initialize λ 0, X <- X F independently each client
Clients compute in SMC [S = [
for t = 1 to & for E (1 -,
0g v A, :) 0. .) Clients compute in SMC [S,] Σx (k..gl end for
Clients compute in SMC [A.] sgn [n][ 2[[5o[[Sbo] [[[ [S=12/n) [2/)]]
Clients reveal A,
Xe : EXPUPDATE() (
A,) independently each client
end for
A_ Atmax
Clients compute in SMC [u] = I [kww\/n and [o²] = I v=l/n -1=2
Clients reveal ph <- and a +- o²
Output: The fitted triplet (Na,)lo; o
10a
Median Max
Median
109
Max
10-2
10
10a4
10-1
8
X
1014.
10
10
10
10-10
20
25
30
35
40
45
50
20
25
30
35
40
45
50
Decimal precision
Decimal Decimall precision
Homogeneous
(h) Heterogenenus
Figure 3: Comparison of SECUREFEDYJ and JXPYJ for various fixed-point floating precisions used SMC, with f + and fmax = 40. The data distributed across 10 clients, either randomly (homogeneous, left), per decile (heterogeneous, right, i.e. each client gets one decile of the data). report the maximum, median, 25%-75% and 10%-90% percentiles of the relative error 5A \ASECPEdY] > across the 108 features described in Appendix E.1
that for any set of datasets {æu,)) if (( is the result of SECUREFEDY] { then (Xe, A7 Ar)t=1,.. Aamas =
Proof. This proposition comes from the fact that all gradient signs A, revealed during the algorithm can retrospectively inferred from A.. Indeed, 0 log LYJ < 0 for A and Ōg log LYJ > for À A.. Besides, the successive values of X; explored each step can be deterministically inferred from the initial value λz=o and and the final fitted value X, We construct such a function. F and numerical verify this proposition in Appendix
Performance of SECUREFEDYJ We implement SECUREFEDY] in Python, using the MPyC library 12 based on Shamir Secret Sharing 44. We refer to Appendix for more details on our implementation. To represent signed real-valued numbers in an SMC protocol, representation (see Appendix D.2 using / bits, among which f bits are used for the decimal parts. This means that consider floats ranging from 2 to 2 and that we have absolute precision 2 in our computations.
use exed-point
In order to ensure the accuracy of SECUREFEDY] results, we need to make sure that 1 and f are large enough. Figure shows the accuracy of SECUREFEDY] when compared to ExPYJ for various values of According to these numerical experiments, taking f = 50 and 100 provides reasonably
7

---

log(1 + u) accurate results. Moreover, by construction, the outputs of SECUREFEDY] do not depend on how the data split across the clients, up to rounding numerical errors. Therefore this algorithm is resilient to data heterogeneity, as long as the numerical decimal precision f is large enough, as shown in Figure
Performing SECUREFEDYJ with mmax = 40 takes 726 rounds of communication (see Appendix D.6). During these communication rounds, each client sends overall about Mb per feature to every other client (see also Appendix (.6). SECUREFEDY: can be applied independently and in parallel to each feature. Therefore, the overall number of rounds does not depend on the number of features being considered, and the communication costs grow proportionally to the number of features. In a realistic cross-silo FL setting as described in [9]] the bandwidth of the network is 1 Gb per second with delay of 20 ms between every two clients. In this context, the execution of SECUREFEDY! with max = 40 on p features would take about 726 x 20 ms = 15 8 due to the communication overhead, in addition to 8 Mb/1 Gbps = 8p ms due to the bandwidth. This shows that SECUREFEdY? is indeed real-world scenario.
viable algorithm
the binary search in the exponential search can be replaced by k-ary search. In such a setting, the sign of the negative log-likelihood of the YJ transformation is computed for k -- different values of À at each round. Such a modification would reduce the number of communication rounds required obtain given accuracy, while increasing the size of the data exchanged over the network at each round.
As pointed out in Appendix
B
5 Applications
Genomic data: TCGA We start by showing the benefits of YJ preprocessing in survival analysis experiments on lung (LUAD+LUSC), pancreas (PAAD), and colorectal (CRC) cancers. Given gene pression raw counts (features) and censored survival data (responses) from patients hav- ing either of those three cancers, we aim to fit a Cox Proportional Hazards (CoxPH) model ICI with the highest possible con- cordance index (C-index) 251. which mea- sures how well patients are ranked with respect to their survival times. We refer to (25) for a more thorough introduction to survival analysis. In Figure I we compare three different preprocessing methods: (i) whitening, (ii) log normalization, and (iii) YJ, each followed by PCA dimensionality
0.75
Whitering
Log
0.70
Y)
C-index
0.35
0.50
0.45
LUAD i LUSC
PAAD
CRC
Cross-validation survival analysis perfor- mance (higher is better) of CoxPH model with dif- ferent normalization methods. The YJ transformation yields either a better par performance, and further stabilizes results compared to other approaches.
Figure 4:
reduction step. More precisely, whitening (i) consists in centering and reducing to unit variance the to- tal read counts of all genes across all samples, log normalization consists in applying to raw read counts before applying global whitening, and YJ is a global YJ transform on the total read counts. We then evaluate each strategy using 5-fold cross-validation with 5 different seeds. We refer to Appendix F222 for experimental details. While this experiment is performed in a pooled environment, note that, importantly, each step has federated pooled-equivalent version: apart from the proposed SECUREFEDYJ for YJ, see e.g. (21) for PCA, and Webdisco [31] for Cox model fitting. This simplified setting allows us to understand the importance of the Yeo-Johnson transformation in an ideal setting, independently of other potential downstream federated learning artifacts.
In Figure 4 we see that YJ is better prediction results for colorectal cancer, while yielding results which are on par with the best results for lung and pancreas cancers, with smaller variance.
par with the best method for each cancer: YJ improves
Synthetic data We show how applying YJ may help improving performance compared no or basic preprocessing, and how SECUREFEDYJ yiclds improvements compared to local YJ transforms

---

0.9
0.98-
0.8
0.96
0.7
R2 score
0.94
R2
0.5
0.90-
0.4
0.88
Local
Federated YJ
None
Whitening
Local Yj
Federated Y]
10
Number of samples per center
Figure 5: Comparison of different preprocessing methods for linear regression on synthetic federated data. Left: performance with 200 samples (20 on each the 10 centers) on 1000 independent draws, showing the interest of using YJ preprocessing. The R2 score of the models are computed another dataset of 200 samples not seen during the training. Right: Comparison of local and federated YJ over 1000 independent draws. In local YJ, single center is randomly chosen to fit A, which is then shared with other centers, to ensure that the same transformation is applied everywhere. Full lines correspond to the means, error bars + std of the R2 scores of the model unseen test dataset.
in federated linear regression. To do we generate covariates X and responses
X (exp(r1), + 2), (2z)) with X 22,, (0, y - 3" + with s ~ _/ 0.1),
(4)
where (-) is the sigmoid function and B 211,3, 0.87) was randomly chosen. The goal is then to fit linear model from i.i.d. samples (X;, Ni), 1,..., following 4 after an optional preprocessing step. Similarly to the previous example, for the preprocessing steps, and the linear model is then fitted in a pooled setting for simplicity. All the details of the numerical experiments are provided in Appendix
simulate cross-device FL setting only
We suppose that the
samples (X,, i = I,..., n are homogeneously split across 10 centers. The responses y/i have a highly nonlinear dependency on the covariates X,, but depend linearly on the X;'s (up to Gaussian noise), which not observed. Hence, we expect that applying suitable preprocessing step before training a linear model will transform back the X, 's into the normally-distribbe high performance, compared to no transformation. The results of our experiments are summarized Figure 5 The left figure shows that the YJ transformation indeed capable of roughly inverting the X;'s 's into the X;'s, yielding a major improvement compared to no preprocessing or standard centering and reduction to unit variance. Besides, the right figure shows that even in this homogeneous setting where the data is i.i.d. across centers, using a federated version of YJ compared to a local version of YJ leads to better average performance, and reduced variance.
X; 's and lead to
6 Conclusion
Summary of our contributions In this work, we introduce SECUREFEDYJ, a method to fit YJ transformation on data shared by different clients in a cross-silo setting. SECUREFEDYJ is an SMC version of its pooled equivalent EXPYJ which builds upon the convexity of the negative log-likelihood of the YJ transformation, a novel result introduced by this work, and on the fact that the sign of its derivative can be computed in stable way. We show that SECUREFEDY! has the same accuracy as standard YJ transformation on pooled data. In particular, the results do not depend on how the data is split across the clients, making SECUREFEDY! resilient to data heterogeneity. Besides, the quantities disclosed by SECUREFEDYJ during the training to the central server do not leak any other information than what is contained in the final parameters (M+, (u,,),,00 of).
Limitations and future work While Brent minimization has a super-linear convergence, our approach only has a linear convergence, relies exponential search. This can be an issue if the

---

communication costs between the clients and the server are high. Acceleration could be achieved by either adapting Brent minimization cross-silo setting, or applying a second-order method. We leave the development of a faster SMC methods using either of those two approaches to future work.
Another limitation that even if our approach reveals only information that would be contained in the final fitted parameters, such parameters themselves might leak information about individual samples, as our approach is not differentially private (DP) [6. By adding Gaussian Laplacian noise each sample's s features when computing the S terms one could, in principle, make the resulting algorithm DP [ However it unclear to what extent the noise would impact the final accuracy of the method.
Finally, we only consider an honest-but-eee but-curious setting. We do not explore the threat malicious participant that would purposely deviate from the protocol either gain more information to jeopardize the convergence. We leave this investigation to future work.
Acknowledgement
The authors would like to thank the four anonymous reviewers, as well as the anonymous area chair reviewer for their relevant comments and ideas which significantly improved the paper.
References
[1] Martin Abadi, Andy Chu, Ian Goodfellow, H Brendan McMahan, Ilya Mironov, Kunal Talwar, and Li Zhang. Deep learning with differential privacy. Proceedings of the 2016 ACM SIGSAC Conference Computer and Communicutions Security, pages 308-318, 2016
[2] Mathieu Andreux, Jean Ogier du Terrail, Constance Beguier, and Eric W Tramel. Siloed federated learning for multi-centric histopathology datasets. In Domain Adaptation and Rep- resentation Transfer, and Distributed and Collaborative Learning, pages 129-139. Springer, 2020.
[3] Jon Louis Bentley and Andrew Chi-Chih Yao. An almost optimal algorithm for unbounded searching. Information processing letters, S(SLAC-PUB-1679, 1976.
[4] Keith Bonawitz, Vladimir Ivanov, Ben Kreuter, Antonio Marcedone, Brendan McMahan, Sarvar Patel, Daniel Ramage, Aaron Segal, and Karn Seth. Practical secure aggregation for privacy-preserving machine learning. In Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security, pages 191,
[5] George EP Box and David R Cox. An analysis of transformations. Journal of the Royal Statistical Society: Series B (Methodological) (2(2):211-243) 1964.
[6] Richard P Brent. Algorithms for Minimization Without Derivatives. Prentice Hall, 1973.
[7] Franziska Brunner, Paul Sehmid-Hempel, and Seth M Barribeau. Immune gene expression in bombus terrestris: signatures of infection despite strong variation among populations, colonies, and sister workers. PloS one, 8(7))e6818)) 2013.
[8] Octavian Catrina and Amitabh Saxena. Secure computation with iixed-point numbers. In International Conference Financial Cryptography and Data Security, pages 35-50, Springer, 2010.
[9] Li-Chu Chien. A rrnk-based normalization method with the fully adjusted full-stage procedure in genetic association studies. PloS one, 15(6))e0233847, 2020.
[10] David R Cox. Regression models and life-tables. Journal of the Royal Statistical Society: Series
B (Methedologicall) 34(2):88-2022 1972.
[11] Ivan Damgård, Martin Geisler, Mikkel Kroigaard, and Jesper Buus Nielsen. Asynchronous multiparty computation: Theory and implementation. International workshop on public key cryptography, pages 160-179. Springer, 2009.
10

---

[12] Ivan Damgård, Marcel Keller, Enrique Larraia, Valerio Pastro, Peter Scholl, and Nigel P Smart. Practical covertly secure MPC for dishonest majority-or or: breaking the SPDZ limits. In European Symposium on Research in Computer Secuurtt pages 1-18. Springer, 2013.
[13] Thiago Alexandre das Neves Almeida, Luís Cruz, Eduardo Barata, and Isabel-Maria Garefa- Sánchez. Economic growth and environmental impacts: An analysis based on a composite index of environmental damage. Ecological Indicators, 76:119 19-1109130, 2017.
[14] Daniel Demmler, Thomas Schneider, and Michael Zohner. ABY-A framework for efficient mixed-protocol secure two-party computation. In NDSS, 2015.
[15] Dheeru Dua and Casey Graff. UCI machine learning repository, 2017. URL http:///rccc ics.uci. www..eeu//eeeeee ml
[16] Cynthia Dwork, Differential privacy. International Colloquium on Automata, Languages, and Programming, pages 1-12. Springer, 2006.
[17] Daniel Escudero, Satrajit Ghosh, Marcel Keller, Rahul Rachuri, and Peter Scholl. Improved primitives for mpc over mixed arithmetic- binary circuits. In Annual International Cryptology conference, pages 823-852 Springer, 2020.
[18] Ronald Fisher. The of multiple measurements in taxonomic problems. Annals of eugenics, 179-1199-888, 1936.
[19] David Froelicher, Juan R Proncoso-Pastorizze Jean Louis Raisaro, Michel Cuendet, Joao Sa Sousa, Jacques Fellay, and Jean-Pierre Hubaux. Truly for precision medicine with multiparty homomorphic encryption. Nature Communications, 12 (1):5910, 2021.
federated analytics
[20] Ian Goodfellow, David Warle-Farley, Mehdi Mirza, Aaron Courville, and Yoshua Bengio. Maxout networks. In International Conference Machine Learning, pages 1319-1327. PMLR, 2013.
[21] Andreas Grammenos, Rodrigo Mendoza Smith, Jon Crowcroft, and Cecilia Mascolo. Federated principal component analysis. In H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin, editors, Advances in Neural Information Processing Systems, volume 33, pages 6453- 6464. Curran Associates, Inc., 2020.
[22] Ian Jolliffe. Principal component analysis. Encyclopedia of statistics in behavioral science, 2005.
[23] Peter Kairouz, H Brendan McMahan, Brendan Avent, Aurélien Bellet, Mchdi Bennis, Ar- jun Nitin Bhagoji, Kallista Bonawitz, Zachary Charles, Graham Cormode, Rachel Cummings, al. Advances and open problems in federated learning. Foundations and Trends in Machine Learning, 14(1-2):1 1-00 2021.
[24] Sai Praneeth Karimireddy, Satyen Kale, Mehryar Mohri, Sashank Reddi, Sebastian Stich, and Ananda Theertha Suresh. Scaffold: Stochastic controlled averaging for federated learning. In International Conference on Machine Learning, pages 5132-5143, PMLR, 2020.
[25] David G Kleinbaum and Mitchel Klein, Survival analysis, volume 3. Springer, 2010.
[26] Elies Kouider and Hanfeng Chen. Concavity of Box-Cox log-likelihood function. Statistics and probability leterr, 171-151 1995.
[27] Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009.
[28] Tian Li, Anit Kumar Sahu, Ameet Talwalkar, and Virginia Smith. Federated learning: Chal- lenges, methods, and future directions. IEEE Signal Processing Magazine, 37(3):00-60, 2020.
[29] Tian Li, Anit Kumar Sahu, Manzil Zaheer, Maziar Sanjabi, Ameet Talwalkar, and Virginia Smith. Federated optimization in heterogeneous networks. Proceedings of Machine Learning and Systems, 2:429 4500, 2020.

---

[30] Thomas Lorünser and Florian Wohner. Performance comparison of two generic MPC- frameworks with symmetric ciphers. In ICETE (2), pages 587-594, 2020.
[31] Chia-Lun Lu,, Shuang Wang, Zhanglong Ji, Yuan Wu, Li Xiong, Xiaoqian Jiang, and Lucila Ohno-Machado. Webdisco: a web service for distributed cox model learning without patient- level data sharing. Journal of the American Medical Informatics Association, 22(6):212-1119, 2015.
[32] Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Aguera y Arcas. Communication-effin learning of deep networks from decentralized data. In Artificial intelligence and statistics, pages 1273-1282. PMLR, 2017.
[33] Payman Mohassel and Peter Rindal. ABY3: A mixed protocol framework for machine learning. In Proceedings of the 2018 ACM SIGSAC Conference on Computer and Communications Security, pages 35-52, 2018.
[34] Payman Mohassel and Yupeng Zhang. SecureML: A system for scalable privacy-preserving machine learning. In 2017 IEEE symposium on security and privacy (SP), pages 19-38. IEEE, 2017.
[35] Andrew Paverd, Andrew Martin, and Ian Brown. Modelling and automatically analysing privacy properties for nonest-but-curinn adversaries. Tech. Rep, 2014.
[36] F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and E. Duchesnay. Scikit-learn: Machine learning in Python. Journal of Machine Learning Research, 12:2825-288 2830, 2011.
[37] Tord Ingolf Reistad. A general framework for multiparty computations. 2012.
[38] Tord Ingolf Reistad and Tomas Toft. Secret sharing comparison by transformation and rotation. In International Conference on Information Theoretic Security, pages 169-180. Springer, 2007.
[39] M Sadegh Riazi, Christian Weinert, Oleksandr Tkachenko, Ebrahim M Songhori, Thomas Schneider, and Farinaz Koushanfar. Chameleon: A hybrid secure computation framework for machine learning applications. In Proceedings of the 2018 on Asia Conference on Computer and Communications Security, pages 707-721, 2018
[40] Nicola Rieke, Jonny Hancox, Wenqi Li, Fausto Milletari, Holger R Roth, Shadi Albarqouni, Spyridon Bakas, Mathieu N Galtier, Bennett A Landman, Klaus Maier-Hein, et al. The future of digital health with federated learning. NPJ digital medicine, 3(1):177, 2020.
[41] Théo Ryffel, Pierre Tholoniat, David Pointcheval, and Francis Bach. AriaNN: Low-interaction privacy-preserving deep learning via function secret sharing. Proceedings on Privacy Enhancing Technologies, 1:291-316, 2022.
[42] Berry Schoenmakers. MPyC-python package for secure multiparty computation. In Workshop on the Theory and Practice of MPC., 2018. URL https: www.coomm////c http//sschheepppp /mpyc
[43] Muhammad Shafiq-ul Hassan, Geoffrey G Zhang, Kujtim Latifi, Ghanim Ullah, Dylan C Hunt, Yoganand Balagurunathan, Mahmoud Abrahem Abdalah, Matthew B Schabath, Dmitry G Goldgof, Dennis Mackin, et al. Intrinsic dependencies of CT radiomic features on voxel size and number of gray levels. Medical physics, 1000):1050-106,, 2017.
[44] Adi Shamir. How to share a secret. Communications of the ACM, 22(11):612-613, 1979.
[45] Reza Shokri and Vitaly Shmatikov. sirvacy-ressevvig deep learning. In Proceedings of the 22nd ACM SIGSAC Conference on Computer and Communications Security, pages 1310-1321,
2015.
[46] Jost Tobias Springenherg, Alexey Dosovitskiy, Thomas Brox, and Martin Riedmiller. Striving for simplicity: The all convolutional net. arXiv preprint arXiv:1412.6806 2014.
12

---

[47] David Tellez, Geert Litjens, Péter Bándi, Wouter Bulten, John-Melle Bokhorst, Francesco Ciompi. and Jeroen van der Laak. Quantifying the effects of data augmentation and stain color normalization in convolutional neural networks for computational pathology. Medical image analysis, 58:101544, 2019.
[48] Sameer Wagh. Divya Gupta, and Nishanth Chandran. SecureNN: Efficient and private neural network training. IACR Cryptol. ePrint Arch., 2044:4 2018.
[49] Sameer Wagh, Shruti Tople, Fabrice Benhamouda, Eyal Kushilevitz, Prateek Mittal, and Tal Rabin. FALCON: Honest-majority maliciously secure framework for private deep learning. arXiv preprint urXiv:200..02229 2020.
[50] QJ Wang and DE Robertson. Multisite probabilisticc forecasting of seasonal flows for streams with zero value occurrences. Water Resources Research, 47(2), 2011.
[51] QJ Wang, Andrew Schepen, and David E Robertson. Merging seasonal rainfall forecasts from multiple statistical models through bayesian model averaging. Journal of Climate, 25(16): 5524-5537, 2012.
[52] In-K Yeo and Richard A Johnson. A new family of power transformations to improve normality symmetry. Biometrika, 8744):954-959, 2000.
[53] Lina Zhang, Bizheng Wang. and Qingeum Zeng. Impact of the Malden-Julian oscillation on summer rainfall in southeast China. Journal Climate, 22(2):201-22 2009.
[54] Bo Zhao, Konda Reddy Mopuri, and Hakan Bilen. iDLG: Improved deep leakage from gradients. arXiv preprint arXiv:2001.02611 2020.
[55] Wenting Zheng, Raluca Ada Popa, Joseph Gonzalez, and lon Stoica. Helen: Maliciously secure coopetitive learning for linear models. In 2019 IEEE Symposium Security and Privacy (SP), pages 724-738. IEEE, 2019.
[56) Ligeng Zhu, Zhijian Liu, and Song Han. Deep leakage from gradients. In Advances in Neural Information Processing Systems, volume 32. Curran Associates, Inc., 2019.
[57] Olga Zolotareva, Reza Nasirigerdeh, Julian Matschinske, Reihaneh Torkzadehmahani, Moham- mad Bakhtiari, Tobias Frisch, Julian Späth, David Blumenthal, Amir Abbasinejad, Paolo Tieri, et al. Flimma: federated and privacy-aware tool for differential gene expression analysis. Genome biology, 22(1):1 2021.
[58] Isabella Zwiener, Barbara Frisch, and Harald Binder. Transforming RNA-Seq data to improve the performance of prognostic gene signatures. PloS one, 9(1 (11)x885150 1550 2014.
Checklist
1. For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's
contributions and scope? [Yes]
(b) Did you describe the limitations of your work? [Yes] We described them in Section 0
(c) Did you discuss any potential negative societal impacts of your work? [N/A]
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes]
2. If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results? [Yes]
(b) Did you include complete proofs of all theoretical results? [Yes] The full proof of the main theoretical results, i.e. the convexity the Yeo-Johnson negative log- likeliodd (Proposition 3.1 is provided in Appendix C
3. If you experiments...
13

---

(a) Did you include the code, data, and instructions needed to reproduce the main exper- imental results (either in the supplemental material as a URL)? [No] The code of the experiments is not provided, but a detailed pseudo-code of the newly proposed algorithms are provided.
(b) Did you specify all the training details (e.g., data splits, hyperparameters. how they were chosen)? [Yes] We specified all the hyperparameters and the details of the numerical experiment in Appendix
(c) Did you report error bars (c.g., with respect to the random seed after running exper- iments multiple times)? [Yes] Standard deviations quantiles of the results with respect to the seed are provided (cf plots)
(d) Did you include the total amount of compute and the type of resources used (c.g., type of GPUs, internal cluster, or cloud provider)? [No] The experiment are not heavy and easily personal computer, on CPU
4. If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a) your work uses existing assets, did you cite the creators? [Yes] The datasets used are open datasets available online and are systematically cited.
(b) Did you mention the license the assets? [Yes] The licence of the datasets used are provided in Appendix E.
URL? [No] (d) Did you discuss whether and how consent obtained from people whose data you's rre'e using/curating? [N/A] We are using open datasets available online. The genomic dataset from TCGA have been previously anonymised by its creator before publication
(c) Did you include any assets either in the supplemental material
(c) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content? [N/A]
5. If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, applicable? [N/A]
(b) Did you describe any potential participant risks, with links Institutional Review Board (IRB) approvals, if applicable? [N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation? [N/A]
14


Reviews:
[{"review_id": "jN0HJno7gCx", "reviewer": "Reviewer_Kq6t", "summary": "The authors propose ExpYJ, a new algorithm for feature Gaussianization which beats Brent algorithm in terms of numerical stability but asymptotically slower than Brent algorithm. The authors them combine ExpYJ with secure aggregation to obtain FedYJ.", "questions": "1. Please clarify how the algorithm is customized to deal with heterogeneity if any?\n2. Have the authors considered discretization in SMC when performing the empirical evaluation?\n\n", "limitations": "1. Please include empirical evaluation of computation and communication cost on real clusters.\n2. Please add details about discretization for SMC.\n3. Please validate the numerical stability advantage on more real-world datasets.\n4. Please refine the proof for Proposition 4.1 (for example, simulation-based proof).\n5. Please clarify how the algorithm is customized to deal with heterogeneity if any?", "rating": 5, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "The authors propose a federated feature Gaussinaization algorithm by first proposing a new optimization algorithm to optimize the hyper-parameters for YJ transformation and then combining it with secure aggregation in FL.\n\nStrength:\n1. The paper is well written and easy to follow.\n2. The authors evaluate their algorithm on several downstream applications.\n3. The improvement on numerical stability is a good contribution. However, I think evaluation on only one dataset is not enough. I encourage the authors to validate this advantage on more datasets.\n\nWeakness:\n1. The technical novelty is limited. The proof of strict convexity is a contribution, but with the convexity of Box-Cox transformation, the proof is not too challenging.\n2. FedYJ requires multiplication on secret shared values so benchmarking the system performance (how long does it take to perform the computation and communication) is a necessity.\n3. The authors highlight the heterogeneity challenge in the introduction but I don't see how they customize their algorithm to deal with heterogeneity. If I understand correctly, the application itself is not sensitive to heterogeneity. For example, I cannot tell any difference between Figure 3(a) and 3(b).\n4. The threat model is too weak, which makes the whole setting degenerate to distributed learning instead of FL.\n5. Proposition 4.1 is correct but the proof is too sketchy.\n6. To use SMC, the data points need proper discretization. The authors do not mention any details about the discretization process like the hyper-parameters.\n7. Some minor points: 7.1 There are many references to Appendix C. I recommend the authors to refer to subsections in Appendix C which will make the flow clearer. 7.2 The secure multiparty computation background is almost taking up a whole page. Consider moving it to the appendix.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "MdU9DtLjATH", "reviewer": "Reviewer_5G9f", "summary": "The paper addresses the Yeo-Johnson feature transformation, and how it can be computed in the federated setting, where privacy and security restrict how information can be shared. The paper demonstrates the convexity of the log-likelihood function, which enables an efficient search procedure in the multiparty computation model that does not leak any information beyond the revealed answer.  Experiments are included that show the behavior inpractice. ", "questions": "An alternative approach in the cross silo setting would before each client to apply the YJ transformation locally to their data, on the assumption that the distribution is common across clients.  Can you show examples that demonstrate where this heuristic will fail? \n\n", "limitations": "The conclusions section has a nice discussion of limitations and future work that is appropriate for the paper. ", "rating": 6, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths \n\n- There is growing interest in secure machine learning that goes beyond the core training part of the ML pipeline, and provides security for data preparation in addition\n\n- The paper shows a nice result about the convexity of the Yeo-Johnson transformation which enables its secure computation\n\n- The proof of minimum leakage (proposition 4.1) is a very nice contribution, and clearly makes the argument that there is no overall leakage from the algorithm\n\nWeaknesses\n\n- The paper does not offer insights that go beyond the immediate question of the YJ transformation. \n\n- The search algorithm requires some operations which are potentially costly to perform in the SMC setting\n\n- The paper glosses over the technical details of applying the algorithm in the SMC setting: there are some moderately complex polynomial operations on data that require multiplications / exponentiations, but the cost of these is not discussed in much detail -- Appendix D gives only a very high level overview of basic secret sharing.  Moreover, the paper does not indicate where the challenge lies: once algorithm 1 is presented, it seems that Algorithm 3 is very similar except that data access is performed through secret sharing.  \n\n- The communication overhead is high (8MB per client per feature), and there is no discussion of whether this could be reduced or traded off  -- this is why the method is motivated in the cross-silo rather than cross-device FL setting", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "Gh6_OpSmxnR", "reviewer": "Reviewer_W31E", "summary": "This paper proposes an exponential search method for YJ transformation in centralized and federated learning settings. It proved that the negative log-likelihood of YJ transformation is strictly convex with respect to \\lambda. Building upon that, they provide a method to do an exponential search to find the best \\lambda, \\mu, and \\sigma for YJ transformation. The paper also applies the Secured Multiparty Computation to provide a privacy-preserving guarantee in the process of Federated YJ transformation.", "questions": "- In Algorithm 3, who will run the algorithm, and what is the job of the clients and the server?\n- What is the reason you only consider cross-silo FL since, in my opinion, this method can be generalized to all kinds of FL.\n- Can you please explain the log-likelihood in Eq. 1? I followed reference 50 in the text, but there's no information in the reference. \n- What is the advantage of the proposed method for FL?", "limitations": "- Unclear threat model\n- The proposed method will destroy the heterogeneity of Federated Learning.\n- There's no theoretical guarantee of the clients' data privacy leakage from the output of the proposed method.", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n- Provide a novel exponential search method for YJ transformation.\n- The proposed method achieves high accuracy compared to Brent minimization, which is widely used.\n- The proposed method has better numerical stability than existing methods (Brent minimization).\n\nWeakness:\n- Unclear threat model: can not understand the purposes or capabilities of the threat model.\n- The proposed method will destroy the heterogeneity of Federated Learning: After the YJ transformation process, it seems like every client will have the same distribution since they share the same \\lambda_*, \\mu_*, \\sigma_*. This poses a risk to FL: if the adversary knows the data distributions of every client (since they have the same distribution), the adversary will quickly deploy the attack of the tails [1]. Moreover, since the threat model is \"honest-but-curious\", using the YJ transformation will pose a privacy risk to the clients' data since the server can deploy the membership inference attacks [2].\n- There's no theoretical guarantee of the clients' data privacy leakage from the output of the proposed method: since \"the fitted parameter \\lambda_* contains all the information revealed during the intermediate steps,\" whether it can leak any information and whether we should protect it? \n\n[1] Wang, Hongyi, et al. \"Attack of the tails: Yes, you really can backdoor federated learning.\" Advances in Neural Information Processing Systems 33 (2020): 16070-16084.\n\n[2] Shokri, Reza, et al. \"Membership inference attacks against machine learning models.\" 2017 IEEE symposium on security and privacy (SP). IEEE, 2017.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "GWEx3nQzeWj", "reviewer": "Reviewer_wXmX", "summary": "The paper proposes a preprocessing protocol based on multi-party computation that works independently of how the data is skewed among a few parties.\n", "questions": "n/a\n", "limitations": "yes", "rating": 6, "confidence": 2, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "The algorithm makes clever use of SMPC to minimize cost while preserving security. While it reminds me of secure median computation (Aggarwal et al., J. Cryptol. 2010), it is clearly worthy of independent publication. The interesting aspect is the following: The desired algorithm is iterative. This iteration is replicated in the secure protocol as follows: In every step, there is a relatively simple computation, the result of which (one bit) is revealed. This bit is then used for the next step in the manner of bisectional search. The revealing operation raises the question of the security of doing so. The security is given by the fact that the final result (which is revealed anyway) implies all the intermediate results, and thus, the adversary does not learn any extra information as long as all parties follow the protocol.\n\nThe definition of SMC in Section 2 is more limited than the generally used one. The authors only consider computation on local shares but not computation involving communication (line 130). General multi-party computation crucially involves a protocol without which only very limited functionality could be achieved.\n\nThe evaluation of the secure computation (Appendix D.3) does not correspond to the state of the art. This might suffice given the low complexity, but I would like to note that there are more efficient ways for secure comparisons (Escudero et al., Crypto 2020).\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}]

Author rebuttal:

Decision: Accept