SecureFedYJ: a safe feature Gaussianization protocol for Federated Learning
Abstract
The Yeo-Johnson (YJ) transformation is a standard parametrized per-feature unidimensional transformation often used to Gaussianize features in machine learning.
In this paper, we investigate the problem of applying the YJ transformation in a cross-silo Federated Learning setting under privacy constraints.
For the ﬁrst time, we prove that the YJ negative log-likelihood is in fact convex, which allows us to optimize it with exponential search.
We numerically show that the resulting algorithm is more stable than the state-of-the-art approach based on the Brent minimization method.
Building on this simple algorithm and Secure Multiparty
Computation routines, we propose S ECURE F ED YJ, a federated algorithm that performs a pooled-equivalent YJ transformation without leaking more information than the ﬁnal ﬁtted parameters do.
Quantitative experiments on real data demonstrate that, in addition to being secure, our approach reliably normalizes features across silos as well as if data were pooled, making it a viable approach for safe federated feature Gaussianization.
Introduction
Federated Learning (FL) [45, 32] is an approach that was recently proposed to train machine learning (ML) models across multiple data holders, or clients , without centralizing data points, notably for privacy reasons.
While many FL applications have been proposed, two main settings have emerged [23]: cross-device FL, involving a large number of small edge devices and cross-silo FL, dealing with a smaller number of clients, with larger computational capabilities.
Due to the sensitivity and relative local scarcity of medical data, healthcare is a promising application of cross-silo FL [40], e.g. to train a biomedical ML model between different hospitals as if all the datasets were pooled in a central server.
In this paper, we focus on the cross-silo setting.
The constraints of cross-silo FL
Although cross-silo FL resembles standard distributed learning, it faces at least two important distinct challenges: privacy and heterogeneity.
Due to data sensitivity, clients might impose stringent security and privacy constraints on FL collaborations.
This arises in coopetitive FL projects, where models are jointly trained on industrial competitors’ datasets [55], as well as medical FL applications, where conservative data regulations might apply.
In this setting, using standard FL algorithms such as F ED A VG [32] might not provide enough privacy guarantees, as privacy attacks such as data reconstruction can be carried out based on the clients’ gradients [56, 54].
Various protocols based on Secure Multiparty Computation (SMC) (see Section 2 for more details), such as Secure Aggregation [4], can mitigate this shortcoming by disclosing only the sum of the gradients from all clients to the server, without disclosing each gradient individually.
An additional constraint is that data might present statistical heterogeneity across clients, i.e. the local clients’ data distributions may not be identical.
In the case of medical applications, such heterogeneity may be caused e.g. by environmental variations or differences in the material that was used for acquisition [43, 47, 2].
While different ways of adapting federated training algorithms have been proposed to automatically tackle heterogeneity [28, 29, 24], these solutions do not address data harmonization and normalization prior to FL training.
Preprocessing in ML Data preprocessing is a crucial step in many ML applications, leading to important performance gains.
Among others, common preprocessing methods include data whitening, principal component analysis (PCA) [22] or zero component analysis [27, 20, 46].
However, linear normalization methods might not sufﬁce when the original data distribution is highly non-Gaussian.
For tabular and time series data, a popular approach to Gaussianize the marginal distributions is to apply feature-wise non-linear transformations.
Two commonly-used parametric methods are the
Box-Cox [5] transformation and its extension, the Yeo-Johnson (YJ) transformation [52].
Both have been used in multiple applications, such as climate and weather forecast [53, 50, 51], economics [13] and genomic studies [7, 58, 9].
Problem and contributions In this paper, we investigate the problem of data normalization in the cross-silo FL setting, by exploring how to apply the YJ transformation to a distributed dataset.
This problem arises frequently in medical cross-silo FL, e.g. when trying to jointly train models on genetic data (see e.g. [19, 57]).
Due to data heterogeneity, no single client can act as a reference client:
indeed, there is no guarantee that transformation parameters ﬁtted on a single client would be relevant for other clients’ data.
Hence, it is necessary to ﬁt normalization methods on the full federated dataset.
Moreover, in this setting, data privacy is of paramount importance, and therefore
FL protocols should be carefully designed.
Our main contributions to this problem are as follows:
We prove that the negative YJ log-likelihood is convex (Section 3), which is a novel result, to the best of our knowledge.
Building on this property, we introduce E XP YJ, a method to ﬁt the YJ transformation based on exponential search (Section 3).
We numerically show that this method is more stable than standard approaches for ﬁtting the YJ transformation based on the Brent minimization method [6].
We propose S ECURE F ED YJ (Section 4), a secure way to extend E XP YJ in the cross-silo FL setting using SMC.
We show that S ECURE F ED YJ does not leak any information on the datasets apart from what is leaked by the parameters minimizing the YJ negative loglikelihood (Section 4 and Proposition 4.1).
By construction, S ECURE F ED YJ provides the same results as the pooled-equivalent E XP YJ, regardless of how the data is split across the clients.
We check this property in numerical experiments (Section 4).
The core ideas behind the resulting algorithm, S ECURE F ED YJ, are summarised in Figure 7.
Finally, we illustrate our contributions in numerical applications on synthetic and genomic data in
Section 5.
Background
The Yeo-Johnson transformation The YJ transformation [52] was introduced in order to Gaussianize data that can be either positive or negative.
It was proposed as a generalization of the Box-Cox transformation [5], that only applies to non-negative data.
The YJ transformation consists in applying to each feature a monotonic function (  , · ) parametrized by a scalar  , independently of the other features.
Thus, there are as many  ’s as there are features.
For a real number x , (  , x ) is deﬁned as:
[Equation]
0 ,  6 = 0 , ln( x + 1) , if x  0 ,  = 0 ,  [(  x + 1) 2    1] / (2   ) , if x < 0 ,  6 = 2 ,  ln(  x + 1) , if x < 0 ,  = 2 .[-- -   4 x - -    ( , x ) = 0.
 =0.0=0. =1.0=1.-]
[Figure]
Figure 1a shows the shape of the YJ function for various values of  .
The Yeo-Johnson likelihood Let us consider real-valued samples { x i } i =1 , ··· ,n , and let us apply the YJ transformation (  , · ) to these samples to Gaussianize their distribution.
The log-likelihood that { (  , x i ) } i =1 , ··· ,n comes from a Gaussian with mean µ and variance  2 is given by (derivation details are provided in Appendix A.1):
[Equation]
For a given  , the log-likelihood is maximized for µ ⇤ = 1 n P ni {+=
+}
(  , x i ) and  2 ⇤ =
n P ni
=1 ( ( x i ,  )  µ ⇤ ) 2 .
Once we replace µ and  2 by µ ⇤ and  2 ⇤ , it becomes:
[Equation]
see [52].
Maximizing the YJ log-likelihood is therefore a 1-dimensional problem for each feature.
Once the optimal  ⇤ is found, the transformed data (  , x i ) is usually renormalized by subtracting its empirical mean µ ⇤ and dividing by the square root of its empirical variance  2 ⇤ .
Figure 1b shows an example of the YJ transformation applied to a skew-normal distribution.
Note that in a typical application, the triplet (  ⇤ , µ ⇤ ,  2 ⇤ ) is ﬁtted on the training data only, and is then used to Gaussianize the test dataset during inference.
Minimization methods in dimension 1 As seen above, ﬁtting a YJ transformation can be reduced to a 1D optimization problem.
To tackle this problem, we introduce two standard 1D minimization methods: (i) Brent minimization [6] and (ii) exponential search [3].
Brent minimization [6] (not to be confused with the Brent-Dekker method, see [6], chapters 3 and 4) is a widely used method for 1D optimization.
It is based on golden section search and successive parabolic interpolations, and does not require evaluating any derivatives.
This algorithm is guaranteed to converge to a local minimum with superlinear convergence of order at least 1.3247.
Standard implementations of the YJ transformation, in particular the scikit-learn implementation [36], are based on the Brent minimization method to minimize the negative log-likelihood provided by Eq.
Exponential search [3] is a dichotomic algorithm designed for unbounded search spaces.
The idea is to ﬁrst ﬁnd bounds, and then to perform a classic binary search within these bounds.
This algorithm can be used to ﬁnd the minimum of convex differentiable functions with linear convergence, as explained in Appendix B.
In this work, we build on exponential search to propose a federated version of the YJ transform, for two main reasons: (i) it is more numerically stable than Brent minimization, as shown in Section 3 and Figure 2, (ii), it may conveniently be adapted to a federated setting, as shown in Section 4, and (iii), this latter federated adaptation offers strong privacy garantees, as shown by Proposition 4.1.
Secure Multiparty Computation As illustrated by various privacy gradient attacks [56, 54], sensitive information on the clients’ datasets can be leaked to the central server during an FL training.
One way to mitigate this risk is to use Secure Multiparty Computation (SMC) protocols to hide individual contributions to the server.
SMC enables one to evaluate functions with inputs distributed across different users without revealing intermediate results and is often based on secret sharing.
SMC protocols tailored for ML use-cases have been recently proposed [12, 14, 34, 39, 48, 33, 49, 41].
These protocols are either designed to enhance the privacy of FL trainings, or to perform secure inference, i.e. to enable the evaluation of model trained privately on a server without revealing the data nor the model.
A popular FL algorithm relying on SMC is Secure Aggregation (SA) [4].
Schematically, in SA each client adds a random mask to their model update before sending it to the central server.
These masks have been tailored in such a way that they all together sum to zero.
Therefore, the central server cannot see the individual updates of the clients, but it can recover the sum of these updates by adding all the masked quantities sent from them.
More generally, an SMC routine schematically works as follows (we refer to Appendix D for further details).
Let us consider the setting where K parties k
= 1 , . .
, K want to compute g = f
( h (1) , . . .
, h ( K ) ) for a known function f , where ( h (1) , . . .
, h ( K ) ) denote private inputs.
Each party k knows h ( k ) and is not willing to share it.
During the ﬁrst step, secret sharing , each party splits its private input h ( k ) into K secret shares h ( k ) 1 , . .
, h ( k ) K , and sends the shares h ( k ) k 0 to the party k 0 .
These secret shares are constructed in such a way that (i) knowing h ( k ) k 0 does not provide any information on the value of h ( k ) , and (ii) h ( k ) can be reconstructed from the vector ( h ( k ) 1 , . .
, h ( k ) K ) .
For simplicity, we denote J h ( k )
K = ( h ( k ) 1 , . .
, h ( k ) K ) the vector of share secrets.
In a second step, the computation , each party k 0 computes the quantity denoted g k 0 using the secret shares they know along with intermediate quantities exchanged with the other parties.
The way to compute g k 0 depends on f and on the SMC protocol that is used, and is chosen so that g = f ( h (1) , . .
, h ( K ) ) can be reconstructed from ( g 1 , . . .
g K ) .
Said otherwise, g k 0 are secret shares of g :
J g K = ( g 1 , . . .
g K ) .
Finally, during the reveal step, each party k reveals g k to all other parties, and each party can reconstruct g from ( g 1 , . . .
g K ) .
Threat model
In this work, we consider an honest-but-curious setting [35].
Neither the clients nor the server will deviate from the agreed protocol, but each party can potentially try to infer as much information as possible using data they see during the protocol.
This setting is relevant for cross-silo
FL, where participants are often large institutions whose reputation could be ternished by a more malicious behaviour.
A novel method to optimize the Yeo-Johnson log-likelihood: E XP YJ Algorithm 1 E XP YJ
Input: data x i , total data size n , number of steps t max
Initialize  t = 0 ,  + t = 1 ,   t = 1 Compute S ' for t = 1 to t max for g 2 { (  , · ) , (  , · ) 2 , @  (  , · ) , @  (  , · ) 2 }
Compute S g end for  t = sgn h nS @ 2  2 S S @
2 S ' ⇣ S 2  S 2 n ⌘i  t ,   t ,  + t E XP U PDATE (  t  1 ,   t  1 ,  + t  1 ,  t ) end for  ⇤  t max Compute µ ⇤ =
S /n and  2 ⇤ =
S 2 /n  µ 2 ⇤
Output: The ﬁtted triplet (  ⇤ , µ ⇤ ,  2 ⇤ )
In this section, we leverage the convexity of the negative log-likelihood of the
YJ transformation (see Proposition 3.1) to propose a new method to ﬁnd the optimal  ⇤ using exponential search.
While this method only offers linear convergence, compared to the super-linear convergence of Brent minimization method, we demonstrate two of its advantages: (i) it is more numerically stable, and (ii) it is easily amenable to an FL setting with strong privacy guarantees.
The method proposed in this section is based on the following result.
Proposition 3.1.
The negative loglikehood  [Equation] is strictly convex.
The proof of Proposition 3.1 builds upon the work of [26] which shows that the negative log-likelihood of the Box-Cox transformation [5] is convex.
The complete proof is deferred to Appendix C.
The exponential YJ algorithm
The pseudo-code of the proposed algorithm is presented in Algorithm 1, and relies on the exponential search presented in Algorithm 2 (cf Appendix B for more details on exponential search).
An illustration of E XP YJ is shown in Figure 6 in Appendix B.
Due to the strict convexity of the negative log-likelihood of the YJ transformation, we may perform the exponential search described in Section 2 and
Appendix B.
To do so, it is enough to obtain the sign of the derivative.
Let @  (  , · ) 2 = 2 (  , · ) @
(  , · ) and ' ( x ) = sgn( x ) + log( | x | + 1) .
Further, for g 2 { (  , · ) , @  (  , · ) , (  , · ) 2 , @  (  , · ) 2 , ' } , let us deﬁne S g def = [Equation] g (
[Equation]
The derivative of the log-likelihood is available in closed form (see Appendix A.3):
[Equation]
Notice that S 2  S 2 /n can be expressed as a variance, hence is non-negative.
We may therefore obtain sgn [ @  log L YJ ] while avoiding performing division by computing
[Equation]
Avoiding this division is crucial to make the overall procedure more numerical stable, as explained below, and eases the use of SMC routines.
Algorithm 2 E XP U PDATE
Input:  ,  + ,   ,  2 {  1 , 1 } if  = 1 then     (  + +  ) / 2 if  + <else  max(2  , 1) else  +   (   +  ) / 2 if   >  else  min(2  ,  1) end if
Output: Updated  ,  + ,
Accuracy of E XP YJ
We check the accuracy of E XP YJ on the datasets presented in Appendix E.1.
In particular, we compare the results provided by E XP YJ with the outputs of the scikitlearn algorithm based on Brent minimization.
For 2 of the 108 features present in the datasets, the scikitlearn implementation leads to numerical instabilities discussed hereafter.
Therefore, we focus our comparison on theremaining features, that we aggregated regardless of the dataset.
Figure 2 reports the relative difference  between the results obtained by E XP YJ and by the scikit-learn implementation as a function of the number of iteration t max (as deﬁned in
Algorithm 1).
These results show that this relative difference is of order less than 10  6 when t max = 40 .
Numerical stability of E XP YJ
Our experiments demonstrate that E XP YJ is numerically more stable than Brent minimization.
Indeed, for some values of  and some datasets { x i } , the transformation (  , · ) concentrates all data points in a small interval such that the values of (  , { x i } ) are all equal up to machine precision.
In that case, the log-likelihood is not well-deﬁned and the term log  2  takes the value 1 , which prevents Brent minimization from converging.
This phenomenon does not affect the E XP YJ routine as we do not compute directly the sign of @  L = @
2  /  2
P i ' ( x i ) , but rather the sign of  2  @
L =  @
2    2  P i ' ( x i ) , see Eq.
Figure 2 illustrates this in the case of a feature of the Breast Cancer Dataset .
The  ⇤ returned by the Brent minimization method of scikit-learn is  14 .
53 while the minimizer of the negative log-likelihood found by the E XP YJ is  0 .
In particular, Figure 2 shows the values of the negative log-likelihood as a function of  computed using 64-bit ﬂoat precision.
The orange vertical full bands correspond to values for which  2  is zero within the machine precision, resulting to a negative log-likelihood of 1 .
This instability happens for 2 of the 108 features used in numerical experiments, where blindly applying the Brent-based YJ transformation leads to all data points collapsing to zero, while E XP YJ succeeds in transforming the data distributions to more Gaussian-like ones.
Appendix E.4 shows that this issue also arises in other real-life datasets.
Applying the Yeo-Johnson transformation in FL
So far, we only considered the centralized setting, where data is accessible from a single server.
Yet, as mentioned in Sections 1 and 2, many real-world situations require working with heterogeneous data split between different centers, and to take privacy constraints into account.
When the data is split across centers k = 1 , . .
, K and the function to optimize is separable, i.e. of the form
F (  ) =
P
Kk =1 f k (  ) where each f k can be computed from data present in the center k exclusively,
Federated Learning techniques were recently proposed.
In short, they consist in repeatedly performing a few rounds of local optimization in each center, before aggregating local parameters in the server.
We refer to [23] for an overview of recent advances in FL.
In our case, however, the YJ negative log-likelihood (2) is not separable, due to the log-variance term.
Indeed, turning the variance into a separable term would require sharing the global YJ mean µ  to all centers at each iteration.
Compared to the method we propose in this section, this would lead to more privacy leakage.
We now introduce S ECURE F ED YJ, a secure federated algorithm that builds upon E XP YJ to apply YJ transformations.
This algorithm satisﬁes the two following properties: (i) it is pooled-equivalent , i.e. it yields the same results as if the data were freely accessible from a single server, and (ii) it leaks as little information as possible about the underlying datasets, as shown by Proposition 4.1.
Finally, it converges in a limited number of iterations, thanks to the linear convergence of the underlying exponential search.
S ECURE F ED YJ
S ECURE F ED YJ is a federated adaptation of E XP YJ presented in Section 3 to ﬁnd the best parameters (  ⇤ , µ ⇤ ,  2 ⇤ ) of the YJ transformation when training datasets are split across different clients.
It relies on SMC to ensure that only the ﬁnal triplet (  ⇤ , µ ⇤ ,  2 ⇤ ) ﬁtted on the training datasets is revealed, without leaking any other information apart from the overall total number of training samples n .
Indeed, at each intermediate step, only the sign of @  log L YJ is revealed, and the mean and variance of the transformed data is only revealed at the last step.
The pseudo-code of the resulting algorithm is presented in Algorithm 3, and relies on the exponential search presented in
Algorithm 2.
A functional representation of S ECURE F ED YJ is displayed Figure 7.
In Algorithm 3,we label the clients by k = 1 , . . .
, K and each client k holds data { x k,i : i = 1 , . . . , n k } .
We suppose that the total number of samples n = P Kk =1 n k is public and shared to all clients.
For a given function g , we denote S k,g the sum S k,g def = P n k i = 1 g ( x k,i ) .
As introduced in Section 2, we use double brackets J · K to indicate an SMC secret shared across the clients (see
Appendix D for more details).
Privacy leakage In Proposition 4.1 we show that Algorithm 3 only reveals information already contained in the ﬁtted triplet (  ⇤ , µ ⇤ ,  2 ⇤ ) .
In comparison, turning the YJ negative log-likelihood (2) into a log-separable function before using off-the-shelf FL methods would require sharing µ and its gradient and centrally computing  2 for intermediate values of  at each iteration.
This could potentially lead to uncontrolled privacy leakage.
Proposition 4.1.
The ﬁtted parameter  ⇤ contains all the information revealed during the intermediate steps of S ECURE F ED YJ .
More precisely, there exists a deterministic function F such
Algorithm Input: Data { x k,i } , total data size n , number of steps t max Notations: J ·
K indicates a SMC secret shared across the clients.
Any operation such as J · K = f J ·
K , J · K , · · · ) where f can be the sum, product, or the sign, designs an SMC routine across the clients as described in Appendix D.5.
Initialize  t = 0 ,  + 1 ,   1 independently on each client Clients compute in SMC
J S ' K =
P k J S k, '
K for t = 1 to t max for g 2 { (  , · ) , (  , · ) 2 , @  (  , · ) , @ (  , · ) 2 }
Clients compute in SMC J S g
K = P k J
S k,g K , end for
Clients compute in SMC J  t
K = sgn [ n
[-∂ Ψ 2 (cid:75) − 2 (cid:
-]J S @ K
J S KJ
S @ K
J S '
K ( J
S K  J
S
K 2 /n )
⇤ Clients reveal  t  t ,   t ,  + t E XP U PDATE (  t  1 ,   t  1 ,  + t  1 ,  t ) independently on each client end for  ⇤  t max Clients compute in SMC
J µ K =
P k J S k, K /n and
J
K =
P k
J
S k,
K /n  J µ K Clients reveal µ ⇤ µ and  2 ⇤
Output: The ﬁtted triplet (  ⇤ , µ ⇤ ,  2 ⇤ )
[Figure]
that for any set of datasets { x k,i } , if  ⇤ ( { x k,i } ) is the result of S ECURE F ED YJ on { x k,i } , then {  t ,  + t ,   t ,  t } t =1 , ··· ,t max = F [  ⇤ ( { x k,i } )]
Proof.
This proposition comes from the fact that all gradient signs  t revealed during the algorithm can be retrospectively inferred from  ⇤ .
Indeed, @  log L YJ < 0 for  >  ⇤ and @  log L YJ > 0 for  <  ⇤ .
Besides, the successive values of  t explored at each step t can be deterministically inferred from the initial value  t [-=
-]=0 and and the ﬁnal ﬁtted value  ⇤ .
We construct such a function F and numerical verify this proposition in Appendix F. ⌅
Performance of S ECURE F ED YJ
We implement S ECURE F ED YJ in Python, using the MPyC library [42] based on Shamir Secret Sharing [44].
We refer to Appendix D for more details on our implementation.
To represent signed real-valued numbers in an SMC protocol, we use a ﬁxed-point representation (see Appendix D.2) using l bits, among which f bits are used for the decimal parts.
This means that we consider ﬂoats ranging from  2 l  f to 2 l  f and that we have an absolute precision of 2  f in our computations.
In order to ensure the accuracy of S ECURE F ED YJ results, we need to make sure that l and f are large enough.
Figure 3 shows the accuracy of S ECURE F ED YJ when compared to E XP YJ for various values of f .
According to these numerical experiments, taking f = 50 and l = 100 provides reasonably accurate results.
Moreover, by construction, the outputs of S ECURE F ED YJ do not depend on how the data is split across the clients, up to rounding numerical errors.
Therefore this algorithm is resilient to data heterogeneity, as long as the numerical decimal precision f is large enough, as shown in
Figure 3.
Performing S ECURE F ED YJ with t max = 40 takes 726 rounds of communication (see Appendix D.6).
During these communication rounds, each client sends overall about 8 Mb per feature to every other client (see also Appendix D.6).
S ECURE F ED YJ can be applied independently and in parallel to each feature.
Therefore, the overall number of rounds does not depend on the number of features being considered, and the communication costs grow proportionally to the number of features.
In a realistic cross-silo FL setting as described in [19], the bandwidth of the network is 1 Gb per second with a delay of 20 ms between every two clients.
In this context, the execution of S ECURE F ED YJ with t max =on p features would take about 726 ⇥ 20 ms ' 15 s due to the communication overhead, in addition to p ⇥ 8 Mb / 1 Gbps ' 8 p ms due to the bandwidth.
This shows that S ECURE F ED YJ is indeed a viable algorithm in a real-world scenario.
As pointed out in Appendix B, the binary search in the exponential search can be replaced by a k -ary search.
In such a setting, the sign of the negative log-likelihood of the YJ transformation is computed for k  1 different values of  at each round.
Such a modiﬁcation would reduce the number of communication rounds required to obtain a given accuracy, while increasing the size of the data exchanged over the network at each round.
Applications
Genomic data: TCGA We start by showing the beneﬁts of YJ preprocessing in survival analysis experiments on lung (LUAD+LUSC), pancreas (PAAD), and colorectal (CRC) cancers.
Given gene expression raw counts (features) and censored survival data (responses) from patients having either of those three cancers, we aim to ﬁt a Cox Proportional Hazards (CoxPH) model [10] with the highest possible concordance index (C-index) [25], which measures how well patients are ranked with respect to their survival times.
We refer to [25] for a more thorough introduction to survival analysis.
In Figure 4, we compare three different preprocessing methods: (i) whitening, (ii) log normalization, and (iii)
YJ, each followed by a PCA dimensionality reduction step.
More precisely, whitening (i) consists in centering and reducing to unit variance the total read counts of all genes across all samples, log normalization consists in applying u 7! log(1 + u ) to raw read counts before applying global whitening, and YJ is a global YJ transform on the total read counts.
We then evaluate each strategy using 5-fold cross-validation with 5 different seeds.
We refer to Appendix E.2 for experimental details.
While this experiment is performed in a pooled environment, note that, importantly, each step has a federated pooled-equivalent version: apart from the proposed S ECURE F ED YJ for YJ, see e.g. [21] for PCA, and Webdisco [31] for Cox model ﬁtting.
This simpliﬁed setting allows us to understand the importance of the Yeo-Johnson transformation in an ideal setting, independently of other potential downstream federated learning artifacts.
In Figure 4, we see that YJ is better or on par with the best method for each cancer:
YJ improves prediction results for colorectal cancer, while yielding results which are on par with the best results for lung and pancreas cancers, with a smaller variance.
Synthetic data We show how applying YJ may help improving performance compared to no or basic preprocessing, and how S ECURE F ED YJ yields improvements compared to local YJ transforms in federated linear regression.
To do so, we generate covariates e X and responses y as
[Equation]
where  ( · ) is the sigmoid function and
= (  1 . 3 , 2 . 4 , 0 . 87) was randomly chosen.
The goal is then to ﬁt a linear model from i.i.d.
samples ( e
X i , y i ) , i = 1 , . .
. , n following (4), after an optional preprocessing step.
Similarly to the previous example, we simulate a cross-device FL setting only for the preprocessing steps, and the linear model is then ﬁtted in a pooled setting for simplicity.
All the details of the numerical experiments are provided in Appendix E.3.
We suppose that the samples ( e
X i , y i ) , i = 1 , . . .
, n are homogeneously split across 10 centers.
The responses y
i have a highly nonlinear dependency on the covariates e
X i , but depend linearly on the X
i ’s (up to Gaussian noise), which are not observed.
Hence, we expect that applying a suitable preprocessing step before training a linear model will transform back the [Equation]
into the normally-distributed X i ’s and lead to a high performance, compared to no transformation.
The results of our experiments are summarized in Figure 5.
The left ﬁgure shows that the YJ transformation is indeed capable of roughly inverting the e
X i ’s into the X i ’s, yielding a major improvement compared to no preprocessing or standard centering and reduction to unit variance.
Besides, the right ﬁgure shows that even in this homogeneous setting where the data is i.i.d.
across centers, using a federated version of YJ compared to a local version of YJ leads to better average performance, and reduced variance.
Conclusion
Summary of our contributions In this work, we introduce S ECURE F ED YJ, a method to ﬁt a
YJ transformation on data shared by different clients in a cross-silo setting.
S ECURE F ED YJ is an SMC version of its pooled equivalent E XP YJ which builds upon the convexity of the negative log-likelihood of the YJ transformation, a novel result introduced by this work, and on the fact that the sign of its derivative can be computed in a stable way.
We show that S ECURE F ED YJ has the same accuracy as a standard YJ transformation on pooled data.
In particular, the results do not depend on how the data is split across the clients, making S ECURE F ED YJ resilient to data heterogeneity.
Besides, the quantities disclosed by S ECURE F ED YJ during the training to the central server do not leak any other information than what is contained in the ﬁnal parameters ( µ ⇤ ,  ⇤ ,  2 ⇤ ) .
Limitations and future work While Brent minimization has a super-linear convergence, our approach only has a linear convergence, as it relies on exponential search.
This can be an issue if the communication costs between the clients and the server are high.
Acceleration could be achieved by either adapting Brent minimization to a cross-silo setting, or applying a second-order method.
We leave the development of a faster SMC methods using either of those two approaches to future work.
Another limitation is that even if our approach reveals only information that would be contained in the ﬁnal ﬁtted parameters, such parameters themselves might leak information about individual samples, as our approach is not differentially private (DP) [16].
By adding Gaussian or Laplacian noise to each sample’s features when computing the S g terms one could, in principle, make the resulting algorithm
DP [1].
However it is unclear to what extent the noise would impact the ﬁnal accuracy of the method.
Finally, we only consider an honest-but-curious setting.
We do not explore the threat of a malicious participant that would purposely deviate from the protocol to either gain more information or to jeopardize the convergence.
We leave this investigation to future work
Acknowledgement
The authors would like to thank the four anonymous reviewers, as well as the anonymous area chair reviewer for their relevant comments and ideas which signiﬁcantly improved the paper.
Checklist
For all authors... (a) Do the main claims made in the abstract and introduction accurately reﬂect the paper’s contributions and scope?
[Yes] (b) Did you describe the limitations of your work?
[Yes] We described them in Section(c)
Did you discuss any potential negative societal impacts of your work?
[N/A] (d) Have you read the ethics review guidelines and ensured that your paper conforms to them?
[Yes] 2.
If you are including theoretical results... (a) Did you state the full set of assumptions of all theoretical results?
[Yes] (b) Did you include complete proofs of all theoretical results?
[Yes] The full proof of the main theoretical results,
i
e. the convexity of the Yeo-Johnson negative log-likelihood (Proposition 3.1) is provided in Appendix C. 3.
If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?
[No] The code of the experiments is not provided, but a detailed pseudo-code of the newly proposed algorithms are provided.
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)?
[Yes] We speciﬁed all the hyperparameters and the details of the numerical experiment in Appendix E. (c)
Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?
[Yes] Standard deviations or quantiles of the results with respect to the seed are provided (cf plots)
(d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)?
[No] The experiment are not heavy and run easily on a personal computer, on a CPU 4.
If you are using existing assets (e.g., code, data, models) or curating/releasing new assets... (a) If your work uses existing assets, did you cite the creators?
[Yes] The datasets used are open datasets available online and are systematically cited.
(b) Did you mention the license of the assets?
[Yes] The licence of the datasets used are provided in Appendix E. (c)
Did you include any new assets either in the supplemental material or as a URL?
[No] (d) Did you discuss whether
and how consent was obtained from people whose data you’re using/curating?
[N/A] We are using open datasets available online.
The genomic dataset from TCGA have been previously anonymised by its creator before publication (e) Did you discuss whether the data you are using/curating contains personally identiﬁable information or offensive content?
[N/A] 5.
If you used crowdsourcing or conducted research with human subjects... (a) Did you include the full text of instructions given to participants and screenshots, if applicable?
[-Let us consider the task of ﬁnding the smallest element u i 
-]
[-Then it performs a binary search between i = i max / 
-]
[-It is important to note that this algorithm is correct even if f is not increasing, as long as f ( s ) < 
-]
[-Let { a i } i =
-]
[-As f i > 0 , we can conclude from Eq. 
-]
[-i γ i and σ 
-]
[N/A] (b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable?
[-We just need to show that T ( a, λ ) > 
-]
[-Note that if a > 1 , then D > 
-]
[N/A] (c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation?
[N/A]