Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: The paper proposes a preprocessing protocol based on multi-party computation that works independently of how the data is skewed among a few parties.

strengths_and_weaknesses: The algorithm makes clever use of SMPC to minimize cost while preserving security. While it reminds me of secure median computation (Aggarwal et al., J. Cryptol. 2010), it is clearly worthy of independent publication. The interesting aspect is the following: The desired algorithm is iterative. This iteration is replicated in the secure protocol as follows: In every step, there is a relatively simple computation, the result of which (one bit) is revealed. This bit is then used for the next step in the manner of bisectional search. The revealing operation raises the question of the security of doing so. The security is given by the fact that the final result (which is revealed anyway) implies all the intermediate results, and thus, the adversary does not learn any extra information as long as all parties follow the protocol.

The definition of SMC in Section 2 is more limited than the generally used one. The authors only consider computation on local shares but not computation involving communication (line 130). General multi-party computation crucially involves a protocol without which only very limited functionality could be achieved.

The evaluation of the secure computation (Appendix D.3) does not correspond to the state of the art. This might suffice given the low complexity, but I would like to note that there are more efficient ways for secure comparisons (Escudero et al., Crypto 2020).

questions: n/a

limitations: yes
review_title: null

final_score: rating: 4: Borderline reject: Technically solid paper where reasons to reject, e.g., limited evaluation, outweigh reasons to accept, e.g., good evaluation. Please use sparingly.
review_content: summary: This paper proposes an exponential search method for YJ transformation in centralized and federated learning settings. It proved that the negative log-likelihood of YJ transformation is strictly convex with respect to \lambda. Building upon that, they provide a method to do an exponential search to find the best \lambda, \mu, and \sigma for YJ transformation. The paper also applies the Secured Multiparty Computation to provide a privacy-preserving guarantee in the process of Federated YJ transformation.
strengths_and_weaknesses: Strengths:
- Provide a novel exponential search method for YJ transformation.
- The proposed method achieves high accuracy compared to Brent minimization, which is widely used.
- The proposed method has better numerical stability than existing methods (Brent minimization).

Weakness:
- Unclear threat model: can not understand the purposes or capabilities of the threat model.
- The proposed method will destroy the heterogeneity of Federated Learning: After the YJ transformation process, it seems like every client will have the same distribution since they share the same \lambda_*, \mu_*, \sigma_*. This poses a risk to FL: if the adversary knows the data distributions of every client (since they have the same distribution), the adversary will quickly deploy the attack of the tails [1]. Moreover, since the threat model is "honest-but-curious", using the YJ transformation will pose a privacy risk to the clients' data since the server can deploy the membership inference attacks [2].
- There's no theoretical guarantee of the clients' data privacy leakage from the output of the proposed method: since "the fitted parameter \lambda_* contains all the information revealed during the intermediate steps," whether it can leak any information and whether we should protect it? 

[1] Wang, Hongyi, et al. "Attack of the tails: Yes, you really can backdoor federated learning." Advances in Neural Information Processing Systems 33 (2020): 16070-16084.

[2] Shokri, Reza, et al. "Membership inference attacks against machine learning models." 2017 IEEE symposium on security and privacy (SP). IEEE, 2017.
questions: - In Algorithm 3, who will run the algorithm, and what is the job of the clients and the server?
- What is the reason you only consider cross-silo FL since, in my opinion, this method can be generalized to all kinds of FL.
- Can you please explain the log-likelihood in Eq. 1? I followed reference 50 in the text, but there's no information in the reference. 
- What is the advantage of the proposed method for FL?
limitations: - Unclear threat model
- The proposed method will destroy the heterogeneity of Federated Learning.
- There's no theoretical guarantee of the clients' data privacy leakage from the output of the proposed method.
review_title: null

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: The authors propose ExpYJ, a new algorithm for feature Gaussianization which beats Brent algorithm in terms of numerical stability but asymptotically slower than Brent algorithm. The authors them combine ExpYJ with secure aggregation to obtain FedYJ.
strengths_and_weaknesses: The authors propose a federated feature Gaussinaization algorithm by first proposing a new optimization algorithm to optimize the hyper-parameters for YJ transformation and then combining it with secure aggregation in FL.

Strength:
1. The paper is well written and easy to follow.
2. The authors evaluate their algorithm on several downstream applications.
3. The improvement on numerical stability is a good contribution. However, I think evaluation on only one dataset is not enough. I encourage the authors to validate this advantage on more datasets.

Weakness:
1. The technical novelty is limited. The proof of strict convexity is a contribution, but with the convexity of Box-Cox transformation, the proof is not too challenging.
2. FedYJ requires multiplication on secret shared values so benchmarking the system performance (how long does it take to perform the computation and communication) is a necessity.
3. The authors highlight the heterogeneity challenge in the introduction but I don't see how they customize their algorithm to deal with heterogeneity. If I understand correctly, the application itself is not sensitive to heterogeneity. For example, I cannot tell any difference between Figure 3(a) and 3(b).
4. The threat model is too weak, which makes the whole setting degenerate to distributed learning instead of FL.
5. Proposition 4.1 is correct but the proof is too sketchy.
6. To use SMC, the data points need proper discretization. The authors do not mention any details about the discretization process like the hyper-parameters.
7. Some minor points: 7.1 There are many references to Appendix C. I recommend the authors to refer to subsections in Appendix C which will make the flow clearer. 7.2 The secure multiparty computation background is almost taking up a whole page. Consider moving it to the appendix.
questions: 1. Please clarify how the algorithm is customized to deal with heterogeneity if any?
2. Have the authors considered discretization in SMC when performing the empirical evaluation?


limitations: 1. Please include empirical evaluation of computation and communication cost on real clusters.
2. Please add details about discretization for SMC.
3. Please validate the numerical stability advantage on more real-world datasets.
4. Please refine the proof for Proposition 4.1 (for example, simulation-based proof).
5. Please clarify how the algorithm is customized to deal with heterogeneity if any?
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: The paper addresses the Yeo-Johnson feature transformation, and how it can be computed in the federated setting, where privacy and security restrict how information can be shared. The paper demonstrates the convexity of the log-likelihood function, which enables an efficient search procedure in the multiparty computation model that does not leak any information beyond the revealed answer.  Experiments are included that show the behavior inpractice. 
strengths_and_weaknesses: Strengths 

- There is growing interest in secure machine learning that goes beyond the core training part of the ML pipeline, and provides security for data preparation in addition

- The paper shows a nice result about the convexity of the Yeo-Johnson transformation which enables its secure computation

- The proof of minimum leakage (proposition 4.1) is a very nice contribution, and clearly makes the argument that there is no overall leakage from the algorithm

Weaknesses

- The paper does not offer insights that go beyond the immediate question of the YJ transformation. 

- The search algorithm requires some operations which are potentially costly to perform in the SMC setting

- The paper glosses over the technical details of applying the algorithm in the SMC setting: there are some moderately complex polynomial operations on data that require multiplications / exponentiations, but the cost of these is not discussed in much detail -- Appendix D gives only a very high level overview of basic secret sharing.  Moreover, the paper does not indicate where the challenge lies: once algorithm 1 is presented, it seems that Algorithm 3 is very similar except that data access is performed through secret sharing.  

- The communication overhead is high (8MB per client per feature), and there is no discussion of whether this could be reduced or traded off  -- this is why the method is motivated in the cross-silo rather than cross-device FL setting
questions: An alternative approach in the cross silo setting would before each client to apply the YJ transformation locally to their data, on the assumption that the distribution is common across clients.  Can you show examples that demonstrate where this heuristic will fail? 


limitations: The conclusions section has a nice discussion of limitations and future work that is appropriate for the paper.
review_title: null

Meta-review: metareview: This paper studies data preprocesing in the federated learning setting.
It proposes a simple and elegant algorithm for performing a Yeo-Johnson (YJ) power transform on univariate numerical data. This is a nonlinear transform intended to make the data more like a Gaussian.

The paper shows that the likelihood objective is convex and that we can sign its derivative using linear aggregates, which can be performed using secure multiparty computation. This permits us to optimize the transformation using binary search / exponential search. In the honest-but-curious setting, this exponential search does not reveal any superfluous information.

Overall, the reviewers thought this was a valuable contribution and that the paper is well-written and technically sound and that the experimental evaluation is adequate.
However, the reviewers did identify some limitations and directions for further work:
 - *Trust Model.* The paper only considers the honest-but-curious setting. What goes wrong if we consider a more powerful adversary? Can we provide further guarantees?
 - *Leakage via final output.* There is still the possibility of leaking sensitive information via the final parameters $\lambda$, $\mu$, & $\sigma$. This is beyond the scope of MPC guarantees, but it raises the question of whether these techniques can be combined with tools like differential privacy to address this concern.
 - *Efficiency.* The overhead of the system is still quite high in terms of rounds of interaction and total data transfer. The paper appropriately discusses this limitation.
 - *Heterogeneous data.* The paper discusses data heterogeneity, but it is unclear how relevant this is. The algorithm only uses linear aggregates over the coalition, so it shouldn't matter if the data is homogeneous or heterogeneous. Figure 3 compares these two settings, but the results appear to be exactly the same. As such, this figure could be removed from the camera ready version.

On balance, this paper seems appropriate for acceptance at NeurIPS. Data preprocessing is an important task (that is often underappreciated) and the paper proposes and interesting method for doing this in the federated learning setting.
confidence: Less certain
Decision: Accept