Title: Vertical Federated Feature Screening
Venue: NeurIPS 2025
Accepted: yes
Keywords: Vertical federated learning

Feature screening

Ultrahigh dimensionality

Imbalance data

Subsampling
Abstract: With the rapid development of the big data era, Vertical Federated Learning (VFL) has been widely applied to enable data collaboration while ensuring privacy protection. However, the ultrahigh dimensionality of features and the sparse data structures inherent in large-scale datasets introduce significant computational complexity. In this paper, we propose the Vertical Federated Feature Screening (VFS) algorithm, which effectively reduces computational, communication, and encryption costs. VFS is a two-stage feature screening procedure that proceeds from coarse to fine: the first stage quickly filters out irrelevant feature groups, followed by a more refined screening of individual features. It significantly reduces the resource demands of downstream tasks such as secure joint modeling or federated feature selection. This efficiency is particularly beneficial in scenarios with ultrahigh feature dimensionality or severe class imbalance in the response variable. The statistical and computational properties of VFS are rigorously established. Numerical simulations and real-world applications demonstrate its superior performance.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces a feature selection method to help eliminate unnecessary features in vertical federated learning (VFL). The benefit is to reduce the communication overhead without sacrificing the accuracy of the trained model. It is possible because in ultra-high dimensional settings, only a small number of features truly contribute to the model prediction, and other features can be pruned. The idea is to first ask the passive participants to group the features they own and calculate a group-wise score for the server to remove those surely unimportant ones. Then, a similar process is done on individual features of those remaining groups. Theoretical studies and experiments on simulated and real-world datasets have been conducted to show the effectiveness of the proposed method.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths\n+ VFS supports secure VFL and extremely imbalanced datasets. These are practical considerations that are commonly overlooked in the VFL literature.\n+ The hierarchical pruning (i.e., group->individual) is sound. Encryption overhead is often a concern in practical deployment of FL systems. The reduction in such overheads can be of broad impact.\n+ Theoretical studies have been conducted to analyze the properties of VFS.\n\nWeaknesses\n- The paper writing could be improved. Technical details are insufficient. For instance, how Y_{enc} is being used by the clients to compute the encrypted statistic is not clearly articulated.\n- The choice of real-world datasets is not clearly justified. While those datasets are not simulated, it is unclear whether they are for VFL. As shown in prior work [1], VFL datasets are of different characteristics than HFL, and randomly distributing features to different \"clients\" can create an artificial dataset that does not represent VFL.\n\n[1] Wu, Zhaomin, Junyi Hou, and Bingsheng He. \"Vertibench: Advancing feature distribution diversity in vertical federated learning benchmarks.\" ICLR 2024.\n\nQUESTIONS:\n1. Could you clarify how to compute the statistic? It is unclear how to make use of Y_{enc} to measure the statistic for pruning.\n2. Could you elaborate on the choice of the real-world datasets used in the experiments? How are those features being partitioned and distributed across different parties. The reviewer is particularly concerned about whether this partitioning is natural or artificial. If it is artificial, whether it can be a representative of real-world VFL problems.\n\nLIMITATIONS:\nThe authors discussed the limitations of existing methods. However, they did not discuss the potential impact of limitations of the method they propose.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 2, "CLARITY": 2, "IMPACT": 4, "ORIGINALITY": 3, "DATE": "2025-06-16"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors address the problem of high computational, communication, and encryption costs in Vertical Federated Learning (VFL) under ultra-high dimensional feature spaces and imbalanced (rare event) response variables. Existing federated feature selection methods struggle to scale in these settings due to the enormous feature dimension and the resource-intensive secure multi-party computations involved. To solve this, the authors propose the Vertical Federated Feature Screening (VFS) algorithm, a two-stage coarse to fine screening procedure that first eliminates irrelevant feature groups and then performs refined screening on individual features, all under secure encryption. They combined subsampling technique to address class imbalance with a model-free statistic compatible with encrypted data, they significantly reduce the resource burden for downstream federated learning tasks. The authors achieve theoretical guarantees of consistency and asymptotic properties for their method, and demonstrate through both simulations and real-world experiments that VFS can maintain high predictive performance while reducing encryption and computation costs by an order of magnitude compared to classical approaches.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\n1. The authors present a strongly motivated problem. In practice, datasets with very high feature dimensionality and severe class imbalance can create serious challenges for federated learning, especially when participating devices have limited computational resources.\n\n2. The application of U-statistics to define a model-free VFS statistic is an elegant choice, providing an asymptotically consistent way to measure associations between features and rare-event labels without relying on restrictive parametric assumptions.\n\n3. The paper offers a solid theoretical analysis, rigorously establishing convergence, screening consistency, and bounded model size properties of the proposed method.\n\n\nWeakness:\n\n1. The authors did not implement a full vertical federated learning algorithm (such as SplitNN or SecureBoost) to directly verify how their proposed feature screening would impact downstream VFL model training.\n\n2. The paper does not adequately investigate how robust the feature selection is when coupled with an actual VFL training pipeline, leaving open questions about its real-world effectiveness.\n\n3. The paper is not studied the scalability of their algorithm across a VFL framework. With N number of clients. \n\n4. The work does not analyze potential privacy leakage risks from the intermediate screening statistics themselves, which might expose sensitive label information to adversaries under certain attack models.\n\nQUESTIONS:\n1. In Stage 2, the fine screening depends on the features retained from Stage 1. Could cascading errors accumulate if Stage 1 removes too many features? Did the authors investigate how robust the method is to Stage 1 mistakes? Since the group-level screening uses a hypothesis-testing framework, how are the type I and type II errors controlled across potentially many groups, and is there a multiple testing correction strategy built in? \n\n2.  Would it be feasible to skip Stage 2 altogether if the initial grouping was fine-grained enough? In other words, is there a principled trade-off between group granularity and the necessity of a second stage?\n\nLIMITATIONS:\n1. One limitation of the work is that authors have not tested their method in a VFL setup with multiple clients involved with a diversed amount of features and classes (multi-class). Therefore, it is difficult to say at this moment the applicability of the algorithm in a real scenario.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a Vertical Federated Feature Screening (VFS) algorithm designed to filter out spurious features that may arise in Vertical Federated Learning (VFL) settings. The authors support the effectiveness of their method through both theoretical results and empirical evaluation.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths\n- The proposed VFS algorithm can be used as a preprocessing step before existing VFL algorithms such as LESS-VFL to improve overall efficiency. Notably, VFS appears to be more efficient than classical screening methods under class imbalance condition.\n- The authors propose a model-free VFS statistic inspired by generalized U-statistics, and they prove theoretical properties of the proposed method.\n(Due to time constraints, I was unable to fully verify the theoretical proofs provided in the appendix. I assume that the proofs are correct and base my review on the theoretical results presented in the main text.)\n\nWeaknesses\n- While feature screening based on generalized U-statistics has been studied in the statistics literature, it may be unfamiliar to a general machine learning audience, including myself. I suggest providing a brief background on this topic in the main text or appendix, which would help clarify the novelty of the proposed method.\n- Given recent trends in the ML community, more extensive experiments on real-world datasets are expected.\n- In the experiments, the authors still apply an additional federated feature selection method such as LESS-VFL. Is it difficult for the proposed method to perform sufficiently accurate feature screening to enable the use of a vanilla VFL algorithm? If possible, it would be helpful to include results of applying vanilla VFL directly after VFS.\n\nQUESTIONS:\n- In Section 6, is there a communication cost difference between using LESS-VFL alone and using VFS + LESS-VFL?\n- Please clearly define m_0 and m_1 in Equation (1).\n- The statement of Theorem 1.1 does not appear sufficient to justify the claim in line 203. In particular, tilde{C_2}>1/C_1 is required where C_1 is defined as C_1 = liminf n_1^gamma/log n > 0 (the positivity comes from assumption (C3)). Such a result would be needed to ensure that the second term on the left-hand side of Theorem 1.1 tends to zero as n_1\\to\\infty.\n- Are the datasets used in Section 6 class imbalanced?\n\nLIMITATIONS:\nNo. I would encourage the authors to include a dedicated section that discusses the limitations of their approach.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-01"}

comments: SUMMARY:
The proposed method attempts to address the ultrahigh dimensionality of features and the sparse data structures inherent in large-scale datasets in a VFL context. The gist of the method lies in a two-stage approach by which features are selected in a coarse-to-fine manner: first by groups then by individual features.

STRENGTHS AND WEAKNESSES:
Stength: 
+ The idea, which is based on Generalized U-statistic, seems straightforward and effective. 
+ Theoretically analysis elucidate the effectiveness of the method. It also provides kind of performance estimation which seems agree with the empirical results. 

Weakness: 
- empirical results on real datasets are limited, in terms of the number of features tested (50K features are only considered moderated in real-life applciations)
- feature selection methods are not necessarily limited to VFL. It would be good to compare the proposed with more baseliens e.g. those referred in Sect.2.

QUESTIONS:
1. how different options in the grouping method of stage one would affect the overall performance?

2. in terms of scalbility, what will happen if apply the method to features more than 100K?

3. how would privacy-preserving mechanisms e.g. DP or HE affect the proposed method?

LIMITATIONS:
yes
TITLE: Official Review
CLARITY: 2
IMPACT: 2
ORIGINALITY: 3
DATE: 2025-07-05