PubSub-VFL: Towards Efficient Two-Party Split Learning in Heterogeneous Environments via Publisher/Subscriber Architecture
Authors: Yi Liu
Yang Liu
Leqian Zheng
Jue Hong
Junjie Shi
Qingyou Yang
Ye Wu
Cong Wang
Venue: NeurIPS 2025 poster ()
Keywords: Vertical Federated Learning
Publisher/Subscriber Architecture
Computational Resource Utilization
Asynchronous Mechanism
Primary area: infrastructure
TL;DR: We propose PubSub-VFL, a novel VFL paradigm with a Publisher/Subscriber architecture optimized for two-party collaborative learning with high computational efficiency.

Abstract: With the rapid advancement of the digital economy, data collaboration between organizations has become a well-established business model, driving the growth of various industries. However, privacy concerns make direct data sharing impractical. To address this,  Two-Party Split Learning (a.k.a. Vertical Federated Learning (VFL)) has emerged as a promising solution for secure collaborative learning. Despite its advantages, this architecture still suffers from low computational resource utilization and training efficiency. Specifically, its synchronous dependency design increases training latency, while resource and data heterogeneity among participants further hinder efficient computation. To overcome these challenges, we propose \texttt{PubSub-VFL}, a novel VFL paradigm with a Publisher/Subscriber architecture optimized for two-party collaborative learning with high computational efficiency. \texttt{PubSub-VFL} leverages the decoupling capabilities of the Pub/Sub architecture and the data parallelism of the parameter server architecture to design a hierarchical asynchronous mechanism, reducing training latency and improving system efficiency. Additionally, to mitigate the training imbalance caused by resource and data heterogeneity, we formalize an optimization problem based on participants’ system profiles, enabling the selection of optimal hyperparameters while preserving privacy. We conduct a theoretical analysis to demonstrate that \texttt{PubSub-VFL} achieves stable convergence and is compatible with security protocols such as differential privacy. Extensive case studies on five benchmark datasets further validate its effectiveness, showing that \texttt{PubSub-VFL} compared to state-of-the-art baselines not only accelerates training by $2 \sim 7\times$ without compromising accuracy but also achieves computational resource utilization by up to 91.07\%.

PubSub-VFL: Towards Efficient Two-Party Split Learning in Heterogeneous Environments via Publisher/Subscriber Architecture
Yi Liu', Yang Liu?** Leqian Zheng', Jue Loong",Junji Shi?, Qingyou Yang", Ye Wu?, and Cong Wang
'Department of Computer Science, City University of Hong Kong BByteDance Inc.
{yiliu247-c, legizheng2-c>0my cityu. edu. congwangecityu edu (liuyang fromthu, tanzhuo. 107, (yangqingyou, wuye wuye. 20ye.2000)0bytedaacc. com
beyp:eeeggggeeeeeteeeeeeaaaae com
Abstract
With the rapid advancement of the digital economy, data collaboration between organizations has become a well-established business model, driving the growth various industries. However, privacy concerns make direct data sharing impractical. To address this, Two-Party Split Learning (a.k.a. Vertical Federated Learning (VFL)) has emerged as promising solution for secure collaborative learning, De- spite its advantages, this architecture still suffers from low computational resource utilization and training efficiency. Specifically, its synchronous dependency design increases training latency, while resource and data heterogeneity among participants further hinder efficient computation. To overcome these challenges, propose PubSub- VFL, a novel VFL paradigm with a Publisher/Subseribe architecture op- timized for two-party collaborative learning with high computational efficiency. PubSub-VFL. leverages the decoupling capabilities of the Pub/Sub architecture and the data parallelism of the parameter server architecture design hierarchical asynchronous mechanism, reducing training latency and improving system effi- ciency. Additionally, to mitigate the training imbalance caused by resource and data heterogeneity, we formalize an optimization problem based on participants' system profiles, enabling the selection of optimal hyperparameters while preserving pri- vacy. We conduct a theoretical analysis to demonstrate that PubSub- -iabb-VV VFL achieves stable convergence and is compatible with security protocols such as differential privacy. Extensive case studies on five benchmark datasets further validate its effectiveness, showing that, compared to state-of-the-a baselines, PubSub- VFL not only accelerates training by 2 ~ 7x without compromising accuracy, but also achieves computational resource utilization rate of up to 91. 107%.
Introduction
In the digital economy, data has become crucial resource driving technological advancements in sectors like autonomous driving [1], healthcare [2], and e-commerce [3]. In this context, enterprises increasingly collaborating to aggregate diverse data sources to train Machine Learning (ML) models, improving user experience and efficiency [4, 5). However, centralized data collection poses significant privacy risks and potential breaches [6]. Additionally, regulations like the General Data Protection Regulation (GDPR) [7] impose strict limits on data aggregation and usage, requiring explicit consent and robust safeguards.
Cong Wang and Yang Liu are the corresponding authors.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

To address the above privacy concerns, enterprises generally deploy Two-Party Split Learning [8-10] (a.k.a. Vertical Federated Learning (VFL) [11-14]) to collaboratively train ML models without accessing the original data, as shown in Fig. 1. Specifically, in VFI,, the data is partitioned vertically, such that each party stores data corresponding to the same sample IDs but with different feature spaces [13, 15, 16]. This allows for comprehensive analysis through a secure exchange of intermediate results, protecting sensitive data while facilitating joint model training [1I, 17J. A notable example is the cooperation between banks and insurance companies aiming train a model to predict a customer's credit score, a common interest for both entities. Each party holds data on the same individuals but in different feature spaces: banks maintain financial transaction records, while insurance companies retain car accident reports. Due to privacy concerns, regulatory constraints (such as GDPR and HIPAA [18]), and communication network limitations, these features must remain localized within each party. In such scenarios, the VFL. approach becomes essential. Therefore, many efforts [19, 2, 20] have been invested in VFL research to further unlock the commercial value of data.
In SL.-based VFL.2, each party trains a partial deep net- work up to specific layer known as the cut layer (called bottom model), which maps raw data features into mean- ingful vector representations or embeddings for prediction tasks. These embeddings are securely transmitted (using methods like differential privacy [21]. homomorphic cryption [4], or secure two-party computation [2]) to the active party, which holds the labeled data. The active party completes the training using the remaining part of the net- work (called top model) without accessing the raw data of the other parties, thus completing a forward propagation round. Subsequently, gradients are back-propagated from the final layer of the top model to the cut layer. Only the gradients at the cut layer are sent back to the passive party, who then continues the backpropagation process locally. This iterative process is repeated until the model converges.
Tap MadelA
Secure Prenaod
(AE, DP)
Drsknti
Aligessen
Oppankiation
Batton
Organlintion
Bolion
Meddl
Figure I: Overview of the split learning (SL.)-based VFL. framework.
To reduce training costs and resource consumption, researchers and developers have been dedi- cated to designing efficient VFL architectures to enhance computational resource utilization and improve training efficiency. From a system-level perspective, previous works such as FATE [22] and PaddleFL. [23] have focused on achieving efficient data parallelism by employing Parameter Server (PS) [24] architectures, thereby significantly improving computational resource utilization. At the algorithmic level, considerable effort has been devoted to developing effective asynchronous VFL training protocols to optimize computational efficiency [25]. However, these methods still face limitations. Some approaches may not fully address computational bottlenecks, while others might fail to maximize computational efficiency under resource and data heterogeneity conditions. We highlight the reasons behind the above design deficiencies as follows:
Neglect of the Decoupling of the System. Since the VFL with PS setup is still synchronous and tightly coupled, there a waiting bottleneck between workers on different parties [26], where they either wait for each other or have computational dependencies (see Fig. 6 in Appendix B). Intro- ducing an asynchronous mechanism could potentially improve system decoupling. However, the unique characteristics of the VFL ID alignment [27] present significant challenges in implementing such a mechanism directly (see the detailed analysis in Appendix A).
Neglect of the Resource and Data Heterogeneity. In VFI., computational resource and data feature dimensions often vary significantly among partics, leading to imbalances in computing time [28, 29]. Existing methods typically focus incrementally improving the computational efficiency of individual parties, overlooking these discrepancies. Thus, the overall computational resources are underutilized due to the lack of holistic approach to addressing this issue.
To address the limitations of existing approaches, we propose an efficient VFL framework named PubSub-VFL, designed to achieve better computational resource utilization and improve training effi- ciency. PubSub-VFL combines the loose coupling advantages of the Publisher/Subseriber (Pub/Sub) architecture with the data parallelism benefits of the PS architecture, enabling flexible asynchronous communication and significantly enhancing system throughput. To further boost computational effi- ciency and ensure convergence, we design an adaptive cemi-asynchronous mechanism within the PS, forming a hierarchical asynchronous mechanism in conjunction with the Pub/Sub asynchrony outside
For the sake of convenience, the term "VFL" is used in the context refer the term "SL-based VFL".

---

the PS. Additionally, to tackle load imbalance issues caused by resource and data heterogeneity, we develop an optimization model that leverages system profile information to determine optimal hyper- parameters, thereby further improving resource utilization. Extensive evaluations on five benchmark datasets demonstrate that PubSub-VFL. achieves 2 ~ 7x improvement in computational efficiency over
baselines while maintaining model accuracy.
2 Related Work
Vertical Federated Learning with PS. The integration of PS [24] architectures into (V)FL systems has garnered attention due to its potential to improve scalability and flexibility. Early efforts such as [30] explored using PS architecture to achieve efficient aggregation and distribution of model parameters, simplifying the coordination multiple participants, but these were primarily focused on horizontal FL settings. More recently, some solutions demonstrated the applicability of PS architectures VFL contexts, showing improvements in resource utilization and dynamic participant management [26, 22, 23]. Castiglia et al. in [31] extended this approach by introducing hierarchical PS architectures, allowing for better load balancing and reduced latency in VFL. However, these solutions still grapple with issues such as decoupling from the unique step in VFL (i.e., ID alignment) to further unlock the potential of PS.
Asynchronous Vertical Federated Learning. Asynchronous training [32-34] methods have gained traction in VFL address the inefficiencies caused by straggler effects and the need for strict synchronization. Asynchronous VFL (AVFL) enables participants to train and exchange intermediate results independently, significantly improving system throughput and scalability. Recent works, such as those by [34], have demonstrated the efficacy of AVFL in mitigating delays caused by slow participants, thereby enhancing the overall efficiency of the learning process. Nevertheless, asynchronous methods introduce new complexities, including staleness in gradients and potential divergence of models. To tackle these issues, recent research has focused on developing consistency models [35, 36] and adaptive asynchronous strategies [37]. We summarize the comparison between existing methods and PubSub- VFL in Table in Appendix A.
3 Problem Formulation
We consider a realistic scenario where two organizations collaboratively train an ML model to perform a prediction task. We then formalize framework for VFL, with the objective of maximizing the utilization of computing resources between two parties. In VFL, data is vertically partitioned such that each party holds different features for the same set of samples. Specifically, Active Party Pa possesses dataset D₁ = {(x", y4 b)) where x, E Rd represents its feature vectors and y e R denotes the labels, while Passive Party Pe holds dataset D2 = {x," J with x E IR4,, Note that the features of the two parties are disjoint. The model architecture is split into two parts: the top model and the bottom model, where the bottom models (-)) and f-)-) are held by Pa and Pp. respectively, while the top model -(- is held by P. which holds the label. Before training begins, P. and Pe must identify data samples with matching identifiers (such as ID). To preserve the privacy of both parties, Private Set Intersection (PSI) [38] technique is typically employed to securely obtain the "shared"' dataset without revealing any private information. During the forward pass, P. computes an intermediate representation 2!¹ = fx(") (i.e., embeddings) and securely sends it to P. by using methods like DP protocol, which then computes 20t = fa (x") and aggregates these representations using 9()) to produce gr. i..., ú; (f (xf()x)) fx(xx)) The loss function L( 31) is calculated Pa. For example, for classification task, the loss function can be the Cross-Entropy Loss:
C(gr, vk) =
(yi log(ū.) + (1 yi) log(1 x)))
(1)
In the backward pass, Pa computes gradients ve L and Vo, L. sending V.r L back to P,, which uses it to compute Vo, C. Both parties update their local model parameters using gradient descent:
0 + 0 - 7/ Ve, C, 02 + 02 Ve,C. where 7) is the learning rate, and 0, and 0g the network parameters of f1 and f2, respectively. To maximize the utilization of computational resources, we need to balance the computational load between the two parties. Let Cost denote the computational cost of P. for processing fa and
(2)
3

---

System Design
System Workflow
Profiling Phuse
Planning Plase
Chime Barkel
Callhnaian Datrser
Tep-llttton Made
System Profile
Optiniration
Prablem
Griiber
Leateel
Publisker
Bock
Profiling
Generate Crofiguratios
Hlynaiic Programing
Chosel
Purty A Party
Hlantrar Capacitis
Wacker
Model Infa
Butch Allocation
Worker Allecation
Channe Allocation
Camollis Nuc
Weige Sbs
Cratn Bastritt
Misearr Badact
Execution Phase
Figure 2: System architecture and workflow of PubSub-VFL.
Costp denote the computational cost of P. for processing fp. We use TA to denote the total time taken by Pa for one iteration and use Tp to denote the total time taken by Pp for one iteration. The goal is to minimize the maximum time taken by either party, i.e., minimize max(TA, Tp). Assuming that the communication overhead is not negligible, we can write:
Ta = Cost + 'go Tp Coslp + le-
(3)
where ig is time for sending gradient Vo,C and fe is time for sending 2.. To balance the load, we aim to equalize TA and Tp: Ta = Tp. This can be achieved by adjusting the training efficiency of and fp or dynamically allocating computing resources. For example, if Cost > Costp, we can improve the training efficiency of fa or increase the training efficiency of fp to balance the workload. The formal objective is to minimize the maximum time taken by either party while ensuring convergence of the model:
min max (TA; Tr).
subject to: 1.2 EE C(ú,, 31) I K. where k is a small tolerance for the loss.
Discussion. Optimizing Eq. (4) is challenging task due to two key factors: 1) Resource and Data Heterogeneity. In practice, Pa and Pp often differ significantly in their computing resources and data dimensions. Additionally, Pp computational overhead in VFL is notably lower than Pa 's as P. does not need perform a backward pass of the top model. Therefore, is essential to balance resource allocation while accounting for the computational complexities of both parties. 2) Privacy Restrictions. Due to privacy concerns, it not feasible to manage and control resource allocation in a centralized manner. This limitation distinguishes the problem from traditional resource scheduling in VFL., requiring more nuanced approaches to ensure privacy while optimizing resource utilization.
System and Algorithm Design
4.1 System Design
Key Idea. To address the scarecrow solution's limitations (more details can be found in Appendix A), our key idea is to decouple the data ID alignment lask from the training lasks of workers across different parties. This approach allows workers to focus solely on local training without the need to ensure data ID alignment beforehand. To implement this, we introduce the Publisher/uubscrite (Pub/Sub) [39] architecture, which effectively separates the training process from the data ID matching task, enabling asynchronous operations while maintaining the necessary alignment. By doing so, can climinate waiting delays, improve system concurrency, and ensure seamless data ID alignment without burdening the workers with additional coordination tasks. More information about the Pub/Sub can be found in the Appendix B.
To this end, we design the PubSub- VFL system to support efficient and scalable VFL. As illustrated in Fig. 2, we introduce two types of communication channels: embedding channels and gradient channels, responsible for transmitting embeddings and gradients, respectively. To decouple data alignment from model training, we assign unique batch IDD to each training batch. This batch ID used to label both embedding and gradient channels, enabling precise coordination of intermediate

---

results across parties. Each worker sends its outputs to the appropriate channel based on the batch ID. avoiding synchronization delays. Given a total of n training samples and a batch size B, the system maintains [n/B] embedding and gradient channels. This design ensures consistent data alignment while supporting asynchronous training, thereby improving efficiency. In more practical scenario, computationally efficient workers may generate excessive embeddings or gradients, leading to channel congestion, which can impede model convergence or even cause training failures. To address this, we propose two mechanisms:
Each channel buffer can store up to p embeddings and q gradients, with each entry timestamped. When the buffer reaches its capacity, it discards the oldest embedding or gradient based on First-In-First-Ou (FIFO) [40] prineiple to prevent stale updates from affecting the training process.
Buffer Mechanism:
If subscribing worker does not receive the embedding or gradient from the publishing worker within a predetermined time Taa, it discards the current batch and notifies the other party to proceed with the next batch. The system then reassigns the batch to any available pair of workers for retraining, ensuring the continuity of the training process and mitigating delays caused by congestion.
Waiting Deadline Mechanism:
Intra-party Semi-asynchronous Mechanism. Building on the Pub/Sub architecture, we achieve inter-parte asynchronous communication, effectively eliminating worker-side waiting delays. To further improve computational efficiency within each party (i.e., between the PS and its workers), we extend this design with an intra-party asynchronous mechanism. However, this hierarchical asynchrony can hinder model convergence if not properly controlled. To mitigate this, we propose a dynamic sami-asynchronous mechanism that adaptively regulates the synchronization interval based on training feedback. Specifically, the synchronization interval AT, decreases as the model approaches the target accuracy, striking a balance between computation speed and convergence stability. The interval AT, defined as:
AT- = ATo tanh 2
2- ATo
ATo 2
- 2
+
(5)
where ATh is the initial asynchronous interval, is the current training epoch, and tanh(.) is the hyperbolic tangent function. Initially, when the model is far from the target accuracy, AT, is small, allowing the model achieve stable learning. As the accuracy increases, the interval increases and the synchronization frequency is reduced to fine-tune the model and ensure faster convergence.
Differential Privacy Protocol. To prevent the embedded information sent by Pp from being in- ferred by attackers (such as labels or features), specific perturbations must be added the embeddings. To balance privacy and utility effectively, we adopt Gaussian Differential Privacy (GDP) (see App- denix C for more details) as outlined in [21, 33] to safeguard the embedding information. addition, we prove in Appendix D that PubSub-VEL integrated GDP protocol can still converge stably.
4.2 System Profiling
Key Idea. PubSub MabSub-VFL effectively integrates Pub/Sub and PS architectures to harness the compu- tational capabilities of participating parties, substantially improving training efficiency. Nevertheless, the system remains susceptible to latency caused by resource and data heterogeneity. A key chal- lenge lies in optimizing resource allocation collaboratively without violating privacy constraints. To address this, we propose optimal configurations, e.g., the number of workers, batch size, and core allocation, based on each party's system profile, including model characteristics and hardware capabilities. By adapting these parameters to individual constraints, the system achieves balanced workload distribution, reduced latency, and improved overall efficiency, all while preserving data privacy.
parameter optimization strategy that determines
To achieve this goal, we first model the computation and communication delays of both the active and passive parties in the designed PubSub VFL system. For the Pa, we denote the number of workers as wa E [P, Q) batch size B, and total computing cores C.. For the Pp. the number of workers is Wp E [M, N). batch size as B, and total computing cores as Cp- Let (Ca,s i= (where Die, Ca.s < Ca. be the CPU cores allocated to each of the wa and {Cps )32 (where Eee Cpis S Cp) be the CPU cores allocated to each of the up. Therefore, the computation delay
5

---

of the forward pass of both parties can be formally defined as follows:
(B)
AB
My (B)
A,
(6)
Up
ca.
Σ Cp.ó
where 7//0) (B) is the time for forward pass of the Po on batch of size B, 7 (B) is the time for forward pass of the Pp on batch of size B, Ya. Xar 7p. and Ap are the proportionality constants. For simplicity, if all wp workers are assigned equally, it might become T/ (B) = Aa a and To (B)
Co
A= Similarly, let Ba and 3p be the constant for the backward pass. Thus, we have: Cp
T PaBau wa I (B) = Bp.BSuup (B)
Co
Cp
We give the forward and backward propagation time of the top model part of Pa as follows:
BiBiowa B'B.... I top (B) = Co
(8)
Ca
Next, consider the communication delay within the system pipeline. Let E denote the size of the embedding sent by the Pd and G the size of the gradient sent by the Pa- Each iteration involves two primary communications: Pp sends the embedding Pa. and Pa sends the gradient back Pp- Therefore, the total communication delay for each iteration can be expressed as:
E. Bb"
G B,
Temb
T Tgred
where B3 denotes the bandwidth capacities of the system. Since the semi-as asynchronous aggregation within the PS is performed internally within each participant, the communication delays from this step can be ignored, as they are generally fixed constants.
Pub/Sub. With a Pub/Sub architecture, one can sometimes overlap the next batch's 's forward pass at the P, with the Pa 's backward pass from the previous batch. In a fully pipelined system with enough buffering, the overall iteration time can be lower than the naive sum above. Nonetheless, many system analyses still approximate iteration time by "critical path" sum or maximum of partial sums [41, 42]. For simplicity, we continue with the additive formula here, but in practice, pipelining would reduce that total somewhat.
Remark on Hipelining/Asynchronn
4.3 System Planning Phase
Optimization Problem Formulation. Due to privacy constraints and restricted network access, participants cannot collaboratively execute fine fine-grained pipeline operations. To overcome this limitation, we estimate the computation and communication times in the Pub/Sub setting using observations from a synchronous baseline, illustrated in Fig. 2. These estimations enable the construction of an optimization model to determine the optimal initialization hyperparameters, specifically, the number of workers and batch size. This model aims to balance computation and communication costs while maintaining system efficiency under privacy and network constraints. Based on Eq. (4), the formal expressions for TA and Tp can be rewritten
TA = T + TT()) + Tic + Torad, Tp = TVT)) + TTp)) + Temb- Then, we rewrite Eq. (4) as follows:
(10)
min max(T) + TT())
I!() + Tyrad, T((2) + +T(P) Temb).
(11)
In Eq. (11), we want to choose the optimal hyperparameters, i.e., w. E (
Wp (M, M +I,..., Nj. an integral (or real) batch size < B < Bmaz (bounded above by some feasible maximum, i.e., memory constraints), to optimize the goal. We assume that each party has its own memory constraint per worker. Thus, we follow [41] to specify the memory usage functions as
MA(B)= MA + PABx MP MP + ppBX
(12)

---

where MA and Mp are the base memory consumptions at the active and passive parties respectively. and Pa. Pp (with exponent X) capture the extra memory needed per worker as a function of the minibatch size B. If the maximum available memory per worker at the Pa is MA and at the Pp is
Ma-Man 1/x
1/x
Mp, then the memory constraints become B <
.BS
Mr- Men
Thus, we
70
define the overall feasible maximum batch size due memory as
Ma (M MAO)
{
1/x
MM Mro
1/x
Bmax min
(13)
PA
PP
We assume that the candidate minibatch sizes taken from discrete set, i.e., B [ B2...., BR), but only those values that satisfy B S Bmax are feasible. Since both par- ties must finish their computations before proceeding to the next iteration, the overall per-iteration delay is the maximum of the two computation delays plus the communication delay:
T(p) + T + + (T()) + T((a) + T((a) top
min Ow up, B)
min wh (Mp. BS Denax
max
BB
(14)
Dynamic Programming Algorithm Design. Because the decision space (over Wa, "pp, and B) discrete, we now describe a dynamic programming approach to search for the optimal configuration.
We define a dynamic programming state by the triplet (i, j. r), where, indexes the candidate active worker count: Wa M +
for 1,2, ... (N M + 1), indexes the candidate passive worker count: Wpp = P j for 1,2...... (Q P + 1), and indexes the candidate minibatch size: B B, for 1,2,.... ... R, with the additional constraint B, Bmax- The cost associated with state (i.j. r) is defined as
Bo + N,B. + + P. + v. B }( + +i-
Cost(i,j,r) = max
C.
(2p p.
(15)
Cp
The objective is to find the state (i", """ r*) that minimizes this cost. The above dynamic programming solution (Algo. 2) and the pseudo code of PubSub-VFL. (Algo. I) can be found in the Appendix E.
5
Experiment
5.1 Experiment Setup
To evaluate the performance of our PubSub- -VLL system, conduct extensive experiments five datasets. All experiments are developed using Python 3.9 and PyTorch 1.12 and evaluated on a server with an (R)EL(R) XEON(R) GOLD 6530 (64-core CPU).
Datasets. We evaluate PubSub-VFL on four public benchmark datasets (see Table in Ap-
Table 1: Accuracy comparison results.
Dataset
Metric
VEL
VFL-PS
AVFL
AVFL-PS
Ours
pendix F) spanning both regression and classi- fication tasks, along with a large-scale synthetic dataset. For regression, we use the Energy [43] (19,735 samples, 27 features) and Blog [44] (60,021 samples, 280 features) datasets. For
Energy
RMSE
84.58
85.39
Blog
RMSE
23.20
23.12
23.38
23.45
22.34
Bank
AUC
96.54
Credit
AUC
81.90
81.34
80.83
80.34
82.34
Synthetic
AUC
91.27
91.31
90.97
91.21
92.87
classification, we adopt the Bank [45] (40,787 samples, 48 features) and Credit [46] (30,000 samples, 23 features) datasets. To assess scalability, we generate synthetic dataset with million samples and 500 features using Scikit-learn [47]. Each dataset is split into 70% training and 30% testing, with training data approximately evenly distributed between two parties. To simulate feature heterogeneity, vary the number of features assigned to each party.
Models. For the top model, we use Multi-Layer Perceptron (MLP) with two layers. For the bottom model, we use two models of different sizes, namely a ten-layer MLP and ResNet [48], which can verify the performance of PubSubVFL under different model sizes.
7

---

VAL
VEL-PS
AVEL
AVEL-PS
Ours
CALEM
103
WaatN) Teve
75
15
Crrdi
Datstet
Detantt
(a) Running Time
(b) CPU Utilization
(c) Waiting Time
(d) Comm. Cost
Figure 3: Comparison with existing baselines in computation and communication efficiency.
VFL
VFL-PS
AVFL
AVFL-P5
Ours
100e
1006
456.50
306.192
100.200
120x
Canre
Case
Running Time
(b) CPU Utilization
(c) Running Time
(d) CPU Utilization
Figure 4: Comparison with existing baselines on computation efficiency in resource and data heterogeneous scenarios.
Bank
Credict
Synthetic
116
1500
(a) Accuracy
(b) CPU Utilization
(c) Comm. Cost
(d) ASR
Figure 5: The impact privacy budget on the performance, efficiency, and security of PubSub VFL.
Table 2: Effect of the number of workers.
Table 3: Effect of the different batch size.
of Workies
10
20
30
52
512
A9c.15)
12.13
01.05
$2.05
02.23
32.00
52.35
02.21
Ausz (33)
01.75
91.87
92.55
1430.32
1067.57
344.76
124.01
573.09
(8))
43.77
(9))
1,688
Comm. (MB))
875.91
19966
1867 A8
1558.00
2197.27
For series of constants, set AT, 5, Tas: 10s, p 9 5. For the constants in the optimization model, we determined them through empirical experiments (see the set the learning rate 0.001, the number of workers to
Parameters.
Appendix H for details). In addition, Wa/wp E [2, 50), the batch size B E {16, 32, 64, 128, 256, 512, 1024), and C. + Cp 64.
this paper, we adopt the following baselines: 1) Pure VFL.: This is a classic VFL architecture that does not involve the PS architecture asynchronous mechanisms. 2)VFL. with PS: This is the most widely adopted VFL architecture in the industry. implemented in mature frameworks such FATE [22] and PaddleFL [23]. By leveraging the PS architecture, it enhances computational resource utilization and efficiency, enabling more effective parallel processing. 3) Asynchronous VFL. Building on traditional VFL, developers can integrate asynchronous mechanisms [32, 34] to implement Asynchronous VFL (AVFL), enhancing system efficiency by reducing idle time and improving parallelism. 4) Asynchronous VFL with PS: Building on AVFL, developers can integrate the PS mechanism [26] to further enhance VFL efficiency. Notably, the asynchronous implementation this architecture achieved through inter-party communication between parties.
Baselines.
Evaluation Metrics. For the four public benchmark datasets, we evaluate classification tasks using the Area Under the ROC Curve (AUC) and regression tasks using the Root Mean Square Error (RMSE). For the synthetic dataset, we use AUC as the evaluation metric classification tasks. To fairly assess computational resource utilization, we measure running time, CPU utilization, and waiting time/epoch. Additionally, we record communication cost to compare the communication efficiency of different methods. Results of additional experiments can be found in Appendix H.

---

5.2 Numerical Results
System Performance. We evaluate the performance of PubSub-VFL and baseline methods on five datasets across classification and regression tasks. To ensure a balanced allocation of resources and data, we evenly distribute CPU cores and feature sizes between the two parties. Additionally, to assess the impact of different bottom model architectures, we employ both ML.P and ResNet models. We report the best performance of each method under optimal hyperparameter configurations in Table (small size model) and Table 7 (large size model). The experimental results demonstrate that PubSub-VFL achieves accuracy comparable to or even surpassing bascline methods the Bank, Credit, and Synthetic datasets. This confirms that integrating the Pub/Sub architecture and semi-asynchronous mechanism does not compromise system performance or convergence.
Comp. & Comm. Costs. We evaluate the computational and communication efficiency of PubSub-VFL against baseline methods. Using our strategy, we set the hyperparameters to B = 256, Wa = 8, and w/p 10. For computational efficiency, we measure total running time, CPU uti- lization, and per-epoch waiting time required to reach a target accuracy of 91 %. Communication efficiency is assessed via the total communication cost. Experiments on the synthetic dataset show that PubSub-VFL. significaat outperforms all baselines in both computation and resource utilization. As shown in Fig. 3, PubSub-VFL achieves 7x reduction in running time and 35% higher CPU utilization compared to the best-performing baseline, AVFL-PS. These gains stem from reduced worker idle time and improved parallelism. Moreover, the hicrarchical asynchronous mechanism enhances convergence efficiency, leading to lower communication cost than other methods.
Resource and Data Heterogeneity Scenarios. We evaluate the computational efficiency of PubSub-VFI VFL and the baseline methods under varying resource allocation and feature size distri- bution scenarios. For the resource heterogeneous scenario, we set different CPU core ratios between Pa and Pp: 50:14. 48:16 40:24, and 36:28 (where the first value represents Pa 's CPU cores). For the data heterogeneous scenario, we set different feature size ratios: 50:450, 100:400, 150:350, and 200:300. In each scenario, we apply our optimization method to determine the best hyperparameters and configure them for PubSub-VFL. The experimental results, shown in Fig. 4, reveal that in the resource heterogeneous scenario, imbalanced computational efficiency between parties significantly increases waiting time and decreases CPU utilization in baseline methods. Specifically, the CPU utilization of PubSub-VEr VFL is still as high as 87.42% when the CPU core ratio 50: 14, while that of AVFL-PS is only 42.129% 12%. This happens because resource disparities exacerbate computational imbal- ances, further extending training latency. In contrast, PubSub- VFL effectively balances computational efficiency, reducing running time and maintaining high CPU utilization. In the data heterogeneity scenario, we observe similar trends. Additionally, we find that reducing the data dimension processed by Pa can further decrease running time (as shown in Fig. 4 (d). This is because it helps balance the computational load between both parties, aligning with our optimization model design approach.
Security Performance Evaluation. We evaluate it from two dimensions: system performance and defense against embedded inversion attacks [49]. We follow the above hyperparameter configuration to configure PubSub-VFL For the privacy parameter, we set E (0.1, 1, 2, 4, 8, 10, +
System Performance. We evaluate the impact of on PubSub- by recording its accuracy, CPU utilization, and communication cost on the Bank, Credit, and Synthetic datasets. The results, presented in Fig. 5, demonstrate that introducing the DP protocol has minimal effect accuracy and CPU utilization, indicating that PubSub-VEs VFL maintains its computational efficiency even with privacy protection. However, observe notable increase in communication cost due to the added DP noise, which leads to a slower convergence. Nevertheless, the results confirm that LubSub-VFL seamlessly integrates with security protocols while maintaining strong performance.
Defend Against Embedding Inversion Attacks (EIA). Similarly, we record the performance results (i.e., Attack Success Rate (ASR)) of PubSub-VER -FLL against EIA (we adopt it in [49]) on these datasets (more details can be found in the Appendix G). The results are shown in Fig. 5, showing that the introduction of the DP protocol can help PubSub- -VLL defend against EIA well.
Parameter Sensitivity Evaluation. We evaluate the impact of different numbers of workers and batch sizes on the performance of PubSub-VFI VFL under the same setting. Specifically, we set wa- = Wp E {4, 5, 8, 10, 20, 30, 50} and B {16, 32, 64, 128, 256, 512, 1024).

---

Table Comparison of Different Methods on Various Datasets
Method
Energy
Blog
Bank
Credit
Synthetic
All (PubSub-VFL)
83.94
22.14
96.97
86.07
94.17
w/o Tau
84.35
23.17
95.26
85.74
92.86
w/o Dynamic Programming
84.07
22.16
96.33
85.79
93.82
w/o AT
85.68
24.11
95.01
84.45
92.07
w/o PubSub
83.98
22,66
95.17
85.93
93.52
w/o Tax and A
85.81
24.24
94.32
82.69
91.73
VFL
84.24
23.18
94.97
83.42
92.74
VFL--S
84.14
23.07
94.74
85.44
92.67
AVEL
83.91
22.97
95.02
84.23
91.54
AVEL-PS
84.29
23.15
95.06
82.27
92.21
Effect of the Numer of Workers. We conduct experiments on the Synthetic dataset with B 32, and the experimental results are shown in Table 2. The results show that simply increasing the parallel factor (i.e., w) does not always improve computational efficiency. For example, we find that when w = 8, its computation and communication efficiency is the highest. This is because a large parallel factor will lead to slower convergence.
Efffett of the Batch Size. Similarly, we conduct experimens on the Synthetic dataset with Wa = Wpp Table 3 records the impact of different on the performance of PubSub- We find that blindly increasing B cannot always improve the computational efficiency of PubSub-VFL. For example, we find that when B 256, its computation and communication efficiency is the highest. This is because too large a batch size will also lead to slower convergence. The above results verify that we need a suitable method to find the optimal hyperparameters in the VFL task.
Ablation Studies. We conduct comprehensive ablation studies following the above experimental setup, including the same model architectures, number of clients, and heterogeneous resource settings. Experiments are performed on the Energy, Blog, Bank, Credit, and Synthetic datasets evaluate the individual contributions of PubSub-VEr VFL's key components. Specifically, to assess the impact of the waiting deadline mechanism, we set the deadline to Tail Os, effectively disabling the mechanism. To evaluate the dynamic programming algorithm, we adopt fixed worker allocation (i.e,, equal numbers of workers on both party sides), removing the adaptive scheduling enables. For the intra-party cemi-asynchronous mechanism, we remove this component while retaining the PS architecture to isolate its effect. Finally, to study the role of the PubSub architecture, we replace it with the AVFL-PS architecture while keeping all other components unchanged. These ablation experiments allow us to disentangle the influence of each design choice on overall system performance. The experimental results are shown in the Table 4. The Waiting Deadline and Intra-party Semi- asnchhoynnh Mechanisms are pivotal to PubSub- VFL's performance, effectively balancing synchronization and asynchrony mitigate gradient staleness and ensure timely updates. Their removal incurs significant degradation, e.g., up to 10% AUC drop on Synthetic and 1.7 % on Bank highlighting their role in stable convergence. The PubSub architecture and dynamic programming contribute more modestly to performance but enhance robustness by alleviating resource heterogeneity and coordination overhead, improving stability under coupled training dynamics.
6 Conclusion
The paper addressed the problem of underutilization of computational resources in VFL by proposing a new framework named PubSub- Specifically, PubSub-VEs enhanced computational efficiency by leveraging Pub/Sub architecture with hierarchical asynchronous mechanism. Furthermore, we theoretically prove the convergence of PubSub-VFL. reduces training latency, improves resource utilization, and maintains strong convergence and privacy guarantees, achieving up to 2 ~ 7x faster training and x 35% better resource efficiency than state-of-the-art baselines.
Limitations. One limitation of PubSub-VFL. is that it only supports two-party learning and has not yet been able to support multi-party learning. In future exploration work, we will seek ways to support efficient multi-party learning.
10

---

Acknowledgements
We thank all anonymous reviewers for their constructive comments, This work was supported in part by the Hong Kong Research Grants Council under Grants CityU 11218322, 11219524, R6021 -0F;; R1012-21, RFS21221S04, C2004-21G, C1029-22G, C6015-23G, and N CityU139/21 and part by the Innovation and Technology Commission (TTC) under the Joint Mainland Haong Kong Funding Scheme (MHKJFS) under Grant MHP/135/23, This work was also supported by the InnoHK initiative, The Government of the HKSAR, and the Laboratory for Al-Powered Financial Technologies (AIFT).
References
[1] T. Zheng, A. Li, Z. Chen, H. Wang, and J. Luo, "Autofed: Heterogeneity-aware federated multimodal learning for robust autonomous driving." Proc. of MobiCom, 2023.
[2] J. Ogier du Terrail, S.-S. Ayed, E. Cyffers, F. Grimberg, C. He, R. Loeb, Mangold, T. Marc- hand, O. Marfog, E. Mushtaq, et al., "Flamby: Datasets and benchmarks for cross-silo federated learning in realistic healthcare settings," " in Proc. of NeurIPS, 2022.
[3] R. Li, Y. Shu, Y. Cao, Y. Luo, Q. Zuo, X. Wu, J. Yu, and W. Zhang, "Federated cross-view e-commerce recommendation based feature rescaling," Scientific Reports, vol. 14, no. pp. 1-19, 2024.
[4] C. Zhang, S. Li, J. Xia, W. Wang, F. Yan, and Y. Liu, "(BatchCrypt): Efficient homomorphic encryption for {Cross-Silo) federated learning," Proc. of USENIX ATC, 2020.
[5] Y. Li, Y. Sun, Z. Cui, P. Shen, and S. Shan, "Instance-consiste fair face recognition," " IEEE Transactions Pattern Analysis and Machine Intelligence, 2025.
[6] K. Liu, S. Hu, S. Z. Wu, and V. Smith, "On privacy and personalization in cross-silo federated learning," in Proc. of NeurlPS, 2022.
[7] P. Voigt and A. Von dem Bussche, "The eu general data protection regulation (gdpr)," A Practical Guide, 1st Ed., Cham: Springer International Publishing, vol. 10, no. 3152676, pp. 10-5555, 2017.
[8] N. Chandran, D. Gupta, A. Rastogi, R. Sharma, and S. Tripathi, "Ezpe: Programmable and efficient secure two-party computation for machine learning." in Proc. of EuroS&P, 2019.
[9] N. Agrawal, A. Shahin Shamsabadi, M. J. Kusner, and A. Gascón, "Quotient: Two-party secure neural network training and prediction," " in Proc. of CCS, 2019.
[10] M. Poirot, Vepakomma, K. Chang, learning for collaborative deep learning in healthcare," " arXiv preprint arXiv: arXiv: 2019.
Salpathy-Crames R. Gupta, and R. Raskar, "Split
[11] Y. Liu, Y. Kang. T. Zou, Y. Pu, Y. He, X. Ye, Y. Ouyang, Y.-Q. Zhang, and Q. Yang, "Vertical federated learning: Concepts, advances, and challenges, IEEE Transactions on Knowledge and Data Engineering, 2024.
[12] Q. Yang, Y. Liu, T. Chen, and Y. Tong. "Federated machine learning: Concept and applications." ACM Transactions on Intelligent Systems and Technology (TIST), vol. 10, no. 2, pp. 1-19, 2019.
[13] G. Wang, B. Gu, Q. Zhang, X. Li, B. Wang, and C. X. Ling, "A unified solution for privacy and communication efficiency in vertical federated learning," " in Proc. of NeurlPS, 2024.
[14] T. Qi, Wu, C. Wu, L. Lyu, T. Xu, H. Liao, Z. Yang, Y. Huang, and X. Xie, "Fairvfl: A fair vertical federated learning framework with contrastive adversarial learning," in Proc. of NeurlPS, 2022.
[15] C.-j. Huang, L. Wang, and X. Han, "Vertical federated knowledge transfer via representation distillation for healthcare collaboration networks," in Proc. of WWW, 2023.
[16] Y. Xing, Z. Zheng, and F. Wu, "Preventing strategic behaviors collaborative inference for vertical federated learning," in Proc. of KDD, 2024.

---

[17] X. Jin. P.-Y. Chen, C.-Y. Hsu, C.-M. Yu, and T. Chen, "Cafe: Catastrophic data leakage in vertical federated learning." in Proc. of NeurIPS, 2021.
[18] L. O. Gostin, L. A. Levit, and S. Nass, "Beyond the hipaa privacy rule: enhancing privacy, improving health through research," 2009.
[19] Q. Zhang, B. Gu, C. Deng, S. Gu, L. Bo, J. Pei, and H. Huang, "Asysqn: Faster vertical federated learning algorithms with better computation resource utilizationn" in Proc. of KDD, 2021.
[20] B. Gu, Z. Dang, X. Li, and H. Huang. "Federated doubly stochastic kernel learning for vertically partitioned data," in Proc. of KDD, 2020.
[21] C. Dwork, "Differential privacy." in International colloquium on automata, languages, and programming, pp. 1-12, Springer, 2006.
[22] FederatedAI, "Fate: A federated learning framework."
FATE. Accessed: 2023-10-01.
[23] PaddlcPaddle, "Paddlefl: A federated learning framework based on paddlepaddle." https: //github. com/PaddleePdddle httpee/eeaaddeee/ddlllllllle PPaddleF.. Accessed: 2023-10-01.
[24] M. Li, D. G. Andersen, J. W. Park, A. J. Smola, A. Ahmed, V. Josifovski. J. Long, E. J. Shekita, and B.-Y. Su, "Scaling distributed machine learning with the parameter server," in Proc. of OSDI, 2014.
[25] Z. Li, K. Yang, J. Tan, W.-j. Lu, H. Wu, X. Wang, Y. Yu, D. Zhao, Y. Zheng, M. Guo, al., "Nimbus: Secure and efficient two-party inference for transformers," in Proc. of NewrlPS, 2024.
[26] Y. Wu, N. Xing, G. Chen, T. T. A. Dinh, Z. Luo, B. C. Ooi, X. Xiao, and M. Zhang, "Falcon: urivacy-preserving and interpretable vertical federated learning system." " in Proc. of VLDB. 2023.
[27] Y. He, X. Tan, J. Ni, L. T. Yang. and X. Deng, "Differentially private set intersection for asymmetrical alignment," IEEE Transactions on Information Forensics and Security, vol. 17, pp. 3479-3494, 2022.
[28] T. Castiglia, A. Das. S. Wang, and S. Patterson, "Compressed-vil:: Communication--ffiiiccan learning with vertically partitioned data," in Proc. of ICML, 2023.
[29] Y. Li and X. Lyu, "Convergence analysis of sequential federated learning on heterogeneous data," in Proc. of NeurlPS, 2024.
[30] K. Bonawitz, "Towards federated learning scale: Syste m design," in Proc. of MLSys, 2019.
[31] Castiglia, Wang, and S. Patterson, "Flexible vertical federated learning with heterogeneous parties," IEEE Transactions on Neural Networks and Learning Systems, 2023.
[32] K. Zhang, G. Wang, H. Li, Y. Wang, H. Chen, and B. Gu, "Asynchronous vertical federated learning for kernelized auc maximization," in Proc. of KDD, 2024.
[33] Chen, X. Jin, Y. Sun, and W. Yin, "Vafl: method of vertical asynchronous federated learning." " arXiv preprint 20t.::20000008.1. 2020.
[34] Q. Zhang, B. Gu, C. Deng, and H. Huang, "Secure bilevel asynchronous vertical federated learning with backward updating." " in Proc. of AAAI, 2021.
[35] J. Liu, H. Xu, L. Wang, Y. Xu, C. Qian, J. Huang, and H. Huang, "Adaptive asynchronous federated learning in resource-constrained edge computing." IEEE Transactions on Mobile Computing, vol. 22, no. 2, pp. 674-690, 2021.
[36] A. Koloskova, S. U. Stich, and M. Jaggi, "Sharper convergence guarantees for asynchronous sgd for distributed and federated learning." Proc. of NeurlPS, 2022.
[37] Zhang, I.. Gao, S. Lee, M. Zhang, and S. Avestimehr, "Timelyfl: Heterogeneity-wware asynchronous federated learning with adaptive partial training," in Proc. of CVPR, 2023.
12

---

[38] H. Chen, K. Laine, and P. Rindal, "Fast private set intersection from homomorphic eneryption." in Proc. of CCS, 2017.
[39] M. Srivatsa, L. Liu, and A. Iyengar, "Eventguard: A system architecture for securing publish- subscribe networks," ACM Transactions Computer Systems (TOCS), vol. 29, no. 4, pp. 1-40, 2011.
[40] Q. Herr and P. Bunyk, "Implementation and application of first-in first-out buffers," IEEE transactions applied superconductivity, vol. 13, 2, pp. 563-566, 2003.
[41] S. Ye, L. Zeng, X. Chu, G. Xing, and X. Chen, "Asteroid: Resource-efficiet hybrid pipeline parallelism for collaborative dnn training on heterogeneous edge devices," in Proc. of MobiCom, 2024.
[42] I.. Zheng, Z. Li, H. Zhang, Zhuang, Z. Chen, Y. Huang, Y. Wang, Xu, D. Zhuo, P. Xing, et al., "Alpa: Automating inter-and (Intra-Operator) parallelism for distributed deep learning." in Proc. of OSDI, 2022.
[43] L. All. "Appliances energy prediction." appliances -eeeaaneeeeeeeeyy-eeeaeeea peergy-preiction.
2017. 2017.
[44] I. University of California, "Blogfeedback data set." http://archivv..ess....dd datasets//LloFeeddbad 2014.
[45] I. University of California, "Bank marketing data set." http://aarchiee.tcssu....c datassts/Baak-Waareetinn 2012.
[46] U. I.. Repository, "Default of credit card clients dataset." https://ww. com/ ue uciml/defan wtt.///aaaaat-ooerr----ee----eee card- cl deettsddttsee, 2016.
[47] F. Pedregosa al., "Scikit-leam: Machine learning in python." https://scikit-learr learn, org/stable/, 2011.
[48] K. He, X. Zhang, S. Ren, and J. Sun, "Deep residual learning for image recognition." in Proc. of CVPR, 2016.
[49] C. Song and A. Raghunathan, "Information leakage in embedding models," in Proc. of CCS, 2020.
[50] K. Cheng, T. Fan, Y. Jin, Y. Liu, T. Chen, D. Papadopoulos, and Q. Yang, "Secureboost: A lossless federated learning framework," IEEE intelligent systems, vol. 36, no. 6, pp. 87-98. 2021.
[51] P. T. Eugster, P. A. Felber, R. Guerraoui, and A.-M. Kermarrec, "The many faces of publish/sub- scribe," ACM computing surveys (CSUR), vol. 35, no. 2, pp. 114-131, 2003.
[52] G. Cugola and H.-A. Jacobsen, "Using publish/subscribe middleware for mobile systems." ACM SIGMOBILE Mobile Computing and Communications Review, vol. 6, no. 4, pp. 25-33, 2002.
[53] Zhao, Z. Li, X. Ren, Ding. Yang, and Y. Li, "Vertimrf: Differentially private vertical federated data synthesis," in Proc. of KDD, 2024.
[54] M. Abadi, A. Chu, I. Goodfellow, H. B. McMahan, I. Mironov, K. Talwar, and L. Zhang, "Deep learning with differential privacy," Proc. of CCS, 2016.
[55] C. Qu, W. Kong, L. Yang, M. Zhang, M. Bendersky, and M. Najork, "Natural language understanding with privacy-preserving bert," in Proc. of CIKM, 2021.
13

---

NeurIPS Paper Checklist
The checklist is designed to encourage best practices for responsible machine learning research, addressing issues of reproducibility, transparency, research ethics, and societal impact. Do not remove the checklist: The papers not including the checklist will be desk rejected. The checklist should follow the references and follow the (optional) supplemental material. The checklist does NOT count towards the page limit.
Please read the checklist guidelines carefully for information on how to answer these questions. For each question in the checklist:
You should answer [Yes]. [No] [NA].
[NA] means either that the question is Not Applicable for that particular paper or the relevant information is Not Available.
Please provide short (1-2 sentence) justifiiatin right after your answer (even for NA).
The checklist answers are an integral part of your paper submission. They are visible to the reviewers, area chairs, senior area chairs, and ethics reviewers. You will be asked to also include (after eventual revisions) with the final version of your paper, and its final version will be published with the paper.
The reviewers of your paper will be asked to use the checklist as one of the factors in their evaluation.
While "[Yes] " is generally preferable to "[No]", ", it perfectly acceptable to answer "[No] provided proper justification is given (e.g., "error bars are not reported because it would be too computationally expensive" or "we were unable to find the license for the dataset we used"). In general, answering "[No] or "[NA] is not grounds for rejection. While the questions are phrased in binary way, we acknowledge that the true answer often more nuanced, so please just use your best judgment and write justification to elaborate. All supporting evidence can appear either in the main paper or the supplemental material, provided in appendix. If you answer [Yes] to a question, in the justification please point to the section(s) where related material for the question can be found.
IMPORTANT, please:
Delete this instruction block, but keep the section heading "NeurIPS Paper Check- list".
Keep the checklist subsection headings, questions/answers and guidelines below.
Do not modify the questions and only the provided macros for your answers.
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
We conducted extensive case studies support the claims we make.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation as long as is clear that these goals are not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
14

---

Justification:
We discuss the limitations of the proposed approach in the conclusion.
Guidelines:
The answer NA means that paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model vell-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested few datasets with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?
Answer:
[Yes]
Justification:
This paper provides convergence proof in the appendix.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
Justification:
We provide the complete code and data in the Supplementary Materials.
Guidelines:
The answer NA means that the paper does not include experiments.
15

---

If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. w
For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily new algorithm, the paper should make it clear how to reproduce that algorithm. (b) the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification:
We provide the complete code and data in the Supplementary Materials.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.. cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https: / //nips. //wpps.cc/publli/ https/ccppcccccidddeeeeeeeeeeeee http////ggiieee///uuuuuiiiiiiiii Swop///oddeisssiieooee piooniicc for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
16

---

Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g.. data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: We introduce the environment and settings required for the experiments in detail in the Experiments section.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Ye]]
Justification: We present detailed numerical analysis and discussion in the Experimental Section.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
OK to report I-sigma error bars, but one should state The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that out of range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: We introduce the environment and settings required for the experiments in detail in the Experiments section.
Guidelines:
The answer NA means that the paper does not include experiments.
17

---

The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted the paper conform, in every respect, with the NeurlPS Code of Ethics attps://neurfs cc/ hetoolii/Ethissuiddeeo
Answer: [Yes]
Justification:
We fully follow the NeurlPS'25 policy for manuscript submission.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer:
[Yes]
Justification:
We provide a detailed discussion of impact in the Introduction section.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
NA No, they should explain why their work has no societal impact or why the paper does not address societal impact.
If the authors
Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g.. deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Question:
Does the paper describe safeguards that have been put in place for responsible release of data models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Answer: [Yes]
18

---

Justification: We have carefully checked the published models and data and confirmed that they do not pose any risks.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification:
We fully follow the NeurlPS'25 policy for manuscript submission.
Guidelines:
The answer NA means that the paper does not existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset used and, possible, include URL.
The name the license (e.g.. CC-BY 4.0) should be included for each asset. For scraped data from a particular source (e.g., website). the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. cow. com/dataseta has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re- re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's 's creators.
13. New assets
Question: Are assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification:
The new dataset is documented in the attached material.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details the dataset/code/mooell as part their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
19

---

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification: Not applicable.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country or institution) were obtained?
Question:
Answer: [NA]
Justification: Not applicable.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Answer: [Yes]
Justification:
This paper uses LLM to polish the language.
Guidelines:
The answer NA means that the method development in this research does not involve LLMs as any important, original, or non-standard components.
Please refer to our L.MM policy (https: : /t////nuripppe/ cc//onfferecce//2000 /LLM) for what should or should be described.
20

---

A Scarecrow Solution
To optimize Eq. (4), common approach is to implement a PS architecture within each party to achieve parallel processing. This architecture is widely adopted by industrial-Ievel VFL frameworks like FATE [22] and PaddleFL [23]. The PS architecture VFL typically involves each party setting up PS and a set of workers, each running a VFL executor responsible for model training and security protocols. For a given task, the coordinator, responsible for scheduling system components, determines the parallel factor v, which specifies the number of workers to be created. It then instructs the agents of each party to initiate the PS and workers. This setup involves both inter-party Peer-to- Peer communication among workers from different parties, as well as intra-party communication between the PS and its workers, ensuring efficient coordination and execution of the VFL. task [26].
Specifically, in each iteration, the Pa 's PS selects a batch and divides the instance IDs into q subsets, such II IID]:-- ID,)) PS then broadcasts these subsets the Pp PS, which distributes each subset ID, to its workers. This ensures that during execution, the PS and workers of each party are aligned, with each group of workers processing the same instances. The workers of P. and Pp execute forward propagation concurrently, with the workers of P. completing the remaining forward propagation and performing backward propagation using the top model. Meanwhile, once the workers of Pp receive the corresponding gradients, they carry out backward propagation to update their local models. Afterward, each worker uploads the updated model parameters its respective PS. Finally, the PS aggregates these updates and broadcasts the refined model parameters to its workers, completing the iteration.
Limitations. While deploying the PS architecture in VFL can significantly improve training effi- ciency (i.e., decrease Cost: and Cosl2), it still faces certain limitations. Firstly, due to disparities in computational overhead among the parties, the computation times in VFI. are unbalanced. Since PS must ensure strict Il) alignment, the faster worker in pair of workers must wait for the slower one, or the training process will fail due mismatched embeddings. Moreover, deploying a PS does not fully address the issues of resource and data heterogeneity between parties, the PS architecture cannot facilitate resource collaboration between different parties. This limits its ability to optimize resource allocation and manage data heterogeneity effectively in VFL.
Table 5: Comparison of Different VFL Architectures
F Framework
Architecture
Asynchronou
Computational Emiieeey
Comm. Mechanism
Scalability Fault Tolerance
Implementation Representative Complesity
Frameworks
Pure VIL
Cesuralized
Low
VFT. with
High
Contalived PS Clon-
Metium
Medium
Medium
FATE 122]. Pad- didFi [13]
munication
AVFL.
Centriiizdd
You
Medium
Asynchmmons Poer- to Peer
Mediom
Low
High
SecureBocet [50].
Asy
AVFL
with
PS
Yos
High
Asynchmnons Communicatin
PS
High
Medium
Hfgh
Falcom [26]
PS
PutSub VEL
PubiSob with
Yes
Highest
Efficient Channels
PaleSub
Highest
High
Mediom
Proposed System
B Publisher/Subscriber Architecture
Pub/Sub Architecture. The Pub/Sub model consists of three primary entities: publishers, sub- scribers, and message broker (or middleware). Specifically, the Pub/Sub architecture design pattern that facilitates communication between different parts of a software system by decoupling message senders (publishers) from message receivers (subscribers) [51]. In this model, publishers generate messages and send them to central message broker, specifying topics or channels. Sub- scribers express interest in particular topics and receive messages related to those topies through the broker, without needing to know who the publishers are. This setup allows for scalable, flexible, and asynchronous communication, as publishers and subscribers can operate independently, and additional components can be added without disrupting existing interactions [52]. We summarize the advantages of the Pub/Sub architecture as follows:
Decoupling of Components: Publishers and subscribers operate independently, reducing system complexity and enhancing maintainability.
Sealability: The architecture supports high-throughput message dissemination, making it suitable for large distributed systems.
21

---

Syn. Dependencies
Forward Pass of the Active Party' Botiom Model
Backward Pass of the larrly's Model
ID Alignment
Forward Pass of the Passive
B
Backward Pass the Passive Party's 's Bottom Model
B
B
Party' Bottom Model
Forward Pass of the
Hackward Pass of the Active Party's 's Model
Party's Top
A
B
Aggregation
Forward Communicatin
Backward
Communiction
Parameter
Latency
Communication
Syn. Dependencies
Figure 6: Pipeline overview of the VFL with PS architecture. Synchronization dependencies may cause latency and latency 3, while latency is the computation dependency caused by alignment.
Message brokers often provide mechanisms for persistence, ensuring reliability even the presence of failures. Asynchronous Communication: Subscribers receive messages without blocking publishers, im- proving system responsiveness.
Fault Tolerance:
Traditional Asynchronous Mechanism
PubSub Mechanism
Torvani Computation of the
Furward C Computation
Forward Computation the Active Party's Top
Backward Computation of the Active Party' Modal
Active Party' Bostom Model
Party
Backward Computation of
Active Panty's 's Bottom Model
Barkward Computation the Pussive Party
AlReeuuee
Forward Communication
Communication
Backward
Pammetter Communication
Labency AT
A Asyechonnuus Proquency
Figure 7: Traditional Asynchronous Mechanism Pub/Sub Mechanism.
Traditional Asynchronous Mechanism v.s. Pub/Sub Mechanism. In Fig. we summarize the pipeline of the traditional asynchronous mechanism and the Pub/Sub mechanism in VFL. We found that the traditional asynchronous mechanism still has to decouple from ID alignment. Because the traditional asynchronous mechanism relies direct sender-receiver interactions using message queues, callbaeks, or polling, leading to tight coupling and increased complexity the number of workers grows. In contrast, the Pub/Sub mechanism introduces a decoupled communication model where publishers send messages subscribers receive relevant updates asynchronously, improving scalability and flexibility. Therefore, Pub/Sub can well decoupled from ID alignment, so that the next task can executed without additional waiting.
redundant latency because difficult
broker, and
C Differential Privacy Protocol
Definition C.1 A randomized mechanism M satisfies (p,, d for all measurable sets S and for all adjacent datasets D and D', the following inequality holds:
(16) where is a privacy loss parameter, aap is the standard deviation of the Gaussian noise added, and a small probability allowing for a slight relaxation of the privacy guarantee. We explain the GDP design applicable to the embedding mechanism belows.
P(M(D) E S] S e"F(M(D') E S 5,
Unlike traditional differentially private VFL. [53], which applies noise directly to the gradient, in SL-based VFL, privacy protection must be applied to the embedding sent by Pd to safeguard
22

---

intermediate interaction results. This necessity arises because adversaries commonly exploit em- bedding inversion attacks [49] infer P. private feature information. To ensure DP protection at the embedding level, we leverage the moments accountant technique introduced in [54] to precisely calibrate the DP noise. Specifically, we apply GDP by injecting randomized noise into selected neurons, effectively balancing privacy preservation and model utility while mitigating privacy risks in SL-based VFL. To this end, we set the variance of the Gaussian random neuron at the (-th layer as
dp = O V ((NN),
(17)
where Nm is the size of minibatch used at worker, N is the size of the whole batch, K is the number of queries (i.e., the number of batches processed by h at worker), then PubSub-VFL. satisfies μ-GDP for the data of worker. This method demonstrates the trade-off between accuracy and privacy. To increase privacy, i.e., decrease in (16), the variance of random neurons needs to be increased (cf. (17)). However, as the variance of random neurons increases, the variance of the stochastic gradient also increases, which will in turn lead to slower convergence.
Discussion. In practice, existing VFL. frameworks like FATE often employ homomorphic encryp- tion protocols protect the privacy of intermediate results, such embeddings gradients. However, these cryptographic techniques come with significant computational overhead, leading to increased costs for participants. Given that the participants our scenarios are institutions or enterprises with a low likelihood of engaging in malicious attacks, GDP emerges as a more economical and privacy-friendly solution. Furthermore, our experiments, we thoroughly evaluate the effectiveness of GDP in defending against various advanced inference attacks. The results demonstrate that GDP offers robust protection, making a practical choice for preserving privacy in VFL systems without compromising performance.
D Convergence Proof
D.1 Assumptions
In this section, we provide the necessary assumptions as follows:
Assumption D.1 (Smoothness). If the global loss function f(0) is L-smooth, thus, we have:
f(;) v f(02)l s Lll01 - 02ll. 401, 02.
(18)
Assumption D.2 (Strong Convexity). For the purpose of this analysis, we assume that f(0) is j-stroonly convex:
f(02) 2 f(0)) (V 01) 5110= - 0111..
(19)
Assumption D.3 (Bounded Stochastic Gradient Variance). Let g(0; E) denote the stochastic gradient computed on mini-batch. Then we have:
(I]g(0:E) V |(ON c².
(20)
Assumption D.4 (Bounded Delay/Staleness), Due to semi-asynchronous updates, the gradient used at iteration t is computed at delayed parameter 0,-r()) with r(t) AT. We assume that the delay satisfies
[ - S
Σ
110
8,ll.
(21)
5=t-r(()
Assumption D.5 (Gaussian DP Noise Injection). The Gaussian noise Edp added to each exchanged embedding is independent with
Edp ~ NN0 1) and the variance o determined by the calibration formula provided. Consequently, the effective update noise is increased to
(22)
a iotal = 0²
+ odp
(23)
Assumption D.6 (Reliable Communication under Capacity Constraints). The Pub/Sub mechanism guarantees that embeddings (of dimension d) are delivered within the delay bound T provided that the channel capacity p is not exceeded.
23

---

D.2
Convergence Analysis
Based on the necessary assumptions above, we write the following convergence objective.
Theorem D.1 If Assumptions D.1-D.6 are held and if the update rule
0c+1 00- 100-11100 11-((90000) + Edip.):
(24)
applied with constant learning rate chosen sufficiently small (so that higher-order terms are negligible), there exist constants such that the expected optimality gap satisfies
by (a2 + ozp))
B( - [(07)] (1-2ur o =2) (5(00) =(0*))+ +
(25)
4p
where 9 is the unique minimizer of f(0).
Proof D.1 First, review the gradient update rule. The gradient update with delay and DP noise is written as:
0.+1 0, n (ge-r(1) Editt)
(26)
where Edp,t is the independent DP noise at iteration t. We then use the L-smoothness (refer to Assumption D.1), we have:
I. f(0++1) & f(0)) + (V.(0)), - 01) + 2 110++1- - -ell
(27)
We substitute the gradient update rule, thus, we have:
Ln? f(0c+1) S f(0)) - f(..nn)(V))))...ooo 9t-(() + Edpa) + 119t (199--(1) + Edp,el?.
(28)
Taking expectation conditioned 0, and noting that Edp. is independent of Or with zero mean, get:
[[[f(0++)]] = f(0.) f(O,, (Eg,))),
Laj² E [119- + Edp.ell3]
(29)
+ 2
Since [[()--(())] (0,--((()) and using the independence and zero mean of Edp,t, thus, we have:
[119---(3) + Edp.ell?]
|V/f((===(a))1
+ [[19)-7())- - -V
(30)
+ E [1]]
By assumptions Assumptions D.3 and D.6, this yields
E. [lle--(t) [lp < [[ + 0² + oă.- = [V/((0-7())11 oiuul
(31)
Thus, the expected loss satisfies:
El((0(+)) S s(0,) (VV((())) V5(0L-7(1))
Lnd
(32)
+ 2 (IVf(0=-(0)11-
o o
Because of the delay, we need relate ((())) and (fr-r(()) Under the smoothness assumption and bounded delay (refer to Assumption D.4), one can show (following standard asynchronous update arguments) that:
(V)) 7(00,t()) |VS(0))1 d.,
(33)
with an error term & that is on the order of T. - |10 0,ll [[)))) This error is
typically bounded by term proportional n²² 2 ((,) Thus, for some constant Ct.
V f(0.), vf()--(0) > V - [1
(34)
24

---

Similarly, we can upper bound (()_ in terms of V f(t: plus similar
ervor. For simplicity the derivation, assume that the delay-related emoos yield an additional multipliaative factor of order C,n T,2T.
Strong convexity (refer Assumption D.2) implies:
[Vf(0=)] 2 (p( f(07))
(35)
where 0* is the unique minimizer.
Substitute the bounds back into the expected loss difference:
1) = f(0,) - (I -1t.:/O000--CCLLLLV1IV
Tap
+ 2 (IV tonaat)
(36)
f(0,) - - (1)) CiT
Ln7 [V]((1)1 +
LT²
o
2
Collecting the gradient terms:
B[f(0++1)] & - - (1- (1-CunP
Ln
Ln²
+
Fonut-
(37)
2
2
Using the strong convexity lower bound [[ 2n(s(0.)~ -((0)), we obtain:
1- -1- Ln (.7 )) (f(0))-f(0") f(00))
- (1(00=))-(007)
- 1 1 2pm
(38)
L + 2 diunal
For sufficiently small 1 (and ignoring higher-order terms), this recursion can be written approximately as:
B[f(O==)) - f(0)] (1-m++ f(0))) + C o
where C₁ and C2 are constants that incorporate the delay and smoothness effects. Unrolling the recursion yields:
E[f(0.) - f(0)) s (1- 1p-pp+ mp G7PL17)) (f(Oo) f(())) + C2ul o
(39)
2p
or equivalently.
[(f(0)) f(0*) < (1-
mu C -f(0o)- f(()))
C2nL + o22))
(40)
24
Here, the second term represents the error floor caused by the combined stochastic gradient noise and the additional DP noise.
Remark on DP Noise. The additional term o², in the variance o means that even if the algorithm converges in expectation, the asymptotic error floor is increased by the amount of DP noise injected. In practice, this is the trade-off between privacy (controlled by the noise multiplier and hence 2n and accuracy (since the convergence error floor grows with o2²g)
E Algorithms
The pseudo code of our improved PubSub-VFL. training process and the dynamic programming algorithm are deseribed as follows.
25

---

Algorithm PubSub-VFL Training Framework
Require: Active party Pa with dataset DI, Passive party P. with dataset D2
Ensure: Trained models far fp- and g
1: Initialize models far fp. and g with random weights 01. B2
2: Establish embedding channels Ce and gradient channels Cg with FIFO buffers
3: Set synchronization interval AT; using Eq. (5)
4: for each global epoch 1 to T do
5: Publisher Phase (Passive Party Po):
6: for each worker Wp Pb do
7: Sample batch B, with IDs {IDI, I,... IDB}
8: Compute embeddings z? -pp(p;00 02)
9: Add noise Edp (0, 2 to 2" (GDP protocol)
Publish (2", BB) to embedding channel C. By]
11: end for
12: Subscriber Phase (Active Party Pa):
13: for each worker w. Pa in parallel do
14:
[B,] not empty then
15:
Fetch (2 B,) from C. [B]]
16:
Compute 2 = fx" 01) and 1 = g(=", 22)
17:
Calculate loss Lū y) and gradients Vo, L, V L
18:
Publish 2.rL to gradient channel Cg [B]
19:
else
20:
Trigger waiting deadline mechanism (skip after T
21: end if
22:
end for
23:
Backward Propagation:
24:
for each worker up E Pg do
Fetch 2 from C, [,]]
26:
Compute Ve,C C and update 02 + 02 - 11 Ve C
27:
Push 0g Pt's parameter server
28:
end for
29:
Semi-Asyne Parameter Aggregation:
30:
if mod AT, == 0 then
31:
Aggregate 0, from Pa's 's workers via Broadcast updated 0, to all Pa workers
32:
33:
end if
34:
end for
F Dataset Information
We provide details the benchmark datasets used as follows:
Energy (Appliances Energy Prediction): The Energy dataset consists of 19,735 samples with 27 features. It used for regression tasks and focuses on predicting the energy consumption of appliances based environmental and meteorological variables. This dataset is commonly used in energy efficiency and smart home applications.
Blog (Blog Feedback Prediction): The Blog dataset contains 60,021 samples and 280 features. is regression dataset designed for predicting the number of comments on blog posts based on textual and metadata features. The dataset originates from online blog platforms and is widely used in social media analytics.
Bank (Bank Marketing): The Bank dataset has 40,787 samples with 48 features. It a classification dataset used for predicting whether client will subscribe term deposit based demographic, economic, and campaign-relater features. This dataset is widely used in financial and marketing analytics.
Credit (Credit Card Default Prediction): The Credit dataset consists of 30,000 samples with 23 features. It is classification dataset designed for predicting whether credit card user will default on
26

---

Algorithm 2 Optimal Configuration via Dynamic Programming
Require: CPU cores Ca. Cn. candidate batch sizes B, memory constraints MA. Mp Ensure: Optimal (war upp B*)
Compute Bmax + min MAu 1/x (
1/x
[Mr Men pp
1:
2: Initialize DP table dpli][i][r]
3: for each batch size B, E B where Br S Bmax do
4: for each wa {P. ...,.... Q} do
5: for each ui E [M,...,N] do
6:
Calculate computation delays TA. Tp using Eq. (7)-(9)
Calculate communication delay T Toomm E+G
7:
8:
Total delay O(war Wp: By) max Tp) + Toomm
9:
Owwa, Wp, B,) < dp [a [ Br then Update dp[wa][uv [Br] (wa, Upp B,)
10:
11:
end
12: end for
13:
end for end for
15:
Return (wa, w B²) + arg min dpH-
their next payment. The dataset is sourced from financial institutions and is commonly used in risk assessment and credit scoring.
Table 6: Summary of Benchmark Datasets for PubSub-VFL. Evaluation.
Dataset
Samples
Features
Task Type
Domain
Energy 19,735
27
Regression
Energy Efficiency
Blog 60,021
280
Regression
Social Media
Bank 40,787
48
Classification
Finance/Marketin
Credit
30,000
23
Classification
Finance
G
Embedding Inversion Attacks
Embedding Inversion Attacks (EIA). Embedding inversion attacks aim to recover private feature representations from embeddings shared the VFI. framework. When the bottom model contains only the embedding layer, the attacker predicts the original feature by finding the nearest neighbor of each perturbed embedding in the embedding space [55]. For bottom models with additional layers, more sophisticated optimization-based attack [49] is used. This method iteratively refines word selection vectors by minimizing the distance between the predicted feature's 's representations and observed representations for each input sample. In this paper, follow [49] assume the adversary trains a neural network to directly map embeddings back to their original inputs. Futhermore, we assume that the adversary has access to shadow dataset similar to the target P, 's data.
Table 7: Accuracy comparison on benchmark datasets with large model.
Dataset
Metric
VFL
VFL-PS
AVFL
AVFL-PS
Ours
Energy
RMSE
84.24
86.14
83.97
84.29
83,94
Blog
RMSE
23.18
23.07
22,97
23.15
22.14
Bank
AUC
94.97
94.74
95.02
95.06
96.97
Credit
AUC
83.42
85.44
84.23
82.27
86.07
Synthetic
AUC
92.74
92.67
91.54
92.21
94.17
H Additional Experiments
Empirical Experiments. To determine the constant Xay To, Ap. Tp. Xo; Yas Pa: Bas Pp. 3p: B. Pa in the delay model, we conduct empirical experiments. Specifically, we utilize a ten- layer MLP the bottom model and two-layer MLP as the top model. We set B
27

---

Use
Time baconns)
10-1
Time Seecommoo
Timee teecomas
20
10
siteed stant
30-4
22
24
2
Barch Size
Baach
Batch Size
Top Bacoward and Updata Time
Bacoward and Updata Time
Sottom Backward and Update
10-4
Time (acconba
Time tuecoved
TThee (occondin
10
10
20--
10
10
10-4
10
22
24
22
2
Satch Size
Baach $20
Botch Size
Figure 8: Empirical experimental results.
Table 8: The result of the constant being solved for.
Symbol
Value
Symbol
Value
Symbol
Value
A 0.018
%a
-0.8015
Xp
0.010
7p
-1.0071
X
0.011
Y
-0.7514
Pa 0.066
B
-0.6069
4p
0.038
Bp -1.0546
B.
-0.7834
Paa
0.072
Table 9: Performance Comparison on Criteo ITB Dataset.
Dataset
Metric
VFL
VFL-PS
AVFL
AVEL-PS
Ours
AUC (%) Runtime (h)
81.23
81.45
80.97
81.32
82.15
48.6
32.1
28.9
21.5
6.8
Criteo 1TB
CPU Utilization (%)
42.3
65.7
58.9
72.1
90.8
Waiting Time/epoch (s)
12.8
8.5
6.2
4.1
1.3
Comm. Cost (GB)
1280
950
890
720
450
{2, 4, 16, 32, 64, 128, 256, 512, 1024] to observe the forward and backward propagation times of both participants. The experimental results are presented in Fig. Based on this figure and the delay model, we derive these constants, with their computed value shown in Table 8. Note that the constants solved different operating environments are different.
System Performance with Large Model. We compare the performance of PubSub-VEL and its baselines five datasets using large models. Similar to previous evaluations, we record the best performance results and configure these methods with their optimal hyperparameters. The experimental results are summarized in Table 7. The results show that both PubSub-VEr and its baselines remain unaffected by the large model (i.e., ResNet), with performance showing slight improvement. This outcome demonstrates the robustness of PubSub-VEr in handling large models.
System Performance Large Dataset. To further evaluate scalability, introduce the Criteo ITB Click Logs d a widely used industrial benchmark for online advertising and recommen- dation systems. It contains ~4.5 billion samples with 39 features (13 numerical, 26 categorical, and high-dinensional after one-hot encoding), representing real-world big data characteristies (massive samples and sparse features). Following the evaluation metrics (AUC for classification, runtime, CPU utilization, waiting time per epoch, and communication cost) in our paper, the Table below compares PubSub-VFi with baselines. PubSub-VFL achieves superior performance in accuracy, efficiency, resource utilization, and communication cost. It attains the highest AUC of 82.15% outperforming baselines by 0.7-1.2%, demonstrating strong robustness in large-scale, sparse data scenarios. With hierarchical asynchrony and Pub/Sub decoupling, it reduces runtime by ~ 3x compared to AVFL-PS
criteo httt:://allb.cccccccom///wooooo wwt.iiloocooo/wooooaoooocoo criteo- logiick--ogs-attase
28

---

Table 10: Numerical results of system performance in multi-party setting.
Method (# of Parties)
Running Time
CPU Utilization
Waiting Time
Comm, Cost (MB)
RMSE
(8)
(%)
(8)
PubSub- VFL VFL(10)
141.14
86.32
1.9273
896.34
23.44
PubSub-VEr VFL
121.55
88.36
2.0147
684.71
22.61
PubSub- VFL (6)
118.36
85.69
1.5697
645.34
22.34
PuhSub-
104.72
90.14
1.2254
569.65
23.17
PubSub- VFL (2)
92.54
91.07
1.1389
439.45
22.34
VFL-PS (10)
1324.71
52.24
1.4410
1264.64
24.19
VFL-PS (8)
1374.63
47.64
1.2147
1165.17
22.61
VFL-PS (6)
1245.94
50.36
1.1647
1211.37
22.35
VFL-PS (4)
1174.65
51.24
1.4211
1089.64
23.19
VFL-PS (2)
974.65
41.47
1.2765
874.55
23.07
AVFL (10)
1445,28
27.65
20.3677
1024.34
23.54
AVFL (8)
1274.57
28.41
21.4154
967.57
23.71
AVEL (6)
1198.18
28.67
17.6517
915.16
24.01
AVEL (4)
1181.14
25.63
16.7456
847.65
22.84
AVEL
1068.88
21.74
15.3657
754.77
22.97
AVEL-PS (10)
1274.51
67.51
2.6971
965.59
23.08
AVEL-PS (8)
1165.33
68.14
2.8146
817.55
23.67
AVEL-PS (6)
1017.82
58.59
2.6511
721.38
23.61
AVEL-PS (4)
1197.53
61.23
2.5636
617.45
24.07
AVEL-PS (2)
1057.67
57.68
2.1788
565.24
23.15
(6.8h 21.5h) and 7x compared to VFL (6.8h 48.6h). It achieves high CPU utilization of 90.8%, indicating effective load balancing in heterogeneous environments, and cuts communication cost 450GB- approx-aiii 40% lower than AVFL-PS-through through optimized channel management and reduced stale updates.
System Performance a Multi-party Setting. While PubSub-VFi VFL is currently designed and evaluated two-party VFL setting, its core architectural features suggest potential for extension multi-party scenarios. The decoupled Publisher/Subscribe mechanism inherently supports many- to-many communication patterns, which is advantageous for scaling. Similarly, the hicrarchical asynchronous design and buffering strategies can generalize to handle diverse update timings from multiple parties. However, the core challenges are the complexity ID alignment and the adaptation of optimization algorithms. To address these two challenges, we provide the following two insights:
Complexity of ID Alignment. To address this challenge, we can leverage existing multi-party PSI techniques (c.g., [36]). which can be applied during the system configuration phase.
Adaptation of Optimization Algorithms. Because multiple parties involved, the Dynamic Programming algorithm's s search space becomes large, making it difficult to find the optimal solution. To address this challenge, straightforward approach is jointly model the passive party with the least resources (known from system profile information) and the active party to determine the optimal hyperparameter configuration. The key insight from doing is that the key bottleneck dragging down system efficiency is the efficiency gap between the active party and the passive party with the least resources. This idea is consistent with the original manuscript. Although this approach is not optimal, it remains an option for expansion. In this way, our framework can be straightforwardly extended multi-party scenarios.
The reason did not include experiments in multi-party settings in the main text is due the focused scope of the research topic and the intended application scenarios. We believe that incorporating both settings in a single paper might dilute the clarity and focus of the core contributions. To implement these improvements, we refactored the PubSub- -FLL implementation by modifying several core components. Specifically, we improved the cache mechanism by increasing cache capacity to support more stable training, extended the wait deadline (Taa) to 15 seconds to enhance the reliability of embedding matching, and refined the dynamic programming optimization algorithm. All other system mechanisms were kept unchanged. Furthermore, we conducted a series of comparative experiments on the Blog dataset to evaluate the performance of PubSub- VFL multi-party scenarios. The results are summarized in the Table 10.
29


Reviews:
[{"review_id": "LbrEExHQ5d", "reviewer": "Reviewer_HZmk", "summary": "This paper introduces PubSub-VFL, a novel VFL approach for two-party split learning designed to tackle inefficiencies and computational resource utilization issues arising from synchronous dependencies and heterogeneous resources/data distribution. \nPubSub-VFL combines a Publisher/Subscriber architecture with a parameter server for managing hierarchical asynchronous updates and provides an adaptive strategy for handling resource and data heterogeneity. \nThe proposed method reduces training latency and enhances computational resource utilization. Extensive experiments across multiple benchmark datasets demonstrate that PubSub-VFL substantially improves computational efficiency (2-7× faster training), achieving state-of-the-art results regarding performance, resource utilization, and privacy guarantees.", "questions": "1. The resource and data heterogeneity you mentioned frequently appear as motivating factors. Could the authors clarify how explicitly addressing a two-party scenario aligns distinctly with addressing resource and data heterogeneity?\n2. Can the authors provide further reasons for not explicitly comparing with advanced AVFL frameworks like the ones explicitly mentioned in the paper ([36-39])? How would the proposed method's performance be expected to compare in practice?\n3. Could the authors provide detailed explanations for the observed accuracy improvements (in Tables 1 and 6), given that the primary improvements were purportedly computational efficiency and reduced delays?\n4. Given the various asynchronous and Pub/Sub mechanisms extensively detailed, how easily can PubSub-VFL scale to multi-party scenarios beyond two-party splits?\n5. Is the proposed optimization model (equations and dynamic programming solution) computationally lightweight and practical for real-time training scenarios, or would it introduce considerable overhead during initialization phases?", "limitations": "yes, in the applendix", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strengths:\n- The proposed Pub/Sub architecture combined with parameter servers provides a new solution and a clear design advantage compared to traditional synchronous or conventional asynchronous vertical federated learning frameworks.\n- The paper supports its claims extensively through a thorough empirical evaluation across various benchmarks. These results are clear evidence of real-world relevance.\n- The authors provide theoretical convergence analyses under the stated assumptions, formally demonstrating that their method converges reliably.\n\nWeaknesses:\n- One stated motivation—resource and data heterogeneity—is more relevant in multi-party (rather than two-party) scenarios. Since this paper focuses explicitly on two-party vertical federated learning scenarios, the impact of this \"heterogeneity\" motivation needs better alignment with demonstration.\n- Although the authors explicitly mention leading asynchronous VFL (AVFL) works [36-39], the comparisons in Table 4 (appendix) and the primary tables in the paper lack evaluation against them, leading to insufficient experimental contrast with the actual AVFL state-of-the-art.\n- Baseline methods do not sufficiently represent recent AVFL approaches, especially those referred to explicitly, such as [39].\n- The main contribution addresses computational efficiency, training latency, and resource utilization. However, Table 1 and Table 6 indicate nontrivial accuracy improvements without a detailed explanation, somewhat conflicting or not clearly aligned with the stated contributions.", "was_revised": false, "final_justification": "", "rebuttal": ""}, {"review_id": "V3U1OuQyTZ", "reviewer": "Reviewer_Uesg", "summary": "With the digital economy's growth, data collaboration thrives, but privacy issues hinder direct sharing. Two-Party Split Learning (VFL) offers a solution but suffers from low resource utilization and training efficiency due to synchronous design and heterogeneity. The authors propose PubSub-VFL, a Publisher/Subscriber-based VFL paradigm, using hierarchical asynchronous mechanisms and parameter server parallelism to reduce latency. It also optimizes hyperparameters via system profiles for balanced training. The model ensures convergence and privacy, achieving 2–7× training speedup and 91.07% resource utilization on benchmarks.", "questions": "Q1: Is two-party vertical federated learning (as studied in this paper) currently a mainstream paradigm in collaborative machine learning? Could the authors provide specific real-world applications (e.g., in finance, healthcare, or IoT) to illustrate its practical relevance? Additionally, does the PubSub-VFL architecture allow for natural extension to multi-party federated learning scenarios, and if so, what are the key technical considerations for such an extension?\n\nQ2: While the case studies on five benchmark datasets are informative, could the authors consider conducting a comprehensive evaluation of PubSub-VFL on larger-scale datasets (e.g., with millions of samples or high-dimensional features)? This would help validate the model’s scalability and efficiency in real-world big data environments.\n\nQ3: Regarding the design of the Intra-party Semi-asynchronous Mechanism: What were the primary motivations behind this design (e.g., addressing resource heterogeneity, reducing communication overhead)? Could the authors elaborate on the technical trade-offs considered (e.g., between asynchrony and convergence stability) and provide insights into how the mechanism balances computational efficiency with model convergence?\n\nAnd also, see the weaknesses", "limitations": "yes", "rating": 6, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 3, "significance": 3, "originality": 4, "strengths_and_weaknesses": "Strength:\n\nThe paper presents solid theoretical derivations. The experiments are comprehensive, with results validated across five datasets. By leveraging the decoupling capability of the Pub/Sub architecture and the data parallelism of the parameter server framework, the authors design a hierarchical asynchronous mechanism that reduces training latency and improves system efficiency. The paper thoroughly analyzes bottlenecks caused by heterogeneous computing resources.\n\nWeakness:\n\n1.The caption of Figure 4 should be revised.\n\n2.The legend in Figure 7 also needs adjustment.\n\n3.In terms of writing, the paper could benefit from framing the work as a training scheduling system that integrates both VFL and HFL. You could improve this hybrid system not just VFL.\n\n4.While this work focuses on two-party splitting, it could be extended to the multi-party setting in the future. I could understand this is not your focus, maybe consider it as your future work", "was_revised": true, "final_justification": "Thank you for your response, which addressed most of my concerns, especially regarding the transition from two-party to multi-party settings. This makes me really appreciate the practical nature of this work, so I raise my score to 5.\n\n\n----\nsecond round:\n\n\nI’ve finished reading the responses to all the reviewers’ comments (HZmk,aqzq,AWXK), and I feel that they have addressed the concerns from the other three reviewers as well. \n\nAfter careful consideration, i raise 5 to 6.", "rebuttal": ""}, {"review_id": "YYJpz1rGrs", "reviewer": "Reviewer_aqzq", "summary": "This paper addresses how two parties, each holding different types of information, can collaboratively train a model. While collaborative learning is often needed, sharing information between two parties can lead to privacy leaks. To tackle this, split learning is typically employed. However, inefficiencies arise due to differences in computational capabilities between the two parties that own the bottom and top parts of the model. \nThis paper adopts the Publisher/Subscriber architecture, which implements a hierarchical asynchronous mechanism to reduce training latency. To solve the problem, this paper also designs a max time minimization objective.", "questions": "In a two-party setting where computational resources are typically fixed, is it necessary to dynamically adjust the number of workers? Given this, it may be more practical to pre-negotiate the system profile based on known capabilities rather than dynamically adjusting the number of workers during training. It seems that dynamic client' worker selection may be more relevant in multi-party settings, where client participation varies and straggler mitigation is a more critical concern.\n\nLooking at Equation (5), it seems the proposed method assumes that the performance is improved as the number of rounds increases. Since this assumption may not always hold (e.g., overfitting), can the proposed method guarantee consistent performance improvement? It looks like potentially vulnerable to overfitting.\n\nLooking at the (6), it also appears to lack sufficient explanation. Assuming the equal worker assignment, is the (6) the same as $T_f^{(a)}(B) = \\frac{\\lambda_a B^{\\gamma_a} w_a}{C_a}$? It is unclear whether the derivation is fully justified. Clarification is needed on the notation and assumptions involved in this step. \n\nLastly, regarding the system profile, could sharing such information raise other privacy concerns?", "limitations": "Yes", "rating": 4, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 2, "originality": 2, "strengths_and_weaknesses": "The proposed method introduces an asynchronous approach that allows two parties with differing computational resources to collaborate efficiently. By leveraging Publisher/Subscriber architecture, asynchronous collaboration is enabled, which helps reduce training latency and improve system efficiency. The claims are supported by experimental results.\n\nWeakness (Please also refer to the Questions.)\nThe proposed method is limited to only two-party scenarios and does not address extensions to multi-client environments. The practicality and effectiveness of the proposed buffer and deadline mechanisms in a two-party VFL setting are unclear. \nSome assumptions seem underexplained and may require justification. \nPotential privacy concerns arise from the use of system profiles, which are not thoroughly discussed.", "was_revised": true, "final_justification": "Through the discussion with the author, I received detailed responses that have appropriately addressed this reviewer’s concerns. I have therefore changed my recommendation to Borderline accept.", "rebuttal": ""}, {"review_id": "8r5GnP7Q1I", "reviewer": "Reviewer_AWXK", "summary": "This paper addresses the inefficiency problem in vertical Federated Learning (VFL) due to the tight coupling and heterogeneity of VFL systems and proposes a new framework called PubSub-VFL.\nThe key idea of this paper is to introduce a Publisher/Subscriber (Pub/Sub) architecture combined with a hierarchical asynchronous mechanism. The experiment results show that Pub/Sub-VFL can achieve up to 7× speedup and 91% resource utilization compared with baselines.", "questions": "1. Do authors have any idea, which may not be tested, to generalize PubSub-VFL to multi-party scenarios?\n\n2. Can the system profiling and optimization adapt to runtime changes in resource availability or network bandwidth?\n\n3. While DP is used, further clarification on the utility degradation at low privacy budgets would be helpful.", "limitations": "Yes", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 2, "strengths_and_weaknesses": "Strengths:\n\n1. The proposed system is carefully designed and thoroughly evaluated.\n\n2. The integration of Pub/Sub into VFL and the dynamic optimization for resource balancing are important contributions, which may not by covered by prior works.\n\nWeaknesses:\n\n1. The main idea combines existing architectures (PS and Pub/Sub), which have been widely used in federated learning systems. Also, the asynchronous training mechanism is also well-known and widely-used in VFL. This questions the novelty of this work.\n\n2. The current framework only supports two-party VFL, whereas VFL is typically designed for scenarios involving multiple parties. In most practical applications, data features are distributed across more than two parties, and training with only two feature subsets is relatively uncommon, to the best of my knowledge.\n\n3. The theoretical analysis relies on strong assumptions (e.g., convexity), offering limited new theoretical insight.", "was_revised": true, "final_justification": "After reading authors' rebuttal, W2 has been resolved. But I still have my concerns regarding W1 and W3. I have raised my score to 3 (Borderline reject).", "rebuttal": ""}]

Author rebuttal:

Meta-review: The submission proposes a novel method for vertical federated learning to handle limitations from synchronous dependencies and heterogenous data distributions. HZmk requires more quantitative results to support the advantages of dynamic synchronization, which are provided in the author response and deemed satisfactory. Uesg strongly supports the careful design of the proposed method, while questioning scalability and relevance of the two-party setting, which are adequately addressed in the author rebuttal by including additional experiments and explaining real-world setting where the two-party setting is currently deployed. aqzq raises concerns on the definition of computational delay for active and passive parties, which requires additional analysis and improvements in clarifying the assumptions in the author response, which are deemed acceptable by the reviewer. AWXK requires consideration of multi-party scenarios and utility degradation at low privacy budgets, which are addressed by the rebuttal but did not entirely convince the reviewer, who does slightly raise the score. The AC notes that similar questions are raised by Uesg too and the answers are satisfactory in justifying the importance of the two-party setting in itself, while Figure 5 is an adequate analysis on impact of privacy budget. Overall, the paper proposes a thorough method to solve a relevant problem with a novel approach, thus, is recommended for acceptance. The authors are encouraged to include the clarified definitions and additional experiments from the discussion in the final version.
Decision: Accept (poster)