Title: Accelerated Vertical Federated Adversarial Learning through Decoupling Layer-Wise Dependencies
Venue: NeurIPS 2025
Accepted: yes
Keywords: Vertical Federated Learning

Adversarial Training

Adversarial Sample

Robustness
Abstract: Vertical Federated Learning (VFL) enables participants to collaboratively train models on aligned samples while keeping their heterogeneous features private and distributed.
Despite their utility, VFL models remain vulnerable to adversarial attacks during inference. 
Adversarial Training (AT), which generates adversarial examples at each training iteration, stands as the most effective defense for improving model robustness. 
However, applying AT in VFL settings (VFAL) faces significant computational efficiency challenges, as the distributed training framework necessitates iterative propagations across participants.
To this end, we propose **_DecVFAL_** framework, which substantially accelerates **_VFAL_** training through a dual-level ***Dec***oupling mechanism applied during adversarial sample generation.
Specifically, we first decouple the bottom modules of clients (directly responsible for adversarial updates) from the remaining networks, enabling efficient _lazy sequential propagations_ that reduce communication frequency through delayed gradients. 
We further introduce _decoupled parallel backpropagation_ to accelerate delayed gradient computation by eliminating idle waiting through parallel processing across modules.
Additionally, we are the first to establish convergence analysis for VFAL, rigorously characterizing how our decoupling mechanism interacts with existing VFL dynamics, and prove that _DecVFAL_ achieves an $\mathcal{O}(1/\sqrt{K})$ convergence rate matching that of standard VFLs.
Experimental results show that _DecVFAL_ ensures competitive robustness while significantly achieving about $3\sim10\times$ speed up.


Peer reviews:
comments: SUMMARY:
This paper proposes DecVFAL framework to accelerate vertical federated adversarial training. DecVFAL decouples the bottom modules of clients and backpropagation to eliminate idle waiting times. Experiments on four public benchmarks demonstrate that DecVFAL effectively balances computational efficiency with model robustness.

STRENGTHS AND WEAKNESSES:
- Strengths
1. The paper is well-written and relatively easy to follow.
2. There are extensive theoretical analysis about the convergence analysis.
3. Well-designed ablation studies of the impacts of components and hyperparameters are conducted.

- Weaknesses
1. The problem setting seems niche with only 2 clients in main experiments. Even in ablation studies, the max number of clients is only 7.
2. The model deployed in server side is kept as a single-layer perceptron. Analysis or justification for such choice could be helpful. 
3. The speedup is dependent on the split position. Training time increases with more layers included in bottom layers, which restricts the architecture.

QUESTIONS:
1. Will deploying more complex models on the server side, such as multi-layer perceptron affect the results?
2. Typo: "single-layer perception" in line 859

LIMITATIONS:
yes
TITLE: Official Review
CLARITY: 3
IMPACT: 2
ORIGINALITY: 3
DATE: 2025-07-01

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper proposes DecVFAL, a fast adversarial learning technique for vertical federated learning setting. To accelerate training:\n\n1.\tClient-side performs lazy refinement steps on the small bottom module ‘N’ times while keeping the upper-layer gradients frozen. \n2.\tUse delayed gradients concurrently to update the rest of the modules. \n\nAuthors provide theoretical guarantees for convergence and demonstrate 3-10x speed-up on various datasets.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**: \nAlthough the technique comprises well-known tricks (gradient reuse, async updates), it shows good practical speed-up for training. Provides extensive comparisons with baselines with ablations on number of clients, modules, and other decoupling hyperparameters. \n\n**Weaknesses**: \n\n1.\tNovelty is marginal as technique largely reuses know ideas in this context. \n\n2.\tMost of the compared baselines are outdated (mostly from 2019) and some recent ones (e.g., FastAT, ATAS) are only lightly covered.\n \n3.\tNetworks that are used are small (e.g., 84 % clean accuracy on CIFAR 10). It is unclear if results hold for deeper models. \n\n4.\tDecVFAL sometimes yields higher robust accuracy than full PGD despite using weaker inner maximisers with small module in the client-side. It indeed entails strong clarification and justifications. Even the theoretical analysis suggest degradation in performance due to the added bias. \n\n5.\tThe results are inconsistent. For instance, why does PGD and FreeLB accuracy trend differs strongly in Figure 3 and Figure 4 for different datasets?\n\n6.\tThe results for impact of split position in Table 6 seems counter intuitive. Large bottom modules providing better gradient suggests increase in performance. \n\n7.\tExperiments max out at seven clients and assume perfectly aligned sample IDs. Larger client size or heterogeneous environments are untested.\n\n8.\tThe sensitivity of the parameters M and N should be carefully studied and discussed. \n\n9.\tThe analysis of the communication cost is missing. Using decoupled modules suggest substantial increase in communication of smashed layer’s embeddings and should be quantified.\n\nQUESTIONS:\n1.\tCan you briefly instate the significance of AL in VFL and why does it need acceleration in practical scenarios? Considering most of the applications (e.g., in hospitals) involves collaboration of a few parties and would have enough resources.  \n\n2.\tIn this VFL scenario, a separate server is utilized between parties and assumed to hold the labels. Is this a realistic in VFL? What other threats is this setup exposed to? How would the method adapt when parties have non overlapping user sets and rely on secure entity matching?\n\nLIMITATIONS:\nYes some of them.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 1, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces DecVFAL, a novel framework that addresses the computational efficiency challenges in VFAL through an innovative dual-level decoupling mechanism. Specifically, the framework first employs \"lazy sequential backpropagation\" to decouple the client bottom module from the rest of the network, reducing communication frequency between participants; second, it implements \"decoupled parallel backpropagation\" to enable asynchronous computation across modules, eliminating idle waiting time. The authors provide the first convergence analysis for VFAL, proving that DecVFAL achieves an O(1/√K) convergence rate matching standard VFL. Experimental results demonstrate that DecVFAL achieves 3-10x training acceleration compared to existing methods while maintaining model robustness.\n\nSTRENGTHS AND WEAKNESSES:\n## Strengths\n\n1. The paper addresses a critical computational bottleneck in VFAL, which has significant implications for practical deployment. The proposed dual-level decoupling mechanism represents a novel approach that cleverly resolves the inherent sequential dependencies in VFAL.\n\n2. The paper provides the first convergence analysis for VFAL, considering the interaction of multi-source approximation gradients, decoupling mechanisms, and VFL architecture, which is a significant contribution to the theoretical foundation.\n\n3. The work includes extensive evaluations across multiple datasets, different model architectures, and various attack types, demonstrating the effectiveness and generalizability of the method. The detailed analysis of key parameters (M, N), number of modules, and split positions provides valuable insights into the method's design principles.\n\n## Limitations and Questions\n\n1. While the paper mentions differential privacy guarantees, there is limited analysis of whether the decoupling mechanism (particularly delayed gradients) introduces new privacy vulnerabilities. Could malicious participants extract additional information from these delayed gradients?\n\n2. The proposed decoupling strategies are primarily based on feed-forward network architectures. How would these strategies adapt to models with complex structures (e.g., Transformers or networks with dense skip connections), particularly when these structures are distributed across different participants?\n\n3.  While DecVFAL reduces the number of communication rounds, each round may transmit more information. How does this trade-off affect overall performance in bandwidth-constrained environments?\n\n4. The experiments indicate that M and N parameter choices significantly impact performance. Is it possible to develop adaptive strategies that dynamically adjust these parameters based on task characteristics, communication costs, and computational resources?\n\nQUESTIONS:\nSee above.\n\nLIMITATIONS:\nSee above.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nDecVFAL accelerates adversarial training in VFL via dual-level decoupling (lazy sequential + parallel backpropagation), achieving 3–10× speedup with rigorous convergence guarantees and competitive robustness.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:​​\n\n​​1. Rigorous Technical Foundation​​: Proposes a mathematically grounded dual-decoupling mechanism (lazy sequential + parallel backpropagation) with convergence guarantees (Theorem 1, Corollary 1).\n2. ​​Comprehensive Evaluation​​: Validated across 4 datasets (MNIST, CIFAR-10/100, Tiny-ImageNet), 8 baselines (PGD, FreeAT, YOPO), and 10 attack methods (FGSM, PGD, CW, etc.), demonstrating ​​3–10× speedup​​ without compromising robustness (Tables 1–3).\n3. ​​Theoretical-Experimental Alignment​​: Convergence analysis explicitly accounts for VFL-specific challenges (e.g., gradient delays, compression errors) and aligns with empirical results.\n\nWeaknesses:\n\n​​1. Limited Real-World Data​​: Experiments use partitioned public datasets (Sec. 6.1). Testing on industry VFL benchmarks (e.g., finance/healthcare) would strengthen practicality.\n2. Bias Term Trade-Off: Convergence rate includes \\( \\mathcal{O}(N\\mathcal{M}_K/M) \\) (Corollary 1), implying performance degradation with many modules. Not empirically explored beyond Table 5. \n3. Decoupling Inspiration: Builds on prior work (e.g., Synthetic Gradients [27], ADMM [57]), though VFAL-specific extensions are non-trivial.\n\nQUESTIONS:\n1. While experiments on public datasets (MNIST/CIFAR) demonstrate efficiency, how does DecVFAL perform on industry VFL benchmarks (e.g., financial fraud detection or healthcare datasets with heterogeneous feature distributions)?  \n\n2. Corollary 1 identifies a bias term \\( \\mathcal{O}(N\\mathcal{M}_K/M) \\) that degrades performance with increasing modules  \\mathcal{M}_K . What practical strategies can mitigate this?  \n\n3. DecVFAL assumes synchronous client-server updates (Algorithm 1). How would asynchronous settings (common in cross-device VFL [36]) affect convergence?\n\nLIMITATIONS:\nSee questions", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-06"}