# A Supernet-Only Framework for Federated Learning in Computationally Heterogeneous Scenarios
## 1 Introduction
By allowing only left-to-right truncation within each stage, every sub-network becomes a deterministic prefix of the full residual sequence, avoiding operation-level conflicts and enabling full parameter sharing--a property that conventional NAS and FedNAS designs inherently lack due to their search-space structure. To train this supernet, we propose the Single-Path Random Quit (SPRQ) method, in which different exit lengths are randomly selected at each iteration to simulate different network sizes for various clients. SPRQ not only enhances performance but also reduces pretraining time by 24%, significantly improving training efficiency in large-scale settings. Moreover, as the SPRQ method suffers from initialization difficulties, we suggest using the two-stage optimization approach proposed in the DARTS [7] report to perform a warm-up operation. It is evident that our Supernet-Only framework is independent of FL and can be applied to most common FL frameworks. We perform our pretraining scheme on ImageNet1K [8] and compare the performance of the Supernet-Only framework with that of using lightweight networks and clustering clients to build multiple federations, under identical client communication and computation constraints, on the CIFAR-100, CIFAR-10 [9], and CINIC-10 [10] datasets, as well as on ring-based FL and FedAvg. Our method achieves remarkable results across all datasets and federated learning methods, and its advantages become more pronounced as the general ization requirements of the dataset increase. We also validate the necessity of our proposed SPRQ method and the warm-up method in the pretraining methods. In more challenging scenarios, such as those involving CIFAR-100, large-scale federated settings, or smaller backbone networks, our method consistently achieves more than a 1% improvement in Top-1 accuracy, and it exceeds the second-best method by over 5% in all CIFAR-100 experiments under the FedAvg framework. In less constrained settings, including stronger backbones, smaller-scale federated configurations, and simpler datasets, our method performs on par with the best alternative method in each case, while maintaining a margin of more than 1% over the third-best method. We summarize our contributions as follows. Firstly, we revisit the role of residual connections and propose a method to reformulate any network composed of residual units into a supernet for NAS tasks, effectively reinterpreting the optimization task of any residual network into a NAS task. Secondly, we propose the Single-Path Random Quit (SPRQ) pretraining method to simulate structural heterogeneity in federated learning, and we subsequently design a complete training pipeline based on it. Finally, we outline several common paths in Fed-NAS methods, validate the superiority of our proposed method across multiple datasets, and further compare it with AdaptiveFL [11], a state-of-the-art dynamic FL approach. The remainder of this paper is organized as follows: Section 2 reviews the background and related work. Section 3 describes the motivation for our approach and details our methods, including the supernet framework and the training pipeline. Section 4 evaluates the proposed method through experiments, and Section 5 concludes our contributions and discusses future research directions.