# A Supernet-Only Framework for Federated Learning in Computationally Heterogeneous Scenarios
## 5 Discussion  
Existing Fed-NAS research faces the challenge of balancing comprehensive data anonymization and sharing with the full utilization of client performance. We observe that the output of sequential networks composed of residual units with skip connections can be represented as a weighted sum of the outputs from each nonlinear transformation. This allows us to reinterpret any network composed of residual units into a NAS task by making these learnable weights independent as a metal-earning task. Based on this insight, we introduce a new methodology for converting any residual-based sequential neural network into a NAS task, enabling flexible and efficient architecture search. Even though standard NAS and non-NAS methods can be employed to train the transformed supernet, its particular structure led us to discover a more fitting training strategy, referred to as Single-Path Random Quit (SPRQ). Our method facilitates the extensive sharing of anonymized data while maximizing the performance of each client, considering their varying computational capabilities. This effectively eliminates the previously frustrating trade-off, as shown in Table 8.  
Figure 5: Evaluation results of the warm-up phase: (**a**) Loss curves on the ImageNet1k dataset across training epochs during the main pretraining phase, with different warm-up strategies applied beforehand. The \(x\)-axis represents training epochs, and the \(y\)-axis shows validation loss. (**b**) The corresponding validation accuracy curves, where the \(x\)-axis indicates epochs and the \(y\)-axis indicates validation accuracy. (**c,d**) The mean and standard deviation of client accuracies on CIFAR-100, CINIC-10, and CIFAR-10 using FedSGD and FedAvg, respectively. Markers and colors indicate the warm-up strategies used: blue dots for DARTS, green triangles for SPRQ, and orange squares for Normal. Extensive experiments on CIFAR-10, CIFAR-100, and CINIC-10 demonstrate the effectiveness of the proposed Supernet-Only framework for federated learning. Under both ring-based and FedAvg training schemes, the framework generally achieves superior or at least comparable performance to strong baselines across diverse and resource-constrained client settings. On CIFAR-100, the Supernet-Only framework improves average accuracy by more than 5% over the Max Trainable baseline in the FedAvg setup. Statistical analysis using the Wilcoxon signed-rank test confirms that these gains are significant at the client level, with consistently low p-values and large effect sizes. In large-scale federated learning with 128 clients, the framework maintains strong performance, reaching an average accuracy of 77.84% on CIFAR-100 and outperforming the next-best method by over 5%. On CIFAR-10 and CINIC-10, while performance is sometimes comparable to Max Runnable, Supernet-Only remains among the top-performing methods in all cases. When applied to smaller backbones such as ResNet-18 under the AdaptiveFL configuration, it yields accuracy improvements of up to 18.3%. Additionally, the proposed training pipeline, which combines DARTS-based warm-up with SPRQ-based pretraining, achieves the best overall federated performance with reduced variance and cuts pretraining time by approximately 24% compared to other methods. These findings indicate that the Supernet-Only framework offers a practical, scalable, and reliable solution for federated learning in computational heterogeneous environments. Despite the promising results, our study has several limitations. First, the experiments are conducted on commonly-used public benchmarks such as CIFAR-10, CIFAR-100, and CINIC-10. While these datasets facilitate comparability with prior work, they do not fully capture the complexity, heterogeneity, and privacy challenges of real-world federated learning applications, such as those in healthcare, mobile devices, or industrial systems. Second, our current framework assumes a stable communication environment with synchronous client participation.