# A Supernet-Only Framework for Federated Learning in Computationally Heterogeneous Scenarios
## 4 Experiments
### Evaluation of Supernet-Only Framework
#### 4.1.2 Evaluation of the Supernet-Only Framework Under Federated Averaging (FedAvg)  
We then evaluate the Supernet-Only framework using FedAvg [14] under similar evaluation settings with experiments on the ring-based federated model. FedAvg is the most commonly used FL algorithm to train a global model across distributed clients without centralizing data. The supernet is pretrained and we distribute pruned sub-networks to clients as needed. At the end of each federated training round, the server aggregates the model by averaging the parameter updates from clients. In experiments with smaller models, the parameter update process is relatively simple. However, in the supernet-only framework, client models of different sizes share all parameters. To address this, only the parts used by each client are aggregated. For weights used by \(c\) clients in a certain layer, server-side parameter updates are averaged across these \(c\) clients. After the completion of \begin{table}
\begin{tabular}{c c c c} \hline \hline
**Dataset** & **CIFAR-100** & **CIFAR-10** & **CINIC-10** \\ \hline Supernet-Only & \(\mathbf{78.57\pm 1.78}\) & \(\mathbf{95.12\pm 0.84}\) & \(\mathbf{88.09\pm 0.81}\) \\ Max Trainable & \(77.28\pm 0.23\) & \(94.93\pm 0.19\) & \(88.07\pm 0.15\) \\ Max Runnable & \(76.61\pm 0.19\) & \(93.89\pm 0.20\) & \(86.82\pm 0.25\) \\ No Federated & \(56.01\pm 1.66\) & \(88.95\pm 0.63\) & \(79.60\pm 0.71\) \\ Grouped & \(67.54\pm 1.67\) & \(91.49\pm 0.70\) & \(83.46\pm 0.45\) \\ Peaches & \(63.64\pm 16.68\) & \(85.17\pm 19.67\) & \(78.70\pm 17.60\) \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Accuracy statistic under ring-based FL: Every value illustrates the mean validation accuracy and the corresponding standard deviation across different clients under federated learning settings and dataset correspondents (the best and second-best results are in **bold** and **underlined**, respectively). the federated learning process, the mean and variance of the accuracy for all clients on the validation set are as shown in Table 2.  
As shown in Table 2, our method achieved promising results. In the experiments on CIFAR-100, our methods outperform the secondary best Max Runnable group by more than 5%. In the FedAvg experiments on other datasets, the results differed significantly from those observed in ring-based federated learning. Specifically, for CIFAR-10 and CINIC-10, our method showed a narrow disadvantage compared to the Max Runnable group, while Max Trainable struggled due to overfitting and constraints imposed by the limited model size. In these two datasets, where data are relatively abundant, the advantage of shared knowledge of Supernet-Only may not be fully realized. To validate that our method maintains competitive performance and does not degrade relative to other methods when the federated scale varies, we performed FedAvg experiments on federated clusters consisting of 128 clients. In order to simulate a larger federate of 128 clients, we employed devices with higher GPU memory capacity, which consequently introduced some changes to the computational environment. The other experiments' settings are the same as those described in Section 4.1.2. After the completion of the federated learning process, the mean and variance of the accuracy for all clients on the validation set are shown in Table 3.  
As the scale of the federation increases, the convergence of federated learning approaches that of SGD, and the federated learning performance of the model surpasses those of models of a moderate scale. At the same time, the No Federated method, which does not involve federation, performs poorly. To further evaluate the reliability and significance of the observed performance differences, we applied the Wilcoxon signed-rank test to compare Supernet-Only with two competitive baselines, Max Runnable and Max Trainable, using client-wise paired