# A Supernet-Only Framework for Federated Learning in Computationally Heterogeneous Scenarios
## 3 Method
### Reinterpret Approach
Furthermore, the model can be represented as follows:  
\[\mathcal{N}=\sum_{l=0}^{L}\mathcal{W}_{l}\cdot\mathcal{N}_{l}^{\text{Raw}} \circ\mathcal{N}_{l}^{\text{Pre}} \tag{7}\]  
Since the weights \(\mathcal{W}=\{\mathcal{W}_{0},\mathcal{W}_{1},\cdots,\mathcal{W}_{L}\}\) have been separated from the original training task, the training of this hidden parameter can be considered a meta learning task \(\arg\min_{\mathcal{W}}\mathcal{L}_{\text{val}}(\mathcal{P}^{*}(\mathcal{W}))\) beyond the original network parameter optimization \(\arg\min_{\mathcal{P}}\mathcal{L}_{\text{train}}(\mathcal{W}^{*}(\mathcal{P}))\). To enforce the normalization and non-negativity constraints on the weights, we introduce the canonical link function of the Generalized Linear Model (GLM) with multinomial distribution logits as the bridge between the weights and the meta-learning parameters. Let \(\mathcal{W}^{\text{Raw}}\) denote the parameters corresponding to the architecture weights. The final weights can then be computed as follows:  
\[\mathcal{W}=\text{Softmax}(\mathcal{W}^{\text{Raw}}). \tag{8}\]  
Figure 1a illustrates the computational structure of each stage in the reinterpreted supernet. We observe that the meta-learning component of this two-step optimization process corresponds to neural architecture optimization, while the original task constitutes parameter optimization. This residual network formulation naturally aligns with the bi-level optimization structure commonly adopted in neural architecture search, as represented in Equation (2). In manually designed networks such as ConvNeXt and Transformers, as well as in searched architectures like EfficientNet, the most common macro network structure typically consists of three parts. The network begins with a preprocessing component, such as an Embedding layer or a Stem layer. This is followed by multiple stages, each composed of a downsampler followed by a feature extraction block, where the feature extraction block is entirely composed of a sequence of residual units. The network ends with one or more detection heads. If each stage is rewritten in the form of Equation (7), while keeping all other components unchanged, the sequential network can be reinterpreted as a NAS supernet. Although the model may seem to allow the arbitrary selection of intermediate outputs from nonlinear transformations, each output, in fact, depends sequentially on all preceding layers. Once a layer is dropped, all subsequent layers become uncomputable. As a result, sub-networks can only be formed through truncation, and the structure of each truncated stage in the sub-network is shown in Figure 1b. In the preceding text, we have demonstrated that all architectures with residual connections can be reinterpreted as a NAS problem through a single parameterization step. However, when reinterpreting an arbitrary network with residual connection as a NAS problem, we observe that two issues still need to be addressed. Firstly, modern  
Figure 1: Representation of the supernet stage structure: (**a**) Full stage structure, where each stage consists of a downsampler followed by a sequence of residual layers (L1, L2, and L3), and all outputs are aggregated into the stage output. (**b**) Truncation mechanism used to generate sub-networks, where once a layer (e.g., L3) is skipped, all subsequent layers within the same stage become uncomputable and are excluded from the network path. This design enforces a strict sequential dependency within each stage, aligning the structure with a truncatable residual supernet. The dotted lines represent edges and nodes that are not used during the current computation. network architectures are typically composed of multiple stages, where each stage contains one downsampling module and several residual blocks such as ConvNeXt [34] and SWIN Transformer [35].