# A Supernet-Only Framework for Federated Learning in Computationally Heterogeneous Scenarios
## 1 Introduction  
The Internet of Things (IoT) [1] has become an integral part of daily life and industrial processes, generating vast amounts of distributed data from edge devices. However, leveraging these data presents significant challenges, as follows: privacy concerns arise from sensitive information such as smart home data, medical records, or industrial trade secrets; edge device data often deviate from the Independent and Identically Distributed (IID) assumption due to variations in user behavior or institutional demographics; and edge devices exhibit performance heterogeneity caused by differences in design and technological generations. To address these challenges, Federated Learning (FL) [2] has emerged as a promising approach. FL mitigates privacy risks by performing local model training on edge devices and aggregating updates on a central server, avoiding sensitive data transmission. Additionally, FL enhances usability by adapting to non-Independently and Identically Distributed (non-IID) data distributions through localized training. Despite the potential of FL, its uniform network structures struggle to accommodate device heterogeneity and integrate lightweight techniques [3] such as pruning, sparsification, and quantization to reduce model complexity and align with varying device capabilities. While these techniques partially alleviate the device heterogeneity problem, their compression capabilities remain limited, leaving this issue only partially addressed. Recent advancements have integrated Neural Architecture Search (NAS) [4] with FL to enhance machine learning services on heterogeneous edge devices, giving rise to Federated NAS (Fed-NAS) [5]. Common approaches include searching for networks of varying sizes or designing multi-scale networks through supernets. However, these methods primarily focus on designing networks for different scales of clients without ensuring shared features among all clients for collective benefits, achieving only partial FL objectives. The problem is that using the largest model every client can afford and forming a federated cluster with all clients are difficult to achieve simultaneously. In contrast to prior interpretations of residual connections [6], we introduce a new structural perspective that enables reformulating residual networks as NAS-compatible supernets. We observe that the residual networks can be represented as a sum of outputs from all nonlinear transformations, making these weights learnable, which enables us to convert the residual network into a NAS task. By applying transformations to each stage separately, the size of the sub-network of each client could be determined by a truncation parameter. As long as we keep the entire supernet intact, our model can freely switch between different sub-networks. Although our method introduces a small number of additional parameters corresponding to the number of residual units, these are used to dynamically normalize the structural weights during training. This enables consistent output distributions across sub-networks with different truncation lengths and effectively mitigates the distribution shift problem commonly observed in dynamic neural networks. We propose a novel reinterpretation of residual connections, showing that residual networks can be naturally viewed as continuous supernets, where each residual block contributes as a weighted component in a unified architecture space. This perspective transforms the role of residual connections from simple identity shortcuts into explicit architecture-controlling mechanisms, enabling a new class of NAS-compatible designs grounded in standard residual architectures. Unlike conventional NAS supernets where architectural decisions select between competing candidate operations, our reinterpreted supernet only allows truncation from left to right within each stage.