Constraint-guided Hardware-aware NAS through Gradient Modification
Authors: Gregory De Ruyter
Mathias Verbeke
Hans Hallez
Venue: ICLR 2026 Poster (2026)
Keywords: Neural Architecture Search
Hardware-aware NAS
Constraint-aware Optimization
Edge Machine Learning
Primary area: transfer learning, meta learning, and lifelong learning

Abstract: Neural Architecture Search (NAS), particularly gradient-based techniques, has proven highly effective in automating the design of neural networks. Recent work has extended NAS to hardware-aware settings, aiming to discover architectures that are both accurate and computationally efficient. Many existing methods integrate hardware metrics into the optimization objective as regularization terms, which introduces differentiability requirements and hyperparameter tuning challenges. This can either result in overly penalizing resource-intensive architectures or architectures failing to meet the hardware constraints of the target device. To address these challenges, we propose ConNAS, a novel gradient-based NAS framework that enforces hardware constraints directly through gradient modification. This approach eliminates the need for differentiable hardware metrics and regularization weights. The novelty in ConNAS lies in modifying gradients with respect to architectural choices, steering the search away from infeasible architectures while ensuring constraint satisfaction. Evaluations on the NATS-Bench benchmark demonstrate that ConNAS consistently discovers architectures that meet the imposed hardware constraints while achieving performance within just 0.14% of the optimal feasible architecture. Additionally, in a practical deployment scenario, ConNAS outperforms handcrafted architectures by up to 1.55% in accuracy under tight hardware budgets. Our code is publicly available at https://gitlab.kuleuven.be/m-group-campus-brugge/distrinet_public/connas.

Published as a conference paper at ICLR 2026
CONSTRAINT-GUIDER HARDWARE-AWARE THROUGH GRADIENT MODIFICATION
NAS
Gregory De Ruyter!.2 Mathias Verbeke 1,2,3 Hans Hallez!
'Department of Computer Science, KU Leuven, Belgium
Leuven.AI KU Leuven Institute for AI, Belgium 'Flanders Make @KU Leuven (gregory.deruyten) deruyter, mathias. verbeke, hans. hallez mallez))kuleuvee @ulleuven
ABSTRACT
Neural Architecture Search (NAS), particularly gradient-based techniques, has proven highly effective in automating the design of neural networks. Recent work has extended NAS to hardware-aware settings, aiming to discover archi- tectures that both accurate and computationally efficient. Many existing meth- ods integrate hardware metrics into the optimization objective as regularization terms, which introduces differentiability requirements and hyperparameter tun- ing challenges. This can either result in overly penalizing resource-intensive ar- chitectures or architectures failing meet the hardware constraints of the target device. To address these challenges, we propose CONNAS, a novel gradient- based NAS framework that enforces hardware constraints directly through gradi- ent modification. This approach eliminates the need for differentiable hardware metrics and regularization weights. The novelty in CONNAS lies in modifying gradients with respect to architectural choices, steering the search away from in- feasible architectures while ensuring constraint satisfaction. Evaluations on the NATS- Bench benchmark demonstrate that CONNAS consistently discovers archi- tectures that meet the imposed hardware constraints while achieving performance within just 0.14% of the optimal feasible architecture. Additionally, in practi- cal deployment scenario, CONNAS outperforms handcrafted architectures by up 1.55% in accuracy under tight hardware budgets. Our code is publicly avail- able at httttsts///gttttt http:/itaaa..kkleee...... Eistrinet_publicco/eo kuleuven.ba group campus -brugge/
INTRODUCTION
Deep neural networks have proven to be very successful in numerous applications ranging from image recognition (Krizhevsky et al. 2017) and speech recognition (Hinton et al. 2012) to time se- ries segmentation (Lea et al. 2017). More recently, the use of deep neural networks on constrained hardware has gained significant interest in the context of machine learning the edge (EdgeML), where eesource-constraingg trollers used to perform inference tasks. These devices often have limited computational power, memory, and energy resources. As a result, crucial to design neural network architectures that operate effectively under strict hardware constraints.
devices such as mobile devices, embedded computers, and microcon-
A key challenge in EdgeML is designing neural network architectures that meet hardware constraints while maintaining high performance. In recent years, Neural Architecture Search (NAS) was intro- duced to automate the design of neural networks, and to potentially discover novel architectures that outperform manually designed models by experts (Elsken et al. 2019b). In NAS, search algorithm is used to explore a search space of possible neural network architectures, aiming to find the best- performing architecture for a given task. Models originating from NAS techniques have already outperformed numann-deiggna (ct al.) 2018) and semantic segmentation (Chen et al. 2018). Based on the type search algorithm used, NAS methods can be categorized into four groups: (i) Reinforcement Learning (RL) (Zoph) & Le 2017 Zoph et al. 2018 Baker et al. 2017). (ii) Evolutionary Algorithms (EA) (Real et al.
models on tasks such as image classification (Real et al.) [017] ZZoph
2017
Salimans et al. 2017). (iii Bayesian Optimization (White et al. (021). (fv) and Gradient- based methods (Liu et al., 2019). The latter has gained significant popularity in recent years due to
its efficiency and ability scale to large search spaces compared to the other methods.

---

Published as a conference paper at ICLR 2026
Recently, the use of NAS for designing more efficient neural network architectures has been in- creasingly explored. Prior work, specifically on gradient-based NAS, often integrates hardware metrics as regularization terms into the optimization function (Cai et al. 2019 Wu et al., 2019 Wan et al., (020). While this can help guide the search toward ferdware-friendle architectures, these approaches often face several challenges. First, to be compatible with gradient-based optimization, the hardware metrics must be differentiable. Second, each regularization term requires appropriate weighting, introducing additional hyperparameters that demand careful tuning. Improper weighting can either overly penalize essoucce-ntennsvv architectures (resulting to simpler models with subop- timal performance) or fail to enforce the hardware constraints (yielding architectures unsuitable for deployment on the target device). Consequently, these methods often require multiple search runs with varying weights assigned to the hardware-related terms until feasible architecture is found, a process that can be both tedious and time-consuming.
To overcome these limitations, we introduce CONNAS, novel hardware-aware, gradient-based NAS algorithm. Our work makes the following key contributions:
Direct constraint enforcement: Unlike prior approaches that incorporate hardware metrics as regularization terms in the loss function, CONNAS modifies gradients with respect to architec- tural choices to directly enforce hardware constraints, effectively steering the search away from infeasible architectures.
No need for differentiable: Our method eliminates the requirement for differentiable hardware metrics and the associated techniques needed to enforce differentiability.
Explicit constraint specification: CoNNAS allows for the explicit definition of hardware straints without relying on hyperparameter tuning to balance the importance of hardware metrics.
We evaluate CONNAS on the MATS-Bench benchmark (Dong et al.) (202T), demonstrating its ability to systematically find architectures that satisfy various hardware constraints while achieving perfor- mance close to the optimal architecture available within the search space, reaching as little as a 0.14% difference in accuracy. Our experiments show that CONNAS significantly outperforms ex- isting baseline methods, which often struggle to find feasible architectures under strict hardware constraints. Additionally, validate CONNAS in a practical use case, where it successfully dis- covers high-performing architectures under strict hardware constraints (no more than a model size of 64kB and 18kB memory usage), consistently outperforming the best handcrafted architectures by up to 1.55% accuracy,
2 RELATED WORK
Hardware-aware NAS. Over the years, the focus of NAS has shifted from discovering top- performing novel architectures to emphasizing computational efficiency. The design of neural net- works has increasingly been guided by hardware-aware considerations, particularly in the context of deployment ware metrics which could be estimated based on the type of operations found in the model, such as the number of Floating Point Operations (FL.Oss) (Xie et al. 2019 Zhou et al.) 2018). How- ever, other works have proposed to rely on real hardware metrics for more representative evaluation, which are generally obtained through look-up tables regressors trained on real on-device bench- marks of various neural operations (Cai et al.] 2019 Wu et al. 2019: Wan et al.) 2020 Tan et al.)
resource-constrained devices. Early works primarily focused optimizing hard-
2019 Hu et al. 2020). Additionally, other contributions have more specific focus on NAS tech-
niques and search spaces tailored for ultra-constraing devices, such as microcontrollers (Lin et al.)
2020 Liberis al. 202I).
The most straightforward way to regularize hardware-related objectives is by incorporating hard-
ware metrics as additional terms in the optimization function (Xie et al.) 2019 Cai al., 2019
Wu et al.] 2019 Dong & Yangl 2019a; Wan et al., 2020; Bender et al. 2020, Hu et al.) (020). A
weighting factor is then introduced to adjust the relative importance of these hardware metrics in
comparison to the task-specific loss, effectively determining whether more constrained architectures
should be favored. However, this fixed approach does not offer a direct mechanism for enforcing hardware constraints. Instead, it merely guides the search toward architectures with more favorable hardware metric values (typically simpler architectures with limited representational capacity) re- gardless of whether the current architecture actually satisfies the imposed constraints. In response,
2

---

Published as a conference paper at ICLR 2026
Tan et al. 2019; Zhou al. 20188; Dong & Yang (019a); Bender et al. (2020)
propose dynamic
optimization functions where hardware-relatte related terms are adjusted depending on whether the current architecture meets the hardware constraints. The rationale behind this approach is to find architec- tures that are close to the constraint boundary, which makes it possible to explore more complex architectures that still satisfy the constraints. This is in contrast methods with fixed loss term, which may limit the search to simpler architectures. Mcanwhile, other techniques (Elsken et al. 2019a Liberis et al., (021), typically con-gradient-basee methods, use selection procedure based on hardware metric values, selecting only architectures that satisfy the hardware constraints for fur- ther optimization.
Gradient-basse NAS. Gradient-based NAS techniques, originally introduced by DARTS (Luu jet al. 2019), have been widely adopted recent work to efficiently search for neural network architectures. In gradient-based NAS, all possible architectures are represented within an over- parameterized network, where each candidate path is associated with continuous architecture weight This formulation leads to a bi-level optimization problem, in which both the architecture a and its weights w are jointly optimized using gradient descent. Traditionally, the loss function would be defined as the task-specifi loss, such cross-entropy for classification tasks. However, in hardware-award NAS, the loss function is augmented with regularization terms to account for hardware metries.
While hardware metrics for individual architectures are not differentiable, state-of-the-art techniques ensure that these metrics be fully factorized over the architecture parameters. This enables pressing hardware metrics as a weighted sum of contributions from each candidate operation in the search space, thereby making them differentiable with respect to the architecture parameters. However, this approach requires knowledge of each operation's 's contribution to the overall hardware metric, which becomes computationally intractable in large search spaces. To address this, Xie al. (2099) propose approximating the bardware-relater terms using Monte Carlo estimation. Neverthe- less, if the sampling distribution poorly calibrated, the resulting estimates may inaccurate.
Constraint-aware Training, Some recent NAS approaches incorporate hardware constraints di- rectly into the optimization process, though this area remains relatively unexplored. HardCore- NAS (Nayman et al. [021] addresses hard-constrained optimization by adapting the Frank-Wa -Woffe algorithm to search for sub-networks within a pre-trained one-shot model. HDX (Hong et al. 2022) introduces hardware constraints by adding hardware metrics regularization terms and modifying gradients based on differentiated hardware metrics. Similarly, Constraint Guided Gradient Descent (CGGD) (Van Baelen & Karsmakers] 2023). although not NAS algorithm, enforces inequality constraints during training through gradient modification without penalty terms, proposing an up- date scheme that guides optimization toward weights satisfying the specified constraints. Building on these ideas, our work explores how such gradient-basse -baee enforcement can directly impose hard- ware constraints during the NAS search process, without requiring additional regularization terms or differentiable hardware metrics.
METHOD
3.1 PROBLEM FORMULATION
The search space S is modeled as over-parameterized neural network that encompasses all pos- sible architectures. This is achieved by defining, for each layer in the network, a set of candidate operations (e.g., convolutions with varying kernel sizes) that can be used to construct a neural net- work. Each candidate operation is associated with a continuous architecture weight a, which enables the use of gradient descent to jointly optimize both operation weights w and the architecture weights a. A final architecture A can then be derived by selecting subset of candidate operations based on the learned architecture weights a. More formally, let the over wwer-paaameterized network S be defined as a directed acyclic graph (DAG) with L edges where each edge e is associated with a set of candi- date operations O = {or,1, o1,N) } with corresponding architecture weights on = I aI.N]] and operation weights wi [ .., WI,N]. An architecture E
derived by selecting, for each edge e,, the operation with the highest architecture weight, i.c., A = (01.A) ..... OL,,z) where ki arg max; 0y Let Cj denote the k-th hardware metric function (c.g.. number of parameters), and by its upper bound. The set of hardware constraints is defined as C = (c1, b....... (CM. bar) ). where each constraint is expressed as e (A) & by. The search process can then be formulated as
3

---

Published as a conference paper at ICLR 2026
follows:
arg min (usk(w, s.t. ca(A) bus k 1....... M w,a
In other words, the goal is to find architecture weights that yield an architecture A(a) minimizing the task-specific loss Lusk. while ensuring that the resulting architecture lies within the feasibility region F R. defined as the subset of architectures that satisfy all hardware constraints in C. Formally, we aim to find A(x) E F R. or least ensure that the sampled architectures A generated during optimization converge to F R, i.e., lim; d( A*, ,FR) = O.
The value for hardware metric is obtained using C& (A), where e may be implemented as a look- up table learned regressor. Unlike prior work on hardware-aware, gradient-based NAS (Cail al., 2019: Wu al. 2019; Wan et al. 2020 Dong & Yang. 2019a), CONNAS does not require differentiable hardware metrics with respect to the architecture weights
3.2 GRADIENT UPDATE SCHEME
We adopt the gradient update scheme as proposed in CGGD (Van Baelen & Karsmakers) 2023
to enforce the hardware constraints o₁ (A) S by, k I...... M. However, unlike CGGD, which aims to enforce constraints on the model's output predictions, we use this approach to manipulate the architecture weights a to steer the search process towards feasible architectures A. Specifically, each architecture weight 0r,j is updated as follows:
(2) Here, R a rescale factor that determines the strength of the constraints enforcement, and ( [dire(a,,1), (.......... dirc(ay,x)) is a unit vector indicating the direction in which the architecture weights should be modified to satisfy the constraints. An arbitrarily small constant added to allow adjustments even when the loss gradient is zero.
0n - an T/ (Vor,,
a) R- direlau.)) max fia a))l
3.3 GRADIENT DIRECTION
The choice of and dirc(aa) is crucial, as it determines whether the architecture weights are modified in way that effectively reduces the hardware constraint violations, and ultimately leads to convergence to a feasible architecture. Van Baelen & Karsmakers (023) proved that convergence towards the FR is guaranteed if R > and dire is chosen to be the shortest path with spect to the Euclidean distance to the feasibility region. They also showed that performance is not highly sensitive to the exact value of R, as long it is strietly larger than 1. In other words, this eliminates the need for additional hyperparameter tuning, unlike prior work that relies on weighted regularization terms.
However, computing the exact shortest path is often intractable because it requires knowing the hard- ware metries for all possible architectures in the search space, or least that these metrics are fully factorizable across all edges. To address this, we propose heuristic approach to approximate the direction of the global shortest path by calculating the shortest path for each edge ej independently. However, this means that the same convergence guarantees as in Van Baelen & Karsmakers (2023) do not strictly hold. Nevertheless, our experiments (Section demonstrate that this heuristic is effective in practice.
For each edge ét, first evaluate the hardware metric values of all candidate operations O under the current architecture configuration. Then, we construct set of candidate architectures At = (A,, AI,N) } by replacing the (-th operation with ot,, while keeping the rest fixed:
A
"......,,.................
We then evaluate o (Ar,) for each candidate and constraint (ck, bb) C in order to determine der (at). which the direction the FR for constraint cı (Ar,) < bg. Based on whether the candidate architectures A satisfy the hardware constraint, we distinguish three cases:
1. All candidates on e₁ satisty the hardware constraint (Vj E JJ...,,N, (A,,)))) < by). In this case, set dir (oa) 0 for all candidates, as no modification is needed to satisfy the constraint for this edge.
2. Some candidates on violate the hardware constraint (Jj, ca (Ar,)) > by). In this situation, the direction is computed by aggregating the unit vectors that point away from the candidates that
4

---

Published as a conference paper at ICLR 2026
violate the constraint, while pointing towards those that satisfy it. More specifically, we first identify the set of candidates that satisfy the constraint:
F ca(,),)) s bx)
(4)
For each pair m) where E F and m candidate as shown in Figure la
m)
& F, we compute the unit vector that points from candidate
Uj
0. 0.....
......... -1, ...00....0/V
(5)
m-j-1
Then, the final direction dir (on) determined by summing these unit vectors and normalizing the result to a unit vector:
(ar) (en)=EE mET
E'
"J Σ Σ mgmm
(6)
DCFmEr
No candidate on e₁ satisfies the hardware constraint (Vj (I.....N). (A(,),)) > b²). In this case, the direction is computed in such that candidates with higher hardware metric values are penalized while those with lower values are rewarded. First, the candidates are ranked based on their hardware metric values. Then, for each ranked candidate, compute unit vectors Uj (as defined in Equation 5) that point from candidates with higher hardware metric values to those with lower ones. Specifically:
First, we compute the unit vectors between the candidate with the highest hardware metric value and all other candidates.
Then, we compute the unit vectors between the two candidates with the highest hardware metric values and all others.
This process repeated until all candidates, except the one with the lowest hardware metric value, have been considered, shown in Figure lb Finally, the unit vectors are aggregated in the same manner as in Equation
0.2
0.1
D1, di:2
01.3
Ftw.
all
Go
Gu
0,3
U1.
G.
0,1
U23
-v a)
(a) Both A and Au satisfy the constraint, AL. violates it, so the resulting gradient Ga is decreased for cu,s and increased for ai,1 and 01.2.
0.1 00.3
-Va of
9.1
Ge
2,5
Mt.
G
0
G
M2.
V.llw,e)
-Pg d)
(b) No candidate for ej satisfies the constraint, the resulting gradient G.,, increased decreased based on the ranked hardware metric values of each candidate.
Architecture space for single edge ez with three candidate operations, parameterized by 01,1, 02,2, 01,3, where (A,1) < c* (At,) < Cs (Ar.). Each subfigure shows projection onto the plane defined by two architecture weights. The shaded area represents the feasibility region FR for Ck. Ga, is the gradient after modification projected into each plane.
Figure I:
5

---

Published as a conference paper at ICLR 2026
Algorithm Training procedure under hardware constraints
Input: Search space S, current architecture A derived from a, hardware constraints C, rescale factor R.
epsilon learning rates The Ta,, loss function Lusk, batch of data (X,y)
+ N/w V ww
{Update operation weights}
G. Cmaaa (w, a:) {Store gradients w.r.t. architecture weights}
dirs: = {For each unsatistied hardware constraint, compute direction to feasibility region]
for (cx, bb) E C do
if (x) > by then
Append colculate gradient direction(A, (&) to dirs.
end if end for
dire (ax) Ederb dirtt/ Esir carra dert (Aggregate directions into single unit vector)
Ga + Ga + R dire(a)- max [I[Gall (Galllcc, .c) (Modify gradient toward the feasibility region}
a - a Ija Ga {Update architecture weights}
3.4 CONSTRAINT-GUIDED HARDWARE- NAS
We refer Algorithm| for the modified training procedure. Within each batch, the optimization of operation weights w and architecture weights is performed in an interleaved manner, proposed
in prior work (Cai al., 2019: Wu al. 2019 Wan et al. 2020 Dong & Yang 2019a b). After optimizing w. the gradient V Lunkk (msk(W,, a is computed. but instead of immediately updating a, it stored for later modification. Then, for each hardware metric Cg and its associated constraint Ca. (A) < by. the current architecture A is checked for violations. If constraint is violated, the direction for modification dir is computed as described in Section 3.3 All directions are summed and normalized to unit vector to obtain dirc(a). Finally, the stored gradient is modified, as explained Section 3.2
and used to update the architecture weights a.
EXPERIMENTS AND RESULTS
We evaluate the effectiveness of CONNAS on two benchmark tasks: the NATS-Bench topology and size search spaces (Section 4.1). In addition, apply method to practical use involving edge-based condition monitoring of induction motors, where the goal is to discover architectures suitable for deployment across diverse hardware configurations (Section 4.2
4.1 EXPERIMENTS ON NATS-BENGE
NATS- BeTS-Bench, proposed by Dong et al. (202I), a unified benchmark for NAS designed for image classification tasks, including two search spaces; the topology search space and the size search space. The topology search space is a cell-based search space, which contains 15,625 unique architectures, while size search space is layer-wise search space, consisting of 32,768 unique architectures. We refer to Dong et al. (202I) for a detailed description of the search spaces. A key advantage of NATS- Bench over other NAS benchmarks is that it provides test accuracies for all architectures across both search spaces on CIFAR-10, CIFAR- 100, and ImageNet16- 120. This enables rapid evaluation without the need for retraining. Additionally, we combine both real on-device measurements and proxy-based hardware metrics to define hardware constraints for our experiments. Specifically, we use on-device measurements of inference latency and energy usage on an NVIDIA Jetson TX2 Edge GPU (NVIDIA Inc.) for the topology search space, as provided by WV-NAS-BBendh (Li et al.) 2021). For the size search space, we incorporate widely used proxy-based metrics, including the number of parameters, FLOPs, and peak memory usage, all of which can be computed directly from the architecture definition. detailed description of how these proxy-based hardware metrics are approach is flexible and can integrate any
calculated is provided in Appendix A Importantly, other hardware metric, regardless how it is obtained or computed.
We use a Gumbel-Softmal (Jang et al. 2017), as proposed in Dong & Yang (0119)); Wu et al. (2019); Wan et al. (2020), to relax the categorical distribution of the candidate operations O, At each edge et, the output y/ computed weighted sum of the outputs produced by all candidate

---

Published as a conference paper at ICLR 2026
operations:
N exp((lo((an)) + gT)) y1 Σ
(1)(
exp((log(an3) + 9r.s)/T)
where 9t.j Gumbel(O, 1) and temperature T controls the smoothness of the distribution. A higher temperature is used at the beginning of the search to encourage exploration, making the distribu- tion closer to uniform. As the search progresses, the temperature is gradually lowered, making the distribution sharper and closer to arymaaz, which promotes exploitation. It is worth noting that CONNAS agnostic to the specific relaxation method used. Other approaches, such as proposed by DARTS (Liu et al., 2019) or ProxylessNAS (Cai et al. 2019). although not tested here, could also be applied.
We run CONNAS on each search space and corresponding set of hardware constraints for total of 150 epochs. During the first 100 epochs, the temperature T is linearly annealed from 10 to 0.1. The rescale factor R is set to 1.2, though additional experiments show that the performance is not very sensitive to this choice (see Appendix B Among the final 50 epochs, we select the architecture with the lowest validation loss that satisfies the hardware constraints. Finally, the performance of the selected architecture is obtained from the test accuracies provided by NATS-Bench, Each experiment is repeated 5 times, as in Dong et al. (002I). The mean and standard deviation of the results are reported in Table[[and ]able the topology and size search spaces, respectively. Since NATS-Bench provides performance data for each architecture in the search space, we also report the relative error between parentheses to the optimal architecture satisfying the hardware constraints.
Results of individual runs are shown Figure 2
We further compare CONNAS against several hardware-aware, gradient-based baselines listed in Table We adopt the same experimental setup and training procedure for these baselines to ensure fair comparison. More training details provided in Appendix B
CONNAS consistently identifies architectures that satisfy the specified hardware constraints while maintaining strong predictive performance. The resulting architectures exhibit relative errors of at most -1.1%% -4 -..00% and -6.45% compared to the optimal feasible solution on CIFAR- 10, CI0,R-100, and ImageNet16 6 120, respectively. Compared to the bascline methods using the weighting factors listed in Table D CoNNAS achieves substantially better results than Proxyless- NAS and HDX, and performance comparable to TF-NAS. In contrast, FBNet and TAS do not consistently produce valid architectures under all constraint configurations. Additional experiments under stricter hardware constraints, presented in Appendix C further demonstrate the effectiveness of CONNAS, which continues to find valid architectures and generally outperforms all baselines.
It is important note that the weighting factors used for the baselines may not be optimal under the given constraints, and further tuning could potentially improve their performance. Nevertheless, this underscores a key advantage of CONNAS: it eliminates the need for extensive hyperparameter tuning to balance hardware metrics, thereby eliminating the need to rerun experiments multiple times to find an appropriate weighting factor. To illustrate this, we conduct additional experiments with alternative weighting factors for the baselines (see Appendix(D) (D) and an ablation study the rescale factor R for CONNAS (see Appendix B) Both analyses confirm that CONNAS is more robust to variations in the rescale factor, whereas baseline methods exhibit higher sensitivity to weighting factors, requiring careful tuning for each hardware constraint setting.
4.2 PRACTICAL USE CASE: CONDITION MONITORING OF INDUCTION MOTORS
Condition monitoring of industrial assets has received increasing interest over the years (Surucu ct al. (023), preventing unexpected failures and costly downtimes. By leveraging edge machine learning, data from these assets can be captured and processed locally to detect faults in real-time. To demonstrate the applicability of our work, we validate our approach through a use case focused on condition monitoring of induction motors, aiming to detect eccentricity faults using a neural edge device. An eccentricity fault occurs when the rotor is not perfectly centered within the stator, resulting in an uneven air gap, causing an unbalanced magnetic pull (De- senfans et al.) 2024). The prediction task involves classifying the type of eccentricity, specifically distinguishing between no eccentricity fault, static eccentricity (where the rotor remains consistently off-center relative to the stator), dynamic eccentricity (where the rotor's off-center position rotates
network deployed

---

Published as a conference paper at ICLR 2026
Table 1: Overview baseline methods used for comparison. Here, (a(a) the normalized form of cu (a). following the same procedure described in the original work. The term by represents the upper bound associated with Ck- Ahambware is a weighting factor. We use Aurdom = for ProxylessNAS, TAS, TE-NAS and HDX, and
Ahantware 0.6 for FBNet. Additional experiments with varying weighting factors for Atundam provided in
Appendix D
Name
Regularization
ProxylessNAS (Cai et al.
2019)
L(w, a.) = Link(w; a)) + (Aundwar /M) SK (a))
NB (Wu et al.
2019)
L(w, a) = Lusk (w, a) II (g a M
C(w,a) a) = Lud(w, a) + (Ahandware// /M) -k les(a)
((a,,
if c& (A) by
TAS Dong & Yang
2019a)
where le,(c))
(a),
if (A) < bb
0,
otherwise
TF-NAS Hu et al.
2020
L(w, ()) = Lask (w. ()) + (Ahurdumm/ MM)
Ex= max( Gtaa 1,0)
by
V V.L(w,a), a),
cf S by
Ga
HDX Hong al.
2022)
otherwise where L(w, a) Lenk (w, a) + (Auandware/ /M) 1
Ma + V Val(w,
Table 2: NATS-Bench topolegy search space classification results. Test accuracy (mean std over 5 runs) reported under hardware constraints, using on-device measurements of inference latency and energy usage on NVIDIA Jetson TX2 Edge GPU. The constraints are chosen that approximately 50% of architectures the search space satisfy the constraints. The relative error compared to the optimal architecture satisfying the hardware constraints is reported between parentheses. Best results for each constraint and dataset highlighted in bold. Performance of runs that do not always satisfy the constraints are reported in gray.
Method
Top-1 accuracy (%)
Runs satisfied
CIFAR-10
CIFAR-100
ImageNet16-12
Oslatesey : latency 5.31ms (7814 architectaes naiisfled)
ConNAS
93.18 1.009-1.000
70.12 2 (-3.04
40.59 1 (-300))
5 5/5
ProxylessNAS
92,29 1.000-1.97)
67.45 (-568)
39, 20 2.000
5 5/5
FBNet
93.18 10.12(-1.08)
70.00 (3316)
39.71 1.006
5/5
TAS
93.67 0.00
70.91 0.00
41.02 0.00
0/5
TF-NAS
93.27 2.000
70.06 0.00
39.75 + (-00
5/5
HDX
75.83 36. (-1844))
54.18 + (-18.9)
31.69 (-125))
5/5
Caeenergy energy usage 5. 23.95m.
(7814 anchitecturrs ratitfied)
ConNAS
92.97 0.54(-1.118
69.51
1.48 (-2.43)
40.32 (-5.4)))55)
5/5
ProxylcssNAS
92.29 x 0-000-1.88)
67.48 0.00 (-4.46)
39.40 a (.0077
5/5
FBNet
93.05 (-1.00))
69,91
= 0.00 (-2.03)
39.64 + (-00127/22
5/5
TAS
93.67 2000
70.91 0.00
41.02 2.000
0/5
TE-NAS
93.01 (0.5741.14)
69.48 1.29 (12.45)
39.62 (-.30
5/5
HDX
92.29 0.000-1.86)
67.48 00.00 (-4.45)
39.40 +-000-7.17)
5/5
Ci.comi |laenny
< 5.31mms A energy
usage Σ 23.95m.
(7586 architesturrs malsfied)
ConNAS
93.14 +0.5 (-1.01)
70.21 (-.77))
41.45 1.01
5/5
ProxylessNAS
92.29 (-1.0-)
67.48 (-4.45)
39.40 (-.013))
5/5
FBNet
92.01 1 (-200)
67.07 (-1.77))
39.19 1 0-000
5/5
TAS
93.67 2 0.000
70,91 1:00
41.02 2 0.00
0/5
TF-NAS
93.27 0.00
70.06 (1188))
39.75 0.000
5/5
HDX
92.29 (-1.66)
67.48 0.00 (-4.46)
39.40 (-7013)
5/5
over time relative to the stator), and mixed eccentricity (a combination of static and dynamic eccen- tricity).
The goal in this use case is to discover neural network architectures that are deployable across a wide range of microcontrollers with diverse hardware capabilities. To define realistic deployment constraints, we base our hardware constraints on hand-crafted architectures (listed in Appendix G each having different resource requirements. We construct search space based on ID convolutions consisting of layers. Each layer has varying configurations in terms of the number of filters, convolution types, and the inclusion of skip connections, resulting in total of 1 1.94 billion unique
8

---

Published as a conference paper at ICLR 2026
Table 3: NATS-Bench size search space classification results. Test accuracy (mean è over runs) reported under hardware constraints, including the number of parameters, number of FLOPs, and peak memory usage. The constraints are chosen such that about 50% of the architectures each search space meet them. The relative error compared the optimal architecture satisfying the hardware constraints is reported between parentheses. Best results for each constraint and dataset are highlighted bold. Performance runs that do not always satisfy the constraints are reported in gray.
Method
Top-1 accuracy (%)
Runs satisfied
CIFAR-10
CIFAR-100
ImageNet16-122
Cosparam #parameters <, 261650 (16385 ondeiteehuess sotisfidd
ConNAS
91.74 1 (.66)
66.22 0.47 (-2.0))
39.77 (0.77
5/5
ProxylessNAS
85,87 6.66)
51.11 2.35 (-17.1)
25.27 42.13(11626)
5/5
FBNet
93.22 2020
69.43 0.22
45.68 0.33
0/5
TAS
92.25 +000
67.34 0.54
41.89 0.33
0/5
TE-NAS
91.78 0.00 64.75)
63.64 (-000))
38.00 +.0.0
5/5
HDX
86.10 1.28 (-6-4)
51.71
(-177a))
25.36 (-16667)
5/5
Cs.men peak memory usage 655kB
(20450 architecturrr mitiiied)
ConNAS
93.28 (-1.44
69.70 19.70 1.0 (-.26)
44.58 2..433-1335)
5/5
ProxylesSNAS
91.33 (-299-))
68. 22 (-2.6))
38.49 (-7445)
5/5
FBNet
93.16 =.200
69.69 0.05
45.53 037
1/5
TAS
92.93 (-199)0..99
70.10 (-0.76)
44.98 (5.95)
5/5
TF-NAS
92.99 +0.10(-0.44)
70.21 0.16 (-8.55
45.25 +0.27 (-1.99)
5/5
HDX
91.59 (.199)))88)
68.03 a 0.299.2..88 (-2.83)
38.41 (..077
5/5
Cslllops #FLOPs
< 1196M (16385 anrhitecturs malifedd
ConNAS
91.71 (-1.0)
65.44 (-40)1)
38.33 (5.36))
5/5
ProxylessNAS
89.55 (-2225)
63.51 0 0.67
32,43 0 (-11.0)
5/5
FBNet
93.02 20099
69.28 0.36
45.04 0.60
0/5
TAS
92.15 20199
68.66 0.52
42. 2.822
3/5
TF-NAS
91.10 4:25
66.56 (122
37.41 40.95-6006)
5/5
HDX
84.43 (-9.77)77)
49.02 a 0.35 (-20.52)
23.17 (-20299)
5/5
Gaammb #parameters & 261650 A peak memory usage S 655kBA
#FLOPs < 1196M (13955 erchitectures sotisded)
ConNAS
91.72 0,35
65.95 (-2977
39.52 (0.46(2225)
5/5
ProxylessNAS
84.12 0.28 (-8.25)
49.12 (-1.88))
22.64 (031
5/5
FBNet
92.59 20.3
67.92 0.64
43.09 1.12
0/5
TAS
92.50 =0000
67.02 0.00
41.30 0.00
0/5
TE-NAS
90.93 (-1.44)
65.44 41 +0.37 (-.51)
37.03 +0.466
5/5
HDX
83.99 (-007887
48.64 0.00 (220228)
22.50 (-00027
5/5
architectures. The detailed description of the search space is provided in Appendix F The dataset contains 31,920 instances, split into 25,530 for training and 6,390 for testing. Each instance consists of 256 time steps sampled at 5 kHz, with eight features: stator currents (3), phase voltages (3), rotor speed (1), and rotor angle (I). We use 80% of the training set to train the supernet, while the remaining 20% is used as validation set for architecture selection. The training and selection procedure follows the same approach outlined in Section 4.1 The selected urchitectures are retrained from scratch using five-fold tross-validatin over 200 epochs. Their performance is subsequently evaluated on the test set. We compare the best architectures found by CONNAS with hand-crafted models.
Table 4 compares the performance and resource consumption of architectures discovered by CON- NAS with manually designed baselines. CONNAS consistently identifies architectures that outper- form handcrafted ones in terms accuracy under specific hardware constraints. For example, under constraint C1 CONNAS discovers architectures that achieve accuracy of 99.77%7% which is 1.55% higher than Conv DDReg, the best manually designed model under the same constraint. At c,, the discovered architecture reaches 94.37% accuracy, compared to 93.93% for Cony ID-Reg-Min.

---

Published as a conference paper at ICLR 2026
CarRis
froxy KoSNAS
HSN::
TAS
TP 445
MOX
CIFAR-10
CIRAR-100
ImageNer16-120
Gacaro
Curmen Constrants
Coaare
Culeeay Constranbds
Sooone
Comitra itss
(a) topology search space
CorRAS
PENCE
TF-MS
MOX
CIFAR-10
CIFAR-100
IraggeNet66-13 6-120
S-alcurod (50
Cilhen
Consrrannts
Comtrrattt
Constrante
(b)
size search space
Figure 2: Performance comparison of CONNAS against baseline methods. Performance is measured as the difference in test accuracy (%) between the architecture found cach method and the optimal architecture satisfying the hardware constraints. Each point represents a run, runs violating hardware constraints are marked with a cross.
Table Classification performance and resource usage of models discovered by ConNAS under various hardware constraints, compared to handcrafted baseline models. The constraints c2, C3, and defined as follows: d1: model size S 2MB, peak memory usage I 640kB; oz: model size I 256kB, peak memory usage I 112kB; cat model size < 128kB, peak memory usage < 36kB; model size S 64kB, peak memory usage < 18kB.
Name
Top-1 accuracy (%)
Satisfied constraints
Model size (bytes)
Peak memory usage (bytes)
Cs
ConNAS (unconstrained)
97.36 s0.27
6.06M
524k 0.00
ConNAS + Cı
97.09 20.47
1.93M 1322
524k 0.00
ConNAS + 02
96.17 20,4
122k 30.7
69.6k a.ok
ConNAS +
94,96 20.69
26.0k 13.5k
16.8k 10.1k
ConNAS
94.37
8.69k +2122
12.3k
Cony ID-Reg
95.34 +0,29
135K
16.4k
Conv D-DS
93.07 40.35
51.3k
16.4k
Conv D-Ree-Max
93.31 +1.66
22. IM
524k
Conv DD-DIDDDDSMMS D--D--Max
96.72
7.50M
524k
DonvID-Reg. D---e--Miin
90.15 11.33
3.47k
12.3k
Co-vID-DS DDDD-MMin
93,93 21.64
2.32k
12.3k
CONCLUSION
We introduced CONNAS, a novel hardware-aware, gradient-based NAS technique that explicitly en- forces hardware constraints during the search process. Unlike prior approaches that rely on weighted loss terms for hardware-awar regularization, CONNAS directly modifies the gradients of archi- tecture parameters when hardware constraints are violated, effectively steering the search toward feassbleee solutions. Experiments on NATS-Bena demonstrate that CONNAS consistently discovers architectures that satisfy various hardware constraints, achieving performance close to the optimal and overall outperforming existing baseline methods in both performance and compliance with hardware constraints. Furthermore, we validated CONNAS in a practical use case, where successfully identified high-performing architectures within tight resource budgets. These results highlight CONNAS as a promising approach for the automatic design of deep learning models for deployment in resource-constrained environments.
10

---

Published as a conference paper at ICLR 2026
ACKNOWLEDGMENTS
The resources and services used in this work were provided by the VSC (Flemish Supercomputer Center), funded by the Research Foundation Flanders (FWO) and the Flemish Government.
This research supported by the Research Foundation Flanders (FWO) under the SBO grant program ISH9Y24N.
This research was supported by Internal Funds KU Leuven (C3/23/047, C3 Retrokit).

---

Published as a conference paper at ICLR 2026
REFERENCES
Bowen Baker, Otkrist Gupta, Nikhil Naik, and Ramesh Raskar. Designing neural network archi- tectures using reinforcement learning. In International Conference on Learning Representations, 2017.
Gabriel Bender, Hanxiao Liu, Bo Chen, Grace Chu, Shuyang Cheng, Pieter-Jan Kindermans, and Quoc V. Le. Can weight sharing outperform random architecture search? investigation with TuNAS, In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recogni- 2020.
tion, pp.
Han Cai, Ligeng Zhu, and Song Han. ProxylessNAS: Direct neural architecture search on target task and hardware. In International Conference Learning Representations, 2019.
Liang-Chich Chen, Maxwell Collins, Yukun Zhu, George Papandreou, Barret Zoph, Florian Schroff, Hartwig Adam, and Jon Shlens. Searching for efficient multi-scale architectures for dense image prediction. In Advances in Neural Information Processing Systems, volume 31. Curran Asso- ciates, Inc., 2018
Francois Chollet. Xception: Deep learning with depthwise separable convolutions. In Proceedings of the IEEE Conference Computer Vision and Pultern Recognition, pp. 1251-1255 2017.
Gregory De Ruyter, Bart Vanrumste, Mathias Verbeke, and Hans Hallez. A Fault Recognition Dataset for Eccentricity Faults in Rotor-Eccentric Induction Motor, 2025. URL https://
doi www.10rg11... 48804/NSQJCU
Philip Desenfans, Zifeng Gong, Dries Vanoost, Konstantinos Gryllias, Jeroen Boydens, and Davy Pissoort. The influence of the unbalanced magnetic pull on fault-induced rotor eccentricity in induction motors. Journal of Vibration and Control, 30(5):943-959, 2024.
Philip Desenfans, Zifeng Gong, Dries Vanoost, and Davy Pissoort. A python magnetic equivalent circuit model for rapid data generation of induction motors in faulty condi- tions., 2025. URL https: /ttg//gttaaa//a/ http:///tttaaaal..aaaa.aaaaa kuleuven. group-ccmppoe -xruppccanpus-bruueee wavecore wwwlic/phd-ph..i... http:///occcppppppeeeeeeeeaaaaaa immec
Xuanyi Dong and Yi Yang. Network pruning via transformable architecture search. In Advances in Neural Information Processing Systems, volume 32. Curran Associates, Inc., 201 9a.
Xuanyi Dong and Yi Yang. Searching for a robust neural architecture in four GPU hours. In Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 1761-1770, 2019b.
Xuanyi Dong, Lu Liu, Katarzyna Musial, and Bogdan Gabrys. NATS-bench: Benchmarking NAS algorithms for architecture topology and size. IEEE Transactions on Pattern Analysis and Ma- chine Intelligence, pp. 1-1, 2021. ISSN 0162-8828, 2160-9292, 1939-3539.
Thomas Elsken, Jan Hendrik Metzen, and Frank Hutter. Efficient multi-objective neural architec- ture search via lamarckian evolution. In International Conference on Learning Representations, 2019a.
Thomas Elsken, Jan Hendrik Metzen, and Frank Hutter. Neural architecture search: A survey. Journal of Machine Learning Research, 20(55):1-21, 2019b.
Geoffrey Hinton, Li Deng, Dong Yu, George E. Dahl, Abdel-rahman Mohamed, Navdeep Jaitly, Andrew Senior, Vincent Vanhoucke, Patrick Nguyen, Tara N. Sainath, and Brian Kingsbury, Deep neural networks for acoustic modeling in speech recognition: The shared views of four research groups. IEEE Signal Processing Magazine, 29(6):82 97, 2012. ISSN 1558-0792.
Deokki Hong, Kanghyun Choi, Hye Yoon Lee, Joonsang Yu, Noseong Park, Youngsok Kim, and Jinho Lee. Enabling hard constraints in differentiable neural network and accelerator exploration. In Proceedings of the 59th ACM/IEEE Design Automation Conference, DAC '22, pp. 589-594 Association for Computing Machinery, 2022. ISBN 978- 978-1-4503--114.99
12

---

Published as a conference paper at ICLR 2026
Yibo Hu, Xiang Wu, and Ran He. TF-NAS: Rethinking three search freedoms of Aitency-constraine differentiable neural architecture search. In European conference on computer vision, pp. 123- 139. Cham: Springer International Publishing, 2020.
Eric Jang, Shixiang Gu, and Ben Poole. Categorical reparameterization with gumbel-softmax. In International Conference on Learning Representations, 2017.
Alex Krizhevsky, Ilya Sutskever, and Geoffrey Hinton. ImageNet classification with deep con- volutional neural networks. Communications of the ACM, 60(6):84 90, 2017. ISSN 0001-0782, 1557-7317.
Colin Lea, Michael D. Flynn, Rene Vidal, Austin Reiter, and Gregory D. Hager. Temporal convo- lutional networks for action segmentation and detection. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 156-165, 2017.
Chaojian Li, Zhongzhi Yu, Yonggan Fu, Yongan Zhang, Yang Zhao, Haoran You, Qixuan Yu, Yue Wang, Cong Hao, and Yingyan Lin. HWW-NAS-bench Hardware-aware neural architecture search benchmark. 2021.
Edgar Liberis, Lukasz Dudziak, and Nicholas D. Lane. μmas: Constrained neural architecture search for microcontrollerr. In Proceedings of the 1st Workshop on Machine Learning and Systems, EuroMLSYs '21, pp. 70-79, New York, NY, USA, 2021, Association for Computing Machinery. ISBN 9781450382984.
Lin, Wei-Ming Chen, Yujun Lin, John Cohn, Chuang Gan, and Song Han. MCUNet: Tiny deep learning on IoT devices. In Advances in Neural Information Processing Systems, volume 33, pp. 11711-11722. Curran Associates, Inc., 2020.
Hanxiao Liu, Karen Simonyan, and Yiming Yang. DARTS: Differentiable architecture search, In International Conference on Learning Representations (2019), 2019.
Niv Nayman, Yonathan Aflalo. Asaf Noy, and Lihi Zelnik. Hardcore-nas Hard constrained dif- ferentiable neural architecture search. In Marina Meila and Tong Zhang (eds.). Proceedings of the 38th International Conference on Machine Learning, volume 139 of Proceedings of Machine Learning Research, pp. 7979-7990. PMLR, 18 24 Jul 2021.
NVIDIA Inc NVIDIA jetson TX2: High performance AI the edge. URL https:
7/www. nvidia com/en-us www.://ooooooooooooooooooouu machi nes http///ebeddddddeeeeeeeeeeeeeeee systemss/ssssss///ssosmooosss
extson-tx//
Esteban Real, Sherry Moore, Andrew Selle, Saurabh Saxena, Yutaka Leon Suematsu, Jie Tan, Quoc V. Le, and Alexey Kurakin. Large-scale evolution of image classifiers. In Proceedings of the 34th International Conference on Machine Learning Volume 70, ICML'17. 17, pp. 2902-2911. JMLR.org, 2017.
Tim Salimans, Jonathan Ho, Xi Chen, Szymon Sidor, and Ilya Sutskever. Evolution strategies scalable alternative to reinforcement learning. arXiv preprint arXiv: 038664, 2017.
Onur Surucu, Stephen Andrew Gadsden, and John Yawney. Condition monitoring using machine learning: A review of theory, applications, and recent advances. Expert Systems with Applications, 221:1197331 2023. ISSN 0957-4174.
Mingxing Tan, Bo Chen, Ruoming Pang, Vijay Vasudevan, Mark Sandler, Andrew Howard, and Quoc Le. Mnasnet: Platform-aware neural architecture search for mobile. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 2820-2828, 2019.
Quinten Van Baelen and Peter Karsmakers. Constraint guided gradient descent: Training with in- equality constraints with applications in regression and semantic segmentation. Neurocomputing, 156:12 2023. ISSN 0925-2312.
Alvin Wan, Xiaoliang Dai, Peizhao Zhang, Zijian He, Yuandong Tian, Saining Xie, Bichen Wu, Matthew Yu, Tao Xu, Kan Chen, Peter Vajda, and Joseph E. Gonzalez. FBNetV2: Differentiable neural architecture search for spatial and channel dimensions. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 12962-12971, 2020.
13

---

Published as a conference paper at ICLR 2026
Colin White, Willie Neiswanger, and Yash Savani. Bananas: Bayesian optimization with neural architectures for neural architecture search. In AAAI, pp. 10293-10301, 2021.
Bichen Wu, Xiaoliang Dai, Peizhao Zhang, Yanghan Wang, Fei Sun, Yiming Wu, Yuandong Tian, Peter Vajda, Yangqing Jia, and Kurt Keutzer. FBNet: Hardware-aware efficient ConvNet design via differentiable neural architecture search. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.
2019.
Sirui Xie, Hehui Zheng, Chunxiao Liu, and Liang Lin. SNAS: stochastic neural architecture search. In International Conference on Learning Representations, 2019.
Yanqi Zhou, Siavash Ebrahimi, Sercan Ö Ank, Haonan Yu, Hairong Liu, and Greg Diamos. Resource-efficient neural architect. arXiv preprint arXiv:1806.0991,2220 2018.
Barret Zoph and Quoc Le. Neural architecture search with reinforcement learning. In International Conference on Learning Representations, 2017.
Barret Zoph, Vijay Vasudevan, Jonathon Shlens, and Quoc V. Le. Learning transferable architectures for scalable image recognition. In Proceedings of the IEEE Conference Computer Vision and Puttern Recognition, pp. 8697-8710, 2018.
14

---

Published as a conference paper at ICLR 2026
A COMPUTATION HARDWARE METRICS
In this work, we focus on three hardware metrics: the number of parameters, the number of FLOPs, and the peak memory usage during inference. The following subsections describe how each of these metrics is computed. It is important to note that our proposed method is not limited to these metrics and can be applied to any hardware metric, whether derived analytically, measured through profiling, or predicted by learned models.
A.1 NUMBER OF PARAMETERS
The total number of parameters candidate architecture is computed by summing the number of parameters of each individual layer. To estimate the model size in bytes, multiply the total number of parameters by 4, assuming 32-bit floating-point representation.
A.1.1 CONVOLUTIONAL BLOCK
convolutional block consists of convolutional layer, followed by batch normalization and ReLU activation. We assume that batch normalization is fused into the convolutional layer during inference.
Regular Convolution:
params = (Cin x k2 x kw + 1) x Cout
(8)
where Cin and Cout are the input and output channels, kin and ki are kernel dimensions, and the +1 accounts for the bias term.
Depthwise Separable Convolution:
params (Cin Kh Ki + 1) + (Cin Cout + 1)
where the first term corresponds to the depthwise convolution and the second to the pointwise
volution.
A.2 FULLY CONNECTED LAYER
params (N,, + 1) Nowt
(10)
where Ni and Nout the input and output features, respectively, and the +1 accounts for the bias term.
A.3 NUMBER OF FLOPS
Similar to the parameter count, total number of FLOPs computed by summing the FLOPs of each individual layer.
A.3.1 CONVOLUTIONAL LAYER
FLOPs = 2 x (Oin x kn x kw + 1) x Hout X Wout Cout
(1I)
Here, Hout and W out are the output feature map dimensions.
A.3.2 RELU
FLOPs Hout X Wout X Cout
(12)
A.3.3
LINEAR LAYER
FLOPs 2 x (Nin + 1) x Nout
-oot
(13)
A.3.4 GLOBAL POOLING LAYER
FLOPS (kr, Ki + 1) Hout Wont Cout
(14)
15

---

Published as a conference paper at ICLR 2026
A.4 PEAK MEMORY USAGE
Peak memory usage during inference is estimated by computing the combined size of input and output feature maps for each layer. The maximum of these values across all layers is taken as the peak memory usage.
16

---

Published as a conference paper at ICLR 2026
B ABLATION RESCALE FACTOR
Table 5: NATS-Bench topology search space classification results on ConNAS with different rescale factors. Test accuracy (mean std over runs) is reported under hardware constraints, using on-device surements of inference latency and energy usage on an NVIDIA Jetson TX2 Edge GPU. The constraints are chosen that approximately 50% of architectures the search space satisfy the constraints. relative emor compared to the optimal architecture satisfying the hardware constraints is reported between parentheses.
Rescaling factor
Top-1 accuracy (%)
Runs satisfied
CIFAR-10
SIFAR-100
ImageNeet6-122
Oullatency
latency S, 5.31ms (7814 archirectures sotiffed)
1.2
93.18 (-.00-1.08)
70.12 (-.06
40.59 (-200-601)
5/5
1.5
93.20 10009
70.07 (-300)
40.16 16 a (-644)
5/5
2.0
93.21 +006 (-1.85)
70.19 (-292
41.04 + +1.187-3.56
5/5
5.0
93.25 +004 (-7.22)
70. 10 +0.10
40.18 0.96 (-4422
5/5
10.0
92.74 +0.7 23.95m..
68.83 (.79)))
39.60 (-780-7000
5/5
Ouamenny
energy usage
(7etetcchi antiglid)
1.2
92.97 (-.54()
69.51
(-2.43)
40.32 1.56/4441)
5/5
1.5
93.20 +0.0 (-0.5)
70.07 +0.33
40,16 0.97 (-661)
5/5
2.0
92.95 (-.53(-1.20)
69.55 1.50
40.75 1.66 (-6.2)
5/5
5.0
93.17 1.008
70.28
0.00
41.90 0:90 (-..77)
5/5
10.0
92.72 20.6 (-1.43)
68.87
(:3.07)
40,03 (-6.74)
5/5
latency S. m.8 A energy usage S. 23 22395m..J
(7586 architeetuer sntigfied)
1.2
93. 14 2 (-1.1))
70.21 20.17
41.45 1.01
5
1.5
92.95
69. 1.50 (-2.99)
40.75
5/5
2.0
93.17 2000 (-0.88
70.28 (-1.66)
41,90 = (-.66)
5/5
5.0
92.89 ( (-1.26)
69.45
1.45 (-2.999
40.28 1,58 (-6.55)
5/5
10.0
92.49 0.66 (-1.66)
68. 19 1.81 (-3.75)
39.31 (.58227.22)
5/5
17

---

Published as a conference paper at ICLR 2026
Table 6: NATS-Bench size search space classification results on ConNAS with different rescale factors. Test accuracy (mean std over 5 runs) is reported under hardware constraints, including the number of pa- rameters, number FLOPs, peak memory usage. constraints are chosen so that about 50% of architectures in each search space meet them. The relative error compared to the optimal architecture satisfying the hardware constraints reported between parentheses.
Rescaling factor
Top-1 accuracy (% CIFAR-100
Runs satisfied
CIFAR-10
ImageNet16-122
Css.puram
#parameters S 261650 (16385 orchiteceess satisfed)
1.2
91.74 +0.29
66.22 0 (-2.70)
39.77 +.0777-22777
5/5
1.5
92.09 +0.34)0044)
65.92
40.29 -.106-1.74)
5/5
2.0
91.87 2007
66.22 2000
39,48 (-229)2..35)
5/5
5.0
91.85 (0.08)
65.97 2033 (:2.5)
39.55 (.533-2.488
5/5
10.0
91.67 2008 (-0.00)
65.99 0.36 (-2.23)
39.31 (-273)
5/5
Couman
peak memory usage
655kB
(20490
natigfed)
1.2
93.28 (-0204
69.70
44.58 (.43--.255
5/5
1.5
93.26 (0.15(-0.1))
70.14
2033 (-0.22)
44.67 (.668-11.26
2.0
93.19 20.17
69.54 2.95 (-7.12)
44.42 +0.58
5/5
5.0
93.26 10.16
70.06
20.15
44,65 10018(-1.29)
5/5
10.0
93.29 (0.31)
70.01
0.85)
44.62 (-1.1))
5/5
Css.iops
#FLOPs
1196M (16385 wnchieerwww matigles)
1.2
91.71 (-109)
65
(-.00))
38.33 (.036/-5.13)
5/5
1.5
91.73 (-107)
65.80 (-37
38.73 (-.53)
5/5
2.0
91.47 (-1.3)
65.38 0
38.70 (41777
5/5
5.0
91.48 (-.32
66.03 1.366-3.57)
38.56 1.45(499)
5/5
10.0
91. +..56 (-0.2)
66. 1.20 (-3.36)
38.83 1.07 (-466)
5/5
#parameters
& 261650 A peak memory usage
S 655kBA
#FLOPs < 1196.M (13956 orckieecteesss sutigfed)
1.2
91.72 (.3565)665))
65.95 (-.977
39.52 (.46-22.55)
5/5
1.5
92.00 a 0.21 (-0.37)
66. 76 a0.71 (-2.79)
39.54 1.13
5/5
2.0
91.52 0.61 (-0.55)
65. 70 (-3622))
38.79 (-73772
5/5
5.0
91.73 0.25
65.53 =0229
38.78 (-479-2999)
5/5
10.0
91.88 (-0.99)
65.85 (-300)
39.31 (.044-2.45)
5/5
18

---

Published as a conference paper at ICLR 2026
C EXPERIMENTS WITH STRICTER HARDWARE CONSTRAINTS
Table 7: NATS-Bench topology search space classification results under stricter hardware constraints. Test accuracy (mean + std over runs) reported under hardware constraints, using on-device measurements of inference latency and energy usage NVIDIA Jetson TX2 Edge GPU. Stricter hardware constraints set so that approximately 25% and 10% of architectures in the search space satisfy the constraints. The relative compared to the optimal architecture satisfying the hardware constraints reported between parentheses.
Method
Top-1 accuracy (%)
Runs satisfied
CIFAR-10
CIFAR-100
ImageNet16-120
Cs.mencc22 : latency < 4.14ms (3908 architertures ratiefed)
ConNAS
92.08 6000 (-1.97)
67.09
0.20
39.09 (-5.2)
5/5
ProxylesNNAS
92.29 0000
67.48 (-4.04)
39.40 (-5.57)
5/5
FBNet
93.22 0,009
70.03 0.07
39.73 0.05
0/5
TAS
93.27 + 0.00
70,06 0.00
39.75 0.00
0/5
TF-NAS
92.29 (-100)))
67.48 0.00 (-4.04)
39.40 0.00 (-5.57)
5/5
HDX
10.00 0.000 (-6108))
1.00 (-7002))
0.83 (-4173)
5/5
Ca.nneecy latency
3.03ms (1364 architecturee sattefed)
ConNAS
92.05 0.07 (-0.73)
67.01 0.20 (-1.55)
38,83 (.726)
5/5
ProxylesSNAS
92.29 0,00
67. 67.48 0.00
39.40 0.00
0/5
FBNet
93.22
70.03
39.73
0/5
TAS
92.00 +0001
67.13 0.05
39.13 0.05
0/5
TF-NAS
10,00 0:00 (-8.78)
1.00 a 0.000
0.83 a 0.00 (9.60)
5/5
HDX
10,00 0.00 (-2.78))
1.00 0.00 67.36)
0.83 2.000 (-3.60))
5/5
Cuenogy25 &energy
usage < 18.6m.J
(3908 archiertuuess sotisfed)
ConNAS
92.06 & 0.06 (-1.98)
67.09 a 0.20 (-4.45)
39.04 19.04 (-59)
5/5
ProxylesNNAS
92.29 0.00 (-1.76
67.48 0.00 (-4.04)
39.40 0.00 (-5.57)
5/5
FBNet
93.1 20.12
69.97 0.08
39.69 0.06
0/5
TAS
93.01 0.57
69 69.48
39.62
1/5
TE-NAS
92.29 & 0.00 (-1.76)
67.48 a 0.00 (-4.04)
39.40 0.00 (-5.57)
5/5
HDX
10.00 0.00 (-8.05)
1.00 a (-7052))
0.83 a 0.00 (-44.13)
5/5
Casenergv10 energy usage < 13.3mn
(1364 architectures satigled)
ConNAS
92.05 0.07 (-0.73)
67.01
(-1.33)
38.83 (-726))))
5/5
ProxylesNNAS
92.29 & 0.000 (-0.49)
67.48 a (-0.88)
39.40 0.00 (-1.3)
5/5
FBNet
93.14 20122
69.97 2.008
39.69 0.06
0/5
TAS
92.00 0.01
67.11 +000
39.15 0.05
3/5
TE-NAS
10.00 0.00 (-82
1.00 0..00
0.83 0.000
5/5
HDX
10.00
(-82.78)
1.00 (-077)67.8))
0.83 +.00 (.3.60))))
5/5
Cs.comb25 latency
4.1 4..s A energy usage S 18.6m.
(37K5 architertuees antigled)
ConNAS
92.10 0.06 (-1.95)
67.16 1616 (-4.36)
39.35 0.59 (-5.6)
5/5
ProxylessNAS
92.29 0.00 (-1.76)
67.48 0.00
39.40 0.00 (-5.57)
5/5
FBNet
92.01 10.00 (-2.4)
67.07
39.19 (-.78)
5/5
TAS
93.27 a 0,000
70,06 a
39.75 0.00
0/5
TF-NAS
92.29 = 0.00 (-1.76)
67.48 0.00 (-4.04)
39.40 0.00 (-5.57)
5/5
HDX
10,00 0.00 (-84.05)
1.00 1.000 (-70.52)
0.83 (-44.13)
5/5
Cacombl0 latency
< 3.03ms energy usage S 13.3m..
J (1526 architectures satiglee)
ConNAS
92.08 = 0.06 (-0.70)
67.20 2021 (-1.16)
39.10 -9.10=0.00.0 (-1.33)
5/5
ProxylessNAS
92.29 0.00
67.48 20.00
39.40 0.00
0/5
FBNet
92.01 1.000
67.07 0.00
39.19 1.000
0/5
TAS
91.99
67.15 1.00
39.11
0/5
TF-NAS
10.00 0.00 (-2.78)
1.00 67.3))
0.83 a 4.0.0
5/5
HDX
10.00 /.00
1.00 0..00-67.11)
0.83 + (-79.60)
5/5
19

---

Published as a conference paper at ICLR 2026
Table 8: NATS-Bench size search space classification results under stricter hardware constraints. Test accuracy (mean std over runs) is reported under hardware constraints, including number parameters. number of FLOPs, and peak memory usage. Stricter hardware constraints were set so that approximately 25% and 10% of architectures in the search space satisfy the constraints. The relative error compared to the optimal architecture satisfying the hardware constraints is reported between parentheses.
Method
Top-1 accuracy (%)
Runs satisfied
C-FAR-10
CIFAR-100
Image Net1-111
Caparazs #parameters < 187210
ardlitecturss soniffed)
ConNAS
91.33
64.03
38.25
5/5
ProxylesNNAS
85.87 (.161
51.11 2:335
25.27 (.139-133999
5/5 5
FBNet
93.22 2:20
69 43 0:2:2
45.68 0.277
0/ 5
TAS
91.55 s031
64.09 606.1
39.01 1.39
0/5 5
TE-NAS
89.43 (-2444))
58.75 (-746)
31.99 1,56 (-722))
5 5/5
HDX
79.79 (-000-)
32.30 (-332222
15.53
(-20.33))33
5/5
Casparato #parameters
78906 (8278 architectures sangfeed)
ConNAS
90.04 (-1.0))
60,86 2.01 (-3.34)
34.09 (-2.7))
5
ProXylessNAS
85.87 (.6(35.23)
51.11 2 (-13.09
25.27 22.13-11..3)
5/5
FBNet
93.22
69.43 43
45.68
0/5
TAS
91.08 6040
63.75 0.09
37.04 e083
0/5
TE-NAS
85.62 (-.44))))
47.74 (-066-
25.43 (.00
5/5
HDX
79.79 0.001-11.31)
32.30 r
15.53 10:00
5/5
Cammemoey25 peak
memory usage S 393kB (12298 anchitentures maiified)
ConNAS
92.91 8.8 (-0.21)
69 28 (-1.22)
43.65 + (.10(-L005)
5/5
ProxylessNAS
91.33 (.199)))779
68.22 0.28 (-2.26)
38.49 0.03 (-621)
5/5
FBNet
93.16 0.29
69.69 0.005
45.53 0887
0/5
TAS
92.82 0.225
69.40 (-100)
43.69
5/5
TF-NAS
92.35 40.09 (-.77)
69.52 2.13 (-0.6)
43.09 a 0.61 (-166))
5/5
HDX
79.79 a (-00--3333)
32.30 = 0.00 (-38 78)
15.53 (-00
5/5
Commoncyy peak memory usage < 229 B 13584 architectures Aatictee))
ConNAS
92.30 (4.50)
68.80 0.32 (-0.50)
40.80 (.130)
5/5
ProxylesNNAS
91.33 (-1947))47))
68. 22 0.28 (-1.08)
38.49 (-3666)
5/5
FBNet
93.16 0.29
69 69 0.05
45.53 087
0/5
TAS
92.31 2016
68.67 0.446
41.98 0.9
0/5
TE-NAS
91.33 (-1947).47))
68.22 + (-1.08)
38.4 (.366))1))
5/5
HDX
79.79 +000(-3300)
32.30 (-0000)700)
15.53 6.00426657)
5/5
Cop. : #FLOPs
< 514M (8793 architectures natigfied
ConNAS
90.50 (223--1488)
63.30 0.39 (-5.22)
34.87 0.92 (-6003
5/5
ProxylessNAS
89.55 (-2444)
63.51 0.67 (-5.07)
32.43 0.84 (-847)
5/5
FBNet
93.02
69.2
45.04 0.60
0/5
TAS
91.80 -030
68.0
40.45 0.37
0/5
TF-NAS
89 89.25 (2.22)
63.41 0000 (-5.1))
33.31 a 1.35
5/5
HDX
79.79 0.00
32.30 2..00
15.53 25.0/
5/5
Costoppst #FLOPs
223M
anchitettuess natisfie)
ConNAS
89.85 (-533)
61.35 1.46 (-3.99)
33.57 + 0.89 (-489)
5/5
ProxylessNAS
89,55 10 (-1.61)
63.51 0.67 (-1.3)
32.43 0.84 (-600)
5/5
FBNet
93.02 2009
69.28 0.36
45.04 0.00
0/5
TAS
91.39 0.49
66.80
38.04 0.70
0/5
TF-NAS
89,29 (-1.87)
63.38 0.00 (-3.66
33.81 a (4.60)
5/5
HDX
79.79 (-1.37)
32.30 y
15.53 a 0.00 (-2099)
5/5
20

---

Published as a conference paper at ICLR 2026
Ga.combss #parameters Σ 187210 peak memory usage S 393ABA
#FLOPs & 514M (6666 archnecnurs satighed)
ConNAS
90.87 (-1.7)
64.61 (-1.77)
36.60 1,21 (-266)
5/5
ProxylessNAS
84.12 0.28 (-7.00
49.12 (-17.46
22.64 (-1666))))
5/5
FBNet
92.59 0.32
67.92
43.09 1.12
0/5
TAS
91.31 2022
0.95
37.95 0.34
0/5
TE-NAS
87.42 +042 (-.22 (-4.20)
57.65 035 (-8.3))
27.80 (.331-12.777
5/5
HDX
79.79 0.00--1183)
32.30 =000
15.53 (-00022677)
5/5
Gaaembi #paramelers Σ 78906 A peak memory usage
2291BA
#FLOPs S 223M 977 archiecttres satigled
ConNAS
89.66 0.46 (-8.60)
62.06 (..3.
32.57 (.3(2))))
5/5
ProxylesSNAS
84.12 1022
49.12 (-119))
22.64 (0.11-11.66)
5/5
FBNet
92.59 0.32
67.92
43.09
0/5
TAS
91.53 2014
64.32 0.36
37.10 2018
0/5
TE-NAS
86.46 0.11 (-3.6)
56. 66.76 + (-6.32)
25.81 +..51
5/5
HDX
79.79 (.000-)0.33
32.30 (3000-
15.53 (.00-18877)
5/5
21

---

Published as a conference paper at ICLR 2026
D ABLATION WEIGHTING FACTOR BASELINE METHODS
Table 9: Comparison of classification results on NATS-Bench topology search space across baseline methods and weighting factors. Test accuracy (mean std over runs) is reported under hardware constraints, using on-device measurements of inference latency and energy usage on an NVIDIA Jetson TX2 Edge GPU. The relative error compared the optimal architecture satisfying hardware constraints reported between parentheses. Best results for each method, constraint, and dataset are highlighted in bold. Performance of experiments that do not satisfy hardware constraints are reported gray.
Method
Alurdware
Top-1 accuracy (%) CIFAR-100
Runs satisfied
CIFAR-10
ImageNet16-122
Ohclttncy latency
5.3lms
17814 archinectuees sutigled)
ProXylessNAS
0.5
92.06
(-2.2)))
67.15 0. (-18(-)001)
39.23 0.09 2.009 (-7.97)
5/5
ProxylesNNAS
1.0
92.29 0.00
67.48
(-568-)
39.40 (-00
5/5
ProxylessNAS
1.5
42.92 45.07
27.59
16.26
21.12
5/5
FBNet
0.5
93.10
(-1.16)
69.94 (-3.22)
39.67 (-6.91)
5/5
FBNet
0.6
93.18 0.32 (-1.08)
70.00 = 0.08 (-3.16)
39.71 71 0.06 (-6.99)
5/5
FBNet
0.7
93.27 0.00 (-1.00)
70.06 0.00 (-3.10)
39.75 0J00 (-685)
5/5
TAS
0.5
93.67 0.00
70.91 0.00
41.02 0.00
0/5
TAS
1.0
93.67
70,91
41.02
0/5
TAS
1.5
93.67 &: 0.00
70,91
41.02 a 0,00
0/5
TF-NAS
0,5
93.27 0.00 (-1.00)
70.06 (-300))
39.75 0.00 (-6.55)
5/5
TE-NAS
1.0
93.27 0.00 (-1.00)
70.06 0.00 (.310))
39.75 0J00 (-6855
5/5
TE-NS
1.5
93.01
(-1.25)
69, 48
(-3.68)
39.62 0.30 08)
5/5
HDX
0.5
42.86
(-57.49)
27.51 36.30 (-45.6))
16.22
(-30
5/5
HDX
1.0
75.83
36.00 (-18.43)
54.18 (-18.99
31.69 17.25 (-14.1))
5/5
HDX
1.5
59.37
45.07 (-34.89)
40.89 26.44 (-32.27)
23.97 21.12 (-22.6))
5/5
Onameny energy usage
23.
J
(7814 architectueer vatidfed))
ProxylessNASS
0.5
92.29
(-1.86)
67.48 (-4.46)
39.40 0.00 (-7.37)
5/5
ProxylessNAS
1.0
92.29 0.00 (-1.86)
67.48 0.00 (-4.46)
39.40 = 0.00 (-7.77)
5/5
ProxylessNAS
1.5
26.46
36.80 (-67 69)
14.30 29.73 (-57.64
8.55 17:25 (-38.22)
5/5
FBNet
0.5
93.05 2.000
69.91 0.00 (-2.03)
39.64 (-70)2
5/5
FBNet
0.6
93.05
69.91
39, 64,64
5/5
FBNet
93.22 0009 (093)
70.03 0.07 (-1.92)
39.73 & 0.05 (-7.04)
5/5
TAS
0.5
93.67 0.00
70.91 0.00
41.02 0.00
0/5
TAS
1.0
93.67 0:00
70.91 0.00
41.02 0.00
0/5
TAS
1.5
93.67 0.00
70.91 0:000
41.02 0.00
0/5
TF-NAS
0.5
93.27 (-8899
70,06 0.000
39.75 (-7.02)
5/5
TENAS
1.0
93.01 20.57 (-1.14)
69.48 1.29 (-2.46)
39.62 (-7.1))
5/5
TE-NS
1.5
91.99 0.01 (:2.16)
67, 15 0.04 (-4.99)
39. 0004 (-7.6))
5/5
HDX
0.5
42.92 45.00 (-51.23)
27.59 36341 (-44.55)
16. 21,12 (-3055)
5/5
HDX
1.0
92.29 (-1.66)
67. 17.48
39.40 +
5/5
HDX
1.5
10.00 20.00
1.00 20000077.090
0.83 (-45..3))
5/5
Chacoaa latency I 5.3lmms A energy usage S. 23.95ma.
(2586 orchitectress sutisfed)
ProXylesSNAS
0.5
92.29 0.00 (-1.66)
67.48 0.00 (-1.66)
39.40 0.00 (-7.13)
5/5
ProxylesNNAS
1.0
92.29
67.48
39.40
5/5
ProxylessNAS
1.5
42.92 45.00 (:51.23)
27.59
36.41 (-H.35)
16.26
(-302
5/5
FBNet
0.7
92.01 0.00 (-2.14)
67.07 0.00 (-487)
39.19 = 0.00 (-7.34)
5/5
FBNet
0.6
92.01
0.00 (-2.14)
67.07
0.00 (-487)
39.19 0J00 (-7.34)
5/5
FBNet
0.7
92.01 0.00
67.07 0.00 (-487)
39.19 + (-7.34)
5/5
TAS
0.5
93.67
70.91
41.02
0/5
TAS
1.0
93.67 0.00
70.91 a
41.02 4
0/5
TAS
1.5
93.67 0.00
70,91 00000
41.02 a 0.00
0/5
TF-NAS
0.5
93.27 0,90 (-0-89)
70.06 0.00 (-1.88)
39.75 0.00 (-6.88
5/5
TF-NAS
1.0
93.27 0.00 (-0.89)
70.06
39.75 (-6.88)
5/5
TE-AS
1.5
92.76 0.69 (-1.39)
68.
(-3.05)
39.51 & 0.34
5/5
HDX
0.5
92.23 0.13
67 4.0.1
39.36 0.09
1/5
HDX
1.0
92,29
0.90 (-1.86)
.7,48 0.00 (-4.66)
39.40 0J00 (-7.13)
5/5
HDX
1.5
59.37 45.07
40.89 36.41
23.97 21
5/5
22

---

Published as conference paper at ICLR 2026
Table 10: Comparison of classification results NATS-Bench size search space baseline meth- ods and weighting factors. Test accuracy (mean std over runs) is reported under hardware constraints. including the number of parameters, number of FLOPs, and peak memory usage. The relative error compared to the optimal architecture satisfying the hardware constraints is reported between parentheses. Best results for each method, constraint, and dataset highlighted in bold. Performance of experiments that do not satisfy the hardware constraints are reported in gray.
Method
hardware
Top-1 accuracy (%)
Runs satisfied
CIFAR-10
CIFAR-100
ImageNet16-120
Cagaram #parameters S 261650
sattslie))
ProxylesNNAS
90.87 2002 (-1.66)
61.31 20.1 (-761)
35.34 0.31 (-6.9))
ProxylessNAS
1.0
85.87 =: 1.16 (:6.0)
51.11 22.35
25.27
#2.13 (-10.76)
5/5
ProxylessNAS
1.5
83.47 (-9.00)
45.36 (-956-3)
21,59
1.01 (-20.5)
5/5
FBNet
0.5
93.12 0.17
69.36 0.44
45,43
0/5
FBNet
0.6
93.22 0.20
69.43
45.68
0/5
FBNe
92.91 0.26
69.36
45.40
0/5
TAS
0.5
92,26 a 0.01
67.41 033
42.10 0.15
0/5
TAS
1.0
92.25 0,02
67.34 0.5
41,89 0.33
0/5
TAS
1.5
92.25
67.34
41.89 1.33
0/5
TE-NS
0.5
91.30 (-1.23)
64.88 (-4.04)
40.43 (0.24(-167)
5/5
TE-NS
1.0
91.78 0.00 (-0.75)
63 64 (-520)
38, 38,00 20000 (-400)
5/5
TE-NAS
1.5
91.56 + 0.30 (-0.97)
64.05 +0356 (-487)
38. 18.17 +0.24
5/5
HDX
0.5
88.00
54.74 (-311)))
29.33 (-12771
5/5
HDX
1.0
86. 10 (-6.4))
51.71 +.2.88
25.36 2.42 (-76.67)
5/5
HDX
83.37 0.88 (-9.16)
45.20 (-212222)
21.54 1 (-20.49)
Casmem peak memory usage
S 655k B (20449 or futeecttteet sanigfed)
ProxylessNAS
0.5
91.61
67.90
38.30 (-766)))
5/5
ProxylesSNAS
1.0
91.33 20.19 (-2.9)
68.22
38.49
5/5
ProxylesNNAS
91.33 00.9 (-2.9)
68.22 0..28 (-2.44)
38.49 (.7.45)
FBNet
0.5
93.24 0.17
70.03 0.2
45.43 0.55
0/5
FBNet
0.6
93.16 0:29
69.69 0.05
45.53 0.87
1/5
FBNet
0.7
93.23 1.16
69, 94 1.34
45.71 1
1/5
TAS
0.5
93.08 1..20
70.23 (-.16))
45.14 0.31 (-0.79)
5/5
TAS
92.93 (-044)
70.10 (-.22-
44.98 + (-0.95)
TAS
1.5
93.13 0.19 (-0.99)
70.30 #..19 (40.6)
45.02 0 (-0.91)
5/5
TE-NA
0.5
92.92
(-0.50)
69.94
44.73 +0667
5/5
TE-NAS
1.0
92.99
70.21
45.25
5/5
TENAS
93.02 = 0.28 (-0.49)
70.13 0.34 (-0.33)
45.02 00.6 (-0.91)
5/5
HDX
0.5
49,43 5.40 (.999)
60.85 13.96 (-10.01)
34.23 10.49 (-11.0))
515
HDX
1.0
91.59 0.19 (11.8)
68.03 0.0.9 (22.0)
38. 38.41 2007 (-7.53)
5/5
HDX
1.5
91.41
68.09 0..34
38.47 (.04
5/5
Capopp #FLOPs
1196M
(16083 architectures satiylee))
ProxylessNAS
0.5
91.36 = (-1444
66.93 0.65 (-2.4))
38.55 #
5/5
ProxylessNAS
1.0
89.55 0.22 (-3.25)
63.51 11.51 0.67 (-6.03)
32.43 0.84 (-11.03)
5/5
ProxylessNAS
1.5
87.92 (-1.88)
60.76
29.25 20.4
5/5
FBNet
0.5
92.97 1 0.06
69 29
45.47
0/5
FBNe
93.02 + 0.09
69.28 0.36
45.04
0/5
FBNe
0.7
93.02 + 0.06
69.44 0.49
45.07 0.46
0/5
TAS
0.5
92,46 0.34
69.15 0045
43.31 0.50
0/5
TAS
1.0
92.15
68.66 68.66-22
42.11
3/5
TAS
92.13 : 0.16 (-0.67)
68.60 (-0.94)
41.93 0.88 (-1.53)
TE-NAS
0.5
91.21 0.39 (-1.59)
66.75 (.2779)
37.50 0 (3.9)
5/5
TE-NAS
1.0
91.10 0.25 (-1.70)
66,56 (-12
37.41 0.94 (-6.06
5/5
TE-AS
1.5
90.98 0..3 (-1.82)
66.43 +.052
37.63 1.13 (-5.84))))
5/5
HDX
0.5
83.99 (-8.81)
48.64 0.00
22.50
(-20.07)
5/5
HDX
84.43 = 0.99 (-8.97)
49.02 + 0.85 (-20.52)
23.17 = 1.51 (-20.29)
5/5
HDX
1.5
83,99 0.00 (-8.J)
48.64 a 0.00
22.50 0.00 (-20.97)
5/5
23

---

Published as a conference paper at ICLR 2026
Ca.comb #parameters I 261650 A peak memory usage Σ 655kBA
#FLOPs Σ 1196M (13956 architcctures sailsfied)
ProxylesSNAS
0.5
89.56 (.44
61.22 1.1 (-7.70)
30.29 (-333411447
5/5
ProxylessNAS
1.0
84.12 60.8 (-8.25)
49.12 (-19800)
22.64 (-19111)
5/5
ProxylessNAS
1.5
84,49 +0.28 (-7.88)
50.56 (-18.66)
23.06 +0.31 (-18771
5/5
FBNet
92.47 4:4.5
1.15
42.33 1.448
0/5
FBNe
0.6
92.59 0.552
67.9 0.64
43.09 #:1.22
0/5
FBNet
0.7
92,46 0.18
67.84 0.18
42,41 0.488
0/5
TAS
0.5
92.37 1.1.8
67.40 0:5
41,42
0/5
TAS
92.50 0.00
67.02 A 0.00
41.30
0/5
TAS
1.5
91.97 a 0.07
67.02 0.49
41,08 0.29
0/5
TE-NAS
0.5
91.36 0.38 (-1.7)
66.35 + (-2.57)
38,31 (-.49
5/5
TENAS
1.0
90.93 +0.21 (-1.44)
65.41 +.337 (-3.51)
37.03 +0446
5/5
TE-NAS
1.5
91.16 (-1.21)
65 00 (-3.22)
36.53 0.000 (-5.23)
5/5
HDX
80.63
1.88 (1.74)
35.57 + 7.31 (-30.55)
16.93 3.12 (-24.4)
5/5
HDX
1.0
83,99
0.00 (-8.38)
48.64
a 0.00 (-20.20)
22.50 0.00 (-19.27)
515
HDX
1.5
83.99 0.00 (-8.38)
48.64 0.00 (-20.28)
22.50 0.00 (-19.27)
515
24

---

Published as a conference paper at ICLR 2026
E TRAINING DETAILS
To facilitate reproducibility, we provide additional experimental details in this appendix.
E.1 NATS-BENCH
We adopt the same data augmentation techniques and training procedure as described in Dong et al. (202T) The search process is performed in two stages.
In the first stage, supernet trained on 50% of the CIFAR-10 training set (25,000 images) for 150 epochs, using a batch size of 64. The operation weights are optimized via Nesterov momentum Stochastic Gradient Descent (SGD) with momentum of 0.9 and a weight decay of 5 x 10-1, The initial learning rate is set to 0.0025 and annealed to 0.001 over 100 epochs using cosine schedule. Architecture parameters are optimized using Adam with a learning rate of 0.001, weight decay of x 100 and 81 0.5, 82 0.999. The sampling temperature starts at 10 and is linearly annealed to 0.1 over 100 epochs. The rescale factor is kept constant throughout training (we use R = 1.2 for most experiments; an ablation study the rescale factor is presented in Appendix ().
In the second stage, an architecture selected based on the lowest cross-entropy loss, evaluated on the remaining 50% of the training set, while satisfying the imposed hardware constraints. The NATS-Bench performance lookup table is then used to retrieve the test accuracy of the sampled architecture on CIFAR-10. 10. CIFAR-100, and ImageNet16-120.
All experiments are repeated five times using fixed random seeds. The search is performed on single NVIDIA V100 GPU and took approximately hours complete.
E.2 CONDITION MONITORING USE CASE
For the condition monitoring case, dataset is created from Desenfans et al. (2025). representing readings from a voltage meter, a current sensor and an encoder. (De Ruyter et al. 2025 The com- plete dataset contains 31,920 time series instances, split into 25,530 train for training and 6,390 for testing. Each instance consists of 256 time steps with features: 3-phase current, 3-phase voltage, motor speed, and rotor angle. The instances are labeled either fault, static eccentricity fault, dynamic eccentricity fault, or mixed eccentricity fault. A Fast Fourier Transform (FFT) is applied to the 3-phase current and voltage signals, and all 8 features are normalized to the range [0, 1]. The search process is again performed in two stages.
In the first stage, supernet is trained on 80% of the training set (20424 instances) for 150 epochs, using a batch size of 128. The operation weights are optimized via Nesterov momentum Stochastic Gradient Descent (SGD) with a momentum of 0.9 and a weight decay of x 10-4, The initial learn- ing rate is set to 0.0025 and annealed to 0.001 over 100 epochs using a cosine schedule. Architecture parameters are optimized using Adam with learning rate 0.001, weight decay of 10-3 and 31 0.5, 32 = 0.999. The sampling temperature starts at 10 and is linearly annealed to 0.1 over 100 epochs. The rescale factor R is kept constant at 1.2.
In the second stage, architecture is selected based on the lowest cross-entropy loss, evaluated on the remaining 20% of the training set, while satisfying the imposed hardware constraints. The selected architectures are trained using 5-fold cross-validation, where each fold uses 80% of the training data for training over 250 epochs with a batch size of 16. Optimization is performed using SGD with momentum of 0.9 and weight decay of 1 x 14 The initial learning rate is set to 0.0025 and annealed to 0.0001 using cosine scheduler.
All experiments repeated five times using fixed random seeds. The search is performed single NVIDIA P100 GPU and took approximately 7 hours to complete.
25

---

Published as a conference paper at ICLR 2026
F CONDITION MONITORING SEARCH SPACE
The search space for the condition monitoring use case is defined as a supernet composed of eight consecutive edges. Each edge can select from the following candidate operations:
A ID convolution followed by batch normalization and a ReLU activation
A ID depthwise separable convolution, where each convolution is followed by batch normalization and a ReLU activation
An identity operation (available only even-numbered edges)
The classification head consists of a global average pooling layer followed by fully connected layer.
For each convolution, the number of output channels can be chosen from {16, 32, 64, 256, 512). Additionally, a stride of 2 applied to convolutions on odd-numbered edges. Convolutional filters shared across operations, proposed in Wan al. (2020). to reduce the number of trainable parameters and improve memory and computational efficiency.
26

---

Published as a conference paper at ICLR 2026
G CONDITION MONITORING HAND-CRAFTED BASELINES
To ensure coverage of diverse hardware constraints, we sampled six hand-crafted architectures from the proposed search space. These include the smallest (Conv ID-x-Min) and largest (Conv ID-x- Max) models in terms of latency and energy consumption, for both regular and depthwise-separabl convolutions (Chollet 2017). Additionally, Conv DDggg and ConvID-DD follow common design principles where channel width increases spatial dimensions decrease.
Table 11: Overview of hand-crafted architectures used for comparison. Rach convolutional block consists of a convolutional operation regular or depthwise separable, batch normalization, and ReLU activation.
Name
Number blocks
Kernel
Channels
Strides
Convolution
type
Conv ID-Reg ConvID-DS
4
3
[16, 32, 64, 128]
[2, 2, 2, 2]
Regular
4
3
[16, 32, 64, 128]
[2, 2, 2, 2]
Depthwisc Separable
Conv MonvlD-Reg-Man Max
8
1512, 512,
12, 1,
Regular
512, 512,
2, 2,1,
Conv ID-DS-Max
8
3
[512, 512, 512.
[2, 2,2, 1,
Depthwise Separable
512, 512,
512,
2. 2,
Conv ID-Reg Min ConvID-DS-MN
4
3
[8. 8. 8, 81
[2. 2, 2]
Regular
4
[8. 8. 8]
[2, 2,
Depthwise Separable
27

---

Published as a conference paper at ICLR 2026
H DISCLOSURE ON THE USE OF GENERATIVE ARTIFICIAL INTELLIGENCE
We used a large language model (LLM) as a writing aid. Ideation, scientific content, experimental design, and analysis were conducted entirely by the authors without any contribution of LLM.
28


Reviews:
[{"review_id": "4mrOxGuAaz", "reviewer": "Reviewer_wD6Z", "summary": "This paper presents CONNAS, a constraint-guided gradient modification framework for hardware-aware neural architecture search (NAS). Instead of introducing differentiable hardware regularizers or manually tuned weighting factors, CONNAS modifies the gradients with respect to architectural parameters to directly enforce hardware constraints such as FLOPs, parameter count, and memory usage. The approach is evaluated on NATS-Bench across multiple datasets (CIFAR-10, CIFAR-100, and ImageNet16-120).", "questions": "See Weaknesses", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "1. Clear motivation and practical focus: The paper identifies a well-defined challenge in hardware-aware NAS, balancing accuracy with strict resource constraints, and proposes a solution that directly targets constraint satisfaction.\n\n2. Readable and well-organized writing: The manuscript is well-structured, and the figures and tables are clear, making the technical content accessible without ambiguity.\n\n3. Comparison with relevant baselines: The evaluation includes established gradient-based NAS methods (e.g., ProxylessNAS, FBNet, TAS) re-implemented under a consistent setup, allowing fair comparisons.", "weaknesses": "1. Limited experiments: The experiments are only conducted on NATS-Bench, and only small-scale datasets are evaluated. This makes the generality and scalability of the proposed method unclear.\n\n2. Technical novelty: Based on the method description, the gradient update mechanism is primarily derived from CGGD (Van Baelen & Karsmakers, 2023). More clarification on what is newly introduced in applying this approach to NAS is needed to justify the novelty of the contribution.\n\n3. Lack of real hardware evaluation: The so-called hardware-aware metrics are not measured on actual hardware. FLOPs and the number of parameters are theoretical estimates, and the peak memory usage is computed by summing the input and output feature map sizes for each layer, without considering real deployment environments or memory behaviors on target devices.\n\n4. Typos: “seperable” → should be “separable”", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "DRQoZ0tA1z", "reviewer": "Reviewer_41Ma", "summary": "This paper proposes CONNAS, a gradient-based hardware-aware NAS method that enforces resource constraints through constraint-guided gradient modification rather than loss regularization. Instead of adding differentiable hardware penalties, CONNAS projects architecture gradients toward the feasible region, avoiding hyperparameter tuning of penalty weights.\n\nEvaluations on NATS-Bench (CIFAR-10/100, ImageNet16-120) and a real STM32 MCU deployment show that CONNAS consistently meets hardware limits while achieving accuracy close to or better than regularization-based methods (ProxylessNAS, FBNet, TAS). It is robust to hyperparameters and effective under tight constraints.\n\nOverall: An effective approach to enforce hardware constraints directly in differentiable NAS. The idea is empirically validated and practical for real deployment scenarios.", "questions": "See weaknesses.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 2, "strengths": "1. Clear and well-organized presentation. The paper is written in a clean, logical manner, making both the motivation and method easy to follow.\n\n2. Experimental results on multiple benchmarks and a real hardware deployment convincingly demonstrate the effectiveness and robustness of the proposed CONNAS framework.\n\n3. The idea of modifying gradients directly to satisfy hardware constraints, rather than learning or regularizing them, is unusual, providing a new methodological angle for hardware-aware NAS.", "weaknesses": "1. Outdated baselines and limited impact relative to a mature research field. Most compared baselines (ProxylessNAS, FBNet, TAS) are from 2019–2020, while recent NAS frameworks or one-shot supernet approaches are missing. Given that hardware-aware NAS is now a mature area, the lack of modern baselines (e.g., SPOS variants, DiffNAS, or once-for-all models) somewhat limits the significance and contemporary relevance of the work.\n\n2. Directly modifying the gradients, although rare, in principle seems less elegant than learning-based methods. The authors argue that learning-based approaches require tuning the weights of additional penalty terms, but the proposed gradient modification also introduces extra hyperparameters, such as the rescale factor that determines the strength of constraint enforcement. Therefore, to me, the learning-based methods appear more elegant and principled overall.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "J6JdAXGZt4", "reviewer": "Reviewer_z6HG", "summary": "The paper proposes CONNAS, a gradient-based NAS framework that finds the optimal feasible neural network architecture that satisfies a set of hardware constraints (number of parameters, FLOPS, and peak memory usage, etc.). The problem is defined as choosing the optimal subset of operations from a supernet by optimizing both the architecture weight and the operation weight. Unlike existing differentiable methods that include hardware regularization terms in the loss function, CONNAS directly modifies the gradient direction to favor the architecture that satisfies the hardware constraints. For each architecture weight, they find a normalized unit vector by aggregating all directionalities that point from candidates not satisfying the constraint, to the ones that do. Then, the vector is weighted and added to the gradient to push the update away from constraint-unsatisfying candidates, and towards the satisfying ones.  \nThe method is evaluated on two benchmark tasks, NATS-Bench-TS and NATS-Bench-SS, and against gradient-based NAS methods that use hardware-aware regularization terms. The authors show the searched architecture has better predictability whilst better conforming to the given hardware constraints.", "questions": "In general, please refer to weaknesses section. below are some relatively minor ones.\n1. It seems figure 2 is not explained anywhere. Does the red dashed line denote 100% accuracy? Or is it a handcrafted or manually designed baseline? \n2. Since Fig. 2 excludes infeasible runs, could the authors provide the fraction of runs that reach feasibility within the budget, as well as the epoch at which feasibility is first reached?\n3. Could the authors provide the overhead per epoch of computation dir_C relative to vanilla DARTS-like training, especially for large candidate sets per edge? Since there seems to be no analysis for the overhead on computation dir_C, it could make the paper stronger.\n4. In algorithm 1, the paper sums and normalizes directions from all violated constraints. What happens when there are conflicting constraints, such as a case where memory and FLOPs push different edges in opposite directions? Is there any regulation regarding multi-constraint interactions?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "**[S1]** Simple and modular mechanism for constraint enforcement.\nCONNAS steers the architecture update using a geometrically defined direction dir_{C} computed from local candidate substitutions instead of crafting differentiable hardware surrogates and tuning the weights. In addition, CONNAS demonstrates a clean adaptation of CGGD to NAS without performance degradation.\n\n**[S2]** Solid empirical evidence on multiple constraint types and search spaces\nOn NATS-Bench-TS/SS across CIFAR-10, CIFAR-100, and ImageNet16-120, CONNAS achieves a low relative error compared to the optimal feasible architectures while consistently satisfying constraints. In CSS-MEM, CIFAR-10 shows a 0.14% relative error, and strong results hold for PARAM and FLOPs as well, as shown in Table 2 and Figure 2. Furthermore, the paper reports an ablation on the rescale factor R, demonstrating robustness for moderate R in Appendix B.\n\n**[S3]** Practicality in real scenarios on edge-ML cases.\nBy presenting experiments on induction-motor fault diagnosis under explicit MCU-level (STM32 family) budgets and within a large 1.94B-architecture 1D-conv search space, the paper shows consistent outperformance over handcrafted baselines under comparable budgets. Providing a practical scenario further strengthens the paper’s motivation for edge-ML applications.\n\n**[S4]** Good writing\nThe overall presentation of the paper is good and easy to follow. The authors describe their approach in detail with well-defined terms.", "weaknesses": "**[W1]** Missing baselines and fairness should be stronger. \nThe paper does seem to discuss existing solutions. In my search, it appears there are several existing works on hard-constrained NAS, and some of them are very similar to this work, including Hardcore-NAS [1], HDX [2], or TF-NAS [3]. I believe the authors should clearly discuss the difference and additional novelty over these, preferably with experimental comparison.\n\n[1] Nayman, N. et al., \"Hardcore-nas: Hard constrained differentiable neural architecture search.\" ICML, 2021.\n\n[2] Hong, D. et al., “Enabling hard constraints in differentiable neural network and accelerator co-exploration”, DAC, 2022.\n\n[3] Hu, Y. et al., \"TF-NAS: Rethinking Three Search Freedoms of Latency-Constrained Differentiable Neural Architecture Search\", ECCV, 2020 \n\n**[W2]** The experiments lack comparison under varying difficulty of the constraints. The authors mention “The constraints are chosen such that about 50% of the architectures in each search space meet them”, which seems to be a rather easy bar to meet. In my opinion, methods like this would be much more valuable and convincing when the constraints are sufficiently rigorous that handcrafting or sampling methods fail to find a good candidate. Can the authors evaluate their method under different levels of constraint difficulty (e.g., 25%, 10%, or 5% of architectures satisfying the constraints) to better assess its robustness and effectiveness?\n\n**[W3]** Theoretical fallbacks when moving CGGD to architecture parameters should be discussed more. \nThe convergence statement in Sec3.3 relies on CGGD’s guarantee when R >1 and shortest-path directions yield convergence to the feasibility region. However, when mapping from \\alpha to a discrete architecture via Gumbel Softmax annealing and the local replace-one-edge construction of dir_C, the resulting dir_C could violate the assumptions underlying the original convergence guarantee. A formal theorem should strengthen the claim.\n\n**[W4]** Hardware-aware evaluation relies on analytical metrics not device measurements.\nFor NATS-Bench, hardware metrics are computed analytically, as shown in Appendix A, rather than measured on devices or by learned LUT regressors for a target platform. This condition reduces deployment realism and could introduce bias.\n\n**[W5]** There is a lack of explanation behind how the manually designed baseline architecture was found. When the authors say “best manually designed model”, how was this model obtained? Did it go through a systematic search process? Is it just based on expert intuition? Appendix F only lists the outcome of such handcrafting and does not explain the process. To make the evaluation results more convincing, the authors should detail this process clearly in the paper.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "W6O6y660SE", "reviewer": "Reviewer_Sicz", "summary": "This paper proposes to search for neural network architecture in a differentiable manner, guaranteeing hardware constraint compliance.\nThere are prior works that adopt hardware-related loss to add a hardware related objective.\nHowever, using hardware loss as a penalty doesn't guarantee compliance with the hardware constraint.\nInstead of using auxiliary loss for constraint, the paper proposes to modify the gradients of architecture parameters toward a direction for constraint compliance.\n\nThe novelty of the paper is computing the direction of the gradient that make the searched network comply with hardware constraint.\n\nFor a layer $l$, additional gradients of architecture parameters are achieved in the following manner.\n- If all candidate operations comply with the constraint, the additional gradients are zero.\n- If only some of them comply, split candidate operations into two sets according to the compliance.\nFor all combinations made by extracting one from both sets, a unit vector whose values are zero except for the indices of selected operations is initialized.\nThe absolute values of the indices are the same, while the sign of the value becomes positive if it complies with the constraint, negative if not.\nAnd then, all combinations of the unit vectors are summed, and the result vector is normalized.\n- If all candidate operations don't comply with the constraint, sort the operations according to the hardware metric and get all the combinations of two operations among them.\nThen, follow the steps in the above.\n\nWith the proposed gradient modification, the well-performed network architectures that comply with the given constraint can be searched.", "questions": "- Are the architecture parameters of all layers updated with the proposed gradient modification at once?\nOr, a subset of layers is selected to update their architecture parameters with a single batch?\n\n- The reviewer think that the proposal will be too complicated if there are multiple constraints.\nCan the authors execute additional experiments with multiple constraints, and report how much the experiment time increases  and whether the searched networks comply with constraints well?", "limitations": "", "rating": 6, "confidence": 5, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "- Instead of adopting an auxiliary loss to search models that comply with the given constraint, the paper proposes to modify the gradients of architecture parameters according to the relative relation between arbitrary candidate operations. This idea is strong and valid.\n\n- The performance of searched models is less affected by the hyperparameter that balances the gradient for performance and that for compliance. The hyperparameter doesn't have to be fine-tuned well, and it leads to reduce the search cost, too.\n\n- The paper develops what it contends well, and thus it is easy to follow. Also, the paper shows the effectiveness of the proposal well, with various experiments.", "weaknesses": "- The paper addresses many prior works in the NAS field, but it doesn't address hardware constrained NAS works [1, 2], which are directly related to this paper.\n\n- It is hard to understand Figure 1. Are the $G_{\\alpha_l}$ and $-\\nabla_{\\alpha_l} \\mathcal{L}(w, a)$ of each graph projecting the same vectors in different dimensions? If so, it is not clearly delivered.\n\n\n[1] Nayman, Niv, et al. \"Hardcore-nas: Hard constrained differentiable neural architecture search.\" International Conference on Machine Learning. PMLR, 2021.\n\n[2] Hong, Deokki, et al. \"Enabling hard constraints in differentiable neural network and accelerator co-exploration.\" Proceedings of the 59th ACM/IEEE Design Automation Conference. 2022.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)