TabNAS: Rejection Sampling for Neural Architecture Search on Tabular Datasets
Authors: Chengrun Yang
Gabriel Bender
Hanxiao Liu
Pieter-Jan Kindermans
Madeleine Udell
Yifeng Lu
Quoc V Le
Da Huang
Venue: NeurIPS 2022 Accept (2022)
Keywords: neural architecture search
tabular dataset
reinforcement learning
rejection sampling
Monte-Carlo sampling

Abstract: The best neural architecture for a given machine learning problem depends on many factors: not only the complexity and structure of the dataset, but also on resource constraints including latency, compute, energy consumption, etc. Neural architecture search (NAS) for tabular datasets is an important but under-explored problem. Previous NAS algorithms designed for image search spaces incorporate resource constraints directly into the reinforcement learning (RL) rewards. However, for NAS on tabular datasets, this protocol often discovers suboptimal architectures. This paper develops TabNAS, a new and more effective approach to handle resource constraints in tabular NAS using an RL controller motivated by the idea of rejection sampling. TabNAS immediately discards any architecture that violates the resource constraints without training or learning from that architecture. TabNAS uses a Monte-Carlo-based correction to the RL policy gradient update to account for this extra filtering step. Results on several tabular datasets demonstrate the superiority of TabNAS over previous reward-shaping methods: it finds better models that obey the constraints.

TabNAS: Rejection Sampling for Neural Architecture Search on Tabular Datasets
Chengrun Yang', Gabriel Bender', Hanxiao Liu', Pieter-Jan Kindermans', Madeleine Udell', Yifeng Lu', Quoc V. Le', Da Huang' {chengrun, gbender, hanxiaol, pikinder]@googlee com, udell@staa udell@stanford edu, {yifenglu, qvi, dahua)@google com
Google Research, Brain Team Stanford University
Abstract
The best neural architecture for a given machine learning problem depends many factors: not only the complexity and structure of the dataset, but also on resource constraints including latency, compute, energy consumption, etc. Neural architecture search (NAS) for tabular datasets is an important but under-explored problem. Previous NAS algorithms designed for image search spaces incorpo- rate resource constraints directly into the reinforcement learning (RL) rewards. However, for NAS on tabular datasets, this protocol often discovers suboptimal architectures. This paper develops TabNAS, new and more effective approach to handle resource constraints in tabular NAS using an RL controller motivated by the idea of rejection sampling. TabNAS immediately discards any architecture that violates the resource constraints without training or learning from that architecture. TabNAS uses a Conte-Carlo-basee correction to the RL policy gradient update to account for this extra filtering step. Results on several tabular datasets demonstrate the superiority of TabNAS over previous reward- shaping methods: it finds better models that obey the constraints.
Introduction
To make machine learning model better, one can scale it up. But larger networks are more expensive as measured by inference time, memory, energy, etc, and these costs limit the application of large models: training is slow and expensive, and inference is often too slow satisfy user requirements.
Many applications of machine learning in industry use tabular data, e.g., in finance, advertising and medicine. It was only recently that deep learning has achieved parity with classical tree-based models in these domains
[ 11 For vision, optimizing models for practical deployment often relies on Neural Architecture Search (NAS). Most NAS literature targets convolutional networks on vision benchmarks 14 10 19.. Despite the practical importance of tabular data, however, NAS research on this topic is quite limited X 7 (See Appendix A for more comprehensive literature review.)
Weight-sharing reduces the cost of NAS by training a SuperNet that is the superset of all candidate architectures [2]. This trained SuperNet is then used to estimate the quality of each candidate architecture or child network by allowing activations in only subset of the components of the SuperNet and evaluating the model. Reinforcement learning (RL) has shown to efficiently find the for vision problems.
most promising child networks [16 5
In our experiments, we show that a direct application of approaches designed for vision to tabular data often fails. For example, the TuNAS 3 approach from vision struggles to find the optimal architectures for tabular datasets (see experiments). The failure is caused by the interaction of the search space and the factorized RL controller. To understand why, consider the following toy example
36th Conference on Neural Information Processing Systems (NcurIPS 2022).

---

sampling probabilities of candkdate networks over time
Layer 1 size
~V
15
19
23
11/
3.15
1115
OPS
175
O%S
Layer 2ge
Abs
m
20
25
30
Reward
B ==-1
31%
+
25
31
37
Layer
Layer diee
Layer
Layer size
parameters of candidate networks
Layed JA0
33%
33%
11.5
186
0%
Cife
Abs Reward B = -2
Buijhhus Arooabiii
23R
Layer 1 size
2
0.9
0.9
0.66
m
0.9
0.66 0,3
Lage 30e
30%
5%
rejection w/
0.6
9.3
0.1
6%%
estimated F(V)
Layer size
losses of 9 candidate networks
Layar size
0
epochs
D
250
500
Figure 1: A toy example for tabular NAS in 2-layer search space with 2-dimensional input and limit of 25 parameters. Each cell represents an architecture. The left half shows the number of parameters and loss of each candidate in the search space. Infeasible architectures have red- red-striped cells. The bottom left cell (bold border) is the global optimum with size for the first hidden layer and size 2 for the second. The right half shows the change of sampling probabilities in NAS with different RL rewards. Sampling probabilities are shown both as percentages cells, and intensity indicated by the right colorbar. Orange bars on the top and right sides show the (independent) sampling probability distributions size candidates for individual layers. With the Abs Reward, the sampling probability of each architecture is the product of sampling probabilities of cach layer; with the rejection- based reward, the probability of an infeasible architecture is O, and probabilities normalized to sum to 1. At epoch 500, the cell squared in bold shows the architecture picked by the corresponding RL controller. RL with the Abs Reward Q(y) + (T)(T) 1 proposed in TuNAS B either converges to feasible but suboptimal architecture (B 22, middle row) or violates the resource constraint (3 = -1, top row). Other latency-aware reward functions show similar failures. In contrast, TabNAS converges the optimum (bottom row).
feasible architectures
with 2 layers, illustrated in Figure I For each layer, we can choose a layer size of 2, 3, or 4, and the maximum number of parameters is set to 25. The optimal solution is set the size of the first hidden layer to 4 and the second to 2. Finding this solution with RL is difficult with a cost penalty approach. The RL controller is initialized with uniform probabilities. As result, it is quite likely that the RL controller will initially be penalized heavily when choosing option for the first layer, since two thirds of the choices for the second layer will result in model that is too expensive. As result, option 4 for the first layer is quickly discarded by the RL controller and we get stuck in a local optimum.
This co-adaptation problem is caused by the fact that existing NAS methods for computer vision often factorized RL controllers, which force all choices to be be made independently. While factorized controllers can be optimized easily and are parameter--efficient,, they cannot capture all the nuances in the loss landscape. A solution to this could be to use more complex model such an LSTM However, LSTMs are often much slower to train and are far more difficult to tune.
(e.g., 16
Our proposed method, TabNAS, uses a solution inspired by rejection sampling. It updates the RL controller only when the sampled model satisfies the cost constraint. The RL controller then discouraged from sampling poor models within the cost constraint and encouraged to sample the high quality models. Rather than penalizing models that violate the constraints, the controller silently discards them. This trick allows the RL controller to the true constrained loss landscape, in which

---

0,454
0.447
log loss
0,450
32-144-24
loss
(efference archilecture) architectrres
0.446
0..446
parameters
0.445
0,442
2
6
8
10
0.0
0.5
1.0
1.5
2.0
relative stand training
le5
with rejection-based reward (num, parameters = 41,1533
parameters
TUNAS (hum, parameters random sampling (num,
(nnum_praameter parameters in [40000, 42000]:
skylne (rum, parppatrrterss (40000, 42000]:
Figure 3: Validation loss (logistic) vs. number of parameters Criteo with 3-layer search space. The standard deviation (std) of architecture per- formance for different runs is 0.0002, so archi- tectures whose performance difference is larger than 2std qualitatively different with high probability. The search space and Pareto-optimal architectures are shown in Appendix C.2.1.
TabNAS reward distributionally out- performs random search and resource-aware Abs Reward on the Criteo dataset within a 3-layer search space. All error bars and shaded regions are 95% confidence intervals. The x axis is the time relative to train time for single architec- ture. The axis is the validation loss. More details in Appendix C.2.3.
Figure 2:
having some large layers is beneficial, allowing TabNAS to efficiently find global (not just local) optima for tabular NAS problems. Our contributions can be summarized as follows:
We identify failure cases of existing resource-aware NAS methods on tabular data and provide evidence this failure is due the cost penalty in the reward together with the factorized space.
We propose and evaluate an alternative: a rejection sampling mechanism that ensures the RL controller only selects architectures that satisfy resource constraint. This extra rejection step allows the RL controller to explore parts of the search space that would otherwise be overlooked.
The rejection mechanism also introduces a systematic bias into the RL gradient updates, which can skew the results. To compensate for this bias, we introduce a theoretically motivated and empirically effective correction into the gradient updates. This correction can be computed exactly for small search spaces and efficiently approximated by Monte-Carlo sampling otherwise.
We show the resulting method, TabNAS, automatically learns whether bottleneck structure is needed in an optimal architecture, and if needed, where to place the bottleneck in the network.
These contributions form TabNAS, our RL-based weight-sharing NAS with rejection-based reward. TabNAS robustly and efficiently finds feasible architecture with optimal performance within the resource constraint. Figure shows an example.
2 Notation and terminology
Math basics. We define [n]] {1, "1,,""., n)} for positive integer With Boolean variable X, the indicator function 1 (X') equals
is true, and otherwise. |S] denotes the cardinality of a set S; stop_grad(( denotes the constant value (with gradient 0) corresponding to differentiable quantity f. and is equivalent to tensorflow. sttppo..sssspp.....o..oo..oo.... coop-grdieent (f) in TensorFlow [[ or f. r.detach () in PyTorch [15]. C and C denote subset and striet subset, respectively. V denotes the gradient with respect to the variable in the context.
Weight, architecture, and hyperparameter. We use weights to refer to the parameters of the neural network. The architecture of a neural network is the structure of how nodes are connected; examples of architectural choices are hidden layer sizes and activation types. Hyperparameters the -nnhtrchittttrall parameters that control the training process either stand-alone training or RL, including learning rate, optimizer type, optimizer parameters, etc.
Neural architecture. A neural network with specified architecture and hyperparameters called model. We only consider collecconde feedforward networks (FFNs) in this paper, since they can already achieve SOTA performance on tabular datasets [II]. The number of hidden nodes after each weight matrix and activation function is called hidden layer size. We denote a single network in our search space with hyphen-connecne choices. For example, when searching for hidden layer sizes, in the space of 3-hidden-laye layer ReLU networks, 32-144- 144-24 denotes the candidate where the sizes of

---

the first, second and third hidden layers are 32, 144 and 24, respectively. We only search for ReLU networks; for brevity, we will not mention the activation function type in the sequel.
Loss-resource tradeoff and reference architectures. In the hidden layer size search space, the validation loss in general decreases with the increase of the number of parameters, giving the loss- resource tradeoff' (e.g., Figure 3}. Here loss and number of parameters serve as two costs for NAS. Thus there are Pareto-optimal models that achieve the smallest loss among all models with a given bound on the number of parameters. With an architecture that outperforms others with a similar or fewer number of parameters, we do essource-constrange NAS with the number of parameters of this architecture as the resource target or constraint. We call this architecture the reference architecture (or reference) of NAS, and its performance the reference performance. We do NAS with the goal of matching (the size and performance of) the reference. Note that the RL controller only has knowledge of the number of parameters of the reference, and is not informed of its hidden layer sizes.
Search space. When searching L-layer networks, we use capital letters like X = X1--.-X,, to denote the random variable of sampled architectures, in which X; is the random variable for the size of the i-th layer. We use lowercase letters like x = I1- ... -,, to denote an architecture sampled from the distribution over X, in which z; is an instance of the i-th layer size. When there are multiple samples drawn, k-th sample. The search space 5 = (8t (EL]) has C, choices for the i-th hidden layer, in which 8ıj is the j-th choice for the size of the i-th hidden layer: for example, when searching for mee-hidden-layer network with size candidates (5, 10, 15), we have 813 = 15.
bracketed superscript to denote the index over samples: 2(4)) denotes the
Reinforcement learning. The RL algorithm learns the set of logits {le hef2),fecc) in which Lij is the logit associated with the j-th choice for the i-th hidden layer. With a fully factorized distribution of layer sizes (we learn separate distribution for each layer), the probability of sampling the j-th choice for the i-th layer Psj is given by the SoftMax function: P.j = exp(E,)) E exp(L1)) v.). In each RL step, we sample an architecture to compute the single-step RL objective J(v). and update the logits with Vy(()): an unbiased estimate of the gradient of the RL value function.
Resource metric and number of parameters. We use the number of parameters, which can be easily computed for neural networks, as a cost metric in this paper. However, our approach does not depend on the specific cost used, and can be easily adapted to other cost metrics.
3 Methodology
Our NAS methodology can be decomposed into three main components: weight-sharing with layer warmup, REINFORCE with one-shot search, and Monte Carlo (MC) sampling with rejection.
As an overview, our method starts with a SuperNet, which is a network that layer-wise has width equal to the largest choice within the search space. We first stochastically update the weights of the entire SuperNet to "warm up" over the first 25% of search epochs. Then we alternate between updating the shared model weights (which are used to estimate the quality of different child models) and the RL controller (which focuses the search on the most promising parts of the space). In each iteration, we first sample child network from the current layer- wise probability distributions and update the corresponding weights within the SuperNet (weight update). We then sample another child network to update the layerwise logits that give the probability distributions (RL. update). The latter RL update is only performed if the sampled network is feasible, in which case we use rejection with MC sampling to update the logits with a sampling probability conditional on the feasible set.
To avoid overfitting, we split the labelled portion of a dataset into training and validation splits. Weight updates are carried out on the training split; RL updates are performed on the validation split.
3.1 Weight sharing with layer warmup
The weight-sharing approach has shown success on various computer vision tasks and NAS bench- marks [16 2 3 To search for an FFN on tabular datasets, we build a SuperNet where the size of each hidden layer is the largest value in the search space. Figure 4 shows an example. When we sample a child network with a hidden layer size & smaller than the SuperNet, we only use the first & hidden nodes in that layer to compute the output in the forward pass and the gradients in the

---

backward pass. Similarly, in RL updates. only the weights of the child network are used to estimate the quality reward that is used to update logits.
In weight-sharing NAS, warmup helps to ensure that the SuperNet weights are sufficiently trained to properly guide the RL updates [] With probability p. we train all weights of the SuperNet, and with probability - p we only train the weights of random child model. When we run architecture searches for FFNs, we do warmup in the first 25% epochs, during which the probability p linearly decays from 1 to 0 (Figure (aa). The RL controller is disabled during this period.
3.2 One-shot training and REINFORCE
We do NAS on FENs with a REINFORCE- algorithm. Previous works have used this type of algorithm to search for convolutional networks on vision tasks [8] 5 3]. When searching for L-layer FFNs, we learn a separate probability distribution over C, size candidates for each layer. The distribution is given by C. logits via the SoftMax function. Each layer has its own independent set of logits. With C, choices for the ith layer, where = 1,2,.... ... L, there are ILe C2 candidate networks in the search space but only EiE(L) C₁ logits to learn. This technique significantly reduces the difficulty of RL and make the NAS problem practically tractable [ B
The REINFORCE- based algorithm trains the SuperNet weights and learns the logits {hy [E()))e that give the sampling probabilities (e Je(c,) over size candidates by alternating between weight and RL updates. In each iteration, we first sample a child network æ from the SuperNet and compute its training loss in the forward pass. Then we update the weights in x with gradients of the training loss computed in the backward pass. This weight update step trains the weights of
The weights in architectures with larger sampling probabilities are sampled and thus trained more often. We then update the logits for the RL controller by sampling a child network that is independent of the network from the same layerwise distributions, computing the quality reward Qvy) as 1 - loss(y) on the validation set, and then updating the logits with the gradient of Q) log P(y): the product of the advantage of y's reward over past rewards (usually an exponential moving average) and the log- bog-rrobability of the current sample.
[vy = stop (ro__gra(((())
The alternation creates positive feedback loop that trains the weights and updates the logits of the burge-prbabbility child networks; thus the layer-wise sampling probabilities gradually converge to more deterministic distributions, under which one several architectures are finally selected.
Details of a essource-oblivoou version is shown as Appendix Algorithm 1, which does not take into account resource constraint. In Section 3.3] we show an algorithm that combines Monte-Carlo sampling with rejection sampling, which serves as a subroutine of Algorithm by replacing the probability in J(y) with a conditional version.
3.3 Rejection-basee reward with MC sampling
Only a subset of the architectures in the search space S will satisfy resource constraints; V denotes this of feasible architectures. To find a feasible architecture, a resource target To is often used in an RL reward. Given an architecture y, a resource-aware reward combines its quality Qgy) and resource consumption T(y) into a single reward. MnasNet [18] proposes the rewards (uv) (T))/To) and Q(y) max (T(y)/Tu))) 33 while TuNAS [3] proposes the absolute value reward (or Abs Reward) Q(v) + (T) resource target. these rewards /3 is hyperparameter that needs careful tuning.
1/. The idea behind is to encourage models with high quality with respect the
We find that on tabular data, RI. controllers using these resource-aware rewards above can struggle to discover high quality structures. Figure I
shows toy example in the search space in Figure 4 in which we know the validation losses of each child network and only train the RL controller for 500 steps. The optimal network 4-2 among architectures with number of parameters no more than 25, but the RL. controller rarely chooses it. In Section 4.1 we show examples on real datasets.
This phenomenon reveals gap between the true distribution want to sample from and the distributions obtained by sampling from this factorized search space:
We only want to sample from the set of feasible architectures V. whose distribution is (P(y) yy E V)lev. The resources (e.g., number of parameters) used by an architecture, and thus its fcasibility, is determined jointly by the sizes of all layers.
5

---

representaaion
warmup prob
1.0
laa prob
estimaated (N-2048)
0.5
output
1st
2nd
0.0
15 30 45 60 epochs
30 60 90 120 epochs
hidden layers
(a) Warmup probability
(b) Valid probability
Figure 4: Illustration of weight-sharing on two- layer FFNs for a binary classification task. Edges denote weights; arrows the end of lines denote ReLU activations; circles denote hidden nodes; the square in the output layer denotes the output logit. The size of each hidden layer can be one of (2, 3, 4), thus the SuperNet is two-layer FFN with size 444. At this moment, the controller picks the child network 3-2, thus only the first 3 hidden nodes in the first hidden layer and the first 2 hidden nodes in the second hidden layer, together with the connected edges (in red), are enabled compute the output logits.
Figure 5: Examples of layer warmup and valid probabilities. Figure (a) shows our schedule: lin- early decay from 1 to 0 in the first 25% epochs. Figure (b) shows an example of the change of true and estimated valid probabilities (P(V) and P(V)) in successful search, with 8,000 archi- tectures in the search space and the number of MC samples N 1024. Both probabilities are (nearly) constant during warmup before RL starts, then increase after RL starts because of rejection sampling.
On the other hand, the factorized search space learns separate (independent) probability distri- bution for the choices of each layer. While this distribution efficient to learn, independence between layers discourages an RL controller with resource-awee aware reward from choosing bottleneck structure. A bottleneck requires the controller select large sizes for some layers and small for others. But decisions for different layers are made independently, and both very large and very small layer sizes, considered independently, have poor expected rewards: small layers are estimated to perform poorly, while large layers easily exceed the resource constraints.
To bridge the gap and efficiently learn layerwise distributions that take into account the architecture feasibility, we propose a rejection- based RL reward for Algorithm 1. We next sketch the idea; detailed pseudocode provided as Algorithm 2 in Appendix B.
REINFORCE optimizes a set of logits {fe (e( which define a probability distribution p over architectures. In the original algorithm, we sample random architecture y from p and then estimate its quality (1)). Updates to the logits lij take the form lij +- Egj + 7/ Jt,, (y). where 1) is the learning rate, Q is a moving average of recent rewards, and Jyy) stop_ grad (v) - Q) log P(y). If is better (worse) than average, then (zy) - Q will be positive (negative), so the REINFORCE update will increase (decrease) the probability of sampling the same architecture in the future.
In our new REINFORCE variant, motivated by rejection sampling, we do not update the logits when y is infeasible. When y is feasible, we replace the probability P(r) in the REINFORCE update equation with the conditional probability P2 y E V) = Pwv)/((y E V). So J(v) becomes
(1) We can compute the probability of sampling feasible architecture P(V) P(y E V) exactly when the search space is small, but this computation is too expensive when the space is large. Instead, replace the exact probability (ιy) with differential approximation P'(y) obtained with Monte-Carlo
J(y) stop_ (Q()__rr(((()) Q) log (P(r)/P(x VV)
(MC) sampling. In each RL step, we sample N architectures {2(K)) )RE[N) within the search space with proposal distribution g and estimate P(V) as
(k) ((2(k) EV).
P(V)
Σ N
(2)
N
gill
For each & E [N], ((8) is the probability of sampling (k) with the factorized layerwise distributions and so is differentiable with respect to the logits. In contrast, (**) is the probability of sampling (k) with the proposal distribution, and therefore non-differentiable.
((V) is an unbiased and consistent estimate of P(V); log(P(y)/P(V)) is consistent estimate of v log[P(( y E V)] (Appendix J). A larger N gives better results (Appendix H); in experiments,

---

hidden layer BEN 1
probability
hidden layer 2
probability
hidden layer
probability
0.450
log loss
15
45
15 30 45 60
15 30 45 60
0
15 30 45 60
32-64-96 (41.245 params)
# epochs
# epochs
# epochs
32-144-24 (41,153 params)
(a) Layer 1 (finally 32)
Layer (finally 64)
(c) Layer (finally 96)
(d) retrain performance
Figure 6: Failure of the Abs Reward on Criteo in search space of 3-layer FFNs. The change of sampling probabilities and comparison of retrain performance between the 32-144-24 reference and the 32-64-96 architecture found with the (τy) ( /To 1| Abs Reward, the target for the reward was 1131 parameters. Repeated runs the same search find the same architecture. Figure ((d) shows the retrain validation losses of 32-64-96 (NAS-found) and 32- 12-144-24 (reference).
need smaller than the size of the sample space to get faithful estimate (Figure 5(b) Appendix D and I) because neighboring RL steps can correct the estimates of each other. We set & stop_ge (oop_grad(() p) in experiments for convenience: use the current distribution over architectures for MC sampling, Other distributions that have a larger support on V may used to reduce sampling variance (Appendix J).
At the end of NAS, we pick as our final architecture the layer sizes with largest sampling probabilities if the layerwise distributions are deterministic, sample from the distributions times and pick feasible architectures with the largest number of parameters if not. Appendix B Algorithm 3 provides the full details. We find m 500 and n S 3 suffice to find an architecture that matches the reference (optimal) architecture in our experiments.
In practice, the distributions often (almost) converge after twice the number of epochs used train stand-alon, child network. Indeed the distributions are often useful after training the same number of epochs in that the architectures found by Algorithm are competitive. Figure [igure[[shows shows TabNAS finds the best feasible architecture, 4-2, in our toy example, using P(V) estimated by MC sampling.
Experimental results
Our implementation can be found at hbttps://github com/googgee-rssercch//ooooo all experiments using TensorFlow on Cloud TPU v2 with 8 cores. We use a 1,027-dimensional input representation for the Criteo dataset and 180 features for Volker[]] The best architectures our FFN search spaces already produce near-state-of-the-eee results; details in Appendix C.2. More details of experiment setup and results in other search spaces can found in Appendix and Appendix tabulates the performance of all RL rewards all tabular datasets in experiments. Appendix F shows a comparison with Bayesian optimization and evolutionary search in similar settings; Ablation studies in Appendix show TabNAS components collectively deliver desirable results; Appendix H shows TabNAS has easy-to-tune hyperparameters.
We ran
4.1 When do previous RL rewards fail?
Section 3.3 discussed the resource-aware RL rewards and highlighted potential failure case. In this section, we show several failure cases of three resource-aware rewards, ((v)(Ty))T (T)) Q²)) max (T))))T))², and the Abs Reward Q(v) + +TTT)) 1/, on our tabular datasets.
4.1.1 Criteo -3 layer search space
We use the 32-144-24 reference architecture (41,153 parameters). Figure gives an overview of the costs and losses of all architectures in the search space. The search space requires to choose of 20 possible sizes for each hidden layer; details in Appendix D. The search has 1.7x x the cost of a stand-alone training
OOr paper takes these features given. It is worth noting that methods proposed in feature engineering complementary and can work together with TabNAS.
works like [12] and [3
7

---

Failure of latency rewards. Figure shows the sampling probabilities from the search when using the Abs Reward, and the retrain validation losses of the found architecture 32-64-96. In Figures [(a) [[c] the sampling probabilities for the different choices are uniform during warmup and then converge quickly. The final selected model (32-64-96) is much worse than the reference model (32-144-24) even though the reference model is actually less expensive. We also observed similar failures for the MnasNet rewards. With the MnasNet rewards, the RL controller also struggles to find model within 15% of the constraint despite a grid search of the RL parameters (details in Appendix C). In both cases, almost all found models are worse than the reference architecture.
The RL controller is to blame. To verify that low quality SuperNet was not the culprit, we trained Su- perNet without updating the RL controller, and man- ually inspected the quality of the resulting SuperNet. The sampling probabilities for the RL controller re- mained uniform throughout the search; the rest of the training setup was kept the same. At the end of the training, we compare two sets of losses on each of the child networks: the validation loss from the SuperNet (one-shot loss), and the validation loss from training the child network from scratch. Figure ((a) shows that there is a strong correlation between these accuracies; Figure 7(b) shows RL that starts from the sufficiently trained SuperNet weights in 7(a) still chooses the sub- optimal choice 64. This suggests that the suboptimal search results on Criteo are likely due to issues with the RL controller, rather than issues with the one-shot model weights. In a layer search space we can actu- ally find good models without the RL controller, but in 5 layer search space, we found an RL controller whose training is interleaved with the SuperNet is im- portant to achieve good results.
one-shot loss
0.45
hidden layer ma
0.44
0.4 443 0.448 0.453 stand-alone loss
015
60 epochs
100
Figure 7: Left: 3-layer Criteo SuperNet cali- bration after 60 epochs (search space in Ap- pendix C): Pearson correlation is 0.96. The one-shot loss is validation loss of each child network with weights taken from Super- Net trained with the same hyperparameters as in Figure but with no RL in the first 60 epochs; the stand-alone loss of each child network is computed by training the same ar- chitecture with the same hyperparameters from scratch, and has std 0.0003. Right: change in probabilities in layer 2 after 60 epochs of SuperNet training and 40 of RL. Note the rapid changes due to RL.
4.1.2 Volkert - 4 layer search space
balanced logs
0.35
balanced less
0.35
0.33
0.33
We search for 4-layer and 9-layer net- works on the Volkert dataset; details in Ap- pendix D. For resource-aware RL rewards, we ran a grid search over the RL learning rate and 3 hyperparameter. The reference architecture for the 4 layer search space is 48- 10--1100----22 with 27,882 parameters. Despite a hyperparameter grid search, it was difficult to find models with the right target cost reliably using the MnasNet re- wards. Using the Abs Reward (Figure (8). searched models met the target cost but their quality was suboptimal, and the trend is similar to what has been shown in the toy example (Figure [ architecture that is beyond the reference number of parameters, and a larger [3] gives an architecture that is feasible but suboptimal.
0.31
30
60
90
120
0.31
30
90
120
epochs
# epochs
60-64-32-144 9.44 params)
96-48-32-96 (27.738 params)
(found by Abs
by Reward)
48 160116032210 32-144 (27,882 params) (reference)
48.160. 32. 144 (27,882 params) (referance)
(a) B = - 10, RL learning (b) 8 = 25, RL learning rate 1j = 0.001
rate 0.001
Figure 8: Abs Reward misses the global optimum on Volkert. Figure shows the retrain validation losses of two architectures found by the Abs Reward vs. the 48- 160-32- 144 reference.
a smaller [3] gives an infeasible
4.1.3 A common failure pattern
Apart from Section 4.1.1 and 4.1 more examples in search spaces of deeper FFNs can be found in Appendix D. In cases on Criteo and Volkert where where the RL controller with soft constraints cannot match the quality of the reference architectures, the reference architecture often has a bottleneck structure. For example, with a 1.027-dimensional input representation, the 32- 144-24 reference on Criteo has bottleneck 32; with 180 features, the 48-1 160-32-144 reference on Volkert has bottleneck 48 and 32, As the example in Section (.33 the wide hidden layers around the bottlenecks get penalized harder in the search, and it is thus more difficult for RL with the Abs Reward to find
8

---

hidden layer 2 www.pcchiiiieee
hidden layer
pro probabll
probabllg
hidden layer
valid prob
pro
(V-30721
true
30 60 120
120 # epochs
30 60 90 120 0
30 60 90 120 # epochs
# epochs
# epochs
Layer (finally 32)
(b) Layer 2 (finally 144)
(c) Layer 3 (finally 24)
(d) valid probabilities
Figure 9: Success case: Criteo in search space of 3-layer FFNs, Monte-Carlo sampling with rejection eventually finds 32-144 144-24, the reference architecture, with RL learning rate 0.005 and
number of MC samples 3,072. Figure 9(d) shows the change of true and estimated valid probabilities.
model that can match the reference performance. Also, Appendix C.2.1 shows the Pareto-optimal architectures in the tradeoff points in Figure often have bottleneck structures, so resource-awald RL rewards previous NAS practice may have more room for improvement than previously believed.
4.2 NAS with TabNAS reward
With proper hyperparameters (Appendix H), our RL controller with TabNAS reward finds the global optimum when RL with resource-aware rewards produces suboptimal results.
TabNAS does not introduce resource-aware bias in the RL reward (Section 3.3 Instead, it uses conditional probabilities to update the logits in feasible architectures. We run TabNAS for 120 epochs with RI. learning rate 0.005 and N = 3072 MC samples F The RI. controller converges to two architectures, 32-160-16 (40,769 769 parameters, with loss 0.4457 0.0002) and 32-144-24 (41,153 parameters, with loss 0.4455 + 0.0003), after around 50 epochs of NAS, then oscillates between these two solutions (Figure D. After 120-epochs, we sample from the layerwise distribution and pick the largest feasible architecture: the global optimum 32- 12-144.24
On the same hardware, the search takes 3x the runtime of stand-alone training. Hence, can be seen in Figure? 2 the proposed architecture search method is much faster than random baseline.
4.3 TabNAS automatically determines whether bottlenecks are needed
Previous NAS works like MnasNet and TuNAS (often or only on vision tasks) often have inverted bottleneck blocks [17] in their search spaces. However, the search spaces used there have hard- coded requirement that certain layers must have bottlenecks. In contrast, our search spaces permit the controller to automatically determine whether to use bottleneck structures based on the task under consideration. TabNAS automatically finds high-quality architectures, both cases where bottlenecks needed and in cases where they not. This is important because networks with bottlenecks do not always outperform others on all tasks. For example, the reference architecture 32-144-24 24 outperforms the MANAS-found 32-64 32-64-96 on Critco, but the reference 64-192-48 (64.568 parameters, 0.0662 + LaA----nn-TaaaNSSSS
0.0011) is on par with the 0.0013) on Aloi. TabNAS automatically finds optimal (bottleneck) architecture for Criteo, and automatically finds an optimal architecture that does not necessarily have a bottleneck structure for Aloi. Previous swarddshaping rewards like the Abs Reward only succeed in the latter case.
96-80-96-32 (64,024 parameters, 0.0669 +
4.4 Rejection- based reward outperforms Abs Reward in NATS-Bench size search space
Although we target essource-constraing NAS on tabular datasets in this paper, our proposed method not specific NAS on tabular datasets. In Appendix G, we show the rejection based reward in TabNAS outperforms RL with the Abs Reward in the size search space of NATS-Bende 6 NAS benchmark on vision tasks.
"The 3-layer search space has 20 8000 candidate architectures, which is small enough to compute P(V) exactly. However, MC can scale to larger spaces which are prohibitively expensive for exhaustive search (Appendix D).

---

5 Conclusion
We investigate the failure of reseurce-aware RL rewards to discover optimal structures in tabular NAS and propose TabNAS for tabular NAS in a constrained search space. The TabNAS controller uses rejection mechanism to compute the policy gradient updates from feasible architectures only, and uses Monte-Carlo sampling to reduce the cost of debiasing this rejection- rejection-sampling sampling approach. Experiments show TabNAS finds better architectures than previously proposed RL methods with resource-aware rewards in resource-constrained searches.
Many questions remain open. For example: 1) Can the TabNAS strategy find better architectures on other types of tasks such as vision and language? 2) Can TabNAS improve RI. results for more complex architectures? 3) Is TabNAS useful for ressurce-constrained RL problems more broadly?
Acknowledgments and Disclosure of Funding
This work was done when Madeleine Udell was a visiting researcher at Google. The authors thank Ruoxi Wang, Mike Van Ness, Ziteng Sun, Xuanyi Dong, Lijun Ding, Yanqi Zhou, Chen Liang, Zachary Frangella, Yi Su, and Ed H. Chi for helpful discussions, and thank several anonymous reviewers for useful comments.
References
[1] Martín Abadi, Ashish Agarwal, Paul Barham, Eugene Brevdo, Zhifeng Chen, Craig Citro, Greg S. Corrado, Andy Davis, Jeffree Dean, Matthieu Devin, Sanjay Ghemawat, Ian Goodfellow, Andrew Harp. Geoffrey Irving, Michael Isard, Yangqing Jía, Rafal Jozefowicz, Lukasz Kaiser, Manjunath Kudlur, Josh Levenberg, Dandelion Mané, Rajat Monga. Sherry Moore, Derek Murray, Chris Olah, Mike Schuster, Jonathon Shlens. Benoit Steiner, Ilya Sutskever, Kunal Talwar, Paul Tucker, Vincent Vanhoucke, Vijay Vasudevan, Fernanda Viégas, Oriol Vinyals, Pete Warden, Martin Wattenberg. Martin Wicke, Yuan Yu, and Xiaoqiang Zheng. TensorFlow: Large- scale machine learning on heterogeneous systems, 2015. URI.https: //www. tensorflor org/
Software available from tensorflow.org.
[2] Gabriel Bender, Pieter-Jan Kindermans, Barret Zoph, Vijay Vasudevan, and Quoe Le. Under- standing and simplifying one-shot architecture search. In International Conference on Machine Learning, pages 550-559. PMLR, 2018.
[3] Gabriel Bender, Hanxiao Liu, Bo Chen, Grace Chu, Shuyang Cheng, Pieter-Jan Kindermans, and Quoe V Le. Can weight sharing outperform random architecture search? An investigation with TuNAS. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14323-14332 2020.
[4] Han Cai, Tianyao Chen, Weinan Zhang, Yong Yu, and Jun Wang. Efficient architecture search by network transformation. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 32, 2018
[5] Han Cai, Ligeng Zhu, and Song Han. Proxylessnas: Direct neural architecture search target task and hardware. arXiv preprint arXiv: 000320022.0.,... 2018.
[6] Xuanyi Dong, Lu Liu, Katarzyna Musial, and Bogdan Gabrys. Nats-bench: Benchmarking nas algorithms for architecture topology and size. IEEE transactions on pattern analysis and machine intelligence, 2021.
[7] Romain Egele, Prasanna Balaprakash, Isabelle Guyon, Venkatram Vishwanath, Fangfang Xia, Rick Stevens, and Zhengying Liu. Agebo-tabular; joint neural architecture and hyperparameter search with autotuned data-paralld training for tabular data. In Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis, pages 1-14, 2021.
[8] Nick Erickson, Jonas Mueller, Alexander Shirkov, Hang Zhang, Pedro Larroy, Mu Li, and Alexander Smola. Autogluon-tabulan Robust and accurate automl for structured data, arXiv preprint 2005::2003306505, 2020.
10

---

[9] Yury Gorishniy, Ivan Rubachev, Valentin Khrulkov, and Artem Babenko. Revisiting deep learning models for tabular data. Advances Neural Information Processing Systems, 34, 2021.
[10] Andrew Howard, Mark Sandler, Grace Chu, Liang-Chied Chen, Bo Chen, Mingxing Tan, Weijun Wang, Yukun Zhu, Ruoming Pang, Vijay Vasudevan, al. Searching for mobilenetv3. In Proceedings of the JEEF/CVF International Conference on Computer Vision, pages 1314-1324, 2019.
[11] Arlind Kadra, Marius Lindauer, Frank Hutter, and Josif Grabocka. Well-tuned simple nets excel tabular datasets. In Thirty-Fifth Conference Neural Information Processing Systems, 2021.
[12] Farhan Khawar, Xu Hang, Ruiming Tang, Bin Liu, Zhenguo Li, and Xiugiang He. Autofeature: Searching for feature interactions and their architectures for click-through rate prediction. In Proceedings of the 29th ACM International Conference on Information & Knowledge Management, pages 625 634,,,44220 2020.
[13] Bin Liu, Chenxu Zhu, Guilin Li, Weinan Zhang, Jincai Lai, Ruiming Tang, Xiuqiang He, Zhenguo Li, and Yong Yu. Autofis: Automatic feature interaction selection in factorization models for click-through rate prediction. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pages 2636-2645, 2020.
[14] Hanxiao Liu, Karen Simonyan, and Yiming Yang. Darts: Differentiable architecture search. arXiv preprint MXXiv:8806009053
2018. 2018
[15] Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. Pytorch: An imperative style, high- serhoperffrmecce deep learning library. Advances in neural information processing systems, 32:8026-8037, 2019.
[16] Hieu Pham, Melody Guan, Barret Zoph, Quoe Le, and Jeff Dean. Efficient neural architecture search via parameters sharing. In International Conference on Machine Learning, pages 4095-4104. PMLR, 2018.
[17] Mark Sandler, Andrew Howard, Menglong Zhu, Andrey Zhmoginov, and Liang- Chen. Mobilenetv2: Inverted residuals and linear bottlenecks. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 4510-4520, 2018
[18] Mingxing Tan, Bo Chen, Ruoming Pang, Vijay Vasudevan, Mark Sandler, Andrew Howard, and Quoe V Le. Mnasnet: Platform-awe aware neural architecture search for mobile. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2820-2828, 2019.
[19] Chris Ying, Aaron Klein, Eric Christiansen, Esteban Real, Kevin Murphy, and Frank Hutter. Nas-bench-101: Towards reproducible neural architecture search. In International Conference on Machine Learning, pages 7105-7114. PMLR, 2019.
Checklist
1. For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's
contributions and scope? [Yes]
(b) Did you describe the limitations of your work? [Yes] See Appendix H: the number of MC samples N needs be large enough.
(c) Did you discuss any potential negative societal impacts of your work? [N/A]
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yss]
2. If you including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results? [Yes] See Appendix J.

---

(b) Did you include complete proofs of all theoretical results? [Yes] See Appendix J.
3. If you ran experiments..
(a) Did you include the code, data, and instructions needed to reproduce the main experi- mental results (either in the supplemental material or as URL)? [Yes] The datasets are publicly available. We also provide pseudocode and full details of our hyperparameters to reproduce results in Table Al and A2.
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)? [Yes] See Appendix C.2.
(c) Did you report error bars (e.g., with respect to the random seed after running experi- ments multiple times)? [Yes]
(d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, cloud provider)? [Yes] See Figure A3 and A5 in the appendix, and the beginning of Section 4 in the main paper.
4. If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators? [Yes] See Appendix C.2 for the links to datasets and the citation of the OpenML dataset repository.
(b) Did you mention the license of the assets? [No] But we include links to every asset we used in the footnotes of the Appendix, and we followed the dictates their licenses.
(c) Did you include any new assets either in the supplemental material or as a URL? [No]
We only used public open-source datasets for experiments.
(d) Did you discuss whether and how consent was obtained from people whose data you're using/curating? [N/A]
(e) Did you discuss whether the data you using/curating contains personally identifiable information or offensive content? [N/A]
5. If you used crowdsoureing or conducted research with human subjects...
(a) Did you include the full text of instructions given participants and screenshots, if applicable? [N/A]
(b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, applicable? [N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount spent participant compensation? [N/A]
12


Reviews:
[{"review_id": "k8Ug6T7C66J", "reviewer": "Reviewer_guLP", "summary": "This paper proposes a multi-objective NAS algorithm using RL-controller. They use monte-carlo based rejection sampling to guide the controller towards feasible search space. \nThe search space is divided into possible categorical options for each layer, such as [16, 22, 42] etc for layer1. The controller predicts logits corresponding to each possible value for each layer in the fully connected network (FCNN). So logits_{ij} refers to the value for $j^{th}$ value in the $i^{th}$ layer and corresponds to the probability that it would be sampled. As the choice of each layer is independent of the other, to generate a network adhering to the constraints, they use rejection based reward mechanism.", "questions": "1) (a) Please tabulate the accuracies, number of parameters and time taken of networks obtained from MnasNet, TuNAS, TabNAS and for all the datasets. \n\n(b) MoNas [1] is also an RL based multi-objective algorithm which yields a reward only if the constraint < threshold.  Resource-Efficient Neural Architect[2] uses a different penalty as listed in eqn 3 of their paper. Please compare how these two objectives fare against yours.\n\n(c) While you are focusing only on RL based multi-objective algorithms, it would be good to compare with evolutionary based algos, namely Lemonade and NSGA-NET[3] .\n\n (d) It would also be good to compare it against TabNet. Based on the number of parameters of the best model found by Tabnet, you can modify your search to fit that parameter search space. \n\n2) While rejection sampling is helping us focus the search on only the feasible area, the number of samples required to get a good estimate of the probability distribution seem to be very high. So it might not be scalable to larger search spaces. For ex,  5 * $10^{6}$ samples for 9-layer search space for Volkert dataset is computationally expensive and cannot be used in real world. \n\n\n[1] MONAS: Multi-Objective Neural Architecture Search, Hsu et al.\n[2] Resource-Efficient Neural Architect, Zhou et al.\n[3] NSGA-Net: Neural Architecture Search using Multi-Objective Genetic Algorithm, Lu et al.", "limitations": "The monte carlo sampling is very expensive and this search won't scale for tasks requiring large search spaces. Please see Questions section for further details.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "They showed that their technique is able to find well performing networks.\n\nHowever, they did not compare adequately against all the baselines. Tabulating the results rather than presenting them in a paragraph is preferred. Please see Questions for more details.\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "bip1NVTjM7Z", "reviewer": "Reviewer_w3th", "summary": "This paper proposes a neural architecture search algorithm designed for tabular datasets. Prior work uses an RL-based optimization procedure with penalties for feasibility and resource constraints, but the authors show that this does not work well. To address this issue, the authors add an additional rejection sampling phase to filter out those that do not satisfy constraints, and show that this allows the RL method to better explore the search space of architectures. Evaluation is done on the Criteo and Volkert datasets, where the found architectures consistently achieve lower losses.", "questions": "I think the results could be summarized in a more concise and clear way (e.g. a table) - in the paper the only results are shown in 2 small graphs where it's difficult to tell what the absolute numbers are and which baseline is being run.", "limitations": "The authors did not discuss any potential negative societal impacts of their work. ", "rating": 7, "confidence": 2, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n- I believe that the use of rejection sampling in NAS algorithms is novel. However, I am not familiar with neural architecture search so my knowledge of related work is relatively small.\n- The work is significant, in that it unlocks more NAS applications in a new domain area.\n- In general, the writing is clear and well-organized - the authors clearly present their method and reasoning, and I appreciate that the authors experimented with multiple baselines and multiple datasets and found consistent results between them.\n\nWeaknesses:\n- The experimental results are presented on a domain the baseline methods were not developed for, so it is more difficult to put the results in context. It would be informative to see how well the proposed methods work well on image-based domains, even if it does not perform as well.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "J2ecS2H9NsS", "reviewer": "Reviewer_7VoP", "summary": "TabNAS proposes a modification to Reinforce which instead of modifying the reward function to incorporate resource constraints, estimates the feasible set of architectures (e.g. architectures less than some threshold/reference) and updates the probabilities of the controller accordingly by redistributing the probability mass over the feasible set as more trial-and-error progresses. ", "questions": "1. I am quite confused by the contribution of this paper. The way I read it, the authors show that reward function modification as has been commonly done before is not as good as rejection sampling as proposed in this work in coming up with architectures that better meet resource constraints. I buy this and note that most Pareto-frontier aware algorithms which output the *entire* Pareto-frontier employ rejection sampling to constrain search to only those parts of the search space that the user wants (e.g. give me the Pareto-frontier estimate less than < 100 milliseconds inference latency, or between 200-300 milliseconds inference latency) See https://arxiv.org/abs/2105.01015. But there seems to be nothing specific in the method to tabular datasets. I can see applying this method as-is to any task (e.g. vision amd NLP). Is my understanding correct? If so, then it perhaps happened that the authors ran on tabular datasets to get the ball rolling? Note that one can use any of the excellent NAS benchmarks without using GPUs or large compute to validate this method. My suggestion will be NAS-Bench-Suite which has several different benchmarks under a single interface. https://arxiv.org/abs/2201.13396. \n\n2. My second concern/comment is perhaps a bit unfair to this work: It is a bit baffling to me that a stateful RL algorithm like Reinforce has been widely applied (and continues to be used in the broader AutoML community) to a problem setting where one doesn't need to do long horizon credit assignment. NAS is inherently stateless, one immediately knows the result of their action (sampled architecture or in HPO the sampled hyperparameter). So 1-step RL like contextual bandits or Bayesian optimization will be just as efficient and probably more. Can such rejection constraints be added to BOHB, HyperBand, BANANAS, NASBO etc? If so how will they be different from those in https://arxiv.org/abs/2105.01015? \n\n 3. Related to point 2 above: in the appendix lines 251-254, states that there are open questions on BO and ES for one-shot NAS. But is that necessary or one could train the supernet using the style of training in Once-For-All https://arxiv.org/abs/1908.09791 or use all the insights for training supernets as detailed in https://openreview.net/forum?id=Esd7tGH3Spl and then doing search via any search technique? Note OFA uses evolutionary search, the earlier ENAS paper used Reinforce https://arxiv.org/abs/1802.03268 and indeed the actual search technique varies across papers. In this respect it appears to me that the aim of the experiments in the appendix comparing across RL, BO and ES since BO and ES are being run on the feasible set only. So, a comparison (on tabular task) seems orthogonal to the aim of the paper which is to show a modification of specifically Reinforce to take into account resource constraints better than others who modify the reward function?\n", "limitations": "Yes. ", "rating": 5, "confidence": 5, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths: Paper is clear and well-written. Literature has been adequately cited and discussed. \n\nWeaknesses: Please see my comments in the questions section.\n\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}]

Author rebuttal:

Decision: Accept