Fast MNAS: Uncertainty-aware Neural Architecture Search with Lifelong Learning
Authors: Jihao Liu
Yangting Sun
Ming Zhang
Boxiao Liu
Yu Liu
Venue:  (2021)
Keywords: Neural Architecture Search
AutoML
Reinforcement Learning (RL)

Abstract: Sampling-based neural architecture search (NAS) always guarantees better convergence yet suffers from huge computational resources compared with gradient-based approaches, due to the rollout bottleneck -- exhaustive training for each sampled generation on proxy tasks. This work provides a general pipeline to accelerate the convergence of the rollout process as well as the RL learning process in sampling-based NAS. It is motivated by the interesting observation that both the architecture and the parameter knowledge can be transferred between different experiments and even different tasks. We first introduce an uncertainty-aware critic (value function) in PPO to utilize the architecture knowledge in previous experiments, which stabilizes the training process and reduces the searching time by 4 times. Further, a life-long knowledge pool together with a block similarity function is proposed to utilize the lifelong parameter knowledge and reduces the searching time by 2 times. It is the first to introduce block-level weight sharing in RL-based NAS. The block similarity function guarantees a 100% hitting ratio with strict fairness.  Besides, we show a simply designed off-policy correction factor that enables 'replay buffer' in RL optimization and further reduces half of the searching time. Experiments on the MNAS search space show the proposed FNAS accelerates standard RL-based NAS process by $\sim$10x (e.g. $\sim$256 2x2 TPUv2*days / 20,000 GPU*hour $\rightarrow$ 2,000 GPU*hour for MNAS), and guarantees better performance on various vision tasks.

Under review as a conference paper at ICLR 2021
FAST MNAS: UNCERTAINTY-AWARE NEURAL ARCHI- TECTURE SEARCH WITH LIFELONG LEARNING
Anonymous authors Paper under double-bling review
ABSTRACT
Sampling-bast based neural architecture search (NAS) always guarantees better conver- gence yet suffers from huge computational resources compared with gradient-base based approaches, due to the rollout bottleneck exhaustive training for each sampled generation on proxy tasks. This work provides a general pipeline accelerate the convergence of the rollout process as well as the RL learning process in sampling- based NAS. It motivated by the interesting observation that both the architecture and the parameter knowledge can be transferred between different experiments and even different tasks. We first introduce an uncertainty-awer in PPO to utilize the architecture knowledge in previous experiments, which sta- bilizes the training process and reduces the searching time by times. Further, a life-long knowledge pool together with block similarity function is proposed utilize lifelong parameter knowledge and reduces the searching time by times. It is the first to introduce block-level weight sharing in RL-based NAS. The block similarity function guarantees a 100% hitting ratio with strict faimesss. Besides, we show simply designed off off-policy correction factor that enables "replay buffer" in RL optimization and further reduces half of the searching time. Experiments on the MNAS search space show the proposed FNAS accelerates standard RL-based NAS process by ~~00 (e.g. ~55 2x2 TPUv2 dayss /aas// / 20,000 GPU + 2,000 GPU for MNAS), and guarantees better performance on various vision tasks.
critic (value function)
INTRODUCTION
Neural architecture search (NAS) has made great progress in different tasks such image classifica- tion Tan & Le 2019) and object detection (Tan et al. 2019b). And usually, there are four commonly used NAS algorithms: differentiable, one-shot, evolutional, and reinforcement learning (RL) based method. The Ra-based method, due to its fair sampling and training processes, has often achieved a great performance among different tasks. However, demand for computing resources, which makes it hard to follow by the research community.
of the biggest challenges of it is the high
RL-based NAS consumes large number of computing powers on aspects: a) the need for sampling a large number of architectures to optimize the RL. agent and b) the tedious training and testing process of these samples on proxy tasks. For example, the originator of NAS (Zoph & Le 2016) requires 12,800 generations of architecture and current state-of-the-art MNAS (Tan et al. 2019a) and MobileNet-V3 (Howard al. 2019) require 8000 more generations find the optimal architectures. Besides, each generation is usually trained for 5 epochs. All in all, it costs nearly 64 TPUv2 devices for 96 hours or 20,000 GPU hours on V100 for just one single searching process. With such a severe drawback, researchers start looking for other options like differential (L.iu et al. 2018h Chen et al. 2019), or one-shot based (Bender 2019 Guo et 2019) method for NAS.
The one-shot family has drawn lots of attention recently due to its efficiency. It applies single super-network based search space with that all the architectures, also called sub-networks, share parameters with the super-network during the training process. In this way, the training process is condensed from training thousands of sub-networks into training a super-network. However, this thare-weigh strategy may bring problems for the performance estimation of sub-networks. For example, two sub-netue networks may propagate conflicting gradients to their shared components, and the shared components may converge to favor one of the sub-networks and repel the other randomly. This

---

Under review as a conference paper at ICLR 2021
FNAS
0.250
0.225
0.203
0.175
0.150
0.075
2000
5030 generation
4050
8030
10303
2000 4300 10000
6000 1900 generation
2000
3000
6203 15030 6000
8500 10003
2000
5003
gen.eration
generaaion
Figure I: Reward along sample generation between FNAS and MNAS, Blue dots are the searching result of MNAS, while red dots are the results of FNAS.
Lifelong Knowledge Pool
Uncertainty-Awar Critic
initialize
RL Agent
sample
Uncertainty Network
uncertainty
threshald
Real Sample
Trainer
prediction
Fake Sample
Network
train
rewurd
reward
update
Batch
Architecture Experience Buffer
Data
sample
Figure 2: The pipeline FNAS. The proposed modules are highlighted orange. Architectures are sampled by the RL agent and then passed to Uncertainty-Awer Critic (UAC) for predicted performance and the corresponding uncertainty. Then a decide module will determine whether the sample needs to be trained by Trainer. The Lifelong Knowledge Pool (LKP) helps to initialize new samples for training. Half of the samples one batch come from Architecture Experience Buffer (AEB), the other half come from Trainer or UAC's Value Network.
conflicting phenomenon may result in instability of the search process and inferior final architectures, compared with RL-based methods.
In this work, seek combine the privilege of RL-based methods and one-shot methods, by leveraging the knowledge from previous NAS experiments. The proposed method is based on two key observations: First, the optimal architectures for different tasks have common architecture knowledge. Second, the parameter knowledge can also be transferred across experiments and even tasks.
Based on the observations, for transferable architecture knowledge, we develop -neertainty-Aware Critic (UAC) to learn the centicetttrr--fffrmnncee joint distribution from other experiments even other tasks in an unbiased manner, utilizing the transferability of the structural knowledge, which reduces the sample's 's training time by 50% and the result is shown in Figure (with UAC); For the transferable parameter knowledge, we propose Lifelong Knowledge Pool (LKP) to restore the block-level parameters and fairly share them to new samples' initialization, which speeds up each samples' convergence for 2 times, as shown in Figure 1 (with LKP); Finally, also developed an Architecture Experience Buffer (AEB) with a significant off off-policy correctness factor to store the old models for reusing in RL optimization, with half of the time saved. And this is shown in Figure 1 (with AEB). Under the strictly same environment as MNAS and MobileNet-v3, FNAS speed up the searching process by 10x and the performances are even better.
2

---

Under review as a conference paper at ICLR 2021
RL agent
Global Knowledge Pool
Good architectures imageket
Recogration
Load
Load
Good architectures by RL agent
Search ImageNet
Search
RL search space
Face Recognition
Figure 3: On the left, searching for neural architectures on different tasks leads to different optimal architectures. On the right, different tasks share the same global knowledge pool.
2 REVISITING SAMPLING-BASED NAS
2.1 NAS FAMILY
From the perspective of how to derive the performance estimation an architecture, NAS methods can be split into two categories, sumpling-based and share-weight based.
Sampling-basee methods usually sample many architectures from the search space and train them independently. Based the performance of these well-trainee architectures, several ways can be utilized to fetch the best one, such as Bayesian optimization (Kandasamy et al., 2018). evolutionary algorithm (Real et al. 2019), and training an RL agent (Zoph & Le| (016). The main drawback of these methods is a huge time and resource consumption of training the sampled architectures. To alleviate this issue, a common practice
shorten the training epochs and use proxy networks with fewer filters and cells (Zoph et al.) 2018 Tan et al., 2019a). Liu et al. (Liu et al.) (018a) propose train network predict the final performance. Different from these methods, we leverage the accumulated knowledge to accelerate the training process.
Instead of training many architectures independently. the second kind of methods resorts to train super-networt and estimate the performance of architectures with weights shared from the super- network (Bender 2019 Wu et al.) 2019: Liu et al.] 2018b Chen et al. 2019 Xu al.) 2019 Cai et al. SoT1ooStamouiis et al.) 2019 Guo et al.) 2019). With the easy access of performance estimation, DARTS (Liu et al. 201 proposes a gradient- based method to search for the best architecture in an end-to-end manner. However, as pointed in (Li & Talwalkar] 2019). the performance estimation based on shared weights may be unreliable. Chen et al. (Chen et al. 2019) proposes to progressively shrink the search space that the estimation can be more and more accurate. Cai et al. (Cai et al.) 2019) introduces a shrinking based method to train the supernet so as to generate networks different scales without retraining. We also share weights between architectures but in different ways. We construct a general weight pool with many trained architectures, and when we want to train new architecture, we initialize it by the trained architectures in the pool. this way, the number of training epochs can be reduced without harming reliability.
3 KNOWLEDGE BETWEEN NAS EXPERIMENTS IS TRANSFERABLE
RL-based NAS consumes lot of computing resources. MNAS, as it's said before, trains 8,000 models for the agent to converge, which costs 20,000 GPU hours on V100. And all the samples trained for one experiment will not be used anymore. However, the active differentiable- based NAS demonstrates that with various weight-sharing techniques, the NAS algorithms can accelerated a lot. In this section, we will show that the knowledge of previous searched experiments be reused by the following two observations, which helps the follow-up experiment greatly.
3.1 ARCHITECTURE KNOWLEDGE CAN BE TRANSFERRED
Optimal architectures for different tasks have common architecture knowledge. One always holds the assumption that the performance of model is consistent among different tasks. A common
3

---

Under review as a conference paper at ICLR 2021
magerigl
exclusive Space
space
Face
Figure 4: Expectation of each operator of optimal models of face experiment and ImageNet exper- iment. Calculated by the 100 optimal models of face experiments and ImageNet experiments and sorted by the significance of the difference.
practice of this assumption is applying good ImageNet (Deng et al. 2009) models to COCO object detection (Lin et al. 2014) as the backbone. In NAS, however, this assumption needs to be carefully checked as the huge search space of requires the hypothesis to be well generalized. Here, We statistically verify this assumption.
In Figure 4 we sample 100 optimal models on face recognition and ImageNet classification tasks respectively. For each model, we firstly expand each digit of its embedding (i.e. 35 dimension vector) to one-hot: e.g. from "4" to "1000"; "3" to "0100". In this way, the original embedding is expanded to 112-imm from 35-dim. After that, we calculate the expectation for each digit of this expended vector among the top 100 optimal architectures on the face recognition and ImageNet classification respectively. Shown in Figure we get observation that the operators be divided into two spaces, i.c. exclusive space, where the probability difference is large and common space, where the probability difference small.
Many previous works (Liu et al. 201 Sa Kokiopoulou et al., 2019 Luo et al. 2018 Wen et al. 2019 Luo al. 2020) use a predictor predict a model's performance to speed up the NAS process. However, as the predictor requires thousands of samples to train, they usually implement progressive (Liu et al., 20T or semi-supervised manner (Luo et al., 2020). Inspired by the interesting observation above, we implement it in unified way where different tasks' samples are used together to train a unified value network to predict model's performance. When searching architecture on new task, just use directly the unified network trained by the old data and keep updating in the new task during the search process, which speeds up the convergence of the value network. Showing in Figure Sa when transferring value network trained on ImageNet to face recognition task, the network converges much faster.
3.2 PARAMETER KNOWLEDGE CAN BE TRANSFERRED
Initializing the network by ImageNet pre-trained models and training the model on other tasks has nearly been standard way it can always speed up the convergence process. However, pretraining has been ignored in the NAS area as it may break the rank of different models. In our experiments, we observe that the trained checkpoint, we call it parameter knowledge, can help faster than training from scratch. Besides, this feature holds regardless of the data distribution. We randomly sample 50 models and train them on ImageNet in two ways: from scratch or by initializing with parameter knowledge from face experiment. Then, we compare the rank correlation with real rank (i.e. fully trained rank) along the training process. Showing in Figure 5b and 5c with parameter knowledge from face experiment, one gets more accurate rank in fewer epochs.
get the real rank
4 UNCERTAINTY-AWARE NAS WITH LIFELONG LEARNING
In this section, we introduce how we utilize the observations above design three modules to inherit common structural knowledge and parameter knowledge from task to task without bias.
4.1 UNCERTAINTY-AWARI CRITIC IN PPO
The value function is a common module and is widely used in RL algorithms like PPO but rarely used in traditional NAS. Usually, training value function requires large number of samples to
4

---

Under review as a conference paper at ICLR 2021
1.00
init rundoraly
intt with Bacn tank
Imwgereettoe
6.31
0.65
6.85
6.71
0.00
truining LXP
6.43
truining with
Forch
Beration
Trainioneppnc
(a)
(b)
(c)
Figure 5: On the left, the value function pretrained on face recognition task converges much faster. On the right, Spearman rank-order correlation Zar 2005 along the training process of random initialization and block-level initialization.
converge (e-g. million-level steps in Atari trained by ray which is unbearable for NAS it means thousands of models needed to be trained and it's expensive. our algorithm, alternatively, we propose the Uncertainty-Awer Critic deal with this issue which inspired by Section 3
Given an architecture A, sampled from search space, a value network V is utilized to predict the reward V (A.) of this sample, while R(A;) is the real reward of it. And the loss function update V is
Ly (V(A) - R(A)
(1)
Besides, an uncertainty network is utilized to predict the uncertainty (AA)) of this sample, which used to learn discriminately whether sample is in the distribution of learned samples. In our implementation, the loss function to update U
Lu = (AA) Lvl
(2)
If (A;) is greater than a threshold, the sample may locate in the region which has not been learned by the value network. As a result, it will be trained from scratch to get its reward. Otherwise, one thinks the prediction V (A.) is accurate, and V (AA) will be regarded as the reward of A, to update the RL agent. The whole process is illustrated in Figure
Samples need to be trained from scratch are defined as real sample, while others are defined as fake sample if its reward comes from V. With more real samples getting their rewards in our searched tasks, the value network becomes more accurate, thus the uncertainty value gets from U will always decrease. Considering an extreme case where each sample in batch gets reward with low uncertainty and is classified as fake samples, the agent updated by these samples is easy to be over-fitted, which is not conducive the exploration of the RI. agent and it would lead bad performance. In our implementation, we use the following constraints to balance the exploration and the exploitation of the RL agent speed up its convergence without over-fitting.
Constraint Sample whose uncertainty higher than the threshold needs be trained from scratch.
Constraint2: In each batch, when the number of fake samples is greater than 50% of the batch size, the extra fake samples will be thrown away and the agent will resample until enough real samples are gotten to fill the rest of the batch.
With these two constraints, find that the algorithm can maintain decent performance in an accelerated search process.
4.2 UNCERTAINTY-AWART KERNEL POOL.
Parameter knowledge can be transferred among different tasks to speed up the convergence of network as shown in Section 3.2 However, traditional pretraining is not feasible in NAS, as there
5

---

Under review as a conference paper at ICLR 2021
thousands of different architectures in the search space and we can not afford to pretrain each architecture on different task. To address this problem, we propose to initialize each architecture in factorized way and use fuzzy matching algorithm guarantee the hit ratio. First, define an architecture as a combination of n blocks {br,
J. For any two architectures A, and Aj- although generally, their structures may be quite different, some of their blocks may be similar to each other, (eg. 62 of A, == b2 of Aj), thus the weights of these parts could be shared. So we build Kernel Pool to store all the previously trained models block by block key-value manner, where the key is the expand embedding of each block and the value is the Parameter knowledge of the block.
Recent research has found that fairness in weight sharing has great impact on the final perfor- mance (Chu et al. 2019). So we apply the following two strategies to solve the problem of faimess.
The checkpoints stored in the LKP are trained with equal iterations.
For each block query, the proposed uncertainty function to ensure that the match ratio reaches more than 99%, which means less than 1% blocks have been unfairly initialized.
Given query block b;, we calculate the cosine similarity of the expanded embedding as Section 3.1 between b; and each element in LKP. The block with the highest similarity will be recalled to initialize by. We show the overall process in Figure 3 Using LKP can speed up the search process by 2x. At the same time, in Section we found that parameter knowledge is transferable. We apply the same LKP to different tasks and all the search processes are accelerated, illustrated in Figure 5b
5 ARCHITECTURE EXPERIENCE BUFFER
In general RL. task, there are lot of discussions about sample reuses. However, in RL-bbase NAS, sample efficiency is rarely mentioned. In algorithm, we propose architecture experience buffer to store the sampled models in the form of sechitecture-eerformee
pairs, and for each iteration in the future, the stored samples may be used again update the RL agent speed up its convergence. We call the samples stored in the experience buffer as old samples and the newly generated samples as new samples. Different from the traditional RL works, the proposed experience buffer has the following features:
The buffer size is relatively small (usually 10 in our experiments). As the convergence of the RL agent is much faster than RL tasks, the buffer size is set too large, the agent will focus on old samples and the convergence speed would be slow.
In each batch, both old samples and new samples will be selected. To prevent the RL updating from biasing to the old samples, the percentage of the old samples in one batch is constrained to more than 50%.
Some recent works (Schaul et al. 2013) suggested that the samples in the buffer should have different priorities. We do it similarly, defining different priorities in terms of their reward. Then, we sample from the buffer and reweight those samples with their priorities. For each sample {81, 82, 8n)
exp(r)) with their reward R 71, P2... rn} in AEB, the priority score is defined as: P. Σ, exp(rs)"
Following (Schaul et al. 2015), each sample will be reweighted by importance sampling weight. The reweighted score S can be written in: S; = (N * P.)- where N is buffer size and / is the annealing term and it will increase from 0 as the experiment proceeds.
FNAS ON VISION TASKS
In this section, we conduct different experiments on both ImageNet and million level face recognition tasks to verify the effectiveness of FNAS. The details and results are as follows:
6.1 IMPLEMENTATION DETAILS
Following the standard searching algorithm as NASNet (Zoph et al. 2018), MNAS (Tan et al. 2019a) and AKD (Liu al. 2019), we an RNN- agent optimized by PPO algorithm (Schulman

---

Under review as a conference paper at ICLR 2021
Table Performance Results on ImageNet Classification. FNAS-Imagex 1.3 means scale up FNAS- Image for 1.3x along width.
Seurch Cost (GPU Hours)
models
Type
FL.PPs
Topl Acc. (%)
Top5 Acc.
MBv2
Mannal
300M
72
ProsylessNAS DARTS
320M
74.6
92.2
200
Sharh-weight
96
FairNAS
388M
75.3
92.4
288
Once For-All
327M
75.3
92.6
1200
AmoebaNet
Evolationary
55SM
74.5
92
75,600
MNAS NASNet
315M
75.2
92.5
20,000
564M
74
91.6
43,200
MBv3
219M
75.2
RL-based
EfficienNeeRO FNAS-Image
390M
76.3
93.2
225M
75.5
2000
SNAS-Imagex1.3
392M
77.2
93.5
2000
Table 2: Performance on MegaFace.
Table 3: Performance on COCO.
model
FLOPs
Distractor num
Search Cost (GPU Hours)
models
FLOPs
mAP
le5
le6
MNAS
6.697 G
27.68
MBv2
300M
92.75
88.71
MBv2 1.0
6.675 G
29.79
ShuffleNet
295M
94.15
90.46
MBv3
5.852 G
29.25
MNAS
313M
93.41
89.47
20,000
MBv3x110
9.060
30.02
MBv3
218M
94.15
90.64
ENAS-Face
227M
95.45
92.63
2000
FNAS
8.021 G
30.44
et al. 2017). For ImageNet experiments, sample 50K images from the training set to form the mini-va val set and use the rest as the mini-training set. In each experiment, 8K models are sampled to update the RL agent. Note that when equipped with UAC AEB, not all samples need to be activated, as many samples' rewards are directly returned from these two modules. For face experiments, use MSIM (Guo al. 2016) as the mini-training set, LFW (Huang et al. 20008) as the mini-val set. The final performance is evaluated on MegaFace (Keneliaacher-SShizemmaa et al. 2016
6.2 PROXYLESS FNAS ON IMAGENET
Just MNAS has done, also multi-objective reward directly search ImageNet. After the search process, we retrain the top 10 models with the largest reward near the target flops from scratch verify the search results. Table I we get relatively higher result than the current SOTA network MBv3 (Howard et al.) 2019). Note that the model we search does not through the pruning operation NetAdapt Yang et al., 2018), which can reduce 10 10%~115 % computation and keep performance nearly unchanged. Compared with EfficientNetBO, FNAS improves top I accuracy by point under comparable computation budget. And still, there nearly 10x of acceleration in the entire search process compared to MNAS (Tan et al. 2019a ct al. 2019).
MBv3 Howard
6.3 PROXYLESS FNAS ON FINE-GRAINED FACIAL RECOGNITION
Besides verifying the performance FNAS on ImageNet, we also test it on the fine-grained facial recognition task. As can be seen in Table 2 compared with MBv3, verification accuracy improves points comparable FLOPs under 1e6 distractors. When compared with MBv2, FNAS improves verification accuracy for nearly 4 points with 24% FLOPs reduction. The result shows: 1) FNAS has an obvious acceleration effect on different tasks and 2) the importance of searching directly on the target task.
7

---

Under review as a conference paper at ICLR 2021
Table The effectiveness of the three proposed modules, MBv2x0.38 means scale up MBv2 for 0.38 along width
Activated Samples
Search Epoches
Search Cost (GPU Hours)
Models
LKP
UAC
AEB
MFLOPs
Top1 Ace. (%)
MBv2
81
62.65
72
64.23
8,000
4,000
MNAS
74
65.19
10,000
20,000
75
64.97
8,000
4,000
76
65.22
8,000
10,000
FNAS
64.28
2,300
1,150
2,250
66.25
2,000
1,000
Table 5: Transferability of UAC and LKP
Activated Samples
Scarch Epoches
Scarch Cost (GPU Hours)
models
UAC or LKP
MFLOPs
Topl Acc. (%)
MNAS
181
73.25
8,000
4
16,000
FNAS
UAC init with face exp
153
73.91
2,000
4
4,000
MNAS
285
74.62
8,000
4
16,000
FNAS
LKP init with face exp
292
75.22
4,000
6.4
TRANNFERABILIITY ON OBJECT DETECTION
We combine the model found on ImageNet in Table with the latest pipeline detection to verify its generalization. Table 3 shows the performance of the model on COCO (Lin et al. (2014). It can be seen that compared MBv3, there significant improvement with our searched model.
ABLATION STUDY
7.1 THE EFFECTIVENESS OF THE THREE PROPOSED MODULES.
In this section, the effectiveness of UAC, LKP, AEB is verified when they used alone combined. Details are shown Table Three conclusions can be observed: Sampling with LKP initialization gets real rank faster; 2. Fewer samples required when NAS is equipped with UAC and AEB; and 10x speedup can achieved when NAS is equipped with LKP, UAC, and AEB.
7.2 THE TRANSFERABILITY OF THE PROPOSED MODULES.
In Section 3 mentioned that knowledge between NAS experiments is transferable, which is also verified in the experiment. We use the UAC trained on the face pre-trained model and then transfer it the ImageNet experiment. In the absence of 3/4 of activated samples, the optimal model surpasses baseline by 0.67% with fewer FLOPs, showing in Table 5 In addition, we use LKP with the checkpoints from face experiments and then search on ImageNet. In the absence of 1/2 activated samples, performance increases by 0.6%.
CONCLUSION
This paper proposes three modules (UAC, LKP, AEB) to speed the entire running process of RL-based NAS, which consumes large amounts computing power before. With these modules, fewer samples and less training computing resources needed, making the overall search process 10x faster. We also show the effectiveness of applying those modules on different tasks such as ImageNet, face recognition, and object detection. More importantly, the transferability of UAC and LKP being tested by our observation and experiments, which will guide us in tapping the knowledge of the NAS process.
8

---

Under review as a conference paper at ICLR 2021
ACKNOWLEDGMENTS
REFERENCES
Gabriel Bender. Understanding and simplifying one-shot architecture search. 2019.
Han Cai, Chuang Gan, and Song Han. Once for all: Train one network and specialize for efficient deployment. arXiv preprint arXiv:/908 097911 2019.
Xin Chen, Lingxi Xie, Jun Wu, and Qi Tian. Progressive differentiable architecture search: Bridging the depth gap between search and evaluation. In Proceedings of the IEEE International Conference on Computer Vision, pp. 1294-1303, 2019.
Xiangxiang Chu, Bo Zhang, Ruijun Xu, and Jixiang Li. Fairnas: Rethinking evaluation faimesss of weight sharing neural architecture search. arXiv preprint arXiv: 2019.
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference computer vision and pattern recognition, pp. 248-2552 Icee, 2009.
Yandong Guo, Lei Zhang, Yuxiao Hu, Xiaodong He, and Jianfeng Gao. Ms-celeb- 1m: A dataset and benchmark for large-scale face recognition. In European conference on computer vision, pp- 87-102. Springer, 2016.
Zichao Guo, Xiangyu Zhang, Haoyuan Mu, Wen Heng, Zechun Liu, Yichen Wei, and Jian Sun. Single path one-shot neural architecture search with uniform sampling. arXiv preprint 104.0020,,,00420, 2019.
Andrew Howard, Mark Sandler, Grace Chu, Liang-Chieh Chen, Bo Chen, Mingxing Tan, Weijun Wang, Yukun Zhu, Ruoming Pang, Vijay Vasudevan, al. Searching for mobilenetv3. In Proceedings of the IEEE International Conference on Computer Vision, pp. 1314-1324, 2019.
Gary B Huang, Marwan Mattar, Tamara Berg, and Eric Leamed- Labeled faces in the wild: A database forstudying face recognition in unconstrained environments. 2008.
Kirthevasan Kandasamy, Willie Neiswanger, Jeff Schneider, Barnabas Poczos, and Eric P Xing. Neural architecture search with bayesian optimisation and optimal transport. Advances in Neural Information Processing Systems, pp. 2016-2025, 2018.
Ira Saatimaahhhh-SShiemmmaa Steven M Seitz, Daniel Miller, and Evan Brossard. The megaface benchmark: 1 million faces for recognition at scale. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp.
2016.
Ef Kokiopoulou, Anja Hauth, Luciano Sbaiz, Andrea Gesmundo, Gabor Bartok, and Jesse Berent. Fast task-aware architecture inference. arXiv preprint arXiv:1902.05781, 2019.
Liam Li and Ameet Talwalkar. Random search and reproducibility for neural architecture search. arXiv preprint arXiv: /902 2019.
Tsung-Y Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In European conference on computer vision, pp. 740-755. Springer, 2014.
Chenxi Liu, Barret Zoph, Maxim Neumann, Jonathon Shlens, Wei Hua, Li-Jia Li, Li Fei-Fei. Alan Yuille, Jonathan Huang, and Kevin Murphy. Progressive neural architecture search. In Proceedings of the European Conference on Computer Vision (ECCV), pp. 19-34, 201 8a.
Hanxiao Liu, Karen Simonyan, and Yiming Yang. Darts: Differentiable architecture search. arXiv preprint arXiv:1806.0900 2018b.
Yu Liu, Xuhui Jia, Mingxing Tan, Raviteja Vemulapalli, Yukun Zhu, Bradley Green, and Xiaogang Wang. Search to distill: Pearls are everywhere but not the eyes. arXiv preprint 1xXiv:191.00077. 2019.

---

Under review as a conference paper at ICLR 2021
Renqian Luo, Fei Tian, Tao Qin, Enhong Chen, and Tie- Yan Liu. Neural architecture optimization. In Advances in neural information processing systems, pp. 7816-7827, 2018.
Renqian Luo, Xu Tan, Rui Wang, Tao Qin, Enhong Chen, and Tie- Liu. Semi-supervised neural architecture search. arXiv preprint arXiv:2002. 2020.
Esteban Real, Alok Aggarwal, Yanping Huang, and Quoe Le. Regularized evolution for image classifier architecture search. In Proceedings of the aaai conference on artificial intelligence, volume 33, pp. 4780-4789, 2019.
Tom Schaul, John Quan, Ioannis Antonoglou, and David Silver. Prioritized experience replay. arXiv preprint arXiv:/5/1.00990 2015.
John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347, 2017.
Dimitrios Stamoulis, Ruizhou Ding, Di Wang, Dimitrios Lymberopoulos, Bodhi Priyantha, Jie Liu, and Diana Marculescu. Single-path nas: Designing hardware-efficient convnets in less than hours. arXiv preprint arXiv: 1904.02877 2019.
Mingxing Tan and Quoc V Le. Efficientnet: Rethinking model scaling for convolutional neural networks. arXiv preprint a9Xiv: 11946, 2019.
Mingxing Tan, Bo Chen, Ruoming Pang, Vijay Vasudevan, Mark Sandler, Andrew Howard, and Quoc Le. Mnasnet: Platform-aware neural architecture search for mobile. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2820-2828, 2019a.
Mingxing Tan, Ruoming Pang, and Quoc V Le. Efficientdet: Scalable and efficient object detection. arXiv preprint arXiv: 0091111970000 2019b.
Wei Wen, Hanxiao Liu, Hai Li, Yiran Chen, Gabriel Bender, and Picter-Jan Kindermans. Neural predictor for neural architecture search. arXiv preprint arXiv: 19/2:0992200043 2019.
Bichen Wu, Xiaoliang Dai, Peizhao Zhang, Yanghan Wang, Fei Sun, Yiming Wu, Yuandong Tian, Peter Vajda, Yangging Jia, and Kurt Keutzer. Fbnet: Hardware-aware efficient convnet design via differentiable neural architecture search. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 10734-10742, 2019.
Yuhui Xu, Lingxi Xie, Xiaopeng Zhang, Xin Chen, Guo-Jun Qi, Qi Tian, and Hongkai Xiong. Pc-darts: Partial channel connections for memory-efficient architecture search. In International Conference Learning Representations, 2019.
Tien-Ju Yang, Andrew Howard, Bo Chen, Xiao Zhang, Alec Go, Mark Sandler, Vivienne Sze, and Hartwig Adam. Netadapt: Platform-aware neural network adaptation for mobile applications. In Proceedings of the European Conference on Computer Vision (ECCV), pp. 285-300, 2018.
Jerrold H Zar. Spearman rank correlation. Encyclopedia of Biostatistics, 7, 2005.
Barret Zoph and Quoc V Le. Neural architecture search with reinforcement learning. arXiv preprint arXiv:1611.0157,, 2016.
Barret Zoph, Vijay Vasudevan, Jonathon Shlens, and Quoe V Le. Learning transferable architectures for scalable image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 8697-8710, 2018.
A APPENDIX
You may include other additional sections here.
10


Reviews:
[{"review_id": "vav5GvMpHu", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Summary\n-------\n\nThis paper proposes a fast general framework (FNAS) for neural architecture search (NAS) problem to enhance the processing efficiency up to 10x times. Three interesting strategies (UAC, LKP, AEB) for reinforcement learning (RL) processing are introduced in the proposed FNAS and evaluated by extensive experiments to show their efficacy. In particular, the assumption that architecture knowledge is transferable has been verified by real observation.\n\nHowever, the authors paid more attention to introduce the fact based on observations and the thoughts of the framework design, thus neglected the technical depth for the key component (UAC) that has highest impact on the overall performance.\n   \n\nStrengths\n---------\n\n- The paper is well-organized and well-written thus easy to understand, including motivation, approach, and experiments.\n\n- The proposed framework (FNAS) is general, practical and convincing. The three strategies (UAC, LKP, AEB) for RL processing are based on real observations shown in Figure 4, which positively supports the motivation of the approach.\n\n- The evaluations are conducted on extensive experiments with solid results including ablation studies for the three different components LKP, UAC, and AEB (although it might be not sufficient; see next the weaknesses).\n\n\nWeaknesses\n----------\n\n- The technical depth was neglected. For example, the  Uncertainty-Aware Critic (UAC) should be considered as the key component of FNAS framework because it was shown that the UAC has highest impact on (biggest contribution to) the overall performance in terms of efficiency in Table 4. However, there is no any technical/mathematical introductions about how the uncertainty network $U$ is obtained/prepared, and how the $U$ contribute to the NAS in detail.\n\n- Discussions regarding possible over-fitting are not sufficient. For example, the constraint (threshold $\\delta$) of uncertainty is introduced into the UAC strategy (Sec. 4.1), but without any experiment results to show the impact of such hyperparameters in the FNAS framework that has to be considered as trade-off parameter against over-fitting effects.\n\n- Similarly, in the AEB strategy (Sec. 5), the buffer size ($N$), and the annealing term ($\\beta$) are hyperparameters that should have impact on the over-fitting effects during RL processing. However, the authors did not provide any testing results to confirm their impacts. For example, why did the authors determine the buffer size $N=10$ in the experiments?\n\n- The testing of FNAS on vision tasks are not sufficient. This paper provided results on classification (ImageNet) and face recognition tasks, but how about other tasks such as object detection, tracking, person re-identification, and segmentation?\n\n\nOther Questionable Points\n-------------------------\n\n- The loss functions shown in Eq. (1) and Eq. (2) seem too simple. Are they really sufficient to get high performance? Is there any potential loss functions or improvements that would get better performance?\n\n- In Table 1, why the numbers of \"GPU Hours\" for MBv3 and EfficientNetB0 are not shown? It is inconsistent with textual description in Sec. 6.2 (\"there is nearly 10x of acceleration ...\") that is not able to confirm.\n\n- In Table 1 and 2, regarding the numbers of \"GPU Hours\" like 20,000 and 2,000, do they indicate the real runtime in the experiments or only estimated values? As we know, 20,000 hours are roughly equal to 2.3 years, and 2,000 hours are similarly equal to 2.8 months. The reliability of the experiment results might be doubting.\n\n- In the references, there are too many informal publications cited from arXiv. Instead, they should be replaced by their formal publications at the corresponding conferences or journals.\n", "extra_scores": {}, "extra_text": {}}, {"review_id": "ohASsq4cpnb", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper proposes an RL-based neural architecture search approach to decrease the searching cost by introducing three modules to estimate uncertainty, restore parameters, and store old models. Compared to MNAS, it can significantly reduce the search cost up to x10, while giving competitive accuracy. \n\nThis paper is generally well-written and well-motivated, except for some unclear sentences;\n-      Architecture knowledge is not well described. Compared to parameter knowledge, the authors should clarify what they are and the difference between them. \n-      In Figure 4, it is unclear what the operators are and which operators are similar and different. Moreover, details are missing on how to sample 100 optimal models. \n-      In Equation 1, the definition of the reward is missing.\n-      LKP (the acronym first introduced in page 5) is not described.\n\nEven if it accelerates the search process, it entails additional memory due to the proposed module; it stores learned networks. So, I think there’s a trade-off between search cost and the total memory we need to reserve. From this, I wonder reducing the search cost is more significant compared to increase the required memory.\n\nIn Table 3, why FNAS has higher FLOPs than MNAS? This should be properly elaborated.\n\nIn Table 4, the cases using two modules are missing. It would be great to see the results to see which component actually affects the performance.\n", "extra_scores": {}, "extra_text": {}}, {"review_id": "dZCoTtZSKTN", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "In this paper, the authors propose to use a sampling-based approach to neural architecture search, which combines a life-long knowledge pool, uncertainty aware critic, architecture experience buffer. This approach has been demonstrated with vision tasks involving days of TPU training. \n\nOverall, I rank 5, marginally below the acceptance threshold. NAS is an underexplored topic. But the papers seems like an engineering project that combines multiple existing ideas from the others' work, and there lacks theoretical depth about clear mathematical formulation of the approach, and reasoning on why the approach should work.\n\nPros:\n\n+ Having access to TPU\n+ the NAS topic\n+ Integration of lifelong learning, NAS, and several other ideas\n\nCons:\n\n- Why lifelong learning should work, considering that we are not in multi-task learning and nonstationary environment scenario? How do we know that the experimental results is not some coincidence? \n- Can we put down the whole framework mathematically? It seems that this paper has only two formulas about some loss functions.\n- Can we reason about the math? For example, any ideas to better organize knowledge pool and ideas architecture experience buffer for a large number of architectures and parameters encountered?\n", "extra_scores": {}, "extra_text": {}}, {"review_id": "DpAJQETfm_Z", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Summary: The paper propose a few improvements to the sampling-based NAS using RL: 1) an uncertainty-aware critic to decide whether the sample needs to be trained; 2) a life-long knowledge pool to initialize the sample that needs training; and 3) an architecture experience buffer to reuse old samples for RL training. The experiments are done on ImageNet, facial recognition and transferability on object detection. The proposed methods are compared with related works. Finally the paper finishes with ablation studies on both the effectiveness and transferability of the proposed modules. \n\nStrengths:\n- The paper is well-written with clear flow and structures.\n- The three proposed modules are novel and improve the search cost significantly while achieving better performance. \n- It's great to see the authors has done a comprehensive comparison with the related methods for multiple tasks. The ablation study also demonstrate the effectiveness of the three proposed modules. \n\nWeakness:\n- The improvement of Top1 Acc. on ImageNet is marginal (without the 1.3 scale up) and worse than some of the recently proposed differentiable NAS work which requires far less search cost (comparable to DARTS).\n- There could be more discussion on related work studying uncertainty in RL or in supervised learning, given that it is one of the core modules in the proposed pipeline and uncertainty an important topic in general.", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper presents a compelling mechanism for reducing the neural architecture search process based on accumulated experience  that the reviewers found compelling with significant improvements in performance.  This is an intriguing idea. However, there were concerns about clarity that need to be addressed, and more concerning, the paper lacked technical depth or details in several aspects described in the reviews.  The authors subsequent response and revisions have somewhat addressed these issues.

The reviewer discussion had mixed opinions, with some for weak acceptance and others for weak rejection.  There were compelling points that the contribution is significant, but overall this paper would benefit from thoroughly addressing the shortcomings mentioned in the reviews before it is ready for publication. 
Decision: Reject