Neural Architecture Search in Embedding Space
Authors: chun-ting liu
Venue:  (2020)
Keywords: neural architecture search
nas
automl

Abstract: The neural architecture search (NAS) algorithm with reinforcement learning can be a powerful and novel framework for the automatic discovering process of neural architectures. However, its application is restricted by noncontinuous and high-dimensional search spaces, which result in difficulty in optimization. To resolve these problems, we proposed NAS in embedding space (NASES), which is a novel framework. Unlike other NAS with reinforcement learning approaches that search over a discrete and high-dimensional architecture space, this approach enables reinforcement learning to search in an embedding space by using architecture encoders and decoders. The current experiment demonstrated that the performance of the final architecture network using the NASES procedure is comparable with that of other popular NAS approaches for the image classification task on CIFAR-10. The beneficial-performance and effectiveness of NASES was impressive even when only the architecture-embedding searching and pre-training controller were applied without other NAS tricks such as parameter sharing. Specifically, considerable reduction in searches was achieved by reducing the average number of searching to < 100 architectures to achieve a final architecture for the NASES procedure.

Under review as a conference paper at ICLR 2020
NEURAL ARCHITECTURE SEARCH IN EMBEDDING SPACE
Anonymous authors Paper under double-blind review
ABSTRACT
The neural architecture search (NAS) algorithm with reinforcement learning can be powerful and novel framework for the automatic discovering process of neu- ral architectures. However, its application is restricted by noncontinuous and high- dimensional search spaces, which result in difficulty in optimization. To resolve these problems, proposed NAS in embedding space (NASES), which is novel framework. Unlike other NAS with reinforcement learning approaches that search discrete and high- -ighnsinensionll architecture space, this approach enables re- inforcement learning to search in an embedding space by using architecture en- coders and decoders. The current experiment demonstrated that the performance of the final architecture network using the NASES procedure is comparable with that of other popular NAS approaches for the image classification task CIFAR- 10. The beneficial-perforreeee performance and effectiveness of NASES was impressive even when only the cebhitcttur-embeeddie applied without other NAS tricks such as parameter sharing. Specifically, con- siderable reduction in searches was achieved by reducing the average number of searching to < 100 architectures to achieve final architecture for the NASES pro- cedure.
searching and pre-training controller were
INTRODUCTION
Deep neural networks have enabled advances image recognition, sequential pattern recognition, recommendation systems, and various tasks in the past decades. However, selecting suitable neu- ral architecture frequently arduous because the classical and new neural architectures emerging daily. In general, manual design of network architectures according to the cases is achievable. How- ever, hyperparameter tuning and architecture engineering through manual selection requires con- siderable time. Furthermore, manually designing neural network architecture requires substantial experience in deep learning.
Given the aforementioned reasons, neural architecture search (NAS), automated architecture gineering, has been successful the past years. The NAS algorithm is divided into three dimen- sions, namely search space, search strategy, and performance estimation strategy (Elsken et al., 2019). Outstanding results have been achieved using NAS with the reinforcement learning search strategy (Zoph & Le. (017). Here, recurrent network was used to generate string to form child network. However, such type of network exhibits two problems: noncontinuous and high- dimensional search space. The frequent large strings of action from the recurrent network and the discrete space result in difficulty in optimization. The critical contribution of this study is the im- provement of the dimension and quality of the search space that could provide a more efficient framework to solve the two problems of searching architectures.
vector that can represent network architecture without discrete values determined, then the noncontinuous aforementioned disadvantages can be addressed. We proposed the NAS in embed- ding space (NASES) method, which involves mapping origin architecture to architecture-embedding by using architecture encoder. The advantage of embedding space includes the lower-dimensiooall and continues space, considerably alleviates the difficulty in the optimization problem of the NAS procedure with reinforcement learning. To learn and search the embedding space, developed a mechanism to generate architecture encoder and decoder to promote origin architecture communi- cation with the embedding space, and the autoencoder network was used in the mechanism (Hinton

---

Under review as a conference paper at ICLR 2020
& Salakhutdinov 2006). The architecture simulator simulates the origin architecture space, which assists the real architecture encoder learning. The decoder realizes the relationship between oni- gin architecture and arrhitecture-embedddnn which maps the architecture-embcddinn to the origin architecture.
The NASES procedure implemented in stages. We obtained pretraining architecture decoder and pretraining architecture simulator in the first stage and provided the compression rate between the embedding size and testing loss. In the second stage, we used the NASES procedure for image classification on CIFAR-10 R- by using the network pretrained in the first stage. The results of the experiment were efficient and indicated that NASES was highly efficient and considerably reduced the number of searching architectures to <100 in < 12 GPU hours. Thus, the results were comparable with that of other popular NAS methods.
RELATED WORK
2.1 REINFORCEMENT LEARNING WITH ACTION EMBEDDING
Reinforcement learning is general approach that be applied broadly various areas. How- ever, the large and discrete action space causes problems in function approximation. The majority of studies have focused on two approaches, one approach factorizes the action space into binary subspaces (Pazis & Parr 2011 Dulac-Amold et al. 201T). The other approach involves embedding discrete actions into a continuous action, determining optimal actions the continuous space. and selecting the nearest discrete action to reduce the scaling of action sizes (Dulac-Amok) et al. 2015
Hasselt & Wiering 2009).
2.2 SEARCH STRATEGY WITH REINFORCEMENT LEARNING
Our search strategy is based reinforcement learning. Zoph & Le 2017) provided novel NAS framework, which incorporated reinforcement learning and applied it to two agents of the child network and the controller of the recurrent network. The child network generated neural architecture that can be considered the action of the controller network. Unlike the use of policy gradient by Zoph Le (2017), Williamsu (1992), and Baker al. (2016) used q- q-learning to update weight of the network.
2.3 NAS WITH CONTINUES VECTOR
Most NAS procedures use a discrete search space. Unlike other approaches, such as the learning over discrete and nondifferentiable search space, Liu al. (2018) proposed an approach of differentiable architecture search (DARTS), which was based on the continuous relaxation of the architecture representation. On the basis DARTS, Hundt al. (2019) proposed sharp DARTS, which is more general, balanced, and consistent design. The closest concept to NASES is the approach proposed by Luo et al. (2018) which an encoder and a decoder was map neural architectures in continuous space gradient-basse based optimization and predictor was used to achieve embedding accuracy.
3 METHODS
In this section, to elucidate the NASES procedure, we followed the aforementioned three dimen- sions: search strategy, search space, and performance estimation strategy.
3.1 SEARCH STRATEGY
The search strategy is search method for fast and accurate exploration of the space of neural architectures and involves techniques such as reinforcement learning, evolutionary algorithm, and gradient-based method. These are popular strategies in NAS.
In the NAS with reinforcement learning, which is generally designed with two components of the controller network and child network (Figure 1), the controller network is usually used to control the
2

---

Under review as a conference paper at ICLR 2020
child network architecture and generates string and the child network construct neural architecture by using the output of the controller in each NAS iteration. The controller network calculates the policy gradient to update the network by validating the performance of child network. Thus, severe penalty is imposed when performance is low. The controller that is constructed using multilayer perceptron rather than the recurrent neural layer, generates such continuous value, rather than string, in the NASES. This is discussed in the next subsection.
The Child Network
Controller Network
Overview of NAS with reinforcement learning. The child network receives information from the controller and generates a network architecture to evaluate date. The controller network receives reward from the evaluation of the child network for updating network parameters.
Figure 1:
3.2 SEARCH SPACE
The main contributions of NASES in the search space domain, which resolves the two afore- mentioned problems of noncontinuous and high-dimensional space in reinforcement learning; these problems lead to difficult optimization. NASES similar to the general NAS procedure, which also includes the child network and controller network. The optimization of maximize accuracy also used as a policy gradient method. However, our method differed from the general NAS developed architecture encoder the controller network control the archi- Second, from the ar-
procedure; first, tecture of the child network and projected origin architecture into aechitecture-mmbeddii devised the architecture decoder network, which decodes trchitecture-emeedee chitecture controller network to the origin architecture to ensure the child network can understand and generate network by using mrchitecture-emeedee problems, the architecture decoder functions as translator to translate low-dimension embedding into high- tigh-dimension vector for smooth childcontroller network communication. Furthermore, the search space was bounded using micro search in this study. Thus, we did not apply the cell-based trick hierarchical representation because we attempted to search neural network on the complete architecture and not only on the cell.
(see Figure 2). That is, alleviate these
3.2.1 THREE PRINCIPAL FUNCTIONS OF NASES.
The NASES, has three principal functions. This section describes the functions of the architecture decoder, architecture simulator, and controller network.
Architecture Decoder To obtain architecture eebbbecttrr-eembeddding decoder, first we created an approximate of virtual distribution transformation, which projected the low-dimension space into high-dimension space. That is, we transformed uechtecture-embedding into origin architecture.
So RR" - Rm
= fo(a)
where fe a decode function parameterized by 0, R" is the architecture-embedding space, R" is the origin architecture space, a is the set of origin architecture, à is the set of architectur-emmeedding This function released the controller network architecture and could be projected on another space not bounded the origin architecture space. This function is efficient and provides distribution transformers. To develop this approximator, an architecture simulator is required, which is discussed in the next subsection.
3

---

Under review as a conference paper at ICLR 2020
The Child Network
Architecture Controllar
Compressed
Vector
Architecture
Random
Decodar
Distribution
Architecture
Simulator
Figure 2: Overview of NAS on the embedding space. We used three functions namely architecture controller, architecture decoder, and architecture simulator, rather than the origin controller network for searching the neural architecture. Architecture controller received the origin architecture and generated iehitectureemmbedding Architecture decoder received the wechtecture-embeedinn and generated the origin architecture. Architecture simulator received random distribution and generated architecture-embeddinn..
Architecture Simulator An architecture simulator is an approach of the approximator of the vir- tual distribution transformation; purpose creation of function that simulates distribution to achieve architecture-embeedee Furthermore, the distribution of simulation is not limited to dis- crete or continuous space.
$. R
à (U)
where s, is function parametrized by and a uniform distribution.
Obtaining the architecture encoder and architecture simulator To obtain the architecture coder and simulator, we used the autoencoder network (Hinton & Salakhutdinov (006)). The autoen- coder network is unsupervised algorithm for distribution transformation and dimension reduction. The autoencoder generates representation by using the reduced encoding closest to its original in- put. Therefore, the architecture encoder and simulator were assembled in the autoencoder network. We pretrained an autoencoder network before training the controller, in which input space and tar- get space had the same distribution. We used uniform distribution. For convenient policy gradient learning, the activation function of the middle layer used was sigmoid; which leads the output with Bernoulli distribution.
Controller A controller controls the child neural architecture. The neural architecture and hyper- parameters of the controller are copied from the architecture simulator. The input is the child state of network, and the output is architecture-embee embedding with Bernoulli distribution.
9k : R RD
where Ok is an encode function parameterized by k. R is the architettree-embedding space, R the origin architecture space, is the set of origin architecture, and à is the set of architecture- embedding. We did not retrain the controller weights. The controller network can explore the trehiceture-emmeeddnn if the weights are initial- ized using the pretraining simulator of the autoencoder network because the pretraining simulator
the NASES procedures again and only fine-tuned

---

Under review as a conference paper at ICLR 2020
project uniform distribution into architecture-embeddiee Another advantage is fast search- ing because the controller network not required to leam projecting uniform distribution into architectureeembeddingg again and focuses searching architectures. Moreover, devised reward function by using the accuracy of the child network in training and validation. The reward obtained only from the accuracy of the validation set of the past NAS with reinforcement learning. In this study, the reward function is different from the general function, and it not only considers accuracy on the validation set but also estimates generalization errors (Eq. 1).
Reward =AACColl -ACCC, oCCoat))
(1)
=(2ACCpal ACC())
3.2.2 CHILD MODEL
The child model receives continuous vectors from the controller and generates neural architecture by using the pretrained decoder model. Here, we described the NASES mechanism for creating network architecture. We required four hyperparameters, namely number of filters, filter size, kernel type, and connection coefficient, in a layer. More details are in Appendix A.
3.3 PERFORMANCE ESTIMATION STRATEGY
To reduce the computational burden, we used two approaches of obtaining lower fidelities to estimate performance. First, the learning scheduler follows cosine annealing with 4max 0.05, 4min 0.001 and To epochs (Loshchilov & Hutter 2017). Second, each architecture search was run for 10 epochs on the search phrase, and final architecture run for 700 epochs. The detailed is presented in Alg 1.
Algorithm Neural Architecture Search in Embedding Space
Require:
uniform distribution U: architecture simulator sa; architecture decoder fo: controller g: child network e number of optimization iterations L; number of epochs in training phase e number of epochs in final phase E2:
1: 8, and fo were assembled an autoencoder network
2: Train the autoencoder network and evaluate its performance by uniform distribution U.
3: The controller g weights initialized with architecture simulator pretrained weights e.
4: for 1=1,..., T, do
5: Generate child network e by architecture a.
6: Train and evaluate data child network in epochs e by training and validation
7: Compute reward r from train and validation performance.
8: Train controller network Ge using reward r.
9: Project architecture a into architecture -etiuureeembddiing à by using the controller network Iv.
Decode
à into architecture a by using the architecture decoder fo.
11: end for
Train and evaluate on the child network with the highest reward in epoch e2 by training and testing set.
4 EXPERIMENTS
We describe two stages of the experiment in this section. The first stage involves training archi- tecture decoder and architecture simulator network and selecting an appropriate compression ratio of archttettrr-eembeddnng The second stage involves applying the result of first stage discover novel neural architectures for image classification on CIFAR-10 (Krizhevsky Hinton 2009) by using NASES.
5

---

Under review as a conference paper at ICLR 2020
4.1 FIRST STAGE: PRETRAINING ARCHITECTURE DECODER AND SIMULATOR NETWORK
4.1.1 DATASET
In the first stage of NASES experiment, our goal was to map origin architecture to architecture- embedding. We assembled the architecture simulator and decoder in an autoencoder network and trained this autoencoder network instead of training the simulator and decoder. To mimic the origin architecture space, we sampled 300000 as training set data from uniform distribution and sampled 100000 as the testing set data. In this case, we sampled uniform distribution at the interval [0, 30].
4.1.2 TRAINING DETAILS
The optimization of the autoencoder network was achieved by using Adam (Kingma & Ba 2015) optimizer with a learning rate of 0.00001. During the training of the autoencoder network, the learning schedule is increased the batch size instead of decaying the learning rate (Smith et al. 2018) and saved weights with the lowest test loss during testing. In the network architecture, the policy gradient methods update the probability distribution of actions so that the controllers actions with high expected reward exhibit a high probability for an observed state. Therefore, the activation function is a sigmoid function used in the middle layer hidden of the autoencoder network, and leads the distribution of hidden output to Bernoulli distribution, suitable for computing the policy gradient to update the controller network. Other details regarding the experimental procedures are as follows: Three fully connected layers with the number units of 1000, 500, and 100 were used for the simulator and decoder networks; the first layer exhibited the ReLu activation function, and the second and third layers exhibited the tanh activation function. The architecture decoder network exhibited the same neural architecture and hyperparameters setting as the architecture simulator network. The loss function used least square error.
4.1.3 RESULT
To understand the loss of information from the embedding space, we evaluated the compression ratios of 0.83, 0.67, 0.5, 0.33, 0.17, 0.08, and 0.02 to examine the utility of compression. We set an upper bound of testing loss as baseline, because if the decoder is incapable of decoding on aichiteture-emmedddin
the best strategy always is the predicted average value (average is 15 in this case). The utility of compression ratio on different sizes of embedding by using the autoencoder network is presented Table 1. The testing loss represents the loss of information after compres- sion by the autoencoder network, and the loss of information increases with the compression ratio. Appendix B, Figure illustrates the double-axis plot of the testing loss and embedding size (accord- ing to Table 1). As illustrated in Appendix B, Figure 3, a definite trade-off exists between loss and compression rate; high compression rate leads to high information loss. In this case, we suggested range of 20-30 as the appropriate embedding size by using the double-axis plot with cross-area of testing loss curve and embedding size curve. this range, the compression rate is more than half and information loss is considerably low. For image classification (second experiment stage) we followed the experiment results of the first stage, and the set the compression rate at 0.33, That is, the origin size of 60 was projected into the embedding size of 20.
4.2
SECOND STAGE: IMAGE CLASSIFICATION ON CIFAR-
4.2.1
DATASET
The second stage of the experiment is multiclass classification for assigning class the image object. The CIFAR-10 (Krizhevsky & Hinton] 2009) data set consists of 60000 color images of 32 32 RGB in 100 classes. Each class has 6000 images with 5000 training data and 1000 testing data. Additionally, to achieve standardization and normalization, we applied only three standard data augmentation techniques: (1) Subtracting the mean and then dividing the answer by the standard
deviation, which ensures that all variables have mean zero and standard deviation 1. (2) Centrally
padding on training set to 40 40 and randomly cropping images back to 32 32. (3) Randomly flipping images horizontally.

---

Under review as a conference paper at ICLR 2020
Table I: Performance of the autoencoder network. The left block represents the compression ratio. The right block represents information loss on the training and testing set.
Origin Size
Embedding Size
Compression ratio
Training Loss
Testing Loss
60
0.02
71.78
72.58
60
0.08
59.82
63.37
60
10
0.17
53.38
57.52
60
20
0.33
40.79
46.15
60
30
0.5
30.53
34.77
60
40
0.67
19.14
22.66
60
50
0.83
7.54
10.76
60
60
1.74
2.23
4.2.2 SETTINGS
The spilled validation ratio was 0.9; we then randomly split 45000 and 5000 images for training and validation, respectively, in the neural searching procedure. Finally, we used 50000 images for training and 10000 images for testing when the NASES search procedure was complete. Each architecture search procedure was run for 10 epochs on the search phase, and the final architecture was run for 700 epochs.
The child network described in the paragraph following the method section. The hyperparameters setting of the child network considers ENAS (Pham et al. 2018) as reference. It was trained with Nesterov momentum, momentum of 0.9 (Nesterov) 1983). The learning schedule followed the learning rate decay with cosine annealing for each batch (mex 0.05, Lmii = 0.001, To epochs) (Loshchilov Hutter 2017). batch size 128, weight decay of le-4. We initialize w with He initialization(He et al. 2015) in the child network. We designed 15-layer convolutional architecture by using 60 hyperparameters a layer for four hyperparameters contains: number of filters, filer size, kernel type, and connection coefficient). The NASES mechanism provided effective mechanism, and it only required applying 20 hyperparameters on embedding vector by using the 0.33 compression ratio. By following the controller network perspectives, we took the pretrained parameters of an already trained model from the first phase experiment of the pretrained architecture simulator. The controller hyperparameters setting and neural architecture followed the first stage of the experiment.
According to the reward function engineering perspectives, Eq. was used as our reward function; this function not only considered accuracy of the validation set but also estimated the generalization error. stored the reward and updated the rewards of the reward pool at the end of each iteration. To prevent the dependency of the final architecture on initial architectures, we ran random architectures ten times to collect rewards without updating parameters of the controller network in the beginning. We sampled three architecture examples from reward pool to update the controller network in each NASES iteration. Furthermore, the epsilon-gre -reeddy approach (Watkins) 1989) occurred randomly with the probability epsilon. Therefore, we have 10 to generate architecture randomly to out the reward pool too.
reward pool in each NASES iteration. Furthermore, we normalized
4.2.3 RESULT
We ran the NASES procedure five times by using different random seeds on a single Nvidia V100 GPU, and NASES required approximately 12 hours to determine the final architecture for NASES procedure. Furthermore, the average number of searching architectures to achieve final architecture for the NASES procedure was < <100, the number of searching architectures was reduced consider- ably compared with past NAS approaches. To determine the performance of the architecture, we evaluated the child network by using the final architecture network on the CIFAR-10 test dataset.

---

Under review as a conference paper at ICLR 2020
Table 2: Performance and GPU computing time macro search space of the NAS approach for class classification on CIFAR-10: First block represents the results of the final network. The second and the last blocks represent the results of the final network for more filters to each layer.
Level Method
GPUs
Days
Params
Error
Maero NAS with Q-Learning
Zhong et al. 2018
10
8-10
11.2m
6.92
1
SMASH (Brock et al.
2017
1.5
16.0m
4.03
1
NAS Zoph & Le 2017
800
21-28
7.1m
4.47
1
NASES
0.5
8m
4.07
2
Net Transformation (Cai et al.
2017)
5
19.7m.
5.70
2
ENAS (Pham et al. 2018
1
0.32
21.3m
4.23
2
NASES more filtersl.
0.5
20.4m
3.93
3
NAS more filters
800
21-28
37.4m
3.65
3
ENAS + more filters.
1
0.32
38.0m
3.87
3
NASES more filters2
1
0.5
28.4m
3.71
Table summarizes the performance of NASES and other NAS approaches by using the search algorithms. This final architecture is presented Appendix C, Figure 4.
In Table 2, the approaches have been into three levels based the number of parameters. The first block of Table 2 presents the performances of NAS approaches; the NASES final architecture that achieves 4.07 error rate on the testing uses only million parameters, which is comparable with other NAS approaches. For comparing more approaches and models, we added a number of filters each layer of the final architecture. The second block of Table represents the performance of NAS approaches when the number of parameters was approximately 20 million, and the NASES final architecture can be improved to 3.93 error rate by adding 100 the number filters of each layer. Finally, to evaluate the high parameter network, we added 150 to the number of filters of each layer. Notably, the NASES final architecture that achieved 3.71 error rate only used 28.4 million parameters. which was better than approximately 38 million parameters used by the ENAS (Pham) et al., 2018) and NAS (Zoph Le (0017). NASES required approximately half GPU day to discover the final architecture. The beneticial-perfone performance and effectiveness of NASES impressive when only the architecture-embeddee other NAS tricks such as sharing parameters (Pham al. 2018).
searching and pre-training controller were applied without
5 CONCLUSION
NAS with reinforcement learning is powerful and novel framework for the automatic discover- ing process neural architectures. Here, we designed a novel NAS framework, and this approach alleviated the two problems of noncontinuous and nigh-dimensional search space of NAS with rein- forcement learning. We named this NAS framework NASES, in which the controller can be searched embedding space by using the architecture decoder and architecture simulator. We achieved fa- vorable results for image classification on CIFAR-10: the NASES exhibited efficient performance and high effectiveness when the number of searching architecture was reduced <100 architec- tures. We proposed a simple method to estimate the compression ratio of architecture-embedding
REFERENCES
Bowen Baker, Otkrist Gupta, Nikhil Naik, and Ramesh Raskar. Designing neural network architec- tures using reinforcement learning. 2016.
8

---

Under review as a conference paper at ICLR 2020
Jonathon Shlens Barret Zoph, Vijay Vasudevan and Quoc V. Le. Learning transferable architectures for scalable image recognition. 2018.
Andrew Brock, Theodore Lim, J.M. Ritchie, and Nick Weston. Smash: One-shot model architecture search through hypernetworks. 2017.
Han Cai, Tianyao Chen, Weinan Zhang, Yong Yu, and Jun Wang. Reinforcement learning for architecture search by network transformation. 2017.
Gabriel Dulac-Arnold, Ludovic Denoyer, Philippe Preux, and Patrick Gallinari. Fast reinforcement learning with large action sets using error-correcting output codes for mdp factorization. 2011.
Gabriel Bulac-Arnold Richard Evans, Hado van Hasselt, Peter Sunehag, Timothy Lillicrap, Jonathan Hunt, Timothy Mann, Theophane Weber, Thomas Degris, and Ben Coppin. Deep rein- forcement learning in large discrete action spaces. 2015.
Thomas Elsken, Jan Hendrik Metzen, and Frank Hutter. Neural architecture search: A survey. 2019.
Hado Van Hasselt and Marco A. Wiering, Using continuous action spaces to solve discrete problems. 2009.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Delving deep into rectifiers: Surpassing human-level performance on imagenet classification. 2015.
G. E. Hinton and R. R. Salakhutdinov. Reducing the dimensionality of data with neural networks. 2006.
Andrew Hundt, Varun Jain, and Gregory D. Hager. sharpdarts: Faster and more accurate differen- tiable architecture search. 2019.
Sergey Ioffe and Christian Szegedy. Batch normalization: Accelerating deep network training by reducing internal covariate shift. 2015.
Diederik P. Kingma and Jimmy Lei Ba. Adam: A method for stochastic optimization. 2015.
Alex Krizhevsky and Geoffrey Hinton. Learning multiple layers of features from tiny images. 2009.
Hanxiao Liu, Karen Simonyan, and Yiming Yang. Darts: Differentiable architecture search. 2018.
Ilya Loshchilov and Frank Hutter. Sgdr: Stochastic gradient descent with warm restarts. 2017.
Renqian Luo, Fei Tian, Tao Qin, Enhong Chen, and Yan Liu. Neural architecture optimization. 2018.
Qiang Chen Min Lin and Shuicheng Yan. Network in network. 2013.
Yu. F. Nesterov. A method of solving convex programming problem with convergence rate o1/k2). 1983.
Jason Pazis and Ronald Parr. Generalized value functions for large action sets. 2011.
Hieu Pham, Melody Y. Guan, Barret Zoph, Quoc V. Le, and Jeff Dean. Efficient neural architecture search via parameter sharing. 2018.
Samuel L.. Smith, Pieter-Jan Kindermans, Chris Ying, and Quoc V. Le. Don't decay the learning rate, increase the batch size. 2018.
Christopher John Cornish Hellaby Watkins. Learning from delayed rewards. 1989.
Ronald J. Williamsu. Simple statistical followig-oolooingg algorithms for connectionist rein- force- ment learning. 1992.
Zhao Zhong, Junjie Yan, Wei Wu. Jing Shao, and Cheng-Lin Liu. Practical block-wise neural network architecture generatio. 2018.
Barret 7oph and Quoc V Le. Neural architecture search with reinforcement leaming. 2017.

---

Under review as a conference paper at ICLR 2020
A
THE CHILD MODEL DETAILS
A. THE OPERATIONS.
The 12 operations were provided the controller by using two hyperparameters, namely filter size and kernel type. The filter size represents the amount of neighbor information during the cur- rent layer processing, and kernel type represents the components of the neural network, including the convolution layer, depthwise-separabl convolution layer, maximum pooling layer, and aver- age pooling layer. The child network receives these continuous vectors, and we developed rule that transforms into discrete vectors. For example, in the kernel type, the rule is that S 7 assigns sarrhwise-separable convolution, >7 or < 15 assigns convolution. For filter size, the rule is that < 10 assigns to the network are convolution with filter sizes 3 x 3, 5 x 5, and 7 x 7; deethwise-ssppaahhe with filter sizes x 3, 5 x 5, and x 7; maximum pooling with filter sizes 3 x 3, 5 5, and 7 x 7; and average pooling with filter sizes x 3, 5 x 5, and 7 x 7. In addition, each depthwise-separable
filter size. According this principle, the operations available for the child convolution
convolution was applied twice (Barret Zoph & Le 2018
A.2 SKIP CONNECTION.
The skip connections essential connections that occur from the early layers to the later layers through addition or straight up concatenation. The reasoning behind this skip connection that they exhibit uninterrupted gradient flow from the first layer the last layer, which tackles the vanishing gradient problem. In this study, the hyperparameter the early layer to connect to last multiple layers with closer connection coefficient; these layers were concatenated in the channel dimension at the end of the layer. Therefore, we created another rule, and the layers layer one and two are connection when their connection coefficient is between two and five.
the connection coefficient assigned
connection when the connection coefficient close three; for example, the
A.3 THE ORDER OF THE BLOCKS IN EACH LAYER.
Performance was affected by the order of blocks in each layer. We applied the order of Rel lu-conv- batchnorm (Ioffe & Szegedy] 2015). Moreover, the kernel size of x 1 convolution filters can be applied to change the dimensionality the filter space. We applied the order ReLu-conv- batchnorm to kernel size convolution layers before the convolution layer, except for the first layer.
A.4 GLOBAL AVERAGE POOLING.
We employed a trick into the NASES of the global average pooling (Min Lin & Yan 2013) which is an operation that calculates the average output of each feature map in the final convolution layer for reducing the number of parameters from the full connection layer.
10

---

Under review as a conference paper at ICLR 2020
B DOUBLE-AXIS PLOT OF THE TESTING Loss AND EMBEDDING SIZE
Test loss
Embedding Size
80
60
40
30
20
0
Figure 3: Double-axis diagram that illustrates the testing loss and embedding size, which offer useful information for selecting an appropriate embedding size. Right axis: testing loss. Left axis: embedding size.
C FINAL ARCHITECTURE
1000
COU A 350
man 5x5
CORV 5K5
DOnV 5
CONV 3x3 350
COIN 5x5 350
Softnax
336
5
5N.3
3x.3 420
300
370
/359
310
320
300
340
350
Figure 4: The NASES final architecture discovered on the macro search for image classification. is a 15-layer convolutional architecture by using 20 NASES hyperparameters.
11


Reviews:
[{"review_id": "rkg56JppKr", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 1, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This work searches neural architectures in an embedding space, which is continuous. Overall, it lacks of innovation and experimental results are not strong enough. \n\n\t1. The innovation and technical contribution are weak. The key idea of searching in embedding space has already been proposed and applied in  Luo et al. 2018. The authors do not differentiate this work from Luo et al. 2018, and I didn't find any essential differences between them.\n\n\t2. The proposed method does work well. It is not compared with latest algorithms including NAO and DARTS. The reported numbers in Table 2 are poor, far from SOTA numbers.\n\nBesides, there are many writing issues and typos.", "extra_scores": {"experience_assessment": "I have published in this field for several years.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "N/A"}, "extra_text": {}}, {"review_id": "Hkl24SyG9B", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Summary:\nThis paper borrows the idea of word-to-vector from NLP and applies it in reinforcement learning based Neural Architecture Search (NAS). It suggests a pretrained encoder to transform the search space to a dense and continuous architecture-embedding space. First it trains the architecture-embedding encoder and decoder with self-supervision learning like Auto-Encoder.  Then it performs reinforcement learning based Neural Architecture Search(NAS) in the architecture-embedding space.\n\nStrength:\nThere is no architecture prior, such as cell, in the searching process. Thus it's more general and can explore more architectures possibilities.\nBecause it performs architecture search in a continuous space, a CNN based controller is used instead of a RNN controller. \nThe result of the proposed method on CIFAR-10 is comparable with other popular NAS approaches.\nIt reduces the number of searching architectures to <100 in <12 GPU hours without using tricks such as cell or parameter sharing.\n\nWeakness:\nThe evaluations are highly insufficient. It only performs experiment on CIFAR-10, and the generalization ability on other datasets is unclear. In many NAS works. CIFAR-100 and ImageNet are commonly used to evaluate the performance. \nBesides, there is no comparison with more recent and related important methods such as DARTS and the method proposed by Luo et al. (2018). Actually its performance is not as good as Darts or the best performance reported in ENAS.\n", "extra_scores": {"experience_assessment": "I have read many papers in this area.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory."}, "extra_text": {}}, {"review_id": "BJeoxR3Q5H", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The paper proposes an interesting idea to perform Neural Architecture Search: first, an auto-encoder is pre-trained to encode/decode an neural architecture to/from a continuous low-dimensional embedding space; then the decoder is fixed but the encoder is copied as an agent controller for reinforcement learning. The controller is optimized by taking actions in the embedding space. The reward is also different from previous works which usually only considered validation accuracy but this work also considers the generalization gap.\n\nThe idea is interesting, but there are some problems on both the method's and the experimental sides:\n1. NAO [1] also embeds neural architectures to a continuous space. Different from NAO which applies gradient descent in the embedded space, this paper uses RL. I double that RL can work better than gradient descent in a continuous space. The paper should compare with NAO. Ideally, this paper might work better than NAO if the accuracy predictor in the NAO is not accurate, while this paper uses real accuracy as a reward for search. However, this is not soundly compared.\n2. It is unreasonable to discretize continuous actions to a Bernoulli distribution. Many RL methods, such as DDPG, can handle continuous actions;\n3. The paper uses Eq. 1 as a reward. It's interesting, but it's unclear why the generalization error is needed. Ablation study is required.\n4. As the community makes more progresses in AutoML, a better and better (smaller and smaller) search space is used. It doesn't make much sense to compare the search time under different search spaces. Comparison under the same setting (e.g. NASBench-101) is required. \n\n\nMinors:\n1. missing a number in \"and T0 = epochs\"\n2. missing \"x\" in \"32 32 RGB in 100 classes\", and \"100\" should be \"10\"\n\n[1] Luo, Renqian, Fei Tian, Tao Qin, Enhong Chen, and Tie-Yan Liu. \"Neural architecture optimization.\" In Advances in neural information processing systems, pp. 7816-7827. 2018.", "extra_scores": {"experience_assessment": "I have read many papers in this area.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper thoroughly.", "review_assessment:_checking_correctness_of_experiments": "I carefully checked the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "N/A"}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper proposes a method for neural architecture search in embedding space. This is an interesting idea, but its novelty is limited due to its similarity to the NAO approach. Also, the empirical evaluation is too limited; comparisons should have been performed to NAO and other contemporary NAS methods, such as DARTS. 

Due the factors above, all reviewers gave rejecting scores (3,3,1). The rebuttal did not remove the main issues, resulting in the reviewers sticking to their scores. I therefore recommend rejection.
Decision: Reject