Neural Architecture Search in Embedding Space

1 INTRODUCTION . Deep neural networks have enabled advances in image recognition , sequential pattern recognition , recommendation systems , and various tasks in the past decades . However , selecting a suitable neural architecture is frequently arduous because of the classical and new neural architectures emerging daily . In general , manual design of network architectures according to the cases is achievable . However , hyperparameter tuning and architecture engineering through manual selection requires considerable time . Furthermore , manually designing a neural network architecture requires substantial experience in deep learning . Given the aforementioned reasons , neural architecture search ( NAS ) , an automated architecture engineering , has been successful in the past years . The NAS algorithm is divided into three dimensions , namely search space , search strategy , and performance estimation strategy ( Elsken et al. , 2019 ) . Outstanding results have been achieved using NAS with the reinforcement learning search strategy ( Zoph & Le , 2017 ) . Here , a recurrent network was used to generate a string to form a child network . However , such a type of network exhibits two problems : noncontinuous and highdimensional search space . The frequent large strings of action from the recurrent network and the discrete space result in difficulty in optimization . The critical contribution of this study is the improvement of the dimension and quality of the search space that could provide a more efficient framework to solve the two problems of searching architectures . If a vector that can represent network architecture without discrete values is determined , then the noncontinuous aforementioned disadvantages can be addressed . We proposed the NAS in embedding space ( NASES ) method , which involves mapping origin architecture to architecture-embedding by using an architecture encoder . The advantage of embedding space includes the lower-dimensional and continues space , it considerably alleviates the difficulty in the optimization problem of the NAS procedure with reinforcement learning . To learn and search on the embedding space , we developed a mechanism to generate architecture encoder and decoder to promote origin architecture communication with the embedding space , and the autoencoder network was used in the mechanism ( Hinton & Salakhutdinov , 2006 ) . The architecture simulator simulates the origin architecture space , which assists the real architecture encoder learning . The decoder realizes the relationship between origin architecture and architecture-embedding , which maps the architecture-embedding to the origin architecture . The NASES procedure was implemented in two stages . We obtained a pretraining architecture decoder and a pretraining architecture simulator in the first stage and provided the compression rate between the embedding size and testing loss . In the second stage , we used the NASES procedure for image classification on CIFAR-10 by using the network pretrained in the first stage . The results of the experiment were efficient and indicated that NASES was highly efficient and considerably reduced the number of searching architectures to < 100 in < 12 GPU hours . Thus , the results were comparable with that of other popular NAS methods . 2 RELATED WORK . 2.1 REINFORCEMENT LEARNING WITH ACTION EMBEDDING . Reinforcement learning is a general approach that can be applied broadly to various areas . However , the large and discrete action space causes problems in function approximation . The majority of studies have focused on two approaches , one approach factorizes the action space into binary subspaces ( Pazis & Parr , 2011 ; Dulac-Arnold et al. , 2011 ) . The other approach involves embedding discrete actions into a continuous action , determining optimal actions in the continuous space , and selecting the nearest discrete action to reduce the scaling of action sizes ( Dulac-Arnold et al. , 2015 ; Hasselt & Wiering , 2009 ) . 2.2 SEARCH STRATEGY WITH REINFORCEMENT LEARNING . Our search strategy is based on reinforcement learning . Zoph & Le ( 2017 ) provided a novel NAS framework , which incorporated reinforcement learning and applied it to two agents of the child network and the controller of the recurrent network . The child network generated neural architecture that can be considered the action of the controller network . Unlike the use of policy gradient by Zoph & Le ( 2017 ) , Williamsu ( 1992 ) , and Baker et al . ( 2016 ) used q-learning to update the weight of the network . 2.3 NAS WITH CONTINUES VECTOR . Most NAS procedures use a discrete search space . Unlike other approaches , such as the learning over discrete and nondifferentiable search space , Liu et al . ( 2018 ) proposed an approach of differentiable architecture search ( DARTS ) , which was based on the continuous relaxation of the architecture representation . On the basis of DARTS , Hundt et al . ( 2019 ) proposed sharp DARTS , which is a more general , balanced , and consistent design . The closest concept to NASES is the approach proposed by Luo et al . ( 2018 ) in which an encoder and a decoder was to map neural architectures in a continuous space on gradient-based optimization and a predictor was used to achieve embedding accuracy . 3 METHODS . In this section , to elucidate the NASES procedure , we followed the aforementioned three dimensions : search strategy , search space , and performance estimation strategy . 3.1 SEARCH STRATEGY . The search strategy is a search method for fast and accurate exploration of the space of neural architectures and involves techniques such as reinforcement learning , evolutionary algorithm , and gradient-based method . These are popular strategies in NAS . In the NAS with reinforcement learning , which is generally designed with two components of the controller network and child network ( Figure 1 ) , the controller network is usually used to control the child network architecture and generates string and the child network construct neural architecture by using the output of the controller in each NAS iteration . The controller network calculates the policy gradient to update the network by validating the performance of child network . Thus , severe penalty is imposed when performance is low . The controller that is constructed using a multilayer perceptron rather than the recurrent neural layer , generates such continuous value , rather than a string , in the NASES . This is discussed in the next subsection . 3.2 SEARCH SPACE . The main contributions of NASES is in the search space domain , which resolves the two aforementioned problems of noncontinuous and high-dimensional space in reinforcement learning ; these problems lead to difficult optimization . NASES is similar to the general NAS procedure , which also includes the child network and controller network . The optimization of maximize accuracy is also used as a policy gradient method . However , our method differed from the general NAS procedure ; first , we developed an architecture encoder as the controller network to control the architecture of the child network and projected origin architecture into architecture-embedding . Second , we devised the architecture decoder network , which decodes architecture-embedding from the architecture controller network to the origin architecture to ensure the child network can understand and generate network by using architecture-embedding ( see Figure 2 ) . That is , to alleviate these problems , the architecture decoder functions as a translator to translate low-dimension embedding into high-dimension vector for smooth childcontroller network communication . Furthermore , the search space was bounded using micro search in this study . Thus , we did not apply the cell-based trick of hierarchical representation because we attempted to search neural network on the complete architecture and not only on the cell . 3.2.1 THREE PRINCIPAL FUNCTIONS OF NASES .. The NASES , has three principal functions . This section describes the functions of the architecture decoder , architecture simulator , and controller network . Architecture Decoder To obtain architecture-embedding decoder , first we created an approximate of virtual distribution transformation , which projected the low-dimension space into high-dimension space . That is , we transformed architecture-embedding into origin architecture . fθ : Rn → Rm a = fθ ( â ) where fθ is a decode function parameterized by θ , Rn is the architecture-embedding space , Rm is the origin architecture space , a is the set of origin architecture , â is the set of architecture-embedding . This function released the controller network architecture and could be projected on another space not bounded on the origin architecture space . This function is efficient and provides distribution transformers . To develop this approximator , an architecture simulator is required , which is discussed in the next subsection . Architecture Simulator An architecture simulator is an approach of the approximator of the virtual distribution transformation ; its purpose creation of a function that simulates distribution to achieve architecture-embedding . Furthermore , the distribution of simulation is not limited to discrete or continuous space . sv : U → Rn â = sv ( U ) where sv is a function parametrized by v and U is a uniform distribution . Obtaining the architecture encoder and architecture simulator To obtain the architecture encoder and simulator , we used the autoencoder network ( Hinton & Salakhutdinov , 2006 ) . The autoencoder network is a unsupervised algorithm for distribution transformation and dimension reduction . The autoencoder generates a representation by using the reduced encoding closest to its original input . Therefore , the architecture encoder and simulator were assembled in the autoencoder network . We pretrained an autoencoder network before training the controller , in which input space and target space had the same distribution . We used uniform distribution . For convenient policy gradient learning , the activation function of the middle layer used was sigmoid ; which leads the output with Bernoulli distribution . Controller A controller controls the child neural architecture . The neural architecture and hyperparameters of the controller are copied from the architecture simulator . The input is the child state of network , and the output is architecture-embedding with Bernoulli distribution . gk : Rm → Rn where gk is an encode function parameterized by k , Rn is the architecture-embedding space , Rm is the origin architecture space , A is the set of origin architecture , and â is the set of architectureembedding . We did not retrain the controller on the NASES procedures again and only fine-tuned weights . The controller network can explore the architecture-embedding if the weights are initialized using the pretraining simulator of the autoencoder network because the pretraining simulator can project uniform distribution into architecture-embedding . Another advantage is fast searching because the controller network is not required to learn projecting uniform distribution into architecture-embedding again and focuses on searching architectures . Moreover , we devised a new reward function by using the accuracy of the child network in training and validation . The reward is obtained only from the accuracy of the validation set of the past NAS with reinforcement learning . In this study , the reward function is different from the general function , and it not only considers accuracy on the validation set but also estimates generalization errors ( Eq . 1 ) . Reward = ( ACCval − ( ACCtr −ACCval ) ) 3 = ( 2ACCval −ACCtr ) 3 ( 1 ) 3.2.2 CHILD MODEL . The child model receives continuous vectors from the controller and generates neural architecture by using the pretrained decoder model . Here , we described the NASES mechanism for creating a network architecture . We required four hyperparameters , namely number of filters , filter size , kernel type , and connection coefficient , in a layer . More details are in Appendix A . 3.3 PERFORMANCE ESTIMATION STRATEGY . To reduce the computational burden , we used two approaches of obtaining lower fidelities to estimate performance . First , the learning scheduler follows cosine annealing with lmax = 0.05 , lmin = 0.001 and T0 = epochs ( Loshchilov & Hutter , 2017 ) . Second , each architecture search was run for 10 epochs on the search phrase , and final architecture is run for 700 epochs . The detailed is presented in Alg 1 . Algorithm 1 Neural Architecture Search in Embedding Space Require : uniform distribution U ; architecture simulator sv ; architecture decoder fθ ; controller g ; child network c number of optimization iterations L ; number of epochs in training phase e1 ; number of epochs in final phase e2 ; 1 : sv and fθ were assembled in an autoencoder network 2 : Train the autoencoder network and evaluate its performance by uniform distribution U . 3 : The controller g weights initialized with architecture simulator pretrained weights v. 4 : for l=1 , ... , L do 5 : Generate child network c by architecture a . 6 : Train and evaluate data child network c in epochs e1 by training and validation set . 7 : Compute reward r from train and validation performance . 8 : Train controller network gv using reward r. 9 : Project architecture a into architecture-embedding â by using the controller network gv . 10 : Decode architecture-embedding â into architecture a by using the architecture decoder fθ . 11 : end for 12 : Train and evaluate on the child network with the highest reward in epoch e2 by training and testing set .

Summary: This paper borrows the idea of word-to-vector from NLP and applies it in reinforcement learning based Neural Architecture Search (NAS). It suggests a pretrained encoder to transform the search space to a dense and continuous architecture-embedding space. First it trains the architecture-embedding encoder and decoder with self-supervision learning like Auto-Encoder.  Then it performs reinforcement learning based Neural Architecture Search(NAS) in the architecture-embedding space.