MANAS: Multi-Agent Neural Architecture Search

The Neural Architecture Search ( NAS ) problem is typically formulated as a graph search problem where the goal is to learn the optimal operations over edges in order to maximize a graph-level global objective . Due to the large architecture parameter space , efficiency is a key bottleneck preventing NAS from its practical use . In this paper , we address the issue by framing NAS as a multi-agent problem where agents control a subset of the network and coordinate to reach optimal architectures . We provide two distinct lightweight implementations , with reduced memory requirements ( 1/8th of state-of-the-art ) , and performances above those of much more computationally expensive methods . Theoretically , we demonstrate vanishing regrets of the form O ( √ T ) , with T being the total number of rounds . Finally , aware that random search is an ( often ignored ) effective baseline we perform additional experiments on 3 alternative datasets and 2 network configurations , and achieve favorable results in comparison with this baseline and other methods . 1 INTRODUCTION . Determining an optimal architecture is key to accurate deep neural networks ( DNNs ) with good generalisation properties ( Szegedy et al. , 2017 ; Huang et al. , 2017 ; He et al. , 2016 ; Han et al. , 2017 ; Conneau et al. , 2017 ; Merity et al. , 2018 ) . Neural architecture search ( NAS ) , which has been formulated as a graph search problem , can potentially reduce the need for application-specific expert designers allowing for a wide-adoption of sophisticated networks in various industries . Zoph and Le ( 2017 ) presented the first modern algorithm automating structure design , and showed that resulting architectures can indeed outperform human-designed state-of-the-art convolutional networks ( Ko , 2019 ; Liu et al. , 2019 ) . However , even in the current settings where flexibility is limited by expertlydesigned search spaces , NAS problems are computationally very intensive with early methods requiring hundreds or thousands of GPU-days to discover state-of-the-art architectures ( Zoph and Le , 2017 ; Real et al. , 2017 ; Liu et al. , 2018a ; b ) . Researchers have used a wealth of techniques ranging from reinforcement learning , where a controller network is trained to sample promising architectures ( Zoph and Le , 2017 ; Zoph et al. , 2018 ; Pham et al. , 2018 ) , to evolutionary algorithms that evolve a population of networks for optimal DNN design ( Real et al. , 2018 ; Liu et al. , 2018b ) . Alas , these approaches are inefficient and can be extremely computationally and/or memory intensive as some require all tested architectures to be trained from scratch . Weight-sharing , introduced in ENAS ( Pham et al. , 2018 ) , can alleviate this problem . Even so , these techniques can not easily scale to large datasets , e.g. , ImageNet . More recently , gradient-based frameworks enabled efficient solutions by introducing a continuous relaxation of the search space . For example , DARTS ( Liu et al. , 2019 ) uses this relaxation to optimise architecture parameters using gradient descent in a bi-level optimisation problem , while SNAS ( Xie et al. , 2019 ) updates architecture parameters and network weights under one generic loss . Still , due to memory constraints the search has to be performed on 8 cells , which are then stacked 20 times for the final architecture . This solution is a coarse approximation to the original problem as show in Section 6 of this work and in Doe ( 2019 ) . In fact , we show that searching directly over 20 cells leads to a reduction in test error ( 0.24 p.p . ; 8 % relative to Liu et al. , 2019 ) . ProxylessNAS ( Cai et al. , 2019 ) is one exception , as it can search for the final models directly ; nonetheless they still require twice the amount of memory used by our proposed algorithm . To enable the possibility of large-scale joint optimisation of deep architectures we contribute MANAS , the first multi-agent learning algorithm for neural architecture search . Our algorithm combines the memory and computational efficiency of multi-agent systems , which is achieved through action coordination with the theoretical rigour of online machine learning , allowing us to balance exploration versus exploitation optimally . Due to its distributed nature , MANAS enables large-scale optimisation of deeper networks while learning different operations per cell . Theoretically , we demonstrate that MANAS implicitly coordinates learners to recover vanishing regrets , guaranteeing convergence . Empirically , we show that our method achieves state-of-the-art accuracy results among methods using the same evaluation protocol but with significant reductions in memory ( 1/8th of Liu et al. , 2019 ) and search time ( 70 % of Liu et al. , 2019 ) . The multi-agent ( MA ) framework is inherently scalable and allows us to tackle an optimization problem that would be extremely challenging to solve efficiently otherwise : the search space of a single cell is 814 and there is no fast way of learning the joint distribution , as needed by a single controller . More cells to learn exacerbates the problem , and this is why MA is required , as for each agent the size of the search space is always constant . In short , our contributions can be summarised as : ( 1 ) framing NAS as a multi-agent learning problem ( MANAS ) where each agent supervises a subset of the network ; agents coordinate through a credit assignment technique which infers the quality of each operation in the network , without suffering from the combinatorial explosion of potential solutions . ( 2 ) Proposing two lightweight implementations of our framework that are theoretically grounded . The algorithms are computationally and memory efficient , and achieve state-of-the-art results on Cifar-10 and ImageNet when compared with competing methods . Furthermore , MANAS allows search directly on large datasets ( e.g . ImageNet ) . ( 3 ) Presenting 3 news datasets for NAS evaluation to minimise algorithmic overfitting ; and offering a fair comparison with a random baseline . 2 RELATED WORK . MANAS derives its search space from DARTS ( Liu et al. , 2019 ) and is therefore most related to other gradient-based NAS methods that use the same search space . SNAS ( Xie et al. , 2019 ) appears similar at a high level , but has important differences : 1 ) it uses GD to learn the architecture parameters . This requires a differentiable objective ( MANAS does not ) and leads to 2 ) having to forward all operations ( see their Eqs.5,6 ) , thus negating any memory advantages ( which MANAS has ) , and effectively requiring repeated cells and preventing search on ImageNet . ENAS ( Pham et al. , 2018 ) is also very different : its use of RL implies dependence on past states ( the previous operations in the cell ) . It explores not only the stochastic reward function but also the relationship between states , which is where most of the complexity lies . Furthermore , RL has to balance exploration and exploitation by relying on sub-optimal heuristics , while MANAS , due to its theoretically optimal approach from online learning , is more sample efficient . Finally , ENAS uses a single LSTM ( which adds complexity and problems such as exploding/vanishing gradients ) to control the entire process , and is thus following a monolithic approach . Indeed , at a high level , our multi-agent framework can be seen as a way of decomposing the monolithic controller into a set of simpler , independent sub-policies . This provides a more scalable and memory efficient approach that leads to higher accuracy , as confirmed by our experiments . 3 PRELIMINARY : NEURAL ARCHITECTURE SEARCH . We consider the NAS problem as formalised in DARTS ( Liu et al. , 2019 ) . At a higher level , the architecture is composed of a computation cell that is a building block to be learned and stacked in the network . The cell is represented by a directed acyclic graph with V nodes and N edges ; edges connect all nodes i , j from i to j where i < j . Each vertex x ( i ) is a latent representation for i ∈ { 1 , . . . , V } . Each directed edge ( i , j ) ( with i < j ) is associated with an operation o ( i , j ) that transforms x ( i ) . Intermediate node values are computed based on all of its predecessors as x ( j ) = ∑ i < j o ( i , j ) ( x ( i ) ) . For each edge , an architect needs to intelligently select one operation o ( i , j ) from a finite set of K operations , O = { ok ( · ) } Kk=1 , where operations represents some function to be applied to x ( i ) to compute x ( j ) , e.g. , convolutions or pooling layers . To each o ( i , j ) k ( · ) is associated a set of operational weights w ( i , j ) k that needs to be learned ( e.g . the weights of a convolution filter ) . Additionally , a parameter α ( i , j ) k ∈ R characterises the importance of operation k within the pool O for edge ( i , j ) . The sets of all the operational weights { w ( i , j ) k } and architecture parameters { α ( i , j ) k } are denoted by w and α , respectively . DARTS defined the operation ō ( i , j ) ( x ) as ō ( i , j ) ( x ) = K∑ k=1 eα ( i , j ) k∑K k′=1 e α ( i , j ) k′ · o ( i , j ) k ( x ) ( 1 ) in which α encodes the network architecture . The optimal choice of architecture is defined by α ? = min α L ( val ) ( α , w ? ( α ) ) s.t . w ? ( α ) = arg min w L ( train ) ( α , w ) . ( 2 ) The final objective is to obtain a sparse architecture Z ? = { Z ( i , j ) } , ∀i , j where Z ( i , j ) = [ z ( i , j ) 1 , . . . , z ( i , j ) K ] with z ( i , j ) k = 1 for k corresponding to the best operation and 0 otherwise . That is , for each pair ( i , j ) a single operation is selected . 4 ONLINE MULTI-AGENT LEARNING FOR AUTOML NAS suffers from a combinatorial explosion in its search space . A recently proposed approach to tackle this problem is to approximate the discrete optimisation variables ( i.e. , edges in our case ) with continuous counterparts and then use gradient-based optimisation methods . DARTS ( Liu et al. , 2019 ) introduced this method for NAS , though it suffers from two important drawbacks . First , the algorithm is memory and computationally intensive ( O ( NK ) with K being total number of operations between a pair of nodes and N the number of nodes ) as they require loading all operation parameters into GPU memory . As a result , DARTS only optimises over a small subset of 8 cells , which are then stacked together to form a deep network of 20 . Naturally , such an approximation is bound to be sub-optimal . Second , evaluating an architecture amounts to a prediction on a validation set using the optimal set of network parameters . Learning these , unfortunately , is highly demanding since for an architecture Zt , one would like to compute L ( val ) t ( Zt , w ? t ) where w ? t = arg minw L ( train ) t ( w , Zt ) . DARTS , uses weight sharing that updates wt once per architecture , with the hope of trackingw ? t over learning rounds . Although this technique leads to significant speed up in computation , it is not clear how this approximation affects the validation loss function . Next , we detail a novel methodology based on a combination of multi-agent and online learning to tackle the above two problems ( Figure 1 ) . Multi-agent learning scales our algorithm , reducing memory consumption by an order of magnitude from O ( NK ) to O ( N ) ; and online learning enables rigorous understanding of the effect of tracking w ? t over rounds .

Summary: This work built on top of DARTS. In their setting, each edge on the DAG (same as the one proposed in DARTS) has one agent associated with it and every agent maintains weights to propose operations. The author introduced two ways to update these weights: 1) solving a least squares assuming the validation loss decomposes linearly on the operations (MANAS-LS); 2) only update the weights for the activated operations (MANAS). Due to the usage of bandit framework, theoretical guarantees on the regret can be derived.