We introduce ES-ENAS, a simple neural architecture search (NAS) algorithm for the purpose of reinforcement learning (RL) policy design, by combining Evolutionary Strategies (ES) and  Efficient NAS (ENAS) in a highly scalable and intuitive way. Our main insight is noticing that ES is already a distributed blackbox algorithm , and thus we may simply insert a model controllerfrom ENAS into the central aggregator in ES and obtain weight sharing properties for free . By doing so, we  bridge the gap from NAS research in supervised learning settings to the reinforcement learning scenario through this relatively simple marriage between two different  lines of research , and are one of the first to apply controller-based NAS techniques to RL . We demonstrate the utility  of our method by training combinatorial neural network architectures for RL problems in continuous control, via edge pruning and weight sharing . We also incorporate a wide variety of popular techniques from modern NAS literature , including multiobjective optimization and varying controller methods, to showcase their promise in the RL field and discuss possible extensions . We achieve >90\% network compression for multiple tasks, which may be special interest in mobile robotics with limited storage and computational resources .
We introduce ES-ENAS, a simple yet general evolutionary joint optimization procedure by combining continuous optimization via Evolutionary Strategies (ES) and combinatorial optimization via Efficient NAS (ENAS) in a highly scalable and intuitive way. Our main insight is noticing that ES is already a highly distributed algorithm involving hundreds of forward passes which can not only be used for training neural network weights, but also for jointly training a NAS controller, both in a blackbox fashion . By doing so, we also bridge the gap from NAS research in supervised learning settings to the reinforcement learning scenario through this relatively simple marriage between two different yet common lines of research . We demonstrate the utility and effectiveness of our method over a large search space by training highly combinatorial neural network architectures for RL problems in continuous control, via edge pruning and quantization . We also incorporate a wide variety of popular techniques from modern NAS literature including multiobjective optimization along with various controller methods, to showcase their promise in the RL field and discuss possible extensions .
R
neural architecture search (NAS) algorithm for the purpose of reinforcement learning (RL) policy design, by combining
yet general evolutionary joint optimization procedure by combining continuous optimization via
A
combinatorial optimization via
R
distributed blackbox algorithm , and thus we may simply insert a model controllerfrom ENAS into the central aggregator in ES and obtain weight sharing properties for free
highly distributed algorithm involving hundreds of forward passes which can not only be used for training neural network weights, but also for jointly training a NAS controller, both in a blackbox fashion
A
also
A
yet common
D
, and are one of the first to apply controller-based NAS techniques to RL
A
and effectiveness
R
by training
over a large search space by training highly
R
weight sharing
quantization
D
,
R
and varying
along with various
D
. We achieve >90\% network compression for multiple tasks, which may be special interest in mobile robotics with limited storage and computational resources