  Neural Architecture Search (NAS) is emerging as a new research direction
which has the potential to replace the hand-crafted neural architectures
designed for specific tasks. Previous evolution based architecture search
requires high computational resources resulting in high search time. In this
work, we propose a novel way of applying a simple genetic algorithm to the NAS
problem called EvNAS (Evolving Neural Architecture using One Shot Model) which
reduces the search time significantly while still achieving better result than
previous evolution based methods. The architectures are represented by using
the architecture parameter of the one shot model which results in the weight
sharing among the architectures for a given population of architectures and
also weight inheritance from one generation to the next generation of
architectures. We propose a decoding technique for the architecture parameter
which is used to divert majority of the gradient information towards the given
architecture and is also used for improving the performance prediction of the
given architecture from the one shot model during the search process.
Furthermore, we use the accuracy of the partially trained architecture on the
validation data as a prediction of its fitness in order to reduce the search
time. EvNAS searches for the architecture on the proxy dataset i.e. CIFAR-10
for 4.4 GPU day on a single GPU and achieves top-1 test error of 2.47% with
3.63M parameters which is then transferred to CIFAR-100 and ImageNet achieving
top-1 error of 16.37% and top-5 error of 7.4% respectively. All of these
results show the potential of evolutionary methods in solving the architecture
search problem.
