  Neural networks have now long been used for solving complex problems of image
domain, yet designing the same needs manual expertise. Furthermore, techniques
for automatically generating a suitable deep learning architecture for a given
dataset have frequently made use of reinforcement learning and evolutionary
methods which take extensive computational resources and time. We propose a new
framework for neural architecture search based on a hill-climbing procedure
using morphism operators that makes use of a novel gradient update scheme. The
update is based on the aging of neural network layers and results in the
reduction in the overall training time. This technique can search in a broader
search space which subsequently yields competitive results. We achieve a 4.96%
error rate on the CIFAR-10 dataset in 19.4 hours of a single GPU training.
