  In recent years, neural architecture search (NAS) methods have been proposed
for the automatic generation of task-oriented network architecture in image
classification. However, the architectures obtained by existing NAS approaches
are optimized only for classification performance and do not adapt to devices
with limited computational resources. To address this challenge, we propose a
neural network architecture search algorithm aiming to simultaneously improve
network performance (e.g., classification accuracy) and reduce network
complexity. The proposed framework automatically builds the network
architecture at two stages: block-level search and network-level search. At the
stage of block-level search, a relaxation method based on the gradient is
proposed, using an enhanced gradient to design high-performance and
low-complexity blocks. At the stage of network-level search, we apply an
evolutionary multi-objective algorithm to complete the automatic design from
blocks to the target network. The experiment results demonstrate that our
method outperforms all evaluated hand-crafted networks in image classification,
with an error rate of on CIFAR10 and an error rate of on CIFAR100, both at
network parameter size less than one megabit. Moreover, compared with other
neural architecture search methods, our method offers a tremendous reduction in
designed network architecture parameters.
