  Neural architecture search (NAS) methods rely on a search strategy for
deciding which architectures to evaluate next and a performance estimation
strategy for assessing their performance (e.g., using full evaluations,
multi-fidelity evaluations, or the one-shot model). In this paper, we focus on
the search strategy. We introduce the simple yet powerful evolutionary
algorithm of differential evolution to the NAS community. Using the simplest
performance evaluation strategy of full evaluations, we comprehensively compare
this search strategy to regularized evolution and Bayesian optimization and
demonstrate that it yields improved and more robust results for 13 tabular NAS
benchmarks based on NAS-Bench-101, NAS-Bench-1Shot1, NAS-Bench-201 and NAS-HPO
bench.
