  The looming end of Moore's Law and ascending use of deep learning drives the
design of custom accelerators that are optimized for specific neural
architectures. Architecture exploration for such accelerators forms a
challenging constrained optimization problem over a complex, high-dimensional,
and structured input space with a costly to evaluate objective function.
Existing approaches for accelerator design are sample-inefficient and do not
transfer knowledge between related optimizations tasks with different design
constraints, such as area and/or latency budget, or neural architecture
configurations. In this work, we propose a transferable architecture
exploration framework, dubbed Apollo, that leverages recent advances in
black-box function optimization for sample-efficient accelerator design. We use
this framework to optimize accelerator configurations of a diverse set of
neural architectures with alternative design constraints. We show that our
framework finds high reward design configurations (up to 24.6% speedup) more
sample-efficiently than a baseline black-box optimization approach. We further
show that by transferring knowledge between target architectures with different
design constraints, Apollo is able to find optimal configurations faster and
often with better objective value (up to 25% improvements). This encouraging
outcome portrays a promising path forward to facilitate generating higher
quality accelerators.
