  Realistic use of neural networks often requires adhering to multiple
constraints on latency, energy and memory among others. A popular approach to
find fitting networks is through constrained Neural Architecture Search (NAS),
however, previous methods enforce the constraint only softly. Therefore, the
resulting networks do not exactly adhere to the resource constraint and their
accuracy is harmed. In this work we resolve this by introducing Hard
Constrained diffeRentiable NAS (HardCoRe-NAS), that is based on an accurate
formulation of the expected resource requirement and a scalable search method
that satisfies the hard constraint throughout the search. Our experiments show
that HardCoRe-NAS generates state-of-the-art architectures, surpassing other
NAS methods, while strictly satisfying the hard resource constraints without
any tuning required.
