  The emergence of CNNs in mainstream deployment has necessitated methods to
design and train efficient architectures tailored to maximize the accuracy
under diverse hardware & latency constraints. To scale these resource-intensive
tasks with an increasing number of deployment targets, Once-For-All (OFA)
proposed an approach to jointly train several models at once with a constant
training cost. However, this cost remains as high as 40-50 GPU days and also
suffers from a combinatorial explosion of sub-optimal model configurations. We
seek to reduce this search space -- and hence the training budget -- by
constraining search to models close to the accuracy-latency Pareto frontier. We
incorporate insights of compound relationships between model dimensions to
build CompOFA, a design space smaller by several orders of magnitude. Through
experiments on ImageNet, we demonstrate that even with simple heuristics we can
achieve a 2x reduction in training time and 216x speedup in model
search/extraction time compared to the state of the art, without loss of Pareto
optimality! We also show that this smaller design space is dense enough to
support equally accurate models for a similar diversity of hardware and latency
targets, while also reducing the complexity of the training and subsequent
extraction algorithms.
