  Active learning has been a main solution for reducing data labeling costs.
However, existing active learning strategies assume that a data owner can
interact with annotators in an online, timely manner, which is usually
impractical. Even with such interactive annotators, for existing active
learning strategies to be effective, they often require many rounds of
interactions between the data owner and annotators, which is often
time-consuming. In this work, we initiate the study of one-round active
learning, which aims to select a subset of unlabeled data points that achieve
the highest utility after being labeled with only the information from
initially labeled data points. We propose DULO, a general framework for
one-round active learning based on the notion of data utility functions, which
map a set of data points to some performance measure of the model trained on
the set. We formulate the one-round active learning problem as data utility
function maximization. We further propose strategies to make the estimation and
optimization of data utility functions scalable to large models and large
unlabeled data sets. Our results demonstrate that while existing active
learning approaches could succeed with multiple rounds, DULO consistently
performs better in the one-round setting.
