  Missing information is inevitable in real-world data sets. While imputation
is well-suited and theoretically sound for statistical inference, its relevance
and practical implementation for out-of-sample prediction remains unsettled. We
provide a theoretical analysis of widely used data imputation methods and
highlight their key deficiencies in making accurate predictions. Alternatively,
we propose adaptive linear regression, a new class of models that can be
directly trained and evaluated on partially observed data, adapting to the set
of available features. In particular, we show that certain adaptive regression
models are equivalent to impute-then-regress methods where the imputation and
the regression models are learned simultaneously instead of sequentially. We
validate our theoretical findings and adaptive regression approach with
numerical results with real-world data sets.
