ICML 2025

Stochastic Encodings for Active Feature Acquisition
Alexander Norcliffe 1 Changhee Lee 2 Fergus Imrie 3 Mihaela van der Schaar 4 Pietro Liò 1
Abstract
Active Feature Acquisition is an instance-wise,
sequential decision making problem. The aim is
to dynamically select which feature to measure
based on current observations, independently for
each test instance. Common approaches either
use Reinforcement Learning, which experiences
training difficulties, or greedily maximize the con-
ditional mutual information of the label and unob-
served features, which makes myopic acquisitions.
To address these shortcomings, we introduce a la-
tent variable model, trained in a supervised man-
ner. Acquisitions are made by reasoning about
the features across many possible unobserved re-
alizations in a stochastic latent space. Extensive
evaluation on a large range of synthetic and real
datasets demonstrates that our approach reliably
outperforms a diverse set of baselines.
1. Introduction
The standard supervised learning paradigm is to learn a pre-
dictive model using a training dataset of features and labels,
such that the model can make accurate predictions on unseen
test inputs. A fundamental assumption is that, at test time,
all features are jointly available. However, this assumption
does not always hold. Consider the example of a doctor di-
agnosing a patient (Kachuee et al., 2019b;a). Initially, there
is little to no information available and, while there are many
tests that could be conducted, the doctor will choose which
ones to carry out based on their current understanding of
the specific patient’s condition. For instance, if a patient has
pain in their leg, and the doctor suspects a fracture, a leg X-
1Department of Computer Science and Technology, University
of Cambridge, Cambridge, United Kingdom 2Department of Artifi-
cial Intelligence, Korea University, Seoul, Korea 3Department
of Statistics, University of Oxford, Oxford, United Kingdom
4Department of Applied Mathem...