Evaluation of Active Feature Acquisition Methods under Missing Data
Authors: Henrik von Kleist
Alireza Zamanian
Ilya Shpitser
Narges Ahmidi
Venue: Submitted to ICLR 2023 (2023)
Keywords: Active feature acquisition
active sensing
missing data
reinforcement learning
distribution-shift
off-environment policy evaluation
AFA graph
AFAPE
AFAIS
MCAR
MAR
MNAR
causal inference

Abstract: Machine learning (ML) methods generally assume the full set of features are available at no cost. If the acquisition of a certain feature is costly at run-time, one might want to balance the acquisition cost and the predictive value of the feature for the ML task. The task of training an AI agent to decide which features are necessary to be acquired is called active feature acquisition (AFA). Current AFA methods, however, are challenged when the AFA agent has to be trained/tested with datasets that contain missing data. We formulate, for the first time, the problem of active feature acquisition performance evaluation (AFAPE) under missing data, i.e. the problem of adjusting for the inevitable missingness distribution shift between train/test time and run-time. We first propose a new causal graph, the AFA graph, that characterizes the AFAPE problem as an intervention on the reinforcement learning environment used to train AFA agents. Here, we discuss that for handling missing data in AFAPE, the conventional approaches (off-policy reinforcement learning,  blocked feature acquisitions, imputation and inverse probability weighting (IPW)) often lead to biased results or are data inefficient. We then propose active feature acquisition importance sampling (AFAIS), a novel estimator that is more data efficient than IPW. We demonstrate the detrimental conclusions to which biased estimators can lead as well as the high data efficiency of AFAIS in multiple experiments using simulated and real-world data under induced MCAR, MAR and MNAR missingness.

Under review as a conference paper at ICLR 2023
EVALUATION OF ACTIVE FEATURE ACQUISITION METHODS UNDER MISSING DATA
Anonymous authors Paper under double-blind review
ABSTRACT
Machine learning (ML) methods generally assume the full of features are avail- able at cost. If the acquisition of a certain feature is costly at run-time, one might want to balance the acquisition cost and the predictive value of the feature for the ML task. The task of training an Al agent to decide which features are necessary to acquired is called active feature acquisition (AFA). Current AFA methods, however, are challenged when the AFA agent has to be trained/tested with datasets that contain missing data. We formulate, for the first time, the prob- lem of active feature acquisition performance evaluation (AFAPE) under missing data, i.e. the problem of adjusting for the inevitable missingness distribution shift between train/test time and run-time. We first propose AFA graph, that characterizes the AFAPE problem as an intervention on the envi- ronment used train AFA agents. Here, discuss that for handling missing data in AFAPE, the conventional approaches loff-policy policy evaluation, blocked fea- ture acquisitions, imputation and inverse probability weighting (IPW)) often lead to biased results or are data inefficient. We then propose active feature acquisition importance sampling (AFAIS), a novel estimator that is more data efficient than IPW. We demonstrate the detrimental conclusions to which biased estimators can lead as well as the high data efficiency AFAIS in multiple experiments using simulated and real- world data under induced MCAR, MAR and MNAR missing- ness.
causal graph, the
INTRODUCTION
Machine learning methods generally assume the full set of input features available at run-time at little to no cost. This is, however, not always the case acquiring features may impose significant cost. For example in medical diagnosis, the cost of feature acquisition (e.g. a biopsy test) could include both its monetary cost well the potential adverse harm for patients. In this case, the predictive value of a feature should balanced against its acquisition cost. Physicians acquire certain features via biopsies, MRI scans, lab tests, only if their diagnostic value outweighs their cost or risk. This challenge becomes more critical when physicians aim to predict large number of diverse outcomes, each of which has different sets of informative features. Going back to the medical example, a typical emergency department (ED) able to diagnose thousands of different diseases based on large set of possible observations. For every new emergency patient entering ED with an unknown diagnosis, clinicians must narrow down their search for a proper diagnosis via step by step feature acquisitions. In this case an ML model designed do prediction given the entire feature set is infeasible.
Active feature acquisition (AFA) addresses this problem by designing two AI systems: i) so-called AFA agent, deciding which features must be observed, while balancing information gain vs. feature cost; ii) an MI prediction model, often a classifier, that solves the prediction task based on the acquired set of features. An AFA agent, by definition, induces missingness by selecting only subset of features. We call this AFA missingness which occurs at run-time (e.g. when the AFA agent is deployed at the hospital). In addition, in many AFA applications, retrospective data which use for model training and evaluation also contain missing entries. This is induced by a different feature acquisition process (e.g. by physicians, ordering from a wide range of diagnostic tests). We call this retrospective missingness. While using retrospective data (during craining/evaluattinn only decide among available features. At run-time, however, we make the assumption that the agent
the agent can

---

Under review as a conference paper at ICLR 2023
has the freedom to choose from all features. This corresponds to feature "availability" distribution shift that requires adjustment. Apart from difficulties of training an AFA agent using incomplete data, estimating the real world performance of agents (at run- on-time) using incomplete retrospective data is nontrivial and challenging. Evaluation biases might lead to false promises about the agent's performance, serious risk especially in safety-critical applications.
This paper is the first, to our knowledge, to systematically study the evaluation of AFA agents under this distribution shift. We call this problem active feature acquisition performance evaluation (AFAPE) under missing data. Our work on the AFAPE problem contains 4 major contributions for the field of AFA:
First, we propose the AFA graph, a new causal graph, that characterizes the AFAPE problem under retrospective missingness as off-environment policy evaluation under an environment intervention. This is, in our opinion, very important connection between the fields of missing data and policy evaluation (including reinforcement learning (RL) that will result in cross-fertilization of ideas from these two areas.
Second, we show that the AFA literature only contains approaches to handle missing data that are
either derived from a pure RL perspective (off-polies policy evaluation (OPE) (Chang et al. 2019),
blocking of acquisition actions (Janisch et al., 2020 Yoon et al., (008)) or correspond to biased methods for missing data (conditional mcan imputation (An et al. 2022 Erion et al. 202 Janisch
ct al. 2020). We show that these approaches will almost certainly lead to biased results and/or can be extremely data inefficient. We demonstrate in experiments with exemplary missing completely at random (MCAR), missing at random (MAR), and missing not at random (MNAR) patterns that these biased evaluation methods can lead detrimental conclusions about which AFA agent performs best. This can lead for example to high risks for patients' lives if these methods are deployed without proper evaluation.
Third, we bring the readers' attentions to unbiased estimators from the missing data literature (in- verse probability weighting (IPW) (Seaman White 2013) and multiple imputation (MI) (Sterne) et al. 2009)) which haven't i been applied to AFA previously. These methods, however, do not ac- count for the special structure of the AFAPE problem as not every feature might be acquired by the AFA agent. We show that missing data methods can, therefore, lead to data inefficiency.
Fourth, we instead propose AFAIS (active feature acquisition importance sampling), a new estimator based on the aff-environment policy evaluation view. AFAIS is more data efficient than IPW, but cannot always be used for complex MNAR scenarios. For these cases, we propose modification to AFAIS that allows it to be closer to IPW when required, at the cost of some data efficiency. We demonstrate the improved data efficiency of AFAIS over IPW in multiple experiments.
2 RELATED METHODS
AFA:
Various approaches have been proposed for designing AFA agents and prediction models for
active feature acquisition (AFA) (An al. 20066 Li & Oliva 2021a Li al. 202 Chang al.,
2019 Shim et al. 2018 Yin et al. (020). This work focuses, however, not on any particular AFA method, but on the evaluation of any AFA method under missingness. Nevertheless, we refer the interested reader to Appendix for a more detailed literature review of existing AFA methods and a distinction between AFA and other related fields.
Missing data: AFAPE under missingness can be viewed as missing data problem, and hence methods from the missing data literature can be adopted. There are in general two difficulties for solving missing data problems. The first is identification, i.e. the determination whether the estimand of the full (unknown) data distribution can be estimated from the observed data distribution. The second is estimation, for which there exist generally two strategies which are based importance sampling (IS), i.c. inverse probability weighting (IPW) (Seaman & White| 2013). and multiple imputation (MI) (Sterne al. 2009.. See Appendix A.2 for an in-depth review missing data.
Off-policy policy evaluation (OPE): As we show in Section 3 the AFAPE problem can be formu- lated as aff-policy policy evaluation (Oudk) al. 2011 Kallus & Uchara 2020) problem. The goal in OPE is to evaluate the performance of a "target" policy (here the AFA policy) from data collected under "behavior" policy (here the retrospective missingness induced by e.g. the doctor).
2

---

Under review as a conference paper at ICLR 2023
In this special case, the behavior policy cannot be changed, leading to offline reinforcement learning (Levine et al. 2020). Similar to missing data, there are two general OPE estimation strategies based on IS (Precup et al. (0000 and outcome regression (using (-functions (Munos & Szepesvári (008)). (000))
We show that these approaches are extremely data inefficient AFA settings and can lead to biased results in complex missing data scenarios.
3 METHODS
We open this section by contrasting two views on missingness, which we denote as the set view (which focuses on which set of features was acquired), and the sequence view (which focuses on the order of feature acquisitions). Using the sequence view on AFA missingness, we formulate AFA as a sequential decision problem. We further introduce retrospective missingness in the set view and propose a novel graphical representation, the A/A graph, to describe the AFA problem under retrospective missingness. We continue by formulating the AFAPE problem as a distribution shift problem. This formulation allows to first, discuss different views on the problem, second, study existing solutions, and finally, propose the AFAIS estimator, novel estimator for AFAPE.
3.1 THE SET AND SEQUENCE VIEWS ON MISSINGNESS
The traditional view missingness defines the problem by features X, missingness indicators R and observed proxies X. In AFAPE, we also consider corresponding feature acquisition costs C. We provide glossary in Appendix A.13 to help the reader keep track of the different variables and terms. The relationship between variables in this view is graphically modelled by a missing data graph (m-graph) (Figure [TA] (Mohan et al.) 2013 Shpitser al. 2015). We name this view the set view, as it focuses on the set of acquired features collectively. See Appendix A.2 for an in-depth review on this view on missingness.
A)
B) z
Chess Patn: 1) Tropontre CAG:7
Chest Painc
m-graph (sot view)
Proposit::
g0+0
Troponiar CAG-1
X
Xa
m-graph (sequence view;
cto
R₂
R22
R
PPP
X
giR:
Perform labtest
Perform coronography
Stop obeervation
Figure 1: Missing data graphs for a simplified hospital setting of a heart attack diagnosis: X1: chest
pain; X2: troponin (lab test); X3: coronography (CAG) (invasive imaging). A) m-graph from set view. Presence of chest pain increases the chance for the ordering of a lab test (X, - R2). Based on the results from the lab test or the symptom of chest pain, coronography might be ordered (X1 + R3; X₂ + Ra ; Rs + Rz). The application of coronography produces a large cost Ca due to its invasiveness and radiation. B) m-graph from sequence view, showing (different) MAR missingness policy л (T)(R) ( R(()) FMAR- Based on the acquired features acquisition step k, new observation action (k) is performed (here: (k) (observe(Xxs)))
In order formulate the AFA problem, employ an alternative view missingness, the sequence view, which focuses on the sequence of feature acquisitions (Figure [B). The sequence view arises from inducing a topological ordering m-graphs, as follows. We assume that missingness in- dicator R is formed during sequence of observation actions U (T(1), ..... ((N)), where action (1)) can take on values in ["observe(XX """" """"oeree((Xx"" ..... if l N. The last observation tion is always defined as (N) "stop observation". Thus, R(R) (resp. (())) corresponds to the
3

---

Under review as a conference paper at ICLR 2023
intermediate missingness indicator (resp. intermediate observed proxy) after k steps of observa- tion. We denote the distribution p(t)(k]X, R(()) =(T(k|xx R(*) the missingness policy. In order introduce an AFA policy later, we need to define a restriction to the space of policies as follows. We define the functional space FMAR (for MAR policies TMAR E FMAR) by restrict- ing the missingness policy to be dependent only on data observed up to the current time step k:
THar |X, (t( = TMAR( (0(() X( R(()). This means a MAR policy cannot base acquisition decisions on the values of not (yet) observed features. Note that this is special case of MAR in which observations depend on the earlier variables the m-graph under the sequence view.
3.2 PROBLEM FORMULATION: ACTIVE FEATURE ACQUISITION (AFA)
The AFA problem without retrospective missingness is visualized as causal graph in Figure 2 The goal of AFA is to find i) an optimal AFA agent that decides which features to acquire and ïi) a prediction model which uses the acquired features to perform prediction. We assume here classification task as the subsequent prediction task, but the problem extends naturally to other tasks.
Data (Figure 2 bottom right):
Consider a RDD x (dg + 1) dataset comprised np i.d.. realizations of a de di-dimensional random vector of features X E Rd and 1-dimensional random variable of the label Y E N. We consider X and Y fully observed for now and denote them as the environment of the AFA agent. The AFA graph shows arrow X Y (the features cause the label), but all results in this paper also hold under a reversed causal relationship (X <- Y).
AFA missingness (Figure top left): The AFA missingness is induced by the AFA agent that decides which subset of features ac- quire. Let the missingness indicator R denote the AFA missingness in X (which produces ob- served proxies X). The corresponding policy o (()()) (k(), ((())) is the AFA missingness pol- icy. Our MAR assumption holds for the AFA missingness policy (i.e. π (((()) (x(), f((k))
AFA missingness
Classification
C
C
R
Y
r (( (k)), ((())
FMAR). Further, consider a predefined, fixed acquisition cost (i of ob- serving feature X;. The acquisition cost ran- dom variable Ò represents the cost that the AFA policy exerts when observing features in X: Ó = El= R, (i) We make one important as- sumption about the application of the AFA agent run-iime/deploomment:
target available
X
Y
to be optimized
Data
Figure 2: Causal graph for AFA: The AFA agent decides which of the features (which produces ft) and pays feature acquisition costs Ć. Based on the acquired set of features, the classifier predicts Y and pays a misclassifi- cation cost Ö if Y and Y mismatch.
to acquire
Assumption The AFA agent has full control of missingness at run-time.
Assumption implics that there are no other mechanisms that prevent the feature acquisition when the AFA agent requests the acquisition of feature, i.c. the features are available for the agent, when asked for.
Classification (Figure top right):
Consider predicted label random variable Y S(,,x) f(î, X) that predicted based on the observed features X using classifier f. Further, consider wrong predictions produce misclassificattinn cost Č Cme 1 Y), where Cme (Mt Vmn) N22 + R the cost of predicting label of class y/ as belonging to class ym.
AFA objective:
The goal of AFA is to find missingness policy (0(0 f(0)), parameterized by and classifier fa (R,, X), parameterized by that minimize the total sum of acquisition and

---

Under review as a conference paper at ICLR 2023
misclassification costs C Č + C. The associated objective function is defined as
[ p.
arg min [ 0) arg min E 0.8
(1)
6,8
where J(b 0) denotes the expected total cost after applying AFA missingness policy and classifier. Based on the described sequential nature of the AFA problem, may find optimal AFA agent using a method of choice, for example RL. In this work, however, we focus on the important issue of AFAPE, i.e. the unbiased evaluation of AFA agents (via estimating J) under retrospective missing- ness, which arises when the available dataset contains missingness. For ease of reference, we drop and 0, the arguments of the J notation.
3.3 PROBLEM FORMULATION: ACTIVE FEATURE ACQUISITION PERFORMANCE EVALUATION (AFAPE) UNDER MISSING DATA
The objective of evaluating AFA agents is to estimate the expected cost in Eq. For fully observed dataset, estimation is trivially done by sampling from the causal graph in Figure 2 (i.e. running both the agent and the classifier on the dataset) and averaging the resulting costs. Under retrospective missingness, however, one cannot sample from p(X, Y). some of the features in X are missing. We assume retrospective missingness scenarios without missingness in the label Y and direct influence of Y missingness (i.e. arrow Y - R), but these restrictions can also be relaxed. Note that, in contrast the AFA missingness (denoted by R, Ú and X), the retrospective missingness (denoted by R, U and X) is not necessarily restricted to FMAR-
A possible strategy still the agent under retrospective missingness is to allow the agent to acquire only the available features and block the acquisition action if the feature is not available (i.e. if R; = O, then R₂ 0). This results in a modified causal graph, the AFA graph (Figure 3). which comprises the retrospective missing- ness indicator R as an environment variable. Appendix(A.3 we provide details about sampling from the AFA graph. The arrow R + Å restricts the AFA policy to observe only features that are available. The arrow X - R corresponds the retrospective missingness mechanism which is it- self represented by an m-graph (as described in Figure TA).
AFA missingness
Classification
C
C
+
R
Y
R
X
Y
Retrospective missingnass
Data
Figure 3: AFA graph: Causal graph for AFA setting with retrospective missingness. The dis- tribution shift corresponds to an intervention do(R= II. I).
Assumption T which assumes full control of the AFA agent over the missingness at run-time, im- plies that the AFA agent is allowed to implement causal interventions situation shown Figure where It depends on X. This causal intervention is mathematically ex- pressed as do R = I). As result, we define the problem of active feature acquisition performance evaluation (AFAPE) under missing data as the task of evaluating
Rt This is contrary to the
J Joo(R=D) E [C [C is]
In the causal graph in Figure X a confounder (there exists a path R C) which requires adjustment.
3.4 SOLUTIONS TO AFAPE
We now examine different views and estimation strategies for the AFAPE problem that make use of the different views missingness and the variables in the AFA graph.
5

---

Under review as a conference paper at ICLR 2023
3.4.1 BLACK-BOX MISSING DATA VIEW
A)
B)
We call the first view the black-box missing data view. It corresponds to the traditional view on missingness adjustment from the missing data lit- erature, which hasn't t been applied in AFA set- tings previously. It reduces the AFA graph to the three variables X, R and C, as shown in Figure (A). We denote it as "black-box' since makes no use of the special structure of the AFA prob- lem (knowledge about R) and treats the AFA agent and classifier as a black-box function that maps R and X to C.
R
R
R
Figure 4: Reduced AFA graph for different timation strategies. A) Black-box missing data view. B) Off-environment bandit view.
The black-box missing data view leads to the following IS-based estimator for Jdo(f=iyy called inverse probability weighting (IPW) (Seaman & White 2013):
[ I(R - 1) 0 p = [[x)
= B. C I[ [[R =I
JIPW = E,,x,n
(3)
p(R = 1 Xads
where Xadj is subset of features that blocks all backdoor paths between X and C. Intuition behind IPW:
The importance ratio PiPw pf x
I(R=T) T)
(where p(R = I|x) denotes the propensity score) is derived
from the importance ratio p(X dol R The idea of this IS approach is to sample from the known
p(X It=1)
distribution p(X R 1) instead of the unknown target feature distribution p(X|do(F) T)) and to adjust for the difference using the so-called importance ratio.
Advantage of IPW:
big advantage of IPW is its completeness property, i.e. any function of the target distribution is identified if and only if the propensity score is identified (Nabi et al., 2020). Therefore, if identifi- cation for a particular missingness scenario fails for this approach, no other approach can perform unbiased estimation.
Disadvantage of IPW:
Under IPW, AFA agents can only be tested on the complete subset of the data (the IS distribution is P Ī)). This is severe disadvantage,
agent that never acquires certain feature could nevertheless be evaluated only on data points where that feature has observed entries.
We point out another common estimation strategy in the missing data literature: multiple imputation (MI) (Sterne et al., 2009) which we review in Appendix A.44 This approach can be adopted to over- come the data infffccieccy of IPW. However, it suffers from the disadvantage of potentially very complicated identification step and the requirement for fitting many conditional densities, which increases the risk of model misspecification. In Appendix A.4 also discuss common mistakes such as imputing using only features (instead of features and label) and conditional mean imputation, which both can lead biased results.
3.4.2 OFF-ENVIRONMENT BANDIT VIEW
We now propose second view for AFAPE that leverages the knowledge about the AFA feature acquisitions R (Figure 43). Although AFA was defined as sequential decision problem, the order of acquisitions does not influence the evaluation step as the acquisition and misclassification costs are invariant to the order of acquisitions. We therefore treat the whole set of acquisitions f as one multiarmed bandit (Auer al. 2002). Thus, Rt X and Y can be seen
action and the agent the environment, while R denotes the bandit's action and C the return. We thus take the set view for both the retrospective and the AFA missingness. The AFAPE problem can thus be seen as off-environment policy evaluation of bandit under environment intervention.

---

Under review as a conference paper at ICLR 2023
Based the off-environment bandit view, we propose AFAIS (active feature acquisition importance sampling), a novel unbiased and consistent estimator for J do(ft=T)²
C -1|xadi P2) p(R I[Xaut)
JAPAI = N.0
(4)
The derivation is shown in Appendix A.S
Intuition behind AFAIS:
The term PARMS p( R) is derived from the importance ratio N R=11)
pl Xaa) The idea behind this approach is that we switch the adjustment from different feature distributions (as in IPW) to an adjustment for different AFA feature acquisition distributions. The problem mains that cannot sample from the policy p( p[R] R 1) for every data point because we do not have certain features (and have to block those actions). Instead, we sample from Xad) ER.x., p( P(R) Xadj: X R)p(R, (aat)) which is possible to do over all data points (not just fully observed data points). The variable set add corresponds to the subset of features not Xadj-
p(f=
Advantage of AFAIS:
The obvious and main advantage of AFAIS that it uses all data points instead of only complete cases (as in IPW) which increases data efficiency. However, this advantage diminishes in case of "data hungry" agents, i.c agents that often acquire relatively large feature sets. For instance, the AFA agent always acquires all the features, i.e. p(A = X R = I) = VXadj. then the numerator, which we denote as the AFA propensity score, p(R X aujj R. = 0 if R + 1 and AFAIS and IPW will have equal weights.
Disadvantage of AFAIS:
One disadvantage of AFAIS is that requires the fitting of a second function, the AFA propensity score p(R = Xadj, Rt). which increases the chance of model misspecification. Additionally, identi- fication is more demanding for AFAIS. In Appendix A.6 show the identifiability of Jarus under MCAR and simple MAR missingness scenarios, and show how it cannot be evaluated in an MNAR setting.
3.4.3 THE AFAIS-IPW SPECTRUM
AFAIS and IPW can be viewed as two ends of spectrum. In IPW, we only use the complete cases, while AFAIS we use all data points with all missingness patterns. A tuning parameter that allows us to gradually move on this spectrum toward each approach, helps overcome the disadvantages of both (low data efficiency for IPW and problem of evaluation under MNAR missingness for AFAIS). We propose the modified AFAIS estimator as:
P p(R [|xad R. R. İ)I(R, 1)
JAPAIS(5) = E R.
(5)
p(
mdj)
for any desired index subset S -0,..., d.). The derivation is shown in Appendix A.7
Intuition behind the modified AFAIS:
The idea behind this approach is to sample only from data points where important set of features (the features in 8) are always observed while using the off-environment bandit approach for the remaining features. To showease the advantages of using the modified AFAIS, demonstrate in Appendix [A.8] that the modified AFAIS is identified for the MNAR example from Appendix [A.6] and that can be evaluated when appropriate conditioning general identifiability statements for the modified AFAIS estimator, but suggest simple procedure for choosing the set in Appendix A.8
is chosen. We do not give
3.4.4 OTHER VIEWS ON SOLVING AFAPE AND DATA EFFICIENCY
Other views AFAPE include off- aff-policy policy evaluation (OPE) which has been used in the AFA literature before, but very data inefficient in the context AFA and can lead to biased results in complex missing data scenarios. Another approach used in the AFA literature is to just block

---

Under review as a conference paper at ICLR 2023
A)
Different agent: 10%
B)
Different estimators agent:
Jarws
Jasai5
Jow
10"
Lew
Je
Evaluation enorr
Evaluation emor
Imo
00-1
30303
20000
50000
40000
50030
5ODE0
70000
10003
20303
33030
45000
50000
00030
70303
Number of datapoints Ri
Number of datapoints no
Figure 5: Error of expected cost estimation function of the number of data points in synthetic data experiments with MNAR missingness. Plots show two random agents that acquire each costly feature with a probability of A) 10% and B) 90%. Error is defined as the difference between an estimation and the ground truth estimator do( evaluated on the whole dataset (nD 100000).
the acquisitions without adjustment which thus also leads to biased results. We review these two approaches in Appendix A.9 In Appendix AA.10 we quantify the improved data efficiency of the off-environment bandit view over the black-box missing data and OPE views. We also show that the improved data efficiency leads to weaker positivity assumptions required for the off-environment bandit view compared to the other views.
EXPERIMENTS
4.1 EXPERIMENT DESIGN
We evaluate different estimators on synthetic and real world datasets (HELOC (FICO 2018). Retinopathy (Antal & Hajdul 2014), and Income (Newman al. 1998)) under examples syn- thetic MCAR, MAR and MNAR retrospective missingness. We evaluate random AFA policies and vanilla Deep Q-Network (DQN) RL agent (Mnih al. 2015) as AFA agents and use a random forest classifier. We compare the following estimators:
Jimp uses Missforest Stekhooenn & Bühlmann, 2012) to impute missing features.
JR blocks the acquisitions of not available features, but offers no correction.
JR- corresponds complete case analysis and is only unbiased under MCAR.
JJew the stabilized IPW estimator. Stabilization is achieved by normalizing the weights which reduces variance at the cost of small bias (Rubinstein & Kroese 2016).
JAFAIS is the stabilized (modified) AFAIS estimator.
Jao(R=1) is the ground truth, where the agent is run on the fully observed dataset.
We fit the propensity score and AFA propensity score using multi-layer perceptrons. Complete experiment details are given in Appendix A.11.
4.2 RESULTS
Figure SesureBlden desuics depicts convergence rates of different estimators for the MNAR synthetic data experiment for two random policies as a function of evaluation sample size. The results show that the three knowingly-biased estimators Jmmp. JR and J f--r" do not converge correct solutions as the sample size increases. The stabilized AFAIS (and IPW) estimators converge to the correct solution, but at different rates depending costly features, the stabilized AFAIS estimator JAPUS is converging almost as fast as the ground truth J Ro (Figure 5 A). In the case where the agent acquires 90% of the data, the AFAIS and IPW estimators converging similar rate, indicating there is not much benefit of using
the policy. In the case of policy that only acquires 10% of the
8

---

Under review as a conference paper at ICLR 2023
AFAIS over the conventional IPW estimator (Figure 5 B). The increased data efficiency of AFAIS illustrated in Figure on real-world data experiments where it leads to tighter tighter boxplots for AFAIS compared IPW. This benefit is significant for agents that do not acquire much data (i.e. the random 10% agent), and diminishes as the agent becomes more "data hungry". 'he experiments results in Figure further demonstrate the large errors biased estimators, such Jimp- JR and R-T can produce and the potentially wrong conclusions these estimators can lead to. If the policies are compared according to Impp. for example, one would conclude the differences between the random policies in the HELOC dataset are large (Jimp ranges between 6.5 and 8), when in fact they are only small (J A close 6.5 for all random policies). Further experiment results are shown in Appendix A.12
HELOC (MCAR missingnessi
Rctinopathyy (HAR missingness)
Income INNAR missingnesd)
Juras
Jane
Jow
Jee
feet-1
Figure 6: Results for the A) HELOC, B) Retinopathy, and C) Income datasets for different miss- ingness scenarios. Boxplots show variability of the estimate when bootstrapping 300 times without replacement 70% of a total test dataset of size DD (ND = 17'781 for HELOC, D.D.... =575 for Retinopathy and D 161 for Income).
DISCUSSION AND FUTURE WORK
The central argument of this paper is based on Assumption [ the full control of the AFA agent over the missingness at run-time. This assumption might, however, always hold in practice (e.g. due to the breaking of a sensor or a patient who is not cooperative). In this case, a different distribution shift for R should be assumed for evaluation.
In general, the choice of view and estimator for AFAPE must depend on the following questions: 1) Can identification be performed? The identification step might have varying degrees of difficulty depending on the underlying m-graph and the chosen view/ estimator. Identification is for example easier for IPW than for AFAIS.
2) Which estimator is most data efficient?/ Do all positivity assumptions hold? In this paper, we demonstrate that the off-ennironment bandit view more data efficient than the black-box missing data and OPE views. Furthermore, it requires the weakest positivity assumptions.
3) Can the nuisance functions be fitted accurately? The nuisance functions (c.g. the AFA propensity score) need to be fitted based the available data. Fitting the AFA propensity score can thus pose as an additional source of error of the AFAIS estimator that the IPW estimator does not have.
We consider the following suggestions major extensions of this work: i) studying the evaluation of AFA agents under missingness in the temporal setting with time-series features; ii) including the counterfactual expected cost J fo in the training of AFA agents and classifiers.
6 CONCLUSION
We study the problem of active feature acquisition performance evaluation (AFAPE) under missing data which corresponds to an intervention on the enviromment used to train AFA agents. We propose new causal graph, the AFA graph, and demonstrate that current evaluation methods for the AFAPE problem can be strongly biased leading to high risks for model deployment. We show that AFAIS, a novel ba-ed estimator is unbiased estimator for evaluation, and more data efficient than IPW.

---

Under review as a conference paper at ICLR 2023
REFERENCES
Chaojie An, Qifeng Zhou, and Shen Yang. A reinforcement learning guided adaptive cost-
sensitive feature acquisition method. Applied Soft Computing, pp. 108437. January 2022. ISSN 15684946. doi: 10. 20.10016f.as00.2222..00000
URL https: httes////iniinnghu.... elsevier.
com/ http://ee2eeee111115644492220000
Qi An, Hui Li, Xuejun Liao, and Lawrence Carin. Active feature acquisition with POMDP models.
Submitted to Pattern Recognition Letters, 2006.
Bálint Antal and András Hajdu, An ensemble- based system for automatic screening of diabetic retinopathy. Knowledge-bard based systems, 60:20-27, 2014.
Peter Auer, Nicolò Cesa-Bianchi, Yoav Freund, and Robert E. Schapire. The Nonstochastic Multi- armed Bandit Problem. SIAM Journal Computing, 32(1):48-77, January 2002. ISSN 0097- URL (RLpstp::: epubs, s Lam. org/ /tt///oi// /bs//bb/ wtw.137/S00975550 00.1337780091555971338883 Publisher: Society for Industrial and Applied Mathemat-
5397. doi: 10.1 00.11377/000975330019997777
ics.
Christian Beyer, Maik Büttner, Vishnu Unnikrishnan, Miro Schleicher, Eirini Ntoutsi, and Myra Spiliopoulou. Active feature acquisition on data streams under feature drift, An- nals of Telecommunications, (5(9-10):597-111 October 2020. ISSN 0003-4347, 1958-9395. doi: 10..0007/1122444-0000077777 s 912243-020-00713
URL https: linki springer com/ 10. 10007/
Rohit Bhattacharya, Razieh Nabi, Ilya Shpitser, and James M. Robins. Identification In Missing
Data Models Represented By Directed Acyclic Graphs. In Proceedings of The 35th Uncertainty
in Artificial Intelligence Conference. pp. 1149-1158, PMLR, August 2020. URL https://
proceedings mwr. wwwppp/sssww...attteeee /
Chun-Hao Chang, Mingjie Mai, and Anna Goldenberg. Dynamic Measurement Scheduling for Event Forecasting using Deep RI. Proceedings of the 36th International Conference on Machine Learning. pp. 951-960. PMLR, May 2019. URL htttp::/pprcceeeeeoeeooo http:/pooeeediggg// mlr.
press/v97 ht///changggg/ html
Srijita Das, Rishabh Iyer. and Sriraam Natarajan. A Clustering based Selection Framework for Cost Aware and Test-time Feature Elicitation. In 8th ACM IKDD CODS and 26th COMAD, pp. 20-28. ACM, Jamuary 2021. ISBN 978- 4507--8-500-- 34311005
doi: 10.1 1411144430984433310000 URL.
https://di acm www.://///oooooooooo 1145//3...1144333
Miroslav Dudik, John Langford, and Lihong Li. Doubly Robust Policy Evaluation and Learning, May 2011.
httttp://axxiv..rg//
4601 urXiv:103.4660 [cs, stat].
Gabriel Erion, Joseph D. Janizek, Carly Hudelson, Richard B. Utarnachitt, Andrew M. McCoy, Michael R. Sayre, Nathan J. White, and Su-In Lee. CoAl: Cost-Aware Artificial Intelligence for Health Care. Technical report, medRxiv, January 2021. URL
wwwreeiddvvooo org/
content/10. 1t1p://00.111/222111111 01. 19. 21249356v1
FICO. FICO Explainable Machine Learning Challenge. Maachnne-eammnngchhlll
May 2018. URL htttps://commniit: fico. com/s/
expllainable-machie wachinachheeeeeeaaa leaarning challaaaaocc.. lhalgng-chllleeeeeeeeeee Last accessed May 2022.
Wenbo Gong, Sebastian Tschiatschek, Sebastian Nowozin, Richard E Turner, José Miguel Hernández-Lobato, and Cheng Zhang. Icebreaker: Element-wiss Efficient Information Acqui- sition with Bayesian Deep Latent Gaussian Model. In Advances in Neural Information Process- ing Systems, 2019. URL http:ssspppppeeeiigsssooooo
wttpaarw..ppppp.peeeeeeee cc/paperr/2119/haa. htm]
Sheng-Jun Huang, Miao Xu, Ming-Kun Xie, Masashi Sugiyama, Gang Niu, and Songcan Chen. Active Feature Acquisition with Supervised Matrix Completion. Proceedings of the 24th ACM SIGKDD International Conference Knowledge Discovery & Data Mining, pp. 1571-1579, doi: 191111//291999.22222222 URL https: :/tp::://d....
July 2018. ISBN 978-1-4503-5552-4 3220084
/101//00.111//// acm. htt..org/doo/11.1145553
10

---

Under review as a conference paper at ICLR 2023
Jaromír Janisch, Tomáš Pevný, and Viliam Lisý. Classification with costly features sequential decision-making, problem. Machine Learning, 109(8):1587-1615, August 2020. ISSN 0885- doi: com/ 1.0011077 101000011111 0 94 -20---000----------
6125, 1573-0565.
URL http://link springer.
Daniel Jarrett and Mihaela van der Schaar. Inverse Active Sensing: Modeling and Understanding Timely Decision-
1411v [cs, stat], June 2020. URL http://arxiv.
axg/abs/2000 14141
Nathan Kallus and Masatoshi Uehara. Double Reinforcement Learning for Efficient Off-Pooicy Evaluation in Markov Decision Processes. Journal of Machine Learning Research, 21(167):1- 63, 2020. ISSN 1533-7928.
wttp:oopppppppe.e/e//e///m 827 .ttll
Edward H. Kennedy. Semiparametric doubly robust targeted double machine learning: review. URL http://arii..orr/bb//200...
arXiv:2203.06 06469 [stat], March 2022.
arXiv: 2203.06469.
Thomas T. Kok, Rachel M. Brouwer, Rene M. Mandl, Hugo Schnack, and Georg Krempl. Active Selection of Classification Features. arXiv:2/02. 13636 [cs], February 2021. URL http://
arxiv. ww..orggaaa abs//bb/// 221221122.... 13636
arXiv: 2102.13636.
Marine Le Morvan, Julie Josse, Erwan Scornet, and Gael Varoquaux. What's good imputation to predict with missing values? In Advances in Neural Information Processing Systems, vol- ume 34, pp. 11530-11540, 2021. URL httpt:pppppp/ppptpttteeteeeeeeet www.nngg.....rrrueeeeeeeeeeeweee neur psrrips.com http:sscc.u.r.pccccpppppeee cc/paper/
Sergey Levine, Aviral Kumar, George Tucker, and Justin Fu. Offlin Reinforcement Learning: Tutorial, Review, and Perspectives Open Problems. arXiv:2005.0164) [cs, stat], November 2020.
61643
arXiv: 2005.01643.
Yang Li and Junier Oliva. Active Feature Acquisition with Generative Surrogate Models. In Pro- ceedings of the 38th International Conference on Machine Learning, pp. 6450-6459, PMLR, July 2021a. URL 3498.
ngs mllr wttppppsss/////ee/ee/e ht.l ISSN: 2640-
Yang Li and Junier B. Oliva. Dynamic Feature Acquisition with Arbitrary Conditional Flows. arXiv:2006.07701 [cs, stat], March 2021b. URL/attp://azrt org /00pg//aaab0000070011
Yang Li, Siyuan Shan, Qin Liu, and Junier B. Oliva, Towards Robust Active Feature Acquisition.
2007::2107.04163 [cs], July 2021. URLhttp: patttp://ariv... http://wxxvv.r........ wwtp://gg////a.a/.oooo/...... abs 2217 04163
Charles X. Ling. Qiang Yang, Jiamning Wang, and Shichao Zhang. Decision trees with mini- mal costs. In Twenty-first international conference Machine learning ICML '04, pp. 69, 2004. doi:
URI. htttp://ports: www...aaaaaaaaaaooo.m......oooma
.acm wrtp://t///cittt....o/mm.
cfm?do 1=1 01 5330 1015369
Roderick JA Little and Donald B. Rubin. Statistical analysis with missing data, volume 793. John Wiley & Sons, 2019.
Chao Ma. Sebastian Tschiatschek, Konstantina Palla, Jose Miguel Hernandez-Looate Sebastian Nowozin, and Cheng Zhang. EDDI: Efficient Dynamic Discovery of High- Value Information with Partial VAE. In Proceedings of the 36th International Conference on Machine Learn
ing, pp. 4234 4243. PMLR, May 2019. URL htthhtt::: 1 9c. html
ml
Daniel Malinsky, Ilya Shpitser, and Eric J. Tchetgen Tchetgen. Semiparametrie Inference for Non-monotone Missing-Nott -ot-at-Rannde Data: the No Self Sels-Cnns Model. 1XXix:999900144 [math, stat]. May 2020.
org abs/1909. htt.:////gaabbbbb...b 01848 arXiv: 1909.01848
Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Andrei A. Rusu, Joel Veness, Marc G. Bellemare, Alex Graves, Martin Riedmiller, Andreas K. Fidjeland, Georg Ostrovski, Stig Pe- tersen, Charles Beattie, Amir Sadik, Ioannis Antonoglou, Helen King, Dharshan Kumaran, Daan Wierstra, Shane Legg, and Demis Hassabis. Human-level control through deep reinforcement learning. Nature. 518(7540):529- 533, February 2015. ISSN 0028-0836. 1476-4687, doi: 4236
10. (0338/nauure12236 URL http: www. nature. www wwtpeeetea/ettteeeeeeeeeeaae

---

Under review as a conference paper at ICLR 2023
Karthika
Mohan,
Judea Pearl, and Jin Tian. Graphical Models for Inference with Advances in Neural Information Processing Systems, volume 26, URL
Missing Data.
2013.
http:://poceeeeigg.... DFf80333f9947621330 3937633311 c4c455 -bwp::llllccc5St5tas..areeee ract. htmml
Rémi Munos and Csaba Szepesvári. Finite- Fime Bounds for Fitted Value Iteration. Journal of Machine Learning Research, 9(5), 2008.
Razieh Nabi, Rohit Bhattacharya, and Ilya Shpitser. Full Law Identification In Graphical Models Of Missing Data: Completeness Results. arXiv:2004.04872 [es, stat], August 2020. URL http:
Sriraam Natarajan, Srijita Das, Nandini Ramanan, Gautam Kunapuli, and Predrag Radivojac. On Whom Should I Perform this Lab Test Next? An Active Feature Elicitation Approach. In Proceedings of the Twenty-Seventh International Joint Conference Artificial Intelligence, pp. 3498-3505, Stockholm, Sweden, July 2018. International Joint Conferences on Artificial Intelligence Organization. ISBN 978-0-9992411-2-77 doi: 10.24963/jjcai
URL
https:// wwww jjaai. wtt/proceecingg//211..
D. J. Newman, S. Hettich, C. L. Blake, and C. J. Merz. UCI Repository of machine learning databases, 1998. URL http: ics. uci http:///mmmaaee///eeessaaoorrrrr html
Judea Pearl. Causality. Cambridge university press, 2009, ISBN 978-0-521-89560-6
Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Pretten- hofer, Weiss, V. Dubourg, Vanderplas, Passos, D. Cournapeau, M. Brucher, M. Perrot, and E. Duchesnay. Scikit-learn: Machine Learning in Python. Journal of Machine Learning Research, 12:2825-2830, 2011.
Doina Precup, Richard S. Sutton, and Satinder Singh. Eligibility Traces for Off-Policy Policy Eval- uation, 2000.
Reuven Y Rubinstein and Dirk Kroese. Simulation and the Monte Carlo method. John Wiley & Sons, 2016.
Shaun R. Seaman and Ian R. White. Review of inverse probability weighting for dealing with missing data. Statistical methods medical research, 22(3):278-29) 295, 2013. Publisher: Sage Publications Sage UK: London, England.
Burr Settles. Active learning literature survey. 2009.
Victor S. Sheng and Charles X. Ling. Feature value acquisition in testing: a sequential batch test algorithm. In Proceedings of the 23rd international conference on Machine learning, pp. 809- 816, 2006.
Hajin Shim, Sung Ju Hwang, and Eunho Yang. Joint Active Feature Acquisition and Classi- fication with Variable-Size Set Encoding. Advances in Neural Information Processing Sys- tems, 31, 2018. URL ott.aaaae222222222222222-a-aaaaa 6bbe -
neurip cc/ pttp//pppe/////aaai.. html
Ilya Shpitser, Karthika Mohan, and Judea Pearl. Missing data as a causal and probabilistic problem. In Proceedings of the Thirty-First Conference on Uncertainty in Artificial Intelligence, UAI' pp. 802-811, Arlington, Virginia, USA, July 2015. AUAI Press. ISBN 118-0-9966431-0--8
Daniel J. Stekhoven and Peter Bühlmann. for mixed-type data. Bioinformatics, 28(1): 12--118, 2012.
nan-Forrst-non-prrameette missing value imputation
Jonathan A. C. Sterne, Ian R. White, John B. Carlin, Michael Spratt, Patrick Royston, Michael G. Kenward, Angela M. Wood, and James R. Carpenter. Multiple imputation for missing data in epidemiological and clinical research: potential and pitfalls. BMJ, 338:b2393, June 2009, ISSN 0959-8138, 1468-5833. doi: 10.1136/bm 1..11366mjj.bb b23/93m..2233,, UtL www. Jom com/ content/
338 108/bbm) /bmj b2393] Publisher: British Medical Journal Publishing Group Section: Research Methods &amp; Reporting.
12

---

Under review as a conference paper at ICLR 2023
Fengyi Tang, Lifan Zeng. Fei Wang, and Jiayu Zhou. Adversarial Precision Sensing with Healthcare Applications. In 2020 IEEE International Conference on Data Mining (ICDM), pp. 521-530, November 2020. doi: (0.1109/CCDM50008.22000 202.00061..
Stef van Buuren. Multiple imputation of discrete and continuous data by fully conditional specifica- tion. Stulistical Methods in Medical Research, 16(3):219-242, June 2007. ISSN 0962-2802. doi: 10.1177096228220077446
URL / 0ttp://0..1110000222222222202220
Thomas S. Verma and Judea Pearl. Equivalence and Synthesis of Causal Models. Technical Report R-150, Department of Computer Science, University of California, Los Angeles, 1990.
Xiaoyong Chai, Lin Deng, Qiang Yang, and C. X. Ling. Test-cost sensitive naive Bayes classifica- tion. In Fourth IEEE International Conference on Data Mining (ICDM 04), pp. 51-58, November 2004. doi: 10. 1..110//CDMM.000 2004. 10092.
Haiyan Yin, Yingzhen Li, Sinno Jialin Pan, Cheng Zhang, and Sebastian Tschiatschek. Reinforce- ment Learning with Efficient Active Feature Acquisition. 2020. //arxiv orq/ abs////////////// / 2010.11111100 C0825
arXiv:2011.00825 [cs], November
Jinsung Yoon, William R. Zame, and Mihaela Van Der Schaar. Deep sensing: Active sensing using multi-directional recurrent neural networks. In International Conference on Learning Represen- tations, 2018
Jinsung Yoon, James Jordon, and Mihaela Schaar. ASAC: Active Sensing using Actor-Critic models. In Machine Learning for Healthcare Conference, pp. 451 1733. PMLR, October 2019. URL ISSN: 2640-3498.
.mlr
9a. html
Pin Zhang. A novel feature selection method based on global sensitivity analysis with application in machine learning-basse based prediction model. Applied Soft Computing, 85:10585,, 2019. Publisher:
Elsevier.
A APPENDIX
A.1 LITERATURE REVIEW FOR ACTIVE FEATURE ACQUISITION
Active feature acguisition (AFA) An et al. 2006 Li & Oliva 202 Li ct al.
2021 Chang et al.,
2019 Shim et al. 2018 Yin et al.
2020) has been studied under various other
names including,
but not limited to, active sensing" Yoon et al. 2019 2018 Tang et al. 2020
Tarrett & van der
Schaar 2020). "active feature elicitation" (Natarajan an et al. 2018 Das al. 2021), 1. ddynamic feature
acquisition" (Li & Oliva 2021b), "dynamic active feature selection" " (Zhang) 2019), "element-wise
efficient information acquisition" (Gong et al. 2019). "classification with costly features" (Janisch) eet al. 2020) and "test-cost sensitive classification' (Xiaoyong Chai et al. 2004). AFA is different from active learning (Settles) 2009). In active learning, one assumes a classification task with a training data set that contains many unlabeled data points. The active learning task is then to decide which label acquisitions will improve the training performance the most. Similar research also exists for the acquisition of features for optimal improvement of training. This task has been referred to as "active selection of classification features" (Kok et al. (2021), and unfortunately also as "active feature acquisition" (Huang et al., 2018 Beyer et al. 2020), but its objective differs fundamentally from ours. Huang et al. (2018) 8) attempt to find out which missing values within the retrospective data set would improve training the most when retroactively acquired. In this paper, we are, however, interested which features, for a new data point, would improve the individual prediction for that data point the most.
Many AFA approaches are based on greedy feature acquisition strategy wrapping subsequent classification task. An idea is to employ decision tree classifiers and to acquire features sequentially
by traversing the branch of the decision tree (Ling et al., 2004 Sheng & Ling 2006), while the splitting criteria minimizes the combined cost of feature acquisition and misclassification (Ling let al., 2004). Another approach, the test-cost sensitive Naive Bayes (csNB) classifier (Xiaoyong Chai et 20.42 exploits the Naive Bayes assumption of independence among the predictive power of features. This allows for efficient exploration whether acquiring certain feature can reduce
13

---

Under review as a conference paper at ICLR 2023
costs. Das al. (2021) propose clustering- based cost-aware feature elicitation (CATE). In CATE, data points are clustered based on a set of zero-cost features and the optimal, fixed set of features is computed for each cluster. A new partially-obsery
data point is then attributed to a cluster, and the corresponding optimal feature set is acquired for it.
The AFA problem is inherently sequential decision process which one can address using rein- forcement learning (RL). Model-based RL. approaches learn the state-transition function which, for AFA, corresponds to learning an imputation model (Yoon et al.) 2018: Yin et al. 2020 Li & Oliva 202 Ma et al. 2019). The imputation model is used at run-time to simulate possible outcomes of a feature acquisition and derive desired acquisition strategies. Alternaiiely, model-free RL ap- proaches do not require learning a pate-transition function. One variant, Q-learning, relies on mod- eling the ex expected cost of a particular acquisition decision (Chang et al. 2019] Janisch et al. 2020) (Shim et al. (2018) use double Q-learning for the AFA agent with a deep neural network that shares network layers with the subsequent classification neural network.
Shim et al. 2018). As an (xample,[Shim
A.2 MISSING DATA MODELS
Missing data problems relate the full data distribution p(X) and the observed data distribution where variables in may sometimes be unobserved. Each X; E X observed if the corresponding miss- ingness indicator R; assumes value 1. Thus, the observed version of X; which we denote by X; defined to be equal to X, if R, = 1, and is missing (equal to a special value "?") otherwise. We further consider acquisition cost C; for the feature X,. The cost C' is thus deterministically defined by the missingness indicator R. The goal in missing data problems is to estimate some func- tion of p(X), such parameter predictive model, using the i.i.d. samples from the observed distribution p(X, R). where X is the set of all observed proxies, and R is the set of all missing- ness indicators in the problem. Parameters of interest must be identified, i.e. unique functions of p( XX R), in order to make the estimation problem well-posed.
The problem of identification is to determine whether not a function of the full data can be recovered from the observed data distribution. Identification is not possible in general, and relies on assumptions encoded in a missing data model. The simplest identified missing data model is the missing completely at random (MCAR) model, which assumes that p( [[] = ((). The missing at random model (MAR) assumes that missingness only depends on observed features: p(R= F| = p(R = FK[X; : F; = 1)). Finally, any model that is neither MCAR nor MAR is called missing not random (MNAR) where missingness can now depend also on feature values that are unobserved. While parameters of interest are not identified in general under MNAR, large class of identified MNAR models has been derived in the missing data literature (Bhattacharya et al.) 2020 Nabi et al. 2020).
In this paper, we will restrict attention to a special type of missing data models where identifiability restrictions represented by directed acyclic graph (DAG) factorization of the distribution. The restrictions are then formalized as p(X, RR = lIvEXUR P( V ||Ip (V)) for some graph g where (V) selects the parents of the node V in G. The graph G is termed the missing data graph (m-graph) (Mohan et al.) 2013) Shhptterr et al.) 2015) (Figure [A]. In models of missing data, the identification problem for parameters in p(X) from p(X, Rt) may be formulated as an identification result of applying the intervention operator do(.) PParl 2009) to every missingness indicator R, hence p(X) = p(X |do R = Τ)) = x{X, R= In other words, the problem of identification of p(X) is translated to the problem of identification of p( R = T..). If the missing data model is graphical, techniques from causal effect identification theory may be applied to obtain identification of p(X) (Bhattacharya et al. 2020 Nabi al. 2020
problem of causal inference, where p(X) is obtained
A.3 SAMPLING FROM THE AFA GRAPHI
In this section, we provide the details of factorizing the joint distribution over the variables of the AFA process with respect the AFA graph (Figure(3) into distributions from which we can sample. The joint distribution factorizes as
P R. (CC.Y/Y)CCCRR))YXXX.RAA R., R),(RXX,Y)
(A.1)
14

---

Under review as a conference paper at ICLR 2023
AFA missingness
Classification
((K-1)
(CA)
C
2P-1
g(++1
2
Y
Tj(k)
policy restriction
target
available
to be optimized
R
X
Y
not available
to be intervened
Retrospective missingness
Figure A.1: AFA graph from AFA missingness policy viewpoint. The node X represents the intermediate states at step & of the decision process. The state also contains R which was left out for clarity. The AFA policy (0 R()), R) is restricted by R only allow the acquisition of available features, i.c. π T(((k) "observe(X.)" ( R()), R r) 0 if
The sampling procedure thus is defined as follows. First, we sample from the retrospective dataset p( R, X, Y) (i.e. the "environment"). Next, AFA missingness p( (R] R) and the (deterministic) acquisition cost p(C|R) conditional distribu- tions. As AFA missingness is not modelled from a set viewpoint (i.e. we cannot directly sample X R)), we sample equivalently from the corresponding AFA policy π (Ū ((((k))))))) ft( Rt). To visualize this process, we present Figure A..1 the AFA graph from an AFA missingness sequence perspective. Sampling R now corresponds to sampling Ú. We block feature acquisition actions the desired feature not available (i.e. p(A = f|X, R = r) = 0 A = and r; = 0). Finally, apply the (deterministic) classifier and compute the misclassification cost (i.e. "sample" from p( p R)) and ((CYY
the AFA agent. This gives samples from the
A.4 MULTIPLE IMPUTATION (MI) FOR THE AFAPE PROBLEM
An alternative estimation strategy to IPW based the missing data view is multiple imputation (MI). The core idea of imputation to fill (impute) missing entries and then assume fully observed dataset. The procedure can be formulated (Little & Rubin 2019
J Joc(R = Ex,y [4(X, Ex,y [EEx. [h(X, Xmises Y)I]
(A.2)
ΣΣ> 1(X, Xmise, V)p( Xxiis X.Y))((,
X,Y X
where 4(X, = C is the black-box function mapping the data to the costs and X Xmiss represents the unobserved data. Imputation thus corresponds Monte Carlo integration by sampling observed values p(X, X, Y) and imputing missing values using imputation model p(Xmixrl X, Y). To increase accuracy of the estimator, the imputation can be repeated multiple times leading to the MI proce- dure. In the identification step, one determines whether the imputation model can be estimated from retrospective data.
The first drawback of imputation is the requirement of modeling joint distributions which is in practice a complex problem, especially in high-dimensional settings and under complex missingness patterns. As example, the multiple imputation by chained equations (MICE) method (van Buuren 2007) requires fitting K conditional densities for K partially observed features. By comparison, IPW requires only the propensity score, which is often much easier to specify.
A.2 implies imputation by conditioning on the features X and the label Y. This introduces the risk of data leakage, as the imputed features may carry predictive information not because of the true data generation mechanism, but because of the imputation, resulting in potentially overoptimistic estimation of prediction performance. A common alternative, often used in MI., is to impute the data
15

---

Under review as a conference paper at ICLR 2023
without conditioning on Y. This assumption, however, implies that missing feature X, E Xmiss conditionally independent of the label given the observed features (X; TL Y(X). Determining marginal predictive value of feature for predicting Y, is however, the whole task of AFA, which renders this approach impractical.
A simplified imputation approach that reduces the complexity of modelling and that has been applied in AFA settings (An et al. 2022] Erion et al.) 2021] Janisch et al.] (020) is condi-
tional mean imputation. The missing values
thereby imputed using a conditional mean
model for E[X E[Xmiss| [X]. Conditional mean imputation assumes Ex,y [Ex [ Xmiss, Y)]
Ex,y [N,X, Ex,a [imsss Xx, Y)], which does not hold in general and can lead to strongly biased results when is nonlinear as the case in AFA settings.
A.5 DERIVATION OF ACTIVE FEATURE ACQUISITION IMPORTANCE SAMPLING (AFAIS)
The AFAIS estimator is based on IS, a popular method in OPE and missing data to adjust for distri- bution shift. IS assumes a problem setting where one has only access to samples from "behavior" distribution, while the goal is the evaluation of a "target" distribution. This is made possible when knowledge of the ratio of the two distributions (importance ratio) is available. In the setting of this paper, the target distribution is the distribution of C under the intervention do i). The derivation AFAIS is based on the reduced AFA graph from Figure 4 which results from a latent projection (Verma & Pearl 1990) of the full AFA graph from Figure 3 This reduces the graph factorization from equation Eq. (A.1) only the features C, R. R and X. We further consider only a subset of the features Xau that blocks all backdoor paths from R to R and C. Since this yields causal model of a DAG, we can apply the truncated factorization, or the g-formula, to obtain:
p(C, R, Xau) do = I)) = p(C) R)p( Xadj R = (X(()
(A.3)
We propose the following behavior distribution:
P(C, R, Xadi) = p(C]Xad), R))( R))((R) (R]d (ad) )P(Xat)
(A.4)
This means sample from all data points and not only from observed data points for IPW. Sampling from Eq. dA.4 is equivalent sampling from p(C, Rł Xadi, X f R) p(C, R, X, R)
p(C|X, R))((())))))) X,, (X, R) which one can do as shown Appendix A.3
Based on the introduced target and behavior distribution, we now derive the AFAIS estimator:
E cddo( = T)] [ -Cr(C(aoRR= iT)
J Ro]
C
Σ
CplC, Xady (edl Π))
C.
Σ
Cp(CIXadj, R)p( Xaey, R= T)p( (X(ae)))
C,
p(ltl Xudj Kndj R 1)
Σ
Cp(CiXadj 1t)p( Xad,
pl Ri Xail)
C,
p(R = [|Xaet 12)
Σ Cp(CIXadj (2)x((a)Xa)))(((X
p(R = i
Ct
OP(R= i|Xadji R) JAPAIS
= Eic
(A.5)
p(R= i|Xmas
where the equivalence (+) holds due to Bayes rule:
p( (R|XXal) odj. R =
p( R = T|X adj, R?)
p(R i adjj R)
( X R R= i)
p( R, Xmds)
p(
R Xadj, R = 1)p R = i|Xass)
p(R ilXodd)
16

---

Under review as a conference paper at ICLR 2023
A)
B)
AFA missingness
AFA missingness
11
X
&1
X
R1
R2
R3
R1
R₂
R3
X1
X3
X1
X2
X
Data
Data
R3
R2
R3
11
X3
X1
X2
X3
Retrospective missingness (MAR)
Retrospective missingness (MNAR)
Figure A.2: Missing data graph including the AFA and retrospective missingness. A) Retrospective missingness is MAR. B) Retrospective missingness is MNAR.
A.6 IDENTIFICATION EXAMPLE FOR AFAIS
We give an instructive example how to perform identification for the propensity score p( ft = x(ay)) and AFA propensity score p ft = X ft), needed for the AFAIS estimator JAPUS- We show that identification possible for simple MAR (and therefore MCAR) scenario, but that the evaluation of the AFAIS estimator is not possible for an MNAR missing data scenario.
MAR
We assume the missing data graph in Figure A.2 showing both retrospective and AFA missing- ness. The figure shows a simple MAR missing data scenario, where X1 and X₂ are not missing in the retrospective dataset (we leave out R₁ and R₂ for clarity), but influence retrospective missing- ness in X3 (X, + Rs: X2 + Rg)). The blocking of actions leads to: Rg - RG Vi. Furthermore, the AFA agent's decision to acquire feature which translates to X; R. Vi + j. Figure A.2 shows arrows R₁ Ri and li Rg. but this does not hold in general. Other causes between missingness indicators are in general possible, but this does not affect the identification procedure deseribed in the following. The corresponding adjustment set is Xadi (X1, X2).
be made based all previously acquired features
Identification of R i|xaat): The propensity score is identified by:
p(R = i}]Kat) R( X2)
which is function of the observed data (X, and X2 are always observed).
Identification of p(R I|xadt R): The AFA propensity score is also identified:
pR i|Xadi adj R) =
17

---

Under review as a conference paper at ICLR 2023
MNAR
We now assume the missing data graph in Figure A.2 B). The missingness scenario is equal to the
previously described MAR case, except for an additional missingness Rg caused by X1 (X₃ + R2). As the missingness indicator R3 now depends on potentially missing X2, the scenario is MNAR. The corresponding adjustment set remains Xadj (X1, X2).
Identification of R I)xxa)):
The propensity score is still identified by
p( R = ((x))))
=p(R 1|,,,
=p(R2 = 1111 R2g 1|X,, X2)
=[ 11x1)) (IXi)((Pss = 1|11, P2=11
(A.6)
where we used the conditional independence R3 TT Rg|X1, to make the expression a function of the observed data. The problem with using the propensity score for the AFAIS estimator is that it needs to be evaluated for all X and not only for fully observed samples of X (as the case regular IPW). this example, it is not possible evaluate p( R = xax)))) for data points where Rg 0. This means AFAIS cannot be used. One could impute the missing values using the MI procedure. This can, however, be very difficult due to complex identification and modelling. Furthermore, the identification of the AFA propensity score might also be very difficult or potentially impossible in these scenarios.
A.7 DERIVATION OF THE MODIFIED ACTIVE FEATURE ACQUISITION IMPORTANCE SAMPLING (AFAIS)
The derivation of the modified AFAIS estimator that combines importance sampling ideas from the black-box missing data view and the off-environment bandit view, and therefore covers the AFAIS- IPW spectrum, closely follows the derivation done in Appendix A.5 Again, the target distribution
p(C, Xadj do R = I)) = p(C] R)p( R Xndj. R I)p(Xad))
(A.7)
Now, we propose the following behavior distribution:
p(C, R, Xuuj R. I) p(C]) R)p(At)) R. = (X(Sa R, = I)
(A.8)
This means we sample from data points where some subset 8 of the features is fully observed
(Rg 1). Sampling from Eq. A.8) is equivalent to samping from p(C, R, Xadi X R) = I) which one can do shown in Appendix PCC. R, R) PIII A.3
Based the introduced target and behavior distribution, now derive the modified AFAIS esti- mator:
Jdo(fe= E do do( R -i] =
- Cp(C((doRR- T))
Σ Cp(C)Xadi (tpp((])Xat R = T)p(Xat)
fi
Cp(C|Xadi R)p RI Kadj. R. = T)p(X R. ii) p p(R Xati R R= 1) p(Xadt)
Σ
p( Ri Xadj, Rg = 1) p(Xad) Rg = 1)
ft
Cp(C] R)p R Kadj. R, = ()p(X Rg = I). p( = i|X R. R, = T)p((Re =
Σ
p(R = i[Xad)
fi
adj: R1 R₂ = T)T(R. -1]
(sF
(A.9)
p(R- = 1[x
18

---

Under review as a conference paper at ICLR 2023
where the equivalence (+) holds due to Bayes rule: The first fraction is
pl R. X mdj: R R=11 R1
p(R X|xad adj R, R. =1)
(R] R 1)
p [ [ad) R. =1)
p p(R)Xauc Rg I, T)p(R [[ Rs =)
pl x Xadj. R 1) p(R R 1|x R R. T)
p(PtiXadi Xudi R 1) pd R 1| Rs = T)
pl = Xodd, R, Rs = 1)
(A.10)
pf = [|Xess) R2 1)
The second fraction is:
p(Xau)) pl Xadll R. 1)
p(R. =
(A.11)
p(R, = i[Xudt)
Combining both
and
A.11 gives for the importance ratio:
p( [ X add; R = T) p( adi)
p(R aadi Ri, R. =1) p(R, = 1)
p( ((R)Xxxo Xadt, Rs 1) p( (X
R 11
p(R Xuys R. 1) p(R,- = 1lXMat)
p( = a|aid R2 R2g = T)p( Rs =T)
(A.12)
p(R = 1 X(ds)
A.8 IDENTIFICATION EXAMPLE FOR THE MODIFIED AFAIS
Here show that the modified AFAIS estimator with {2) is identified for the MNAR scenario from Appendix [A.6] and that it is possible to evaluate the AFAIS estimator in this case. We follow the same example and see, first of all, that the propensity score now only needs be evaluated on the data points where PS = which is possible. Secondly we show identification of the AFA propensity score:
Identification of p(R T|X R, - i X2, R, R,
x(
The identification of the AFA propensity score p = x R, R2 = 1) is straight-forward, as the conditioning Rs 1 makes this function of observed data only.
How to choose s:
While we do not give general identifiability statements for the modified AFAIS estimator, we present here a simple guide for choosing the set that yields the most data efficient scenario under which the modified AFAIS can be used: First, start with s 9 (full off-environment bandit view) and in- vestigate identifiability and the possibility of evaluation. If identification and evaluation is possible, then the entire dataset can be used for estimation. Otherwise, try larger sets for until identification and evaluation is possible or s {O..... dz} has been reached. When the set s {0. .......... d, reached (full black-box missing data view), then (AFAIS(s) = JIPw and the known identification theories for IPW hold.
A.9 OTHER VIEWS ON AFAPE
OFF-POLICY POLICY EVALUATION (OPE) FOR AFA
A completely different view on the AFAPE problem is off- policy evaluation (OPE). In OPE, one evaluates the performance of a target policy, the AFA policy, from data collected from a behavior policy which in this case corresponds to the retrospective missingness (e.g. the policy the doctors had the hospital when deciding on what features to acquire). This view differs fundamentally from the AFA graph proposed in this paper in that it does not look at retrospective missingness as part of the environment, but rather as actions produced by a "behavior" policy. There are again two approaches that differ in their view missingness. If both retrospective and AFA missingness looked at from a sequence view (Figure/A.3), the problem can be characterized as OPE of RL agents. One drawback of this view is that it requires additional knowledge about the order of acquisitions in
19

---

Under review as a conference paper at ICLR 2023
the retrospective dataset U (T,,),... which is often not known. An OPE alternative, is to look at retrospective and AFA missingness from set perspective (with actions R/R) and therefore consider the problem as OPE of bandit agents. This again uses the fact that the order of acquisitions will, however, continue with
does have an impact the evaluation step. In the following, the OPE RL perspective as this view has been applied to AFA before (Chang et al. 2019), but the described characteristics hold similarly also for the OPE bandit view.
For clarity, we denote the AFA missingness policy as JAFA = (((())))))) R())) and the retro- spective missingness policy as Renr = (ū(() X, R()). Note that Ū and Ú now describe the same variable, but the former is the retrospective distribution that acquired the data, while the latter the interventional distribution that we would like to apply when deploying the AFA agent. A fundamental difference between the two distributions is, however, that Tarm É FMAR in general, which implies arrows X ((k)) vk' (depicted as blue arrows in Figure A.3 that are not mediated by (k).
The evaluation of under any AFA policy nAM from the OPE RL view (and Assumption corresponds to:
= R|CC rAPA)l = Jao(U~RAn)
(A.13)
This reads as: the expected cost had, instead of the retrospective policy, the AFA policy been applied. This shows that the AFAPE problem can be formulated using the AFA graph (with black-box miss- ing data and off-environment bandit views) using this alternative OPE view and that all views can be used to solve the same estimation problem. As the AFA policy does not depend on X, but only on X, the arrows + (K)) Ak' are eliminated under the intervention do(U ~ TAPA). Thus, in this setting, X' is also confounder due to for example the backdoor path (((K)) +- - Č. There- fore, unbiased evaluation requires adjustment X which is complex due to potentially missing X values. However, adjustment is not needed under the assumption TRea E FMAR, which means that retrospective data was potentially also acquired by a relrospective "AFA" agent (e.g. physi- cian in the hospital). A huge drawback of the OPE view (from both RL and bandit perspectives) is that it is very data inefficient as we show in Appendix A.10
Retrospective/AFA missingness
Classification
((R
C
PDO
2(++1
R(K-1
Y
target
available to be optimized
X
Y
not available
to be intervened on
Data
Figure A.3: AFA setting from an OPE RI. view. The graph describes the observational data given by the retrospective missingness policy TRem = π ([)()) XX R())). The state (shown as (*)) also contains (R) which was left out for clarity. The run un-time application of AFA agents then corre- sponds to the intervention do U TAFA) in which arrows X - U eliminated. This necessitates adjustment of the confounder X.
BLOCKING ACQUISITION ACTIONS
The last common approach used handling missing data in AFA settings is to block feature acqui- sition action the corresponding feature is missing (Janisch et al. 2020 Yoon et al.] 2018). Yoon et al. (2018) additionally supplied the AFA agent with the missingness indicators in the retrospective
20

---

Under review as a conference paper at ICLR 2023
dataset, making of "informative missingness". This approach, however, corresponds to evaluat- ing J = C and not ft= and thus is a biased estimator in the setting we consider (due to Assumption Under Assumption| [ there are no "informative missingness" indicators at run-time that could be given to the agent. This method is unbiased only under the alternative assumption that the exact same restrictions for the agent at train/evaluation time apply also at run-time.
A.10 DATA EFFICIENCY AND POSITIVITY ASSUMPTIONS OF DIFFERENT VIEWS ON AFAPE
In this work, we propose different views on the retrospective and the AFA missingness. Here, we derive first the data efficiency of importance sampling approaches of the different views for the AFA setting; and then show the resulting implications the required positivity assumptions of each approach.
DATA EFFICIENCY
We represent data efficiency for AFA by two key quantities: 1) ng: the number of distinct AFA sequence trajectories Ü that can be evaluated using single retrospective data point (defined by Y. X, and U) and 2) Pruns: the number of times we have to run an agent on the data point (i.e. choose the data point the environment) to be able evaluate these trajectories. Table A. presents the views on retrospective and AFA missingness of different approaches, along with their data efficiency. We present the details of derivation and conclusion for each approach below.
We start with a short example of a data point with features with given missingness ū = ("observe(Xx)) "observe(X,)" "stop observation") / (1, 1) before deriving the general data efficiency for each view. We abbreviate with slight abuse of notation: u = (3, 1).
Retrospective missingness
Approach
AFA missingness
Data efficiency
OPE RL view
Sequence view
Sequence view
" Truns
nt |R]|1!
OPE bandit view
Set view
Set view
Ryuns
If
1 : nn 2iz (!(())
Black-box missing data view
Set view (as environment)
Black-box function
Truns 2d
If R + i: ::n Prans
> im
||81| i! =!((IRI.)
Off-environment bandit view
Set view (as environment)
no
Set view
Arums 211
enfeennirnnment
RL view
Set view (as environment)
Sequence view
no Tyuns
E i! ((((i R1))
Σ 1
Table A.I: Different approaches the AFAPE problem with their corresponding views retro- spective and AFA missingness, and the data efficiency. ny denotes the number of AFA trajectories that can be evaluated from single retrospective data point using the respective approach. Mruns denotes the number of times an agent has to be run on the datapoint to evaluate the ny trajectories. The off-environment bandit view is strictly more data efficient than all other views.
Off-policc policy evaluation (OPE) view (RL perspective)
In the OPE view from an RL perspective, the retrospective sequence trajectory á = (3, 1) is used directly evaluate the target trajectory û (3, 1). Thus, in the example and in general, one data point can be used to evaluate one target trajectory (ng = 1) with one run (N = 1). This view thus very data inefficient. It has the fundamental drawback that only trajectories of the behavior
21

---

Under review as a conference paper at ICLR 2023
policy are used to evaluate the target policy. It does not make use of the fact that one also assess what would happen less features were used or if the features were acquired in a different order.
OPE view (bandit perspective)
In the OPE view from a bandit perspective, the retrospective missingness indicator F = (1, 1) is treated as one action (set view). It is then used to evaluate the target action F (1,0, 1). This jointly evaluates the AFA sequence trajectories à E {(1,3), (3, 1)}. Thus, the exemplary data point can be used simuitaneously evaluate two target sequence trajectories (ne = 2) with one run (Nrrnss - 1). The number of trajectories that can be evaluated in general is equal to the number of permutations of the censspectivel-acquiiii features, i.e. no 1 where the L1 norm ||]]] gives the number of acquired features.
This view is similar to the OPE view from the RL perspective shown above, except that it leverages the fact that the order of acquisitions does not have an influence on evaluation. Therefore, it con- siders all possible permutations which result the final set of features. Otherwise, shares the same disadvantages the OPE view from an RL perspective.
Black-box missing data view
In the black-box missing data view, the retrospective missingness is seen as part of the environ- ment and the agent allowed to choose different combinations of the available features. Only fully observed data points can, however, be used for evaluation. Therefore, one can evaluate no AFA sequence trajectories for the exemplary data point as is not fully observed. Con- sider instead (fully- o data point with retrospective missingness iT" (3. F 1.). This data point can be used to evaluate all possible AFA sequence trajectories û" E {(1, 2, 3), (1, 2). (1, (1), (2), (3), O} (16 distinct trajectories). All trajectories that share the same AFA missingness (f*) can be simultaneously evaluated (as in the bandit view). thus requir-
ing one run per possible subset, i.c. f" E {(1, (1, 1, .... (1, 0,0,, .... (0, O, 0)} (8 distinct sets). In general, one obtains ng = Ei al(4)) and Truns = 2d. = I (dz is the total number of features).
The black-box missing data view has the advantage of being a straight-forward approach based on well-established literature. The downside is that it treats the AFA policy as any other function applied to the data, and rejects the chance of evaluating the agent on incomplete data if the missing features is not desired by the agent in the first place. This view is only recommended in complex missing data scenarios where AFAIS fails, namely where the fraction of missing values is very low or when the AFA policy is "data hungry", i.e. the agent often acquires almost all the features.
Off-environment bandit view
Like the black-box missing data view, the off-environment bandit view also treats the retro- spective missingness as part of the environment. The difference is that partially-observed data points can still be used for evaluation.
particular, AFA sequence trajectories that con- tain subset of the available features can be evaluated. For our example, the trajectories are E (11,3), (3, 1), (1), (3), 0} (5 distinct trajectories) which require runs, i.c. one run per unique set (F {(1, 0, 1), (1, 0,,, (0, 0, 1), 0))), In general, one can evaluate no >! il [R))) different sequence trajectories with 7yuns = 2 ft]ls different runs.
This view combines the advantages of the OPE view ppartially-observee data points can be used) and the missing data view (one can evaluate trajectories with subsets of the available features). is strictly more data efficient than the OPE views and the black-box missing data view.
Off-environment RL view The off off-environment RL view also treats the retrospective missingness part of the environment, but looks at the AFA missingness from a sequence view. This view thus also allows the evaluation of
= R|]: :! (1) (IIBll) different sequence trajectories, but requires as many runs Flauss -nj- This view is equal to the off-environment bandit view, with the exception that it does not make use of the fact that the order of acquisitions doesn't 't have an influence on evaluation. It is thus sub-optimal compared to the off-eenviromment bandit view.
ni D
22

---

Under review as a conference paper at ICLR 2023
POSITIVITY ASSUMPTIONS
Importance sampling approaches have to fulfill a positivity assumption that requires the behavior distribution have positive support wherever the target distribution has positive support. Since behavior and target distributions differ between the different views, the required positivity assump- tions will also be different. We first look general positivity assumptions:
General positivity assumptions In order to evaluate any possible AFA policy, the following positivity assumptions must hold:
OPE view (RL perspective): p(ū x|x) AX, E where U the set of all possible sequences of actions
The positivity assumption of the OPE view requires each exact trajectory of actions have positive support (for all possible feature values). It is the strongest of the positivity assumption in this list.
OPE view (bandit perspective): p(R = r|X) > v'X,r {0, 1]dh
The OPE view from bandit perspective requires only each subset of acquired features to have positive support (for all possible feature values).
Black- box missing data view: p(R= [x) > O
The black-box missing data view requires the possibility of complete case (for each possible fea- ture value). This is a much weaker assumption than the assumptions for the OPE views.
Off- onf-reviromment bandit/RL view: p(R = [x)= > X
The off-environmen bandit view has the same positivity assumption the black-box missing data view when the goal is to evaluate any possible AFA policy.
Note that these minimal positivity requirements. A complex missing data scenario might impose additional positivity assumptions for each of these views to allow identification. In the missing data view, for example, identify any identifiable missing data scenario that can be represented by DAG (directed acyclic graph), the following additional requirement must be met (Malinsky et al., 2020): p( R zeros with
x) > 0 VV i.t. i indexes a partially missing feature. ē denotes a vector of position
AFA policy specific positivity assumptions
The general positivity assumptions will relax to some extent, if we are interested in evaluating specific AFA policies only with known support. this case, the following modifications are to be made:
OPE view pt.sppepecctve.:: p(ū= x|X) > O u s.t. p(ú x|X) >0
The OPE view from an RL perspective thus requires the behavior policy to only have positive support for action trajectories that could be taken by the AFA policy.
OPE view (bandit perspective): p(R = r|X) > VX, r s.t. p(f = r|x) >
Similarly, the OPE view from bandit perspective requires the behavior distribution to only have positive support for sets of features that could be acquired by the AFA agent.
Black box missing data view: p(R = Îxx) > Y
The missing data view treats the AFA missingness ploit the scenario where we look at only specific AFA policies. It thus still has the same positivity assumption.
black-box function and can thus not
enffeenirroment bandit/RL view: p(R (r' r.V)Ii)) > 0VX, r s.t. p(u p( r|X) > 0
The off-environment view requires the behavior distribution to have positive support for a superset of the set of features that could be acquired by the AFA agent. This is thus strictly the weakest positivity assumption amongst all views.
A. EXPERIMENT DETAILS
In this section, first describe the general experiment and training setup before explaining each dataset in detail. Afterwards, we provide a detailed list of the parameters and configurations for each experiment.
23

---

Under review as a conference paper at ICLR 2023
Data, costs and synthetic missing data For the experiments, we defined a "superfeature" as a feature that comprises multiple subfeatures, which are usually acquired skipped jointly and which have single cost (e.g. the pixels of an image). Furthermore, we assumed a subset of features available at no cost (free features) and gen- erated random acquisition costs Cacy for the remaining features (from uniform distribution between 0.5 and 1). We simulated misclassification costs to force good policies to find a balance between feature acquisition cost and predictive value of the features. We induced synthetic missingness with MCAR, MAR and MNAR missingness mechanisms similar to the scenarios described in Appen- dices A.6 and A.8 The distribution of the missingness indicators follows the logistic model (e.g. p( = X1) o-xi + 1)). To evaluate convergence of different estimators, we consider the average cost of running the AFA agent on the dataset over all data points the ground truth testset (without missingness) as the true expected cost (J do(R=T))
Training
We used an impute-then-regress classifier (Le Morvan et al. 2021 with unconditional mean impu- tation and a random forest classifier for the classification task and trained it on fully available and randomly subsampled data (where p( R4 1) 0.5). We tested three random acquisition policies that acquire each costly feature with 10%. 50% and 90% probability, respectively. Furthermore, we evaluated vanilla Deep Q-Network (DQN) RL agent (Mnih al. 2015) which during training, blocks feature acquisitions if the respective feature is not available in the retrospective data. The nuisance models (propensity score and afa propensity score) were trained on the test set using multi- layer perceptrons and a cross-fitting approach was employed in the real-world data experiments to make maximum use of the data (Kennedy) (022). cross-fitting, we split the data into K disjoint folds and train K different sets of nuisance models where nuisance model i is trained on all folds except fold i. The estimator is then evaluated on all folds by using the nuisance models if data point is in fold i. We used K = 2 in all real-world experiments.
Synthetic dataset
We evaluated and compared the described methods on synthetic dataset with X R and a bi- nary label Y, simulated using Scikit-lear''s make_ make_class www.ccaassiiii saaifiicatiiii ion function (Pedregosa et al.] 2011) We assumed one superfeature "superX33 to include X3 and X4. The synthetic data ex- periment contains MNAR missingness for which we assumed i) X, (fully observed) causes miss- ingness in X2; ii) X2 causes missingness in super We chose {1) as the conditioning set when using the modified AFAIS estimator to allow evaluation. The fraction of complete cases is p( = T) Rs 12% in this experiment. For the synthetic data experiment we did not use cross-fitting, but instead generated separate datasplit (40' 000 data points) to train the nuisance models (See Table A.2 for full details).
HELOC dataset
We also tested on the HELOC (Home Equity Line of Credit) dataset which is part of the FICO ex- plainable machine learning (xML) challenge (FICO, (FICO)2018). The dataset contains credit applications made by homeowners. The ML task was to predict based on information at the application, whether an applicant is able to repay their loan within two years. The dataset contains 9 superfeatures (See Table A.3 for full details).
Retinopathy dataset
Next, we tested on the Retinopathy dataset (Antal & Hajdu 2014). The ML task was to predict diabetic retinopathy based on extracted image features. We considered image quality assessment and pre-screening free features and manually extracted image features by expert as costly. The dataset contains superfeatures (See Table A.. for full details).
Adult income dataset
We performed evaluation on the Income dataset from the UCI data repository (Newman et al. 1998) The MI. task was predict whether person has income over 5000. We considered age as a free feature and other private information (such as education, work experience, etc.) as costly features. The dataset contains 10 superfeatures (See Table for full details).
24

---

Under review as a conference paper at ICLR 2023
Data and environment
Sample size D
10'000 (plus separate datasplit of DDDiiesss 40'000 for training of the
propensity and AFA propensity scores)
Superfeatures
super.Xo: [uu]. s: [X1]. super.X2: [X2, Xz]
Label
(class 0: 50%, class 1: 50%)
Feature acquisition cost
Cacy = [0.0, 0.84, 0.86)
Misclassification cost
Cmc (0, 1) = Cme (1,0) (1,0 = 10.0
Missingness mechanisms
Ro 1) 1.0,
p( Ri = (1.0 Xo):
MNAR
P(R2 = = 1) -
X1)
Complete cases ratio: p(R 1) 0.12
Models
Classifier
RandomForest (number of estimators: max depth: 3)
Random (10% 50%, 90%)
Agents
DQN (learning rate: 0.001, number of layers: 3,
hidden layer neurons: 50, hidden layer activation function: ReLU)
PS (number of layers: 2, hidden neurons per layer: 32, epochs: 2000,
hidden layer activation function: ReLU)
Nuisance functions
AFAPS (number of layers: 3, hidden neurons per layer: 32, epochs: 8000, hidden layer activation function: ReLU)
Table A.2: Full experiment details: Synthetic dataset
25

---

Under review as a conference paper at ICLR 2023
Data and environment
Sample size n Dast
35562 (data split: 50% train, 50% test)
sup
('ExtemalRiskksssiiate)
super.X1:
[MSiineeOdderrTradeeppe''''' (NSinneMostcccon/Traeepppppt
'AverageMlnFill', "NumSattifaactoryTTaade oaaprraatttniiiuuuuuutsattttroro
superX2:
XyperX3:
PerrennTradeesNeevrDbeeii MMSiceMossRReeenDDiDi MMaxDDllpPPubiccceLLsss1//1/11
Superfeatures
super
('MaxDDelgyver', NumTToaa/Trades)
superXs:
[NummTrade::OpoiinLLss\tMMM Per.:paiii........oeeeererrrrrrr "PercentinstalTTradss, "MSSinecoosskccenttnxe//////s//)
superXe:
superX,:
"NetrractonkeevvviigBBuddde Beer
"Pereet(TradesWBalaee
super (NeeTFrattonnInsaalBuurrdri "NumRevolvingTrrade Baalacce
"XumlosallTTradesWBalaaeee
Label
'RiskPerformance' (class 0: 48%, class 1: 52%)
Feature acquisition cost
Cacy = [0.56, 0.84, 0., 0.58, 0.6, 0.6, 0.64, 0.57, 0.54]
Misclassification cost
Cmc (0, 1) Cmo (1,
(1,0) 0) 15.0
Missingness mechanisms
p(R; = 1.0, i p(R 1) 0.5,
MCAR
(1,3,,7)
3,
Complete ratio:
pR 0.06
p(R = 1.0, i 20.2,4,6,8)
MAR
p(R, 1) (0.00 3.0 ExternaRiisEEsimaate
Complete case ratio: p(R = = 0.21
p( Ro 1) 0.8,
p(R, = 1) = 1.0, i (2,4,6,8) p(R, 1) a0..0 + 3.0 ExternaliiskEstmaaee
MNAR
Complete ratio: p(R 0.17
Models
Classifier
RandomForest (number of estimators: 100, max depth: 15)
Random (10%, 50%, 90%)
Agents
DQN (learning rate: 0.001, number of layers: 3,
hidden layer neurons: 50, hidden layer activation function: ReLU)
PS (number of layers: 2, hidden neurons per layer: 32, epochs: 2000,
hidden layer activation function: ReLU)
Nuisance f'unctions
AFAPS (number of layers: 3, hidden neurons per layer: 32, epochs: 8000, hidden layer activation function: ReLU)
Table A.3: Full experiment details: HELOC dataset
26

---

Under review as a conference paper at ICLR 2023
Data and environment
Sample size Dis
1151 (data split: 50% train, 50% test)
super
('quality assessment"]
super.X;: pprrescreeenig']
super.X2:
['MA detection {i]'), i= {I,..., 6}
Superfeatures
3uperX3: ["exudates_[i)] "] i= (1,..., 8}
superX,; ['dist macula opt disc']
superX;: ['diam opt disc"]
super ['class AM FM']
Label
'DR' (class 0: 47.0%, class 1: 53.0%)
Feature acquisition cost
Carg = 0., 0.55, 0.57, 0.78, 0.52, 0.64)
Misclassification cost
Cinc (0, 1) Cmc (m 12.0
Missingness mechanisms
p(R, = 1.0, i 1,
MCAR
P(R, 1) 0.7,
E (2, 4,5)
Complete case ratio: p(R I) 0.23
p(R = 1.0, i (0.1,6) p(R, = a--10
MAR
+ 2.0 pre-screening). 0.28
j (2,3,4,5]
Complete case ratio: p(R =
P[Rg 1) piR,, 1) = 1.0, p(Re 1) 0.9 p(R, = 1) = 1-100 + 2.0 pre-screening). j E (2, 3,5,)
MNAR
Complete case ratio: p(R 1) 0.22
Models
Classifier
RandomForest (number of estimators: 40, max depth: 6)
Random (10%, 50%, 90%)
Agents
DQN (learning rate: 0.001, number of layers: 3,
hidden layer neurons: 50, hidden layer activation function: ReLU)
PS (number of layers: 2, hidden neurons per layer: 32, epochs: 2000,
hidden layer activation function: ReLU)
Nuisance functions
AFAPS (number of layers: 3, hidden neurons layer: 32, epochs: 4000, hidden layer activation function: ReLUU)
Table A.4: Full experiment details: Retinopathy dataset
27

---

Under review as a conference paper at ICLR 2023
Data and environment
Sample size 11 Dis
32561 (data split: 50% train, 50% test)
workclass:
['Federal-gov', 'Local-gov', 'Never- 'Private', 'Self-emp-inc', "Self-emp-not-ine: 'Stateegov', "Without-pay']
education: ('1st-4th', '5th-6th', '7th-8th', '9th', '10th', '1 '12th', "Assoc-acdm', 'Assoc-voc', 'Bachelors', 'Doctorate,, 'HS-grad 'Masters', 'Preschool', 'Prof-school' 'Some-college', 'nnum']
marital-status:
-aarried-AF-spouse' "Marriedcciv-ppuuse"
MMrried-spouss-abseent" Never-married", ''Separaed", "Widowed', 'relationship Not-in-family', 'relationship Other-relative', 'relationship Own-child', 'relationship Unmarried', 'relationship Wife']
Superfeatures
occupation: ['Adm-clerical', Hamned-Forces 'Craft-repair'.
'Exec-managerial, Barming-fishingg 'Handlers-cleanens:
'Machine-op-inspet"" 'Other-service', Priv-houus-serv', 'Prof-specialty.. 'Protective-serv's 'Sales', Tech-ssppoot', 'Traspoot-mmovigg]
race: "'Asian-Pac-Isladdes 'Black', 'Other'), sex: ['sex Male"]
age: ['age'], J hours-per-week: ['hours-per-wee per--eeker] capital-gain ['capital-gain ]. capital-loss: ['capital-loss"]
Label
'income" (class 0: 76.0%, class 1: 24.0%)
Feature acquisition cost
Cacy [0.53, 0.64, 0.64, 0.53, 0.64, 0.55, 0., 0.82, 0.52, 0.76]
Misclassification cost
Cmel (0, 1) Cmee (1, 0) 20.0
Missingness mechanisms
p( R4 = 1) 0.8, i
MCAR
p(R, = 1.0,
e (1,
Complete case ratio:
p(R - 0.16
y( R4 1) 1.0, E (5,6)
pl R = = a (1.0 + Male + 4.0 age),
j (1,2,3,4)
MAR
R 1) a1.0 + Male + 3.0 age),
b (7,8,9)
Complete case ratio: p(l I) 0.22
p Rs = = 1.0, p Rs = 1) (2.0 + age),
p R
(1.0 Male + 4.0 age),
j (1,2,3,4)
MNAR
p( PR = = 01.0 + Male + 3.0 age),
j J
Complete case ratio: p(lt = I) = 0.20
Models
Classifier
RandomForest (number of estimators: 100, max depth: 8)
Random (10%, 50%, 90%)
Agents
DQN (learning rate: 0.001, number of layers: 3,
hidden layer neurons: 50, hidden layer activation function: ReLU)
PS (number of layers: hidden neurons per layer: 32, epochs: 2000,
hidden layer activation function: ReLU)
Nuisance functions
AFAPS (number of layers: 3, hidden neurons per layer: 32, epochs: 10000, hidden layer activation function: Rel.U)
Table A.5: Full experiment details: Income dataset
28

---

Under review as a conference paper at ICLR 2023
A.12 ADDITIONAL EXPERIMENT RESULTS
Synthetic (MNAR missingness)
Figure 3 in the main text shows the convergence of different estimators for the synthetic MNAR data experiment. The corresponding estimates are shown here in Figure A.4 The experiment fur- ther exemplifies how large the errors of biased estimators can be. Only JIeW and JARUS pro- duce unbiased estimates for all agents. The esti- mates produced by Jimp and JR are far from the ground truth Jdo(R=I)) especially for the DQN agent. This poses high risk for model deploy- ment, especially in safety-critical applications.
JAFNS
5.0
Jon
Ja 1
Ja
Junp
Janth 11
3.5
3.0
The main text also shows the estimates of the HE- LOC (MCAR scenario), Retinopathy (MAR sce- nario) and Income (MNAR scenario) real-world data experiments (Figure (6. We provide the ex- periment results for the remaining missingness scenarios in Figure AA.3
Random 10%Random 50%Ran
Figure A.4: Evaluated performance of differ- ent agents for the MNAR synthetic data exper- iment. Box plots shows variability of the es- timate by bootstrapping 300 times without placement 70% of a total test dataset of size Drar 1000000.
An additional benefit of IS-based estimators that one can further use the estimator to not only estimate the counterfactual cost, but also other quantities of interest, such counterfactual fre- quency of feature acquisitions. Figure A.6 shows
the frequency of acquisitions for the DQN agent in MNAR Income dataset experiment. While the sampling is done on a dataset with missingness and blocked acquisition actions for non-available fea- tures (left plot), the AFAIS estimator (middle plot) gives estimates for the counterfactual frequency of acquisitions, i.e. how many times each feature would have been observed had the retrospective dataset been without missingness (right plot). As the result, the agent acquires the feature 'occupa- tion' the most (among costly features) which is reasonable as it is a good indicator of income. The estimation of counterfactual frequency of acquisition decisions at run-time (e.g. in hospitals) can be a valuable information as it allows for resource management and allocation planning in advance.
29

---

Under review as a conference paper at ICLR 2023
A)
missingness)
B)
Retinopathy ssingness)
Income missingees)
Jeus
70
Justt
Jans
Jne
Jtxl
Ma
JasttD
Rancom 10%sRandom 50% 90%
Random Random Rondom 90%
DON
HELOC missingnes))
Retinopathy missingness)
nooome (MAR m moingners)
Jass
Jess
77
Aew
It-1
25-
Ave
Ive
Joak
Junas
3.4
Jt-
50
Anatt
60
Random 10%6Random 50% Random
10%oRandom 50% 90%
Figure A.5: Results for the HELOC, Retinopathy and Income dataset experiments for differ- ent missingness scenarios. This is a complement to Figure 6 in the main text and shows the re- maining missing data scenarios (MAR/MNAR for HELOC, MCAR/MNAR for Retinopathy and MCAR/MAR for Income). Boxplots show variability of the estimate when bootstrapping 300 times without replacement 70% of total test dataset of size RD (1 17'781 for HE-
LOC, Dr.s 575 for Retinopathy and D.D... = 16'281 for Income).
Feature acquisitions by DQN for the Income dataset
Frequency acquired Teatures (%)
workclass
education
marital-status
occupation
race
sex
age
hours-per-week
capital-gain
capital-loss
R
do(R
Figure A.6: Acquisitions of the DQN agent for the Income dataset. corresponds to the acquisi- tions that were performed by the agent on the retrospective dataset. The acquisitions that were made by the agent when running it on the ground truth dataset (without missingness) are shown on the right (do(R = Ī)). The middle shows the estimate by the AFAIS estimator.
30

---

Under review as a conference paper at ICLR 2023
A.13 GLOSSARY OF TERMS AND SYMBOLS
Term
Description
run-time
Time when the algorithm is applied prospectively (e.g. when de- ployed in the hospital)
retrospective missingness
Missingness in the retrospective dataset used to train/test the Al systems
AFA missingness
Missingness that is created at run-time when the AFA agent de- cides on feature acquisitions
MCAR
Missing completely at random
MAR
Missing at random
MNAR
Missing not at random
AFAPE
Active feature acquisition performance evaluation: The problem of estimating how an AFA agent and classifier would perform at run-time from data with retrospective missingness
sequence view
View missingness with a focus on the sequence of feature quisitions
set view
The traditional view on missingness with a focus on which set of features was acquired
AFA graph
Causal graph describing the AFA problem
black-box missing data view
View on AFAPE where retrospective missingness is seen from a set view and as part of the environment and AFA missingness is seen as a black-box function
off enverovirnttnnt bandit view
View on AFAPE where retrospective missingness is seen from a set view and as part of the environment and AFA missingness also seen from a set view
offeenvronmmnn RL view
View on AFAPE where retrospective missingness is seen from set view and as part of the environment and AFA missingness is seen from a sequence view
OPE
Off-policy policy evaluation
OPE view (RL perspective)
View on AFAPE with a sequence view on both retrospective and AFA missingness
OPE view (bandit perspec- tive)
View on AFAPE with set view on both retrospective and AFA missingness
IPW
Inverse probability weighting
MI
Multiple imputation
AFAIS
Active feature acquisition importance sampling: novel estimator for the AFAPE problem based on the off-environment bandit view
IS
Importance sampling
propensity score
p( = Îlx) (weighting term in the IPW and AFAIS estimators)
AFA propensity score
p( Ī| R, R. I) (weighting term in the AFAIS estima- tors)
nuisance function
Function that needs to be fitted on the data in order to use corre- sponding estimator (examples are the propensity score and AFA propensity score)
31

---

Under review as a conference paper at ICLR 2023
Symbol
Description
X
Ground truth value of features (potentially unknown)
Y
Ground truth value of the label
R
Missingness indicator (retrospective missingness)
X
Observed proxy for X (retrospective missingness)
(k)
Feature acquisition action at acquisition step (retrospective missingness)
R
Missingness indicator (AFA missingness)
X
Observed proxy for X (AFA missingness)
Feature acquisition action at acquisition step & (AFA missingness)
π
Missingness policy, i.e. probability of taking certain feature acquisition action
FMAR
Functional space for missingness policies that restricts decisions of acquiring new features to be based only on previously acquired features
Coog
Vector of feature acquisition costs: element is the cost of acquir- ing feature X;
Cmc(31, Bm)
Misclassificatiinn cost of predicting a label of class yr as belonging to class ym.
8R,
Predicted value for label Y using classifier f from features that have been acquired by the AFA agent
C-Ò+C
Random variable describing the total cost consisting of the accu- mulated feature acquisition Ö and the misclassification cost Ö
J= EC1
Expected cost of applying an AFA missingness policy x and a classifier f on dataset with retrospective missingness
Jdo(R=D) =E Cldol(R = i)]
Expected cost of applying an AFA missingness policy a and a classifier f at run-time (without retrospective missingness)
M(X = E
Expected cost given all features and the label
JAPMS
Active feature acquisition importance sampling (AFAIS) estima- tor for [t=[I==
JIPw
Inverse probability weighting (IPW) estimator for do(A==I)
Jimp
Imputation-bassd estimator for Jao(R=T)
Jn
Estimator for Jao(R=T) based on the blocking of acquisition ac- tions for not available features
JR-Í
Estimator for Jo)(R= that only considers those data points where all features are available (complete case analysis)
s C {o....,dæ)
Hyperparameter of the modified AFAIS estimator that denotes which subset of the features must be complete (i.e. R, = 1) in order for a data point to be used in the evaluation. A smaller set 8 implies higher data efficiency (less restrictions on complete information), while a larger set increases the possibility of iden- tification and evaluation for particular missingness scenario.
32


Reviews:
[{"review_id": "Ytm23dgXTUb", "reviewer": "Reviewer_pwju", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper introduces AFAPE, the first approach active feature acquisition performance evaluation under missing data. It also introduces AFAIS (i.e.,  active feature acquisition importance sampling) a novel estimator that is more efficient than existing approaches. ", "summary_of_the_review": "Overall, the proposed approach has clear potential, but the paper needs quite a bit of re-writing/re-organizing in order to clarify its impact. As most of the required information is already within the appendices, the comments in this review should quite easy and fast to address. ", "strength_and_weaknesses": "The paper introduces a novel approach to an important, real-world problem: active feature acquisition. It sets up the problem, provides a novel framework for solving it, and validates it empirically against existing approaches.\n\nHowever, the paper could be significantly improved by \n(i) a better use of the available space: in its current form, most experimental results (including all on the real-world datasets) are in the appendix. The authors should shrink the current \"section 3\" by at least two pages (from 5 to at most 3) and use the freed space to bring forward the experiments in the appendix and to thoroughly discuss insights on their significance and weaknesses.\n(ii) the paper would greatly benefit from a motivating, real-world dataset. The three datasets in Appendix A-10 appear to be real-world, but the experimental setup is quite vague and confusing (did the author(s) chose arbitrary are the levels of missingness and miss-classification costs? if not, please explain). ", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "XSJINOZ695", "reviewer": "Reviewer_zfWY", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper studies the problem of data acquisition at test time when the training data contains missing data. This work tackles the question: which testing policy should be performed to minimize the cost but maximize performance? The paper shows that traditional ways to handle this problem lead to biased results when training data presents non completely at random missingness. The authors propose to an unbiased estimator of the cost associated with a testing policy.", "summary_of_the_review": "This work tackles a central issue in machine learning for healthcare. However, one of the central assumptions weakens the claims of the paper.", "strength_and_weaknesses": "This paper explores an important problem in machine learning, particularly relevant to healthcare applications. This work provides a clear formalisation of the problem.\n\nHowever, the model makes a key assumption that weakens the paper: \"the order of acquisitions does not matter for the evaluation step\". In the medical context, a first test, if positive, might make all other tests unnecessary. So, could further clarify this assumption?", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "OnsScTfToI", "reviewer": "Reviewer_vxt6", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "- The authors tackled an important problem called active feature acquisition (AFA) which is critical but under-explored.\n- More specifically, the authors focused on the setting where the training data includes missing components which is also practical in the setting when active feature acquisition is necessary (measurement costs are high).\n- The authors do not focus on proposing a new AFA model. Instead, the authors focused on evaluation of AFA models with the training data with missing components.\n- The experimental results are promising, showing consistently better evaluation results in comparison to alternatives in both synthetic and real-world datasets.", "summary_of_the_review": "1. Experiments on real-world data\n- With real world data we cannot do experiments because the assumption 1 does not hold with logged data.\n- So, we should state that the real-world data experiments are actually somewhat semi-synthetic data experiments.\n\n2. IPW\n- When we use IPW, one another way is computing the probability of the agent that selects the same subset of logged features.\n- Then, use that as the propensity score to compute the unbiased estimator of the cost.\n- In other words, we can do weighted average of the logged costs where weights come from the probability that our AFA agent can select the exact subset of logged policy.\n- In that case, we do not need to worry about only utilizing the samples with complete cases and this can significantly improve the sample efficiency of the propensity method. \n\n3. Small number of data points\n- In many medical domains, there are only a small number of samples in the datasets.\n- It would be great if the authors can provide the results with a small number of samples ranging from 0-5000 as well.\n\n4. Number of features\n- It seems like the authors used the synthetic data with 4 features.\n- It would be better if the authors can provide the results with more features. \n- Based on the introduction, the authors mentioned emergency care examples with many possible measurements. To make this result more realistic, it would be better to provide the results with at least 20 features.\n- Note that all the real-world experiments have less than 10 features.\n\n5. Completeness of the experiments\n- As we know, the missing patterns are quite important in this paper.\n- In that case, it would be good if the authors provide MCAR, MAR, MNAR settings for one synthetic data and all 3 real-world datasets.\n- Currently, the authors only provide one set of experimental results.", "strength_and_weaknesses": "Strength:\n- The authors tackled an important problem, active feature acquisition. Especially, the authors focused on evaluation of AFA model with missing components which is highly practical problem.\n- The proposed solution makes sense and have sample efficiency and generally applicable to any missing patterns. \n\nWeakness:\n- The experimental sections are weak. The datasets are too simple; thus, hard to say that the proposed evaluation method can be well generalized to the real-world datasets.\n- In experiments, the authors only focused on one-time AFA. However, in the introduction, the authors mainly discussed about the sequential AFA.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "6rhTZ4okuJF", "reviewer": "Reviewer_qb3R", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The authors consider the task of active feature acquisition, i.e., the requirement of an actor to choose whether and if so which features to acquire during run time, in the context of missing data during the training stage (in all three variants of missingness). Their proposal formulates the task as a causal graph and formulates it as a reinforcement learning problem. Their final estimator of the true cost function is then evaluated in several experimental settings.\n", "summary_of_the_review": "A paper tackling an important task with a principled solution, whose weaknesses lie primarily in its structural presentation and less in its theoretical contribution.", "strength_and_weaknesses": "The authors tackle an important task with a novel and principled approach.  \n\nOne weakness of the paper is in the theoretical section 3. The diversity yet similarity of the notation requires close attention to follow. The authors provide a very helpful glossary in the appendix, however, it is never mentioned in the main text. Adding a reference there would allow for greater readability. The second weakness similarly relates to the structure of the paper. Given the length of Section 3, almost all, especially all experiments with real-world data, are hidden in the appendix.\n\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "12y10jLtKmH", "reviewer": "Reviewer_dNWk", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper tackle the task of evaluating active feature acquisition (AFA) agent when the AFA agent cannot always access the complete set of features during decision making and there is distribution shift between training and evaluation stages. Specifically, it considers a subset of AFA problems where the features acquired by AFA agent is connected to a classification downstream task and the available dataset contains missing entries that could not be acquired. This work mainly tackles non time series AFA problems where the authors consider the task of feature acquisition from a fixed feature set with a sequence view, for which they claim they are the first one to propose such sequence view.\n\nIn the methodology part of this paper, the authors first illustrate the difference between set and sequence views through a hospital example where a task-specific missing data graph (m_graph) is plotted for each view. Then the authors present causal graph for AFA problem, where they identify the feature variable X as the confounder because they claim there exits a path $\\bar{R} \\leftarrow X \\rightarrow C$. They also declare that AFA agent is allowed to implement causal interventions on $\\bar{R}$ so that the problem of AFA PE can be formulated as a task of evaluating the cost objective $\\mathcal{J}$ under causal intervention on $\\bar{R}$. To correct the distributional shift for training and testing stages, the authors propose to integrate an existing technique for unbiased estimation, i.e., importance sampling, to their performance evaluation method.  \n\nThe authors make strong claim that their method could work on various type of missing observation patterns, including missing completely at random (MCAR), missing at random (MAR), and missing not at random (MNAR). In the main paper, they show empirical results only for MNAR.", "summary_of_the_review": "This paper learns an interesting problem of AFAPE, but I have concerns on its writing, problem scope, the capability of tackling all different types of *observational missingness* in one causal graph, the over simplified causal graph and its insufficient empirical evaluation.  There's a possibility the proposed method could work well on a well specified subset of AFA problems with certain property.", "strength_and_weaknesses": "**Strength**\n\n- The problem this paper attempts to tackle is a challenging yet understudied one which is related to many important real-world problem domains such as cost-sensitive diagnosis in healthcare.\n\n**Weaknesses**\n\n[*Scope*] This paper studies the task of evaluating evaluating AFA (Active Feature Acquisition) agents on a very narrow subset of AFA problems where the AFA strategies are defined as reinforcement learning (RL) policies and there is distribution shift between the training and evaluation AFA distributions. I do think the distribution shift for AFA problems is a general challenge existing in both RL-based as well as non RL-based   AFA problems . However, the authors directly connect the AFA problem with off-environment reinforcement learning without claiming the underlying relationship in the two fields, which might mislead the audiences who are new to the AFA literature when reading this paper. Moreover, the main technical solution considered in this work (causal graphs with intervention) is not specifically related to RL. Therefore, only studying performance evaluation (PE) on RL-based AFA problems lacks generality and the work should consider AFAPE with a larger scope.\n\n\n[*Methodology - Sequence View*]   One major property introduced in the work is about the *sequence view* to select the features, instead of a *set view*.  First, the authors make false claims in several places in the paper, stating the *sequence view* presented in the paper is the *new view on missingness*.  To the best of my knowledge, there are some works from the existing literature studying AFA problems with such *sequence view* already, e.g., [Yin et al, 2020] (cited by the paper). The *sequential view* presented in this work is simply a special case of  the *sequence view* from existing literature. It's unfair to claim the *sequence view* is new when the authors are aware of the work. Second, I do not think the *sequence view* is a better design choice than *set view* for the non-time series AFA problems considered in this paper. For example, when test A and B can be taken in any order, the *sequence view* fail to represent homogeneous property of the sequences $A\\rightarrow B$ and $B\\rightarrow A$.  Would it be more appropriate to consider an alternative design which combines the *set view* with *sequence view* and let the AFA policy acquire a subset of features at each step in the *sequence view*? \n\n[*Methodology - Observation Missingness*]  (1)  I suggest the authors to rephrase the term. In conventional AFA literature, people refer *observation missingness* as the natural property for any AFA method, since when employing active learning  by default the algorithm would receive partial observation which comes *observation missingness*. I think the *observation missingness* referred in the paper refers to the missingness between train/evaluation stages which is a different type from that in conventional literature. (2) Though generally *observation missingness* could be classified into missing completely at random, missing at random, and missing not at random, it's suspicious if a performance evaluation (PE) method is able to work perfectly well on all the scenarios. Overall I think it's more possible a PE method could work well on a well defined subrange of AFA problems. \n\n[*Methodology - RL perspective*] I think the RL problem considered in the paper is offline RL problem, rather than off-policy RL, because the authors describe the task as off-environment policy training. I think the authors claim their work as off-policy RL mostly in the main paper, even though offline RL has been briefly mentioned from one place in appendix. The authors even claim offline RL is data inefficient, which I think might not make sense. I'm concerned if the RL part of its methodology is correct/clear. \n\n[*Methodology - Causal Graph*] I feel the presented causal graph is too high-level and not significantly novel. It is simplified for some purpose but I expect to see some reasoning over the distribution shift to play around with, such as representing the AFA policy and other factor leading to distributional shift with different variables to derive the formula with. The authors seem to have combined everything leading to *observational missingness* (AFA policy and sources of distributional shift) in one variable $R$, so that it is still unclear what happens to AFA PE model when we deal with the AFA policy and other assumed sources of distributional shift (other than AFA policy) respectively.\n\n[*Evaluation - Choice of baselines*] I feel the choice of  AFA RL policies are insufficient. So far the authors only adopt CATE and DQN, which is insufficient. CATE is an unpublished work and DQN is a weak off-policy baseline. It would be strong if the authors consider state-of-the-art RL methods proposed for AFA problems like SeqVAE. \n\n[*Evaluation - Results*] Though the paper makes rather strong claim saying the method can tackle various of missing patterns, i.e., missing completely at random, missing at random, and missing not at random, the empirical evidence is quite weak.  (1)  the evaluation results in main paper only consists of the MNAR domain, which is insufficient to support the authors' claim. (2) The authors claim their method could obtain the unbiased estimate with Figure 5, but I do not find so. It seems AFAIS with IPW base results in considerable biases compared with the baseline.  (3) There should be some suggestions on which type of methods fit for wha type of missingness, but the authors only present partial view on MNAR.  \n\n\n[*Evaluation - Scenarios*] I think it would be good if the authors could claim their method could work well under various types of interventions which could well cover the observation missingness types. In the current setting, it's unclear how sensitive the proposed PE method would be. \n\n**Other questions**:\n- a ML - > an ML\n- $\\bar{U}^{(k)} = observe(X_i)$: this definition is ambitious as the authors do not introduce the relationship between $i$ and $k$. ", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "1: The contributions are neither significant nor novel."}, "extra_text": {}}]

Author rebuttal:

Decision: Reject