Risk-Averse Active Sensing for Timely Outcome Prediction under Cost Pressure
Authors: Yuchao Qin
Mihaela van der Schaar
Changhee Lee
Venue: NeurIPS 2023 poster (2023)
Keywords: active sensing
value of information
risk-averse learning
TL;DR: We propose a novel risk-averse active sensing approach that achieves cost-efficient patient outcome diagnosis.

Abstract: Timely outcome prediction is essential in healthcare to enable early detection and intervention of adverse events. However, in longitudinal follow-ups to patients' health status, cost-efficient acquisition of patient covariates is usually necessary due to the significant expense involved in screening and lab tests. To balance the timely and accurate outcome predictions with acquisition costs, an effective active sensing strategy is crucial. In this paper, we propose a novel risk-averse active sensing approach RAS that addresses the composite decision problem of when to conduct the acquisition and which measurements to make. Our approach decomposes the policy into two sub-policies: acquisition scheduler and feature selector, respectively. Moreover, we introduce a novel risk-aversion training strategy to focus on the underrepresented subgroup of high-risk patients for whom timely and accurate prediction of disease progression is of greater value. Our method outperforms baseline active sensing approaches in experiments with both synthetic and real-world datasets, and we illustrate the significance of our policy decomposition and the necessity of a risk-averse sensing policy through case studies.

Risk-Averse Active Sensing for Timely Outcome Prediction under Cost Pressure
Yuchao Qin Univerisity of Cambridge, UK
Mihaela van der Schaar University of Cambridge, UK Alan Turing Institute, UK
Changhee Lee
Chung-Ang University, South Korea
Abstract
Timely outcome prediction is essential in healthcare to enable early detection and intervention of adverse events. However, in longitudinal follow-ups to patients' health status, cost-efficient acquisition of patient covariates is usually necessary due to the significant expense involved in sereening and lab tests. To balance the timely and accurate outcome predictions with acquisition costs, an effective active sensing strategy is crucial. In this paper, we propose novel risk-averse active sensing approach RAS that addresses the composite decision problem of when to conduct the acquisition and which measurements make. Our approach decomposes the policy into two sub-policies: acquisition scheduler and feature selector, respectively. Moreover, introduce novel risk-aversion training strategy to focus on the underrepresented subgroup of high-risk patients for whom timely and accurate prediction of disease progression is of greater value. Our method outperforms baseline active sensing approaches in experiments with both synthetic and real- world datasets, and we illustrate the significance our policy decomposition and the necessity of risk-averse sensing policy through case studies.
1 Introduction
Timely docision-making through accumulated observation history has attracted significant attention in the machine learning community, with broad impact and applications in healthcare [8]. Consider a typical decision-makiin scenario in which an agent employs a sensing policy to actively collect diagnosis-relatee information from an underlying feature trajectory. In the presence of cost pressure, the goal of the agent achieve timely and accurate predictions of a time-varying outcome of interest based on the sensing history, i.e., feature observations accumulated over time, while maintaining reasonable expense of feature acquisition. At each decision step, we refer to the problem deciding when to make new acquisitions and which features to measure active sensing [2 24). The optimal sensing policy can be achieved by negotiating the subjective trade-off between outcome prediction (accuracy and timeliness) and acquisition cost over time.
special class of the active sensing problem has been extensively explored in the optimal stopping literature 22 6 15). In optimal stopping, the agent focuses on the timely diagnosis of terminal status, where observation ceases immediately after confident diagnosis can be made [ Thereby, the acquisition cost in optimal stopping is typically considered as the time span of the information collection process. Recently, Jarrett and van Schaar allow for the consideration of measurement cost of individual feature variables in optimal stopping, enabling better scheduling of the acquisition sequences. However, their approach still focuses on
9 introduced novel sensing framework to
37th Conference on Neural Information Processing Systems (NcurIPS 2023).

---

a static terminal status and is unsuitable for general purpose active sensing where the outcome of interest varies over time.
10T
Scosing History
Disease
Outcome Prediction
Progrest
I-h
O xll
Inlion--
P
Trajectorios the knn S1
G 13 iis
Tox:
10
(Mis Auri)
Alst Evert G
H4
8dadsle of (elineag (Merc A) it
PQv,M2)
0
250
Disgrosis
01000
(a) Active sensing process
(b) Long-tailed sensing deficiency distribution
Figure 1: Overview of active sensing strategies. Active sensing in the dynamical setting (a) involves two distinct decisions on: what features to measure; and when to perform next acquisition. The tradc-off between diagnosis accuracy and acquisition cost usually yields (b) long-tailed tailed sensing deficiency distribution. The sensing deficiency (X) accumulates the weighted sum of the diagnosis error and measurement cost over time. Higher sensing deficiency values indicate failures the sensing process.
In this paper, focus more general healthcare scenarios where timely diagnosis and intervention need to be achieved through active sensing under certain budget. The budget serves as the source of cost pressure and prioritize safe and low-cost measurements in the sensing process. For instance, the management of Alzheimer's disease (AD) requires continuous monitoring and staging of disease progression, which is crucial for early detection and intervention of high-risk patients [8]. However, certain measurements involved in diagnosing AD are notably expensive. A typical example is positron emission tomography (PET) scans. PET scans provide precise information about AD-associated proteins in the brain and are widely used for monitoring AD progression (3]. Nevertheless, the high accuracy of PET scans compared to cognitive tests is accompanied by greater financial costs as well as potentially harmful exposure to radiation 77] A desirable active sensing policy should adaptively balance measurement costs and diagnostic accuracy at different disease stages to provide timely diagnosis and intervention under cost pressure. Such property is not only useful in healthcare but also other applications such as delay-sensitive wireless communication [14]
Another crucial but relatively unexplored aspect of active sensing is the consideration of tail risk. To illustrate this concept, let's examine the prediction of AD progression. active sensing, the trade-off between diagnosis eтror and acquisition cost is usually achieved through the minimization of their weighted sum 23 24). As depicted in Figure ((b) the weighted objective (QT(X) typically exhibits a long-tailed distribution with a notable concentration of high-value patients at the tail side. We consider these patients to be of higher value in the sensing task for two main reasons. Firstly, accurate diagnosis of many high-risk patients heavily relies on biomarkers obtained through expensive PET scans such as fluorodeoxyglucose (FDG) PET and florbetapir (AV45) PET 7 [0]. A vanilla sensing policy optimized for the entire population would easily fail to arrange enough PET scans for these high-risk patients due the high costs, leading to degraded sensing performance (the long tail as shown in Figure ((b)). In the meantime, the majority of patients tend to remain in stable conditions. Their disease stages can usually be diagnosed even with very sparse acquisitions over time. A biased sensing policy may overemphasize the importance of high-risk patients and order frequent PET scans for all patients to avoid false negatives in its diagnosis, which contributes to the long-tailed distribution from another perspective. To avoid such situations, must be risk-averse such that can effectively cater to the needs of both the majority of stable patients and the high-risk subgroups requiring frequent follow-ups, ensuring timely and accurate detection of disease progression under cost pressure.
active sensing strategy
Contributions. In this paper, we propose a novel continuous-time active sensing approach to address the aforementioned desiderata. Our sensing approach adaptively balances measurement costs and diagnostic accuracy by factorizing sensing policy into two sub-policies: acquisition scheduler which

---

determines when to conduct feature acquisitions (i.e., the time interval between adjacent follow-ups), and feature selector that decides which observations to make. Furthermore, our sensing approach adopts the risk-averse strategy and focuses on the trade-off's happen at the long tail, where accurate diagnosis becomes particularly crucial. By dynamically identifying patient trajectories located at the tail side, we optimize our sensing strategy using the conditional value-at-risk (CVaR) loss to provide guarantees on timely and accurate prediction for high-risk patients while avoiding biased decisions for patients with stable conditions.
We validate our approach through experiments on synthetic and performance on high-risk patients is significantly impacted by their relatively small cohort size. With the consideration of acquisition costs, our method able to adaptively identify the patient subgroups at the long tail of sensing deficiency distribution and achieves improved diagnostic performance compared to the state-of-the-are active sensing benchmarks.
datasets, where the sensing
2 Active Sensing in Practice
In this section, with a focus on healthcare applications, we introduce a new active sensing approach, which we refer as Risk-averse Active Sensing (RAS), to solve the decision-making problem of when to conduct feature acquisitions and which features to observe considering prediction accuracy. timeliness, and cost pressure. We will start by providing general framework for active sensing problems.
2.1 Preliminaries: Active Sensing
Notation. Let xx) E Rdd be the characteristic of a patient at time t E [o, T' which is a collection of d (costly) observable time-varying covariates about an underlying disease progression of our interest [ X, = (I(T)]T&& E [0, f]] represents temporal trajectory of observable features until time / S T. For ease of description, we will occasionally abuse the notation and use X to denote the entire patient trajectory Xy. Let y E y be the outcome of our interest which represents the underlying health status of patient (e.g., diagnosis, adverse outcome, etc.) at time f. We assume the joint distribution p(X,,B)) from which the trajectory X. and the outcome y/ are drawn. Hereafter, we focus our description on K-class classification tasks, i.e., y (1,2,...,,K)
We denote an active sensing policy æ. In the sequential outcome prediction task, for each patient X. policy π arranges a set of follow-up times (t1.t2..... ,r] C [0, T] and determines the subsets of features m, to be measured at each time step 4,, where m; is binary vector. The collected observations of patient trajectory X until time form sensing history X It is worth noting that X is piece-wise wise constant over time as it is only updated at discrete follow-ups. At each step. the sensing policy π shall generate a diagnosis of the patient by approximating the conditional ((y.(X7)
Dataset. For the active sensing task, we assume access to an observational dataset 'D containing electrical health records (EHRs) of N patients, i.e., D = {X,, (l4, ((4z), yt, ) of each patient consists observations collected from L follow-ups. It is worth noting that the record X,, E D is equivalent a sensing history.
1.N The record
Following the convention in active sensing literature 9 24), we consider the cost of measurement on each tine-varying observable feature to be time-invariane and thus can be represented cost vector c, where the j-th element, denoted as (c) is the cost for measuring the j-th time-varying feature in vector E X. The central goal of active sensing is then balance between the outcome estimation accuracy based on sensing history X and the accumulated acquisition costs over time.
Challenges. In practice, there are two major challenges that confound the active sensing problem:
1. Adaptive follow-up intervals. Most existing active sensing policies assume a constant follow-up interval. However, in real-world healtheare settings, the desirable acquisition intervals usually change along with disease progression. Sensing policy with constant decision intervals creates redundant follow-ups and may induce extra costs for feature acquisition.
In general, observation of patient's 's underlying disease progression ends at some time 7 > 0 due to death or lost to follow-up.
3

---

2. Failures at the long tail. Vanilla active sensing policies are optimized for the entire population. For diseases like AD, the imbalanced proportion of high-risk and stable patients may lead to over-conservative sensing policies that make no observations. Similarly, sensing policies targeted on high-risk patients may generate biased decision with higher acquisition costs for patients in stable conditions. Both situations lead to failures for samples at the long tail of sensing deficiency distribution.
In the following, we first introduce a generalized sensing policy и with the decomposition of decisions on follow-up interval and feature selection. Then, formulate the active sensing task into risk- averse optimization problem, the solution to which addresses the second challenge.
2.2 Decomposing the Active Sensing Policy
Vast research in active sensing has assumed a constant follow-up interval between two consecutive patient visits and has primarily focused on the sequential feature selection problem, determining which feature variables to measure [6, 24].. However, considering the dynamic and complex nature of disease progression, sensing policies with dynamic follow-up intervals are broadly desired in real-world applications. In this paper, we propose a generalized formulation of the sensing policy, i... π (Tm. A.), where the acquisition scheduler ia determines the time for next follow-up, the feature selector determines which features to measure during the subsequent visit.
Consider the i-th follow-up of patient X time 4.. The next follow-up time 4,+1 is firstly determined
by acquisition scheduler TA as 1,+1 = f; + A; A; ~ TA (X,)) Then, at time 4,41, the feature selector Tm. determines which subset of patient covariates to measure using a binary vector mi+1 Tm (m(,)) M2 E {0, 13d. The new observation is calculated as m;+1 @ (01441) ) where the j-th element
(mi+1 C (x(t++)))) {{
(ar(4441))
(m:41)) =
otherwise.
2.3 Towards Risk-Averse Sensing Policy
Trade-off between accuracy and costs. Consider the i-th follow-up at time 4, The feature selection determined by m, ~ Tm "m(X,, (X
The acquisition cost is evaluated as r. (m.) = c m, where e is the cost vector. The updated sensing history X = -X_,, U(m, ox(t,)} is then used to determine the acquisition interval 4+1 - I = A; T (XE)) Assume a diagnosis error function (X7, A;) that evaluates the mismatch between p(y| and p(y) (x7) during the interval A, The trade-off between diagnosis accuracy and measurement costs of the i-th interval is defined as the following reward signal
r(X7 m,, A,) rm (m.) + (X7, Ai)
(1)
where À= >> 0 is coefficient chosen to balance the two terms.
Sensing deficiency. We define the sensing deficiency of policy æ along a patient trajectory X as the expected cumulative reward as shown in
(*x(x) = E2,, by M7, Me2.... Ma mi
Σ TX m, A.)
(2)
where discount factor y E (0, 1] is used to tackle long trajectories. In this paper, we take y = 0.99.
Trajectories at the long tail. As mentioned earlier, one major challenge for active sensing in practice is the potential sensing failures for patients at the tail of sensing deficiency distribution. Since the distribution tail of (X) changes along with the updates of policy in this paper, we propose to optimize the sensing policy over a dynamic subset of patient trajectories located at the long tail. These tail trajectories
be identified through the upper quantile Pa. where Ex [xQ X) Z pa)] = a. Assume the marginal trajectory distribution p(X). we denote with ST (X X ~ P(X) (X) 2 Pa} the set of tail trajectories under policy π.
Conditioned on the quantile factor a E [0, 1], the sensing deficiency of trajectories S is considered as the value at risk that needs to be improved. Based on the evaluation of CVaR in sensing deficiency distribution, we formulate our active sensing task as a risk-an averse optimization problem.
minimize CVaR & Exes: (x)())).
(3)
x

---

Solutions to problem are considered risk-averse since they reduce the risk of sensing failures in the worst scenarios (5%) Our proposed risk-averse sensing problem has two major distinctions from the class of robust adversarial reinforcement learning (RI.) problems (e.g., [8] ): i) we directly optimize the value at risk C'aRR: and ii) the adversarial agent replaced by the dynamic tail trajectory subset
Sg. As such, our proposed problem requires no online interactions with the data generation process
and thus can avoid the sensing policy и from being biased toward unrealistic or out-of-distribution trajectories generated by simulators.
3 Method: Risk-Averse Active Sensing
Solution to the risk-averse active sensing problem in (3) given observational data is faced with three unique challenges. First, we only have access to observational data that consist of discrete measurements of (continuous) patient trajectories. Second, we need access a baseline conditional outcome estimator, (Xt), to estimate the diagnosis error Ty in (I). Third, the solution of (3) heavily relies on the appropriate construction of the subset, Sa, which, by definition, depends on sensing policy itself.
Handling discrete and sparse observations. Dataset D only contains discrete observations of patient trajectories. However, the adaptive acquisition intervals generated by sensing policy т may require measurements of the underlying patient trajectories at time points not included in D. To solve this problem, we adopt a linear interpolator I as proxy to obtain estimations of these values. In the meantime, policy π by nature generates sparse sensing history with partial observations of patient trajectories. To properly encode the missingness and adaptive observation intervals in sensing history XT, we include the neural CDE as the base encoders in our approach. As enlightened in [T]. CDE is a continuous- time generalization to recurrent neural network (RNN) and offers natural and elegant approach to encode the information in sparse and irregular measurements of a patient trajectory.
Specifically, given a discrete sensing history X,, we first convert it into a continuous trajectory (T)) X() via the interpolator I. Then, CDE encodes the temporal changes in X, as the dynamics of latent variable xtt) E R' via
=(t) = =(1o)+ + / fo(s(t))du(()) for E [4o 4].
(4)
where to 2 O, fo R + I is map with learnable parameters Φ, initial state 20 = z(t = 0)
could be manually specified or learned from data.
Timely and accurate predictions. In this paper, we build CDE-baser outcome estimator fp(t, to approximate the density p(yel X.). where X is sensing history of trajectory X. Details about Jr can found in the Appendix. Given a patient with record X E DD to provide timely and accurate predictions about the underlying disease progression, our sensing policy aims to minimize the mismatch between bascline estimate fP(T, X) and sensing history-based prediction fP X.). For this purpose, in time horizon [t,, t+41), the diagnosis error function for the i-th follow-ups is computed as the cumulative mismatch:
{ DDss(f((TxX))lff(rt..))) 1
A.)
(5)
where Das is the JS divergence, 1++1 14 +.+
The distinction from previous works that utilize a single-step mismatch - such as at a pre-specified time interval [24] is that (5) provides precise proxy to evaluate how timely and accurate the predictions are by gauging the impact with respect to the acquisition interval A,, which is a desired property. Adverse clinical outcomes of the underlying disease progression are likely occur when the outcome probability, p(y. X2), hits a certain threshold, often accompanied by sudden rise or fall. Making too early too late detection of such rise/fall based on the sensing history, i.e., P()x) will drastically increase the cumulative mismatch in (5)
Penalty on invalid visits. We define the follow-up at time 4;+1 as an invalid visit if no patient covariates are measured, i.e., m, = Allowing invalid visits potentially yields degenerated active sensing policy and makes the policy decomposition meaningless. To prevent such situations,
5

---

1-th follow-up
1)-th follow-up
Seasing History
mi.1 x(t-a)
x(fuus
Riss
+
Diagnosis
Festuro Selection
Inaopelacr
Benrjp)
Fn
Pr:
mic
Intepollace
Pemart
Rc.8%
A
Aspialinn Schecake
Policy
Nr
brorpelerr Hittrry Enceder ie
EHR
RAS
Figure 2: The model structure of RAS.
extend the acquisition cost function I'm as I' (m) c m I[ + 0) + v I(m 0), where penalizes the invalid visits scheduled by the sensing policy m
Network structure. Similar to the predictor fr, the neural CDE and interpolator I are also employed in our proposed policy network π to construct embedding of the discrete sensing history X As illustrated in Figure our sensing policy π (parameterized by 0) constructed with the following three networks:
The history encoder, fH. is neural CDE encoder that maps sensing history X," into a summary vector ((T)) = fH(T, ((X]).
The feature selector, fM, is sub-policy network that governs the decision about which subset of patient features to be measured. In a follow-up, Sar takes a summary vector i(T)) as input and outputs a vector p(r) (h(r) that governs the element-wise Bernoulli distribution to generate a binary feature selection vector m, i.e., my Bern (p(r)) for next follow-up 7'.
The acquisition scheduler, fa. is sub-policy network that determines the time for next follow-up. Particularly, after each follow-up, it takes the summary vector h(r) as input and generates two scalar parameters 3 = f& (h(T)) of a Beta distribution, i.e., ~ Beta(a, B). The realizations of the Beta distribution are then used to generate the time interval A [A min A maxx] via A 1ii))- where A min and A max are the minimum and maximum allowed intervals between two follow-ups, respectively. set a, 3 Z to ensure that the time interval follows unimodal distribution.
A A min (A max
Policy gradient. Since the diagnostic error function in (5) is non-differentiable, to back proppapre gradients from the objective CVaR, Exes= [Q* (X] through the active sensing process, we apply the advantage-based approach 21 [22] to estimate the policy gradient of i as the following:
C&CVaR, = E,
(R(X),)) - Ro((3,))) Ve log Anii)
(6)
ST
where 0 is the collection of learnable parameters of policy т. Given the s-th patient in S the cumulative reward R(X,,) 7", m,, A.) is empirical evaluation of the sensing deficiency in (2) for the corresponding sub-trajectory in horizon [t,, T7. Following the actor- framework, we utilize a baseline sensing deficiency estimation Ro(X (X.. in ( to reduce the variance of policy gradient estimation (21).
The baseline estimation Re (X,) is obtained from a critic network fe (parameterized by w)) Taking the embedding h(t) fu (t, I(X?) from the history encoder fH as input. The critic fc is trained

---

to minimize the following mean squared error (MSE) loss.
Le = E,
(((.,))) - Rc(X
S
km
Algorithm Risk-averse active sensing
Input: dataset D. predictor fr. quantile learning rates Ir., Ir.. Initialize: learnable parameters 0 (of πT, a (of fc).
to do if k mod M = then
Po quanti 1e( D,a))
S +- [X.]Q= Pn. E D)
end if
< lr. V.CVaRa
v + lr. Ce
end for
Output: risr-averse policy with parameters
("(X) (X) of policy is optimized over the Given the baseline predictor fp. the sensing deficiency Q"
subset S tail trajectories as illustrated in Algorithm T Specifically, parameters 0 of policy и are updated based on the policy gradient VoCVaR, in (6.. and the critic network Sc is trained with loss function Cc to provide appropriate baselines for the sensing policy т. The collection patient trajectories in ST is iteratively updated every M training epochs such that it can stay informative about the sensing deficiency distribution. Note that the learning rates of critic fc and policy π are intentionally set to be different to facilitate convergence.
4 Related Work
We first provide in Table an overview of the distinction between our proposed approach and other related works in active sensing literature. special class of active sensing task is formulated under the optimal stopping framework. With set of explicitly defined terminal states, optimal stopping approaches focuses on achieving confident diagnosis these states while minimizing the time span The consideration of information acquisition cost was introduced recently into optimal stopping in (9]. Similarly, the trade-off between diagnosis accuracy and costly measurement has been studied in the more general sequential prediction tasks [23] 24]. In this paper, we propose a novel sensing policy decomposition and extend the active sensing analysis into continuous-time settings with adaptive follow-up intervals and highly sparse sensing histories.
of the sensing process 11 6 15]
Table Comparison of active sensing approaches. Typical active sensing approaches are compared based on the problem formulation, sensing history representation, the modelling of acquisition intervals well as the following key concerns highlighted in this paper: (1) supports continuous-time patient trajectory. (2) considers cost pressure on features acquisition. (3) achieves decomposition of feature selection and acquisition scheduling. (4) provides worst-case performance guarantees with risk-averse policy.
Method
Problem
History Eimbedding
Acquisition Interval
(3)
Ahmad and Yu 01
Optimal Stopping Bayesian Update
Fixed (A A)
X
X
X
Jarrett and van der Schoar
Optimal Stopping
Bayesian Update
Fixed (A = A)
X
X
X
Yoon al. 2 2
Sequential Prediction
RNN
Fixed (A A)
X
X
RAS (Ours)
Sequential Prediction
Neural CDB
Adaptive
Temporal feature selection. pointed out by Yoon al. 24
active sensing can be considered
as the temporal generalization of feature selection tasks [3 In each decision step. the sensing policy expected select set of important feature variables to make the prediction. The resulting sensing history marks critical time points and key feature variables for accurate outcome estimation. However, existing active sensing literature primarily focuses on the feature selection perspective and usually assumes fixed follow-up interval for every patient trajectory 24). A fixed acquisition
19

---

interval can lead to delayed diagnosis or extra costs due to redundant feature measurements. As mentioned earlier, a desirable sensing policy shall achieve flexible follow-up intervals for each patient such that the shifts in underlying disease stages can be properly addressed.
Risk aversion. Conventional active sensing strategies typically consider the diagnosis accuracy to be equally important for all trajectories in a dataset. However, in many healthcare applications, there are needs to prioritize smaller subgroup of high-risk patients for whom timely detection of adverse outcomes is partieularly valuable. As a central component of our proposed method, the concept of CVaR [20] enables the selective optimization for patient trajectories in the long tail of sensing deficiency distribution. Optimizing sensing policies over the corresponding tail set S yields risk risk-averse active sensing strategy π that achieves guaranteed diagnosis accuracy for high-risk patients. While rask-aware decision decision-making has been extensively explored in the field of RL 25 to the best of our knowledge, our proposed method the first risk-averse active sensing algorithm for adaptive feature acquisition in continuous-ting settings.
5 Experiment
In the experiments, we evaluate the effectiveness of our proposed risk-averse active sensing approach RAS on both synthetic and real-world healthcare datasets.
Synthetic dataset. We construct a synthetic dataset D5 of N = 2000 samples. Every sample X contains L 20 discrete observations in / E [0, 2]. Each observation yields feature vector F2, 23, =z]", We set 21 (t) min(1, (e"((t-F) w(t -) 1)) 7), where w follows a Gaussian mixture distribution of 0.8 (0.3,, ) 0.2-- 0.1²). Exp(1.0). As proxy of æ1, we set (t) wit r)l[(t 2 r). Variables z3 sin + @) and (t) cos + g). where ~ (0, 12). The outcome E {0, follows a Bernoulli distribution Vo+ ~ Bern(p), and the likelihood for y/ 1 is calculated as p 311-=11² The cost vector is set to be [1.0,0.7 0.1, 1.0, 1.0]", which allows 22 to be measured as a cheap proxy of æ1 for outcome prediction.
T2,
ADNI dataset. The Alzheimer's Disease Neuroimaging Initiative (ADNI) dataset includes records on AD progression of N = 1002 patients with regular follow-ups every six months. We consider the active sensing task to predict patient outcomes in the initial L = 12 follow- ups with four biomarkers from PET (FDG and AV45) and MRI (Hippocampus and Entorhi- nal) imaging, respectively. Based on the disease staging guideline [6 17]. target outcome Ve E (Normal, mild cognitive impairment (MCT), AD} at each time point is determined via the corresponding Clinical Dementia Rating scale Sum of Boxes (CDR-SB) score with cutoff thresholds at 0.5 and 4.5. To reflect the higher cost and harmful radiation exposure in PET scan, measurement costs for PET and MRI biomarkers are set to be 1.0 and 0.5, respectively.
Experiment setup. We first fit the outcome estimator fp on each dataset with 64/16/20
veann/validation/test splits and find the optimal estimator with the best prediction accuracy given sparse observation of test samples as inputs. Then, we freeze the parameters of the optimal fr and evaluate the sensing performance of RAS and other baselines on the corresponding data split. We consider four baselines in the experiments: FO) predictor fr with dense sensing history; ASAC)
24: : NLL.) adaptive sensing with negative log-likelihone (NLL. as diagnosis error function [12] AS) RAS(a 1.0) with constant acquisition interval. Further details of the experiment setup and parameter selection results can be found in the Appendix.
Table 2: Benchmark of sensing performance on the synthetic dataset Ds.
METHOD
ROC
PRC
CoST
de-0.3
dy
do
0.680-10.000 0.000
0.655 :
31.000-10.00
0.502-10.000
0.349 10..000.0000
0.285
ASAC
0.605
0.559:
1.788
1.099:10.644
1.066
1.05240.641
AS
0.671
0.614.10.000
4.501 10.497
0.577
0.522.10.012
0.47910.015
NL.
0.636-10.023
0.588-0.011
2.968-10.774
0.993-10.131
0.974:10.141
0.97540..14
RAS (OURS)
0.600
0.647:10.006
6.077:10.953
0.024
0.264:10.0866
0.246:100771
The 95% confidence interval (CI) evaluated with five diffenent random seeds. Best result in each column are highlighted in bold. The marker 7 indicates valle 0.05.
detps://addi Ioni. wsc. edu
8

---

Timeliness and accuracy. Timely and accurate outcome prediction given sparse observations of patient trajectories is essential for practical applications of active sensing. In the experiments, we report the area under the curve of receiving-operate characteristic (ROC) and area under the curve of precision-recal (PRC) as assessments of the prognostic value of active sensing policies. In the meantime, the timeliness of an active sensing policy и is evaluated with d3- the mean absolute mismatch in diagnosis time of event p(y = _|x;) 2 å compared to ground truth on the test set. The benchmark results of RAS against four baselines on the two datasets are provided in Table 2 and Table respectively.
With fixed acquisition interval of A = 0.2, FO achieves high predictive accuracy at the expense of highest acquisition cost. Despite having the smallest acquisition cost, ASAC suffers significantly from the low diagnosis accuracy and is unable to achieve timely diagnosis of alert events. In contrast, our method (RAS) has reasonable acquisition costs and best timeliness as illustrated in Table 2 The increase in average cost of RAS compared to the AS baseline could be linked to our focus on the tail set 5% The increased acquisition costs are necessary to achieve a significantly lower delay in alert event diagnosis for high-risk patients in Sg. The improved timeliness of RAS over FO suggests that the predictor fP might achieve better generalization with sparse sensing history as inputs. Similarly, according to Table 3 our method RAS outperforms most other baselines in timeliness and cost efficiency except for FO. This is within expectation since the outcome estimations from fr are used as the reference for timeliness evaluation, and these estimations are generally close to the predictions made by the FO baseline.
Table 3: Benchmark of active sensing performance on ADNI dataset.
METHOD
ROC
PRC
COST
dézo.1
da=o.a
da=0.a
FO
0.747:100000
0.577:10.000
26.865-100.00
0.001
0.510:0.000
0.591:1.0000
ASAC
0.521:00116
0.35211.103
0.043.10.186
0.527
3.061
3.00110.000
AS
0.704=0.023
0.519=0.034
0.566=0.854
1.326:40.09
2.31440.348
2.357=20.375
NLL
0.697.0.018
0.512110.020
3.986
1.040:10.199
2.176110.660
2.739.10.135
RAS (OUBS)
0.730=0.007
0.560:0.012
8.61421.157
0.820110.096
1.370::0.227
1.192:10.76
The 95% confidence
evallated with different random The rimeliness dg messoral bosed the likelihood developing Best result each colame are highllghhed in bold. The marker indicates p-value 0.05.
140
8001
NLLL
FO
120
HAS
301
100
A A0.
100
15-
50-
pum 10:40
0005
10
4
0.8
0.5
152
250
Pareto front of sensing policies.
(b) Distributions of sensing deficiency Q" (X).
The impact of risk-aversion and adaptive acquisition scheduling on active sensing policies. Selection of trade-off coefficients, e.g., X, can be performed by scarching for the (a) Pareto front. Pareto optimal models belonging to the front are highlighted with circles. Their parameters are annotated in gray texts. Effectiveness of the risk averse objective in (3) is reflected by the tail size of (b) (x) distribution. The risk-neutral (a = 1.0) version of RAS has degraded sensing performance for tail trajectories. The significantly worse performance of the AS baseline further highlights the henefit of having adaptive acquisition intervals.
Figure 3:
Trade-off between timeliness and acquisition costs. The selection of model parameters could be difficult when two or more criteria are involved in the evaluation. Here, we illustrate the sensing performance of different policies on the synthetic dataset in Figure (3a) highlight the ones the Pareto front with gray circles. These policies are considered Pareto optimal since their timeliness

---

(dz=0.s) and average acquisition cost cannot be simultaneously improved by swapping parameters with other policies. Benefitted from the risk-averse training strategy, most sensing policies obtained via RAS are centered around the knee point of the Pareto front, which helps to explain the outstanding cost efficiency of RAS as reported Table 2
Reshaping the sensing deficiency distribution. To illustrate the effectiveness of the risk-averse objective in 3). we compare the empirical distribution of sensing deficiency (x(X) of RAS with the ablations of risk-neutral sensing (a = 1.0) and AS baseline (a = 1.0, constant acquisition interval 1.0) on the synthetic dataset. All three models are trained with the same trade-off coefficient À 300. As illustrated in Figure336b RAS is able to effectively optimize the sensing performance for trajectories in the tail S% and reduces the upper a quantile of Q (X) to Pa=0.1 10.40, Factor = 1.0 completely disables the risk-aversion training strategy in Algorithm | Thereby, clear increase of sensing deficiency (quantile Pa=0.1 grows from 10.40 to 20.01) is observed with the risk-neutral ablation of RAS. Similarly, without adaptive scheduling of acquisition intervals and risk-averse optimization strategies, the AS baseline illustrates the failure of conventional active sensing paradigms at the long tail of Q* (X) distribution.
6 Conclusion
In this paper, we introduce novel risk-averse active sensing approach RAS to address the chal- lenging continuous- active sensing problem. Through effective decomposition of decisions on feature selection and acquisition intervals, our approach offers valuable insights both feature importance and timeliness of patient follow-ups. The novel risk-aversion training strategy in RAS enables the prioritization of high-value patients the long tail of sensing deficiency distribution and provides guarantees on diagnosis accuracy for worst-case scenarios. The effectiveness of our method is evaluated through experiments synthetic and real-world healthcare datasets, where RAS outperforms all baseline active sensing approaches and achieves accurate and timely outcome diagnosis while maintaining reasonable costs in feature acquisition.
Broader Impacts
The development of advanced active sensing strategy is essential for precision medicine. Appro- priate sensing policies can help clinicians to make tailored follow-up schedules for their patients and effectively utilize the costly and potentially harmful measurements on some important patient characteristics under certain levels of cost pressure. We note that the reduction in acquisition cost is usually associated with degraded diagnosis accuracy and delayed detection of some adverse clinical events, especially for high-risk patients that could benefit from more frequent follow-ups on disease progression. In this paper, we attempt to address this challenge through the risk-averse policy learning strategy in our proposed active sensing approach RAS and achieve improved sensing performance (accuracy and timeliness) on the entire population. Nevertheless, practical applications of active sensing approaches would require careful audits and assessments by human experts to avoid poten- tial negative impacts. The sensing decisions from RAS are only suggestions optimized based on observational data and should not be directly applied without the evaluation of clinicians.
Acknowledgements
Yuchao Qin was supported by the Cystic Fibrosis Trust. Changhee Lee supported by the IITP grant funded by the Korea government (MSIT) (No.2021-0-0134) AI Graduate School Program, CAU). This work supported by Azure sponsorship credits granted by Microsoft's 's AI for Good Research Lab.
References
[1] S. Ahmad and A. J. Yu. Active sensing as bayes-optimal sequential decision-making, In Proceedings of the Twenty-Ninth Conference on Uncertainty in Artificial Intelligence, pages 12-21, 2013.
10

---

[2] A. M. Alaa and M. van der Schaar. Balancing suspense and surprise: Timely decision making with endogenous information acquisition. Advances Neural Information Processing Systems, 29, 2016.
[3] W. Bao, F. Xie, C. Zuo, Y. Guan, and Y. H. Huang. Pet neuroimaging of alzheimer's disease: radiotracers and their utility in clinical research. Frontiers Aging Neuroscience, 13:624330, 2021.
[4] L. Brunke, M. Greeff, A. W. Hall, Z. Yuan, S. Zhou, J. Panerati, and A. P. Schoellig. Safe learning in robotics: From learning-based control safe reinforcement learning. Annual Review of Control, Robotics, and Autonomous Systems, 5:411 144,, 2022.
[5] H. Chernoff. Sequential design of experiments. The Annals of Mathematical Statistics, 30(3):
755-770, 1959.
[6] S. Dayanik and A. J. Yu. Reward-rate maximization in sequential identification under stochastic deadline. SIAM Journal on Control and Optimization, 51 51(4):2922-2948, 2013.
[7] E. Dolgin. Alzheimer's disease is getting easier to spot. Nature, (597775511111101100000 2018.
[8] B. Dubois, A. Padovani, P. Scheltens, A. Rossi, and Dell' Agnello. Timely diagnosis for alzheimer's 's discase: a literature review on benefits and challenges. Journal of Alzheimer's disease, 69-3):617-631, 2016.
[9] D. Jarrett and M. van der Schaar. Inverse active sensing: Modeling and understanding timely decision-making. In International Conference on Machine Learning, 2020.
[10] J. Jing, F. Zhang, L. Zhao, J. Xie, J. Chen, R. Zhong, Y. Zhang, and C. Dong. Correlation between brain 18f-av45 and 18f-fdg pet distribution characteristics and cognitive function in patients with mild and moderate alzheimer's disease. Journal of Alzheimer's 's Disease, 79(3): 1317-1325, 2021.
[11] Kidger, J. Morrill, Foster, and T. Lyons. Neural controlled differential equations for irregular time series. Advances in Neural Information Processing Systems, 33:6696-6777 6607, 2020.
[12] J. Kossen, C. Cangea, E. Vértes, A. Jaegle, V. Patraucean, I. Ktena, N. Tomasev, and D. Belgrave. Active acquisition for multimodal temporal data; A challenging decision- maaking task. arXiv preprint arXiv:2211.05033 2022.
[13] S. M. Lundberg and S.-I. Lee. A unified approach to interpreting model predictions. Advances in neural information processing systems, 30, 2017.
[14] N. Mastronarde and M. der Schaar. Joint physical-layer and system-level power management for delay-sensitinn wireless communications. IEEE transactions on mobile computing, 12(4): 694-709, 2012.
[15] M. Naghshvar and T. Javidi. Active sequential hypothesis testing. The Annals of Statistics, 41 (6):2703-2738, 2013.
[16] S. E. O'Bryant, S. C. Waring, C. M. Cullum, Hall, L. Lacritz, P. J. Massman, P. J. Lupo, S. Reisch, R. Doody, T. A. R. Consortium, et al. Staging dementia using clinical dementia rating scale sum of boxes scores: a texas alzheimer's ' research consortium study. Archives of neurology, 1099119991 2008
[17] S. E. O'Bryant, L. H. Lacritz, J. Hall, S. C. Waring, W. Chan, Z. G. Khodr, P. J. Massman, V. Hobson, and C. M. Cullum. Validation of the new interpretive guidelines for the clinical dementia rating scale sum of boxes score in the national alzheimer's coordinating center database. Archives of neurology, 67(6):746-744 749, 2010.
[18] L. Pinto, J. Davidson, R. Sukthankar, and A. Gupta. Robust adversarial reinforcement learning. In International Conference Machine Learning, pages 2817-2826. PMLR, 2017.
[19] M. T. Ribeiro, S. Singh, and C. Guestrin. why should i trust you?" explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, pages 1135-1 1144, 2016.

---

[20] R. T. Rockafellar and S. Uryasev. Conditional value-at-risk for general loss distributions. Journal of banking & finance, 26(7):1443-1471. 2002.
[21] Schulman, Moritz, Levine, M. Jordan, and Abbeel. High-dimensional continuous control using generalized advantage estimation. arXiv preprint urXiv:/50.002453 2015.
[22] R. S. Sutton and A. G. Barto. Reinforcement learning: An introduction. MIT press, 2018.
[23] Yoon, W. R. Zame, and M. der Schaar. Deep sensing: Active sensing using multi- directional recurrent neural networks. In International Conference on Learning Representations, 2018.
[24] Yoon, J. Jordon, and M. van der Schaar. Asac: Active sensing using actor-critic models, In Machine Learning for Healthcare Conference, pages 451 451-473. PMLR, 2019.
[25] S. Zhang, B. Liu, and S. Whiteson. Mean-variance policy iteration for risk-averse reinforcement learning. Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pages 10905-10913 2021.
12

---

Appendix
A.1 Notation table
A summary of major notations used in this paper is provided below.
x: Feature variable of the patient.
y: Patient's outcome.
X: The continuous- time patient trajectory.
x The sensing history at time
x: Sensing policy.
m Feature selector.
TA: Acquisition scheduler.
m:Feature selection mask vector.
Q (x)::: Sensing deficiency.
CVaR: Conditional value-at-risk.
Tail risk quantile.
T: End time of observation.
Number observations in sensing history.
Σ: Interpolator.
A.2
Limitation
The risk-averse policy learning strategy proposed in this paper relies on the correct identification of tail-risk patients. Data quality of the EHR datasets has direct impact on method. Low quality dataset would lead to poor diagnosis accuracy of the baseline predictor Jp which compromises the validity and usefulness of RAS in practice. In the meantime, in RAS, patients located the long tail of sensing deficiency distribution are selected based on empirical evaluations of (X)). The selection results could be biased due to potential over-fitting of sensing policies on the training set. To tackle this issue, an "honest" and unbiased sensing performance evaluation on the validation set is necessary. However, the inference of acquisition costs from validation samples is beyond the main focus of this paper. We leave this challenge for our future work on risk-averse active sensing tasks.
Code Availability. The source code of RAS be found in the two GitHub repositories listed below:
The der Schaar lab repo: http:e///gthhhoo.cmm////eeeerrrr sensing
The author's personal repo: httppp////thttt.eccmm////eeeeeer
A.3 Estimation of Time-Van Varying Outcome
A baseline outcome predictor fP is required in 5 of the manuscript to evaluate the timeliness and accuracy of a sensing policy. Following the model-based approach in 24 we introduce network fr which we refer to as the baseline predictor to estimate the unknown conditional distributions of patient outcomes. Specifically, we construct the predictor fr with the neural CDE [ model to flexibly handle sensing history with irregular observation intervals compared to the RNN-based counterparts such as 23 24
The predictor consists of four components, i.e., - o I o CDE o I. I is linear interpolator. CDE is neural CDE that takes the continuous trajectories as inputs. fE multi-layer perceptron (MLP) that takes the CDE embedding as input. fy is an MLP with softmax output layer for categorical outcomes. Utilizing the linear interpolator 1 both the discrete EHR record X and sparse sensing history X can be converted into continuous trajectories.
We use Python package torchede for interpolation.
13

---

The solution to the latent CDE provides embeddings of discrete time-series observations and the sensing history as Zn,t = f (X),.)) and u,,t = fe(CDE o I respectively. The corresponding conditional distributions are then estimated via the MLP as fr(Xn,)) = fZ(,).) and fr(X",,) = (y respectively.
The predictor fp is trained to minimize the following loss:
N Σ Kynnta to fy (Zn,t)) + pelynao fr (2
Cp= NL
(8)
where En,t, -
nn. SE SCDEEoI n.t; )), and l is the K-class cross-entropy based on the observed outcome Un, and outputs of the predictor fp. Here, introduce the second term as a regularization for each patient trajectory with random dropouts. More specifically, we introduce random acquisition strategy T0 which randomly collects (sparse) discrete observations from the interpolated trajectories, i.e., I(X,), ). and constructs auxiliary sensing histories with random
dropouts, i.c., X ".t." The strategy n² randomly select feature variables to keep in the observation history based on the drop rate. This regularization improves the generalization the predictor by learning to estimate conditional distributions given randomly drawn sensing histories. Here, μ 0 is a coefficient that controls the strength of regularization.
A.4 Parameter Selection
Outcome predictor. We consider the drop rate p the auxiliary observation strategy T² as the hyperparameter of the outcome predictor. For both the synthetic dataset and ADNI dataset, we select the drop rate of auxiliary strategy TO form the set of E {0.0, 0.3, 0.5, 0.7) The optimal hyperparameter is selected based the average accuracy of the outcome predictor on five test sets of random observations generated by an auxiliary observation strategy nO with drop rate p = 0.7. The hyperparameter selection result is as follows:
Synthetic dataset: drop rate 0.7
ADNI dataset: drop rate = 0.7
Sensing policy. Then, we perform hyperparameter selection for the active sensing upproaches. Among the trained outcome predictors, we select the one with highest ROC score its test set as the shared predictor and keep parameters frozen. The training and test set corresponding to the selected optimal outcome predictor are used for the training and evaluation of the active sensing methods.
In our experiments, the hyperparameters of each active sensing method are selected based on the cost efficiency, i.e., ROC/ Cost, obtained the test
For the synthetic data considered our manuscript, we set the minimum and maximum allowed acquisition intervals min 0.2, A max 1.0, respectively. ASAC is not affected since it needs to directly work on original EHR records). The hyperparameters of each method are reported as follows:
ASAC: coefficient for acquisition cost 0.01 e {0.1, 0.01, 0.005, 0.001
AS: acquisition interval Á 1.0 {0.2, 0.5, 1.0), shared predictor fr with RAS.
NLL: coefficient for diagnostic error 100 (100, 300), shared predictor fp with RAS.
RAS: the coefficient for diagnostic error À 300 E (200, 250, 280, 300, 310, 320, 350, 400). discount factor 7 0.99, tail-risk quantile 0.1, penalty for invalid visits = 10.
For the ADNI data considered in our manuscript, we set the minimum and maximum allowed acquisition intervals A min 0.5, A max 1.5, respectively. ASAC not affected since it needs to directly work on original EHR records).
ASAC: coefficient for acquisition cost μ 0.1 {0.1, 0.01, 0.005, 0.001).
AS: acquisition interval Á 1.5 {0.5, 1.5), shared predictor fP with RAS.

---

NLL:
coefficient for diagnostic enror À 200 E {100, 200, 400). shared predictor fp with RAS.
the coefficient for diagnostic error X 400 {200, 250, 300, 350, 400,000440 450), discount factor = 0.99, tail-risk quantile a = 0.1, penalty for invalid visits v = 10.
RAS:
All methods are trained with K 200 iterations in the experiments. For RAS, we set the tail subset update interval M = 10 for Algorithm I in the manuscript.
Adjustments in experiment results. We have fixed some minor issues with model convergence when using CDE models. Specifically, the negative feedback included the CDE integration as follows:
=t) = z(to) + J (Fo(z(T)) - -1ci))))--1da))(())))))) (iing()))du(r)).oor E [to.
where diag creates a diagonal matrix from the input vector z. Such negative feedback loop avoids the unlimited growth of CDE integration over time and helps with model convergence. This modification causes some minor changes in the experiment results. However, the updated experiment results are consistent with the previous versions and our conclusion is not affected by these changes.
Additional results. We will release more evaluation on real-world datasets on our GitHub repo.
15


Reviews:
[{"review_id": "ZazSimFOsq", "reviewer": "Reviewer_L7Er", "summary": "This paper investigates timely outcome prediction by proposing a novel risk-averse active sensing approach RAS. The proposed RAS decomposes the policy into acquisition scheduler and feature selector to address the composite decision problem of when to conduct the acquisition and which measurements to make. In addition, RAS enables the prioritization of tail-risk patients in the risk-aversion training procedure. The experiments on synthetic and real-world healthcare datasets show its effectiveness. ", "questions": "The result shown in Figure 3 shows that both NRAS and RAS can reduce the long tail in the distribution. It seems NRAS is better than RAS for reducing the long tail, right? Why does this happen? ", "limitations": "The authors have discussed broader impact of this work.\n", "rating": 3, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "This paper is studying a very interesting and important problem - active sensing methods for early detection and intervention of adverse events. The proposed risk-averse active sensing method is technically sound. The experimental results show its effectiveness compared to the three baselines. ", "weaknesses": "Minor comments: 1.there are so many symbols used in the paper writing, it would be helpful to include a notation table in the appendix. 2. The titles of Section 2 and Section 3 are confusing. Probably, Section 2 is Problem Formulation (or General Framework), Section 3 is Methodology. 3. Highlight the best performance in Table 2.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the insightful comments. The reviewer’s comments regarding clarity and experiment results are addressed below.\n\n### 1. Notation table.\n\nWe thank the reviewer for the suggestion of including a notation table to improve clarity. The definition and explanation of major notations used in our manuscript are listed in Table R1.\n\nTable R1. Major notations in the manuscript.\n\n| Notation     | Definition                               | Notation               | Definition                                |\n| ----------   | ------------                             | --------------------   | ----------------------                    |\n| $\\mathbf{x}$ | Patient feature variable                 | $\\mathbf{X}$           | Patient trajectory                        |\n| $y$          | Patient outcome                          | $\\mathbf{\\hat{X}}^\\pi$ | Sensing history under policy $\\pi$        |\n| $\\pi$        | Sensing policy                           | $\\pi_\\Delta$           | Acquisition scheduler                     |\n| $\\pi_m$      | Feature selector                         | $\\mathbf{m}$           | Selection mask                            |\n| $D_{JS}$     | Jensen–Shannon divergence                | $\\mathbf{c}$           | Cost vector                               |\n| $r_y$        | Cumulative mismatch in label prediction  | $r_m$                  | Cost of feature acquisition               |\n| $r$          | Reward signal                            | $Q^\\pi(\\mathbf{X})$    | Expected cost                             |\n| $CVaR$       | Conditional value-at-risk                | $\\alpha$               | Tail-risk quantile                        |\n| $T$          | End time of observation                  | $I$                    | Number of observations in sensing history |\n| $N$          | Number of samples                        | $\\mathcal{I}$          | Interpolator                              |\n\n\nWe will include a more detailed notation table in the final version of our paper.\n\n### 2. Titles of Section 2 and 3\n\nWe apologize for the confusion caused by the similar titles used for Section 2 and 3 in our manuscript. We will modify the title of Section 2 to \"Problem Formulation,\" as recommended by the reviewer.\n\n### 3. Highlight the best performance in Table 2.\nIn our benchmark, both prediction accuracy (AROC, PRC) and acquisition cost (COST) are important performance metrics considering their clinical impacts. Focusing on various perspectives of a sensing approach, there could be multiple ways to rank the four methods in Table 2. Here, we propose one possible criterion, denoted as $\\omega$, which measures the decrease in accuracy per unit reduction in acquisition cost, i.e., $\\omega = \\frac{\\Delta PRC}{\\Delta COST}$, where\n\n- $\\Delta PRC = max(0, PRC_{FO} - PRC)$: drop in accuracy, $PRC_{FO}$ is the PRC score of the FO baseline.\n- $\\Delta COST = COST_{FO} - COST$: reduction in acquisition cost, $COST_{FO}$ is the COST of the FO baseline.\n\nNote that our method, RAS, achieves no loss in accuracy while reducing the acquisition cost from 39.6 to 4.535. Based on the criterion $\\omega$, RAS provides the best performance which can be highlighted in Table R2 below:\n\nTable R2: Benchmark of active sensing performance.\n\n| METHOD  | ROC         | PRC         | COST         | $d_{δ=0.3}$ | $d_{δ=0.5}$ | $d_{δ=0.7}$ |\n| ------  | ----        | ---         | ---          | ----        | ---         | ---         |\n| FO      | 0.668±0.000 | 0.634±0.000 | 39.600±0.000 | 0.582±0.000 | 0.229±0.000 | 0.181±0.000 |\n| ASAC    | 0.582±0.035 | 0.527±0.023 | 9.189±1.895  | 1.052±0.339 | 1.326±0.063 | 1.323±0.065 |\n| FIXED   | 0.655±0.006 | 0.600±0.005 | 0.907±0.034  | 1.384±0.000 | 1.398±0.000 | 1.359±0.000 |\n| **RAS** | 0.678±0.006 | 0.635±0.005 | 4.535±0.088  | 0.142±0.018 | 0.132±0.028 | 0.154±0.032 |\n\n### 4. Comparison of total acquisition cost distribution in Figure 3.\nWe appreciate the reviewer for pointing this out. As mentioned by the reviewer, both RAS and its non-risk-averse version (NRAS) can mitigate the long tail in the distribution of the expected cost function $Q^\\pi(\\mathbf{X})$ compared to the baseline policy with a constant decision interval. Due to the inherent randomness of the stochastic policies, in a single evaluation, the NRAS baseline might have appeared to outperform RAS at some outliers as observed in Figure 3. For a more clear comparison between RAS and NRAS, we have reconducted the experiment in Figure 3 with the average cost $Q^\\pi(\\mathbf{X})$ of each sample averaged over 5 random training/testing splits. The new result clearly demonstrates that our method RAS can further reduce the long tail in $Q^\\pi(\\mathbf{X})$ distribution compared to NRAS, highlighting the importance of our proposed tail-risk minimization strategy. Please check the PDF attachment for the updated figure. \n"}, {"review_id": "H4qn00zp6T", "reviewer": "Reviewer_mJK8", "summary": "This work studies the problem of active cost-aware feature acquisition assuming time varying feature settings via breaking-down feature selection and prediction decision making as two policies. The problem being considered is generally a difficult problem even with the non time-varying feature settings.\n\nI have a few questions about the real-world applicability, assumptions, and the approach. I would like to hear from the authors (see below).", "questions": "I was wondering what was the motivation behind having two policies rather than one with larger action space?\n\nFollowing on the previous question, wouldn’t the two-policy approach result in less optimal solutions compared to the optimization of one joint policy?\n\nIn the real-world settings that we do not have ground-truth values train set (assuming a cost-aware approach or a version of your method was applied at the time of data collection) how would the training process work? Please correct me but my understanding is that we do need fully observed train set for building the baseline predictor.", "limitations": "I have no particular concern", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The problem is well motivated for healthcare domain and is of clear practical value.\n\nThe paper is well-written and clear\n\nI found considering risk and long-tail behavior very interesting and quite important as cost optimization often means dropping precision for minorities which is not acceptable for health domain usecases\n", "weaknesses": "The method used to optimize the cost of acquisition i.e., a linear coefficient to balance cost vs. prediction risk is quite simple and does not necessarily lead to stable or near-optimal results. In the real-world settings the balance point might be different for each sample, and it is even more complicated when we take the time-varying assumption into account.\n\n(see other comments)", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely appreciate the reviewer's valuable comments and suggestions. We address the questions on our problem formulation and experiments as follows.\n\n## 1. Problem formulation\n\n### 1.1 Motivation of policy decomposition\nThe motivation for our sensing policy decomposition is three folds:\n\n- First, the action spaces of the acquisition scheduler and the feature selector are orthogonal and can generate all possible decisions of \"what to observe, and when\" at the next visit given patient trajectory.\n- Second, for each feature variable, the following decision cycle happens repeatedly: i) new acquisition of patient covariates is performed based on the latest sensing history; ii) then, the next follow-up time is determined based on the new sensing history including the newly accrued measurements in step i). The policy decomposition of acquisition scheduler $\\pi_\\Delta$ and feature selector $\\pi_m$ is a straightforward reflection of such a practical sensing cycle.\n- Finally, in most real-world scenarios, multiple selected feature variables are usually measured at the same visit. Thereby, we combine the decisions on individual feature selection at each visit into a binary vector $\\mathbf{m}$ and model its distribution with feature selector $\\pi_m$.\n\n\n\n### 1.2 Optimality of solution\nAs we explained above, the two sub-policies in our formulation have orthogonal action space and are mathematically equivalent to a global policy that determines observation interval $\\Delta$ and feature selection mask $\\mathbf{m}$ simultaneously. Despite the policy decomposition, the solution of our proposed active sensing problem shall be equivalent to the optimal global policy with a larger action space.\n\n\n## 2. Method\n\n### 2.1 Balance between the acquisition cost and prediction accuracy\nWe agree with the reviewer that, in some real-world settings, the balance point between the feature acquisition cost and prediction accuracy may vary across different samples, and the linear equilibrium achieved under our objective function may not be sufficient to tackle such a scenario. However, our method can be easily extended to tackle such challenges. For instance, by varying the linear coefficient $\\lambda$ in some reasonable range, a list of optimal sensing policies can be obtained with our proposed method. Then, we can construct an ensemble of these policies and fine-tune the weights for different policies on an individual sample basis. This approach should be applicable even when the balance between acquisition cost and prediction accuracy is time-dependent. We leave this as our future work as it is out of the scope of our manuscript.\n\n### 2.2 Training set with missing values\nThe reviewer is correct that a baseline predictor is required to train our proposed sensing approach RAS. Leveraging the power of neural CDE in tackling missing data, the baseline predictor can be built on training data with partial observations. Thus, the training set does not need to be fully observed (e.g., MIMIC-III). Indeed, we note that the effectiveness of RAS is highly correlated with the quality of baseline predictor $f_P$. A training set with high missing rates in feature records will generally lead to poor accuracy of $f_P$, which will unavoidably affect the validity of our sensing policy. However, it is worth highlighting that the same baseline predictor was used for the benchmarks (i.e., the variations of our model).\n\n### 2.3 Online training during data collection\nOur method is designed to work with batch data in offline settings. It should not be trained online during data collection time.\n"}, {"review_id": "oZl7lqPVDO", "reviewer": "Reviewer_fZr8", "summary": "This paper proposed an active sensing method that answers the questions of when and what diagnosis test to conduct to optimize the trade-off between the cost of acquisition and the timeliness and accuracy of the predictive model. Compared with the existing active sensing model that assumes a fixed data collection schedule, it builds flexibility to determine the timing of data collection. The paper demonstrates the utility of the proposed using synthetic data and a real-world dataset. ", "questions": "In real-world data set evaluation, it seems to be a three-class classification, while the performance is reported using AUC, etc. which is binary classifier performance metrics, do you weight the multi-class classification in presenting the result? Why it needs to be formulated as classification, is it more relevant if formulating as regression problem instead bin them into discrete categories?\n\n", "limitations": "author could comment on how the bias/quality of the decision in the original dataset may impact the validity of the approach. ", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "The proposed method adds extra flexibility in the decision-making policy to decide when (using continuous time active sensing formulation) to conduct a diagnostic test and what test to complete, which is what the existing model can do. This is a well-motivated problem.\n", "weaknesses": "As shown in the attachment, Table A3 on the empirical experiment with a real-world dataset, the proposed method shows only marginal improvement in accuracy and timeliness, but the cost has almost doubled, as compared with the benchmark; not sure whether these results suggest the better quality of the model overall. This raises questions whether there is practical utility of the proposed method. \n\nIn addition,  there are several places the presentation/notation is confusing and hard to follow; see below.\nThere is some notation confusion - for example, in Figure 2, the arrow from the topic box only sources from the post-interperation sensing history, not the post-interpolation of observational patient trajectory, it is unclear why the exclusion of trajectory history is justifiable, or it is an error of the drawing? \n\nFigure 1, subplot (a) is extremely small; subplot (b), the vertical dot line is not explained. \n\nThe equation number and figure number at someplace are not clearly identified, leaving the reader to guess, which cause confusing interpretation, e.g. Page 5 bottom part\n\n\n\n", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate the reviewer for helping review our paper and providing valuable comments to improve our manuscript. We address the reviewer's concerns on the clarity of our paper and the experimental evaluation of our method as follows. \n\n\n## 1. Clarity\nWe apologize for the confusions. We have fixed the readability and equation numbering issues mentioned by the reviewer and will further improve the clarity of our manuscript in the final version.\n\n### 1.1 Data flow from observational patient trajectory in Fig. 2\nThe reviewer is correct. Data flows should come from both the sensing history and observational patient trajectory. One arrow was missed after the box of interpolator in Fig. 2 of our manuscript. We have corrected this mistake and improved the general readability of Fig 2 in the revision.\n\n\n## 2. Experiments\n\n### 2.1 Benchmark in real-world dataset\nIn our benchmark, both prediction accuracy (ROC, PRC) and acquisition cost (COST) are important performance metrics to take into account. With focuses on different perspectives of a sensing strategy, there could be multiple ways to rank the four methods in Table 2 of our manuscript. As a balanced score function, we propose criterion $\\omega$ as the decrease in accuracy per unit reduction in acquisition cost, i.e., $\\omega = \\frac{\\Delta PRC}{\\Delta COST}$, where\n\n- $\\Delta PRC = max(0, PRC_{FO} - PRC)$: drop in accuracy, $PRC_{FO}$ is the PRC score of the FO baseline.\n- $\\Delta COST = COST_{FO} - COST$: reduction in acquisition cost, $COST_{FO}$ is the COST of the FO baseline.\n\nIn Table A3 of the Appendix, our proposed method RAS achieves nearly no decrease in accuracy while reducing the acquisition cost from 26.4 to 9.89. Based on the criterion $\\omega$, RAS has the best performance. In comparison, ASAC would be the best method when our main target is to simply reduce the acquisition cost. In practice, the criterion for ranking different approaches should be carefully designed based on the specific application scenario.\n\nRegarding the practical utility of our proposed method, we would like to highlight that, one important advantage of our method is the optimization for tail-risk samples. As illustrated in Fig. A.1 of the Appendix, RAS can effectively improve the sensing performance for the tail-risk patients for whom the expected cost $Q^\\pi(\\mathbf{X})$ is significantly higher under the FO and Fixed baselines.\n\n\n### 2.2 AUC score for multi-class classification\nThe reviewer is correct. There are three classes in our considered ADNI dataset. For the accuracy evaluation, we apply the one-vs-the-rest (OvR) strategy to calculate the AUC score for each class and report the average scores across all classes as the summary.\n\n### 2.3 Classification v.s. regression on ADNI\nOn the ADNI dataset, we create patient labels from their cognitive test scores (CDRSB) as the indicator of different stages of the disease. As the reviewer mentioned, the cognitive test scores can be directly used in a regression task. However, we would like to argue that the discrete labels are more appropriate descriptions of the disease progression. According to Table 5 in [1], patients with Dementia conditions may have CDRSB scores in a wide range (4.5 to 18.0). This suggests that the CDRSB scores could be highly noisy and are unsuitable for patient staging on the ADNI dataset.\n\n\n## 3. Discussion of limitations\nWe appreciate the reviewer for the comments on decision biases. Decisions in the data collection process of a dataset indeed have a potential impact on our proposed approach. First, the validity of our method relies on the baseline predictor $f_P$. If the training dataset is of low quality, $f_P$ is likely to have poor predictive power, which can compromise the usefulness of our proposed sensing policy. In the meantime, highly biased decisions during data collection may lead to high missing rates of certain feature variables, which could mislead our method to overlook the importance of such features. We will include a detailed discussion in the final version of our manuscript.\n\n## Reference\n\n[1] O'Bryant SE, Lacritz LH, Hall J, et al. Validation of the new interpretive guidelines for the clinical dementia rating scale sum of boxes score in the national Alzheimer's coordinating center database. Arch Neurol. 2010;67(6):746-749. \n"}, {"review_id": "If2ERlHOgR", "reviewer": "Reviewer_d1SC", "summary": "This paper studies the problem of balancing timely and accurate outcome predictions with acquisition costs. To this end, a risk averse active sensing approach (RAS) is proposed that determines when to perform feature acquisition as well as which features to acquire. The proposed approach decomposes the policy into an acquisition scheduler that decides when to perform feature acquisition, and a feature selector that decides which features to acquire. A risk-averse training strategy is introduced to focus on high-risk patients. The proposed approach is evaluated on a synthetic dataset and a real-world dataset from Alzheimer's Disease Neuroimaging Initiative.", "questions": "1. The problem of active sensing over time (also known as active state tracking) is well-studied, but the related work section does not include any such references. An example set of references are provided below:\n - Krishnamurthy, V., 2002. Algorithms for optimal scheduling and management of hidden Markov model sensors. IEEE Transactions on Signal Processing, 50(6), pp.1382-1397.\n- Krishnamurthy, V. and Djonin, D.V., 2007. Structured threshold policies for dynamic sensor scheduling—A partially observed Markov decision process approach. IEEE Transactions on Signal Processing, 55(10), pp.4938-4957.\n- Krishnamurthy, V., 2013. How to schedule measurements of a noisy Markov chain in decision making?. IEEE Transactions on Information Theory, 59(7), pp.4440-4461.\n- Zois, D.S. and Mitra, U., 2017. Active State Tracking With Sensing Costs: Analysis of Two-States and Methods for $ n $-States. IEEE Transactions on Signal Processing, 65(11), pp.2828-2843.\n- Molloy, T.L. and Nair, G.N., 2021, June. Active trajectory estimation for partially observed Markov decision processes via conditional entropy. In 2021 European Control Conference (ECC) (pp. 385-391). IEEE.\n- Atia, G.K., Veeravalli, V.V. and Fuemmeler, J.A., 2011. Sensor scheduling for energy-efficient target tracking in sensor networks. IEEE Transactions on Signal Processing, 59(10), pp.4923-4937.\n- Beyer, C., Büttner, M., Unnikrishnan, V., Schleicher, M., Ntoutsi, E. and Spiliopoulou, M., 2020. Active feature acquisition on data streams under feature drift. Annals of Telecommunications, 75, pp.597-611.\n-  Kossen, J., Cangea, C., Vértes, E., Jaegle, A., Patraucean, V., Ktena, I., Tomasev, N. and Belgrave, D., 2022. Active Acquisition for Multimodal Temporal Data: A Challenging Decision-Making Task. arXiv preprint arXiv:2211.05039.\n\nA careful literature review and follow-up discussion is needed to place the current paper within the vast prior work on this area and clarify its novelty. \n\n2. The problem studied in this paper also relates to quickest change detection, so it will be informative to include a discussion how the proposed approach contrasts/relates to this line of work. Some example references are provided:\n - Heydari, J. and Tajer, A., 2017, March. Quickest change detection in structured data with incomplete information. In 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) (pp. 6434-6438). IEEE.\n- Chaudhuri, A., Fellouris, G. and Tajer, A., 2021, July. Sequential change detection of a correlation structure under a sampling constraint. In 2021 IEEE International Symposium on Information Theory (ISIT) (pp. 605-610). IEEE.\n\n3. Certain parts of the problem and solution description are unclear or left for the appendix. For example, the discussion on the training of the predictor, which is necessary to fully understand how the proposed approach works, is left in the appendix. In addition, it should be mentioned early on in Section 2.1 that a subset of features are selected when acquisition happens.\n\n4. Certain notation definitions are missing or are confusing. For instance, D_{JS}(.) in Eq. (1) is not defined (I assume it is the Jensen-Shannon divergence). Furthermore, what is the relationship between T and I?  In Eq. (6), m and \\Delta have 3 indices, not one as before, why? In Algorithm 1, A & B should be inputs, right? How are they selected in practice? What is their effect on the performance of the algorithm?\n\n5. The structure of the proposed solution, as illustrated in Fig. 2, is not justified. Why this structure is adopted? Why these neural networks?\n\n6.  Certain decision choices are not justified. For example, why the beta distribution is adopted for the policy?\n\n7. Experiments are provided only on two datasets, out of which 1 is synthetic. These are not enough to justify the performance of the proposed approach. To make matters worse, three baselines are provided, out of which only 1 is from prior work. The results on real-world data are not impressive, and in fact comparable with the baselines, apart from the feature acquisition reduction. No statistical significance is provided. There is no justification of the synthetic dataset setting. \n\n8. The usage of the terminology \"cost pressure on patient trajectory acquisition\" is unclear. Also, somewhere in the paper, it is mentioned that there is a budget on acquisition costs but I could not locate where this budget gets into the problem formulation.", "limitations": "The authors have addressed the limitations of the proposed approach and the potential negative societal impact of their work.", "rating": 5, "confidence": 5, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "+ A continuous-time risk-averse active sensing approach is proposed to balance timely and accurate outcome predictions with acquisition costs.\n+ The proposed approach optimizes timely and accurate prediction for tail-risk patients.\n+ Experimental results on a synthetic and a real-world dataset are provided that illustrate the performance of the proposed approach compared to 3 baselines.", "weaknesses": "- Even though the problem of active sensing over time (also known as active state tracking) is well-studied, the related work section is very thin on references.\n- Certain parts of the problem and solution description are unclear or left for the appendix.\n- Certain notation definitions are missing.\n- Certain decision choices are not justified.\n- The structure of the proposed solution, as illustrated in Fig. 2, is not justified.\n- Experiments are provided only on two datasets, out of which 1 is synthetic.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the helpful comments and efforts towards improving our manuscript. We provide responses in regard to the reviewer's concerns as follows.\n\n## 1. Clarity\n- Following the reviewer's advice, we have moved discussions related to our problem formulation and solutions to the main manuscript. \n- A notation table is available in our response to Reviewer L7Er. Discussion of parameters $A$ and $B$ in Algorithm 1 can be found in Section A.5 of the Appendix.\n- Beta distribution $\\mathrm{Beta}(\\alpha, \\beta)$ is adopted in our model because i) $\\mathrm{Beta}(\\alpha, \\beta)$ is an uni-modal distribution when $\\alpha,\\beta>1$; ii) sampling from a Beta distribution is efficient, which helps to reduce the computational complexity.\n- \"Budget\" and \"cost pressure\". Our considered budget is a constraint on the accumulated feature acquisition cost, which creates the \"cost pressure\" on active sensing policies. For a practical solution to our sensing problem, we relax the budget constraint using a Lagrange approximation (second term of reward signal $r$ in Eq. (2)).\n\n## 2. Related work\nIn the active sensing task, our target is to balance the label prediction accuracy and feature acquisition cost. For this purpose, sensing policies should be able to make decisions on both when to perform new measurements and which feature subset to observe. We thank the reviewer for the detailed list of relevant literature. However, we found that many of these studies are incompatible with our considered active sensing setup. We provide analyses by paper category as follows.\n\n- Hidden Markov models (HMM) [1]: There are two possible ways to align the HMM formulation into our active sensing settings.\n    - Consider label $y$ as the state of a hidden Markov chain. The dependency between sensing history and label $y$ cannot be properly captured by the HMM structure, which makes these HMM-based methods unsuitable for our active sensing task. While the approach proposed in [2] can solve the problem of when to observe in active sensing, it is incapable to handle the feature selection task. \n    - Consider label $y$ as the observation of HMM states. In this case, there is no need to perform active sensing and feature selection.\n- Active learning [3]: This approach focuses on label prediction in static settings and is irrelevant to the setup of our active sensing task.\n- Modality selection [4]: This is the most relevant paper. However, it only addresses the feature selection task and assumes predetermined time interval between observations.\n- Fastest change detection [5]: This paper is closely related to optimal stopping which has already been discussed in our manuscript. Further, the correlation structure in sensor networks is unrelated to our active sensing problem.\n\n## 3. Experiments\n- As requested by the reviewer, we added a new baseline (LL) utilizing the log-likelihood-based reward signal introduced in [4]. The updated benchmark table on the synthetic dataset is provided in the attachment of our global response. \n- Regarding the performance on real-world datasets, we would like to argue that, one important advantage of our method is the optimization for tail-risk samples. As illustrated in Fig. A.1 of the Appendix, our method (RAS) can effectively improve the sensing performance for the tail-risk patients for whom the expected cost $Q^\\pi(\\mathbf{X})$ is significantly higher under the FO and Fixed baselines.\n- We agree with the reviewer that the inclusion of new real-world datasets for evaluation would help to better justify the performance of our method. We will include new evaluations with the MIMIC-III dataset in the final version of our manuscript.\n\n## 4. Justification of the synthetic dataset\nIn typical application scenarios of active sensing, there exist both variables that are highly predictive about the outcome of interest and those that are noisy, less informative proxy variables about the outcome. The acquisition costs of strong predictors are generally higher than the proxies such that balancing between accuracy and observation costs through active sensing is critical. For the synthetic dataset, variable $x_1$ is the strong predictor of outcome $y$, and $x_2$ is the proxy variable with a higher noise level. Thus, the measurement costs for and are set to be 1.0 and 0.1, respectively. The remaining two variables $x_3$ and $x_4$ are irrelevant to outcome $y$. For reference, their acquisition costs are simply set to 1.0.\n\n## Reference\n\n[1] Krishnamurthy, V., 2002. Algorithms for optimal scheduling and management of hidden Markov model sensors. IEEE Transactions on Signal Processing, 50(6), pp.1382-1397.\n\n[2] Krishnamurthy, V., 2013. How to schedule measurements of a noisy Markov chain in decision making?. IEEE Transactions on Information Theory, 59(7), pp.4440-4461.\n\n[3] Beyer, C., Büttner, M., Unnikrishnan, V., Schleicher, M., Ntoutsi, E. and Spiliopoulou, M., 2020. Active feature acquisition on data streams under feature drift. Annals of Telecommunications, 75, pp.597-611.\n\n[4] Kossen, J., Cangea, C., Vértes, E., Jaegle, A., Patraucean, V., Ktena, I., Tomasev, N. and Belgrave, D., 2022. Active Acquisition for Multimodal Temporal Data: A Challenging Decision-Making Task. arXiv preprint arXiv:2211.05039.\n\n[5] Heydari, J. and Tajer, A., 2017, March. Quickest change detection in structured data with incomplete information. In 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) (pp. 6434-6438). IEEE."}]

Author rebuttal:
We would like to thank all reviewers for taking the necessary time and effort to review our manuscript. We sincerely appreciate all your valuable comments and suggestions, which helped us in improving the quality of the manuscript.


## Summary of related work mentioned by reviewer d1SC
We thank reviewer d1SC for providing an extensive list of pertinent literature. We put a summarized table of these papers in Table R1 below for reference.

Table R1. Comparison with additional literature.

| Method     | Focus                 | Observation                           | Time interval         | Action                                                       |
| ------     | -------               | ---------                             | -----------           | --------------                                               |
| RAS (Ours) | Label $y$     | Sensing history $\tilde{\mathbf{X}}$  | Adaptive (continuous) | Determine when and what to observe                           |
| [1,2,4]    | HMM state $e$         | Discrete measurement of $e$           | Constant              | Select most relevant sensor to measure $e$                   |
| [3]        | HMM state $e$         | Scalar measurement of $e$             | Adaptive (discrete)   | Schedule measurement to $e$                                  |
| [5]        | HMM state $e$         | Observation of $e$                    | Constant              | Track the hidden state $e$                                   |
| [6]        | HMM state $e$         | Location of $e$                       | Constant              | Schedule sensors to locate a target object                   |
| [7]        | Model accuracy        | Sample stream                         | Constant              | Select sample and features to improve training set    |
| [8]        | Label $y$             | multi-modal data $\mathbf{\tilde{x}}$ | Constant              | Query most relevant modality data source for prediction      |
| [9,10]     | Correlation structure | Node states                           | Constant              | Detect the change of correlation structure in sensor network |



## Reference

[1] Krishnamurthy, V., 2002. Algorithms for optimal scheduling and management of hidden Markov model sensors. IEEE Transactions on Signal Processing, 50(6), pp.1382-1397.

[2] Krishnamurthy, V. and Djonin, D.V., 2007. Structured threshold policies for dynamic sensor scheduling—A partially observed Markov decision process approach. IEEE Transactions on Signal Processing, 55(10), pp.4938-4957.

[3] Krishnamurthy, V., 2013. How to schedule measurements of a noisy Markov chain in decision making?. IEEE Transactions on Information Theory, 59(7), pp.4440-4461.

[4] Zois, D.S. and Mitra, U., 2017. Active State Tracking With Sensing Costs: Analysis of Two-States and Methods for n-States. IEEE Transactions on Signal Processing, 65(11), pp.2828-2843.

[5] Molloy, T.L. and Nair, G.N., 2021, June. Active trajectory estimation for partially observed Markov decision processes via conditional entropy. In 2021 European Control Conference (ECC) (pp. 385-391). IEEE.

[6] Atia, G.K., Veeravalli, V.V. and Fuemmeler, J.A., 2011. Sensor scheduling for energy-efficient target tracking in sensor networks. IEEE Transactions on Signal Processing, 59(10), pp.4923-4937.

[7] Beyer, C., Büttner, M., Unnikrishnan, V., Schleicher, M., Ntoutsi, E. and Spiliopoulou, M., 2020. Active feature acquisition on data streams under feature drift. Annals of Telecommunications, 75, pp.597-611.

[8] Kossen, J., Cangea, C., Vértes, E., Jaegle, A., Patraucean, V., Ktena, I., Tomasev, N. and Belgrave, D., 2022. Active Acquisition for Multimodal Temporal Data: A Challenging Decision-Making Task. arXiv preprint arXiv:2211.05039.

[9] Heydari, J. and Tajer, A., 2017, March. Quickest change detection in structured data with incomplete information. In 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) (pp. 6434-6438). IEEE.

[10] Chaudhuri, A., Fellouris, G. and Tajer, A., 2021, July. Sequential change detection of a correlation structure under a sampling constraint. In 2021 IEEE International Symposium on Information Theory (ISIT) (pp. 605-610). IEEE.




Meta-review: This paper has been assessed by four knowledgeable reviewers whose recommendations included: weak acceptance (2), borderline acceptance (1) and straight rejection (1), respectively. The main complaints from the reviewers include limited evaluation, concerns about the practical utility of the presented method, important related work - perhaps not directly but conceptually relevant - missing in the discussion and comparisons, and poor organization of the paper whereas some key concepts were relegated to the appendix preventing the main paper to stand firmly on its own. The authors provided rebuttals that alleviated some but not all of the qualms. However, all things considered, and setting aside the most negative but also very curt review short of meaningful arguments, this paper is worth a serious consideration for acceptance for NeurIPS.
Decision: Accept (poster)