Title: Risk-Averse Active Sensing for Timely Outcome Prediction under Cost Pressure
Venue: NeurIPS 2023
Accepted: yes
Keywords: active sensing

value of information

risk-averse learning
Abstract: Timely outcome prediction is essential in healthcare to enable early detection and intervention of adverse events. However, in longitudinal follow-ups to patients' health status, cost-efficient acquisition of patient covariates is usually necessary due to the significant expense involved in screening and lab tests. To balance the timely and accurate outcome predictions with acquisition costs, an effective active sensing strategy is crucial. In this paper, we propose a novel risk-averse active sensing approach RAS that addresses the composite decision problem of when to conduct the acquisition and which measurements to make. Our approach decomposes the policy into two sub-policies: acquisition scheduler and feature selector, respectively. Moreover, we introduce a novel risk-aversion training strategy to focus on the underrepresented subgroup of high-risk patients for whom timely and accurate prediction of disease progression is of greater value. Our method outperforms baseline active sensing approaches in experiments with both synthetic and real-world datasets, and we illustrate the significance of our policy decomposition and the necessity of a risk-averse sensing policy through case studies.

Risk-Averse Active Sensing for Timely Outcome Prediction under Cost Pressure

Abstract

Timely outcome prediction is essential in healthcare to enable early detection and intervention of adverse events. However, in longitudinal follow-ups to patients' health status, cost-efficient acquisition of patient covariates is usually necessary due to the significant expense involved in screening and lab tests. To balance the timely and accurate outcome predictions with acquisition costs, an effective active sensing strategy is crucial. In this paper, we propose a novel risk-averse active sensing approach RAS that addresses the composite decision problem of when to conduct the acquisition and which measurements to make. Our approach decomposes the policy into two sub-policies: acquisition scheduler and feature selector, respectively. Moreover, we introduce a novel risk-aversion training strategy to focus on the underrepresented subgroup of high-risk patients for whom timely and accurate prediction of disease progression is of greater value. Our method outperforms baseline active sensing approaches in experiments with both synthetic and realworld datasets, and we illustrate the significance of our policy decomposition and the necessity of a risk-averse sensing policy through case studies.

1 Introduction

Timely decision-making through accumulated observation history has attracted significant attention in the machine learning community, with broad impact and applications in healthcare [8]. Consider a typical decision-making scenario in which an agent employs a sensing policy to actively collect diagnosis-related information from an underlying feature trajectory. In the presence of cost pressure , the goal of the agent is to achieve timely and accurate predictions of a time-varying outcome of interest based on the sensing history, i.e., feature observations accumulated over time, while maintaining a reasonable expense of feature acquisition. At each decision step, we refer to the problem of deciding when to make new acquisitions and which features to measure as active sensing [2, 24]. The optimal sensing policy can be achieved by negotiating the subjective trade-off between outcome prediction (accuracy and timeliness) and acquisition cost over time.

A special class of the active sensing problem has been extensively explored in the optimal stopping literature [2, 6, 15]. In optimal stopping, the agent focuses on the timely diagnosis of a terminal status, where observation ceases immediately after a confident diagnosis can be made [1, 5]. Thereby, the acquisition cost in optimal stopping is typically considered as the time span of the information collection process. Recently, Jarrett and van der Schaar [9] introduced a novel sensing framework to allow for the consideration of measurement cost of individual feature variables in optimal stopping, enabling better scheduling of the acquisition sequences. However, their approach still focuses on

a static terminal status and is unsuitable for general purpose active sensing where the outcome of interest varies over time.

(a) Active sensing process

0

50

100

150

200

250

Q π

(

X

)

0

10 1

10 2

10 3

# of samples

ρα

Trajectories at the long tail

(b) Long-tailed sensing deficiency distribution

In this paper, we focus on more general healthcare scenarios where timely diagnosis and intervention need to be achieved through active sensing under certain budget. The budget serves as the source of cost pressure and prioritize safe and low-cost measurements in the sensing process. For instance, the management of Alzheimer's disease (AD) requires continuous monitoring and staging of disease progression, which is crucial for early detection and intervention of high-risk patients [8]. However, certain measurements involved in diagnosing AD are notably expensive. A typical example is positron emission tomography (PET) scans. PET scans provide precise information about AD-associated proteins in the brain and are widely used for monitoring AD progression [3]. Nevertheless, the high accuracy of PET scans compared to cognitive tests is accompanied by greater financial costs as well as potentially harmful exposure to radiation [7]. A desirable active sensing policy should adaptively balance measurement costs and diagnostic accuracy at different disease stages to provide timely diagnosis and intervention under cost pressure. Such property is not only useful in healthcare but also other applications such as delay-sensitive wireless communication [14].

Another crucial but relatively unexplored aspect of active sensing is the consideration of tail risk . To illustrate this concept, let's examine the prediction of AD progression. In active sensing, the trade-off between diagnosis error and acquisition cost is usually achieved through the minimization of their weighted sum [9, 23, 24]. As depicted in Figure 1(b), the weighted objective ( Q π ( X ) ) typically exhibits a long-tailed distribution with a notable concentration of high-value patients at the tail side. We consider these patients to be of higher value in the sensing task for two main reasons. Firstly, accurate diagnosis of many high-risk patients heavily relies on biomarkers obtained through expensive PET scans such as fluorodeoxyglucose (FDG) PET and florbetapir (AV45) PET [7, 10]. A vanilla sensing policy optimized for the entire population would easily fail to arrange enough PET scans for these high-risk patients due to the high costs, leading to degraded sensing performance (the long tail as shown in Figure 1(b)). In the meantime, the majority of patients tend to remain in stable conditions. Their disease stages can usually be diagnosed even with very sparse acquisitions over time. A biased sensing policy may overemphasize the importance of high-risk patients and order frequent PET scans for all patients to avoid false negatives in its diagnosis, which contributes to the long-tailed distribution from another perspective. To avoid such situations, an active sensing strategy must be risk-averse such that it can effectively cater to the needs of both the majority of stable patients and the high-risk subgroups requiring frequent follow-ups, ensuring timely and accurate detection of disease progression under cost pressure.

Contributions. In this paper, we propose a novel continuous-time active sensing approach to address the aforementioned desiderata. Our sensing approach adaptively balances measurement costs and diagnostic accuracy by factorizing sensing policy into two sub-policies: acquisition scheduler which

determines when to conduct feature acquisitions (i.e., the time interval between adjacent follow-ups), and feature selector that decides which observations to make . Furthermore, our sensing approach adopts the risk-averse strategy and focuses on the trade-offs happen at the long tail, where accurate diagnosis becomes particularly crucial. By dynamically identifying patient trajectories located at the tail side, we optimize our sensing strategy using the conditional value-at-risk (CVaR) loss to provide guarantees on timely and accurate prediction for high-risk patients while avoiding biased decisions for patients with stable conditions.

We validate our approach through experiments on synthetic and real-world datasets, where the sensing performance on high-risk patients is significantly impacted by their relatively small cohort size. With the consideration of acquisition costs, our method is able to adaptively identify the patient subgroups at the long tail of sensing deficiency distribution and achieves improved diagnostic performance compared to the state-of-the-art active sensing benchmarks.

2 Active Sensing in Practice

In this section, with a focus on healthcare applications, we introduce a new active sensing approach, which we refer to as Risk-averse Active Sensing (RAS), to solve the decision-making problem of when to conduct feature acquisitions and which features to observe considering prediction accuracy, timeliness, and cost pressure. We will start by providing a general framework for active sensing problems.

2.1 Preliminaries: Active Sensing

Notation. Let x ( t ) ∈ R d be the characteristic of a patient at time t ∈ [0 , T ] which is a collection of d (costly) observable time-varying covariates about an underlying disease progression of our interest. 1 X t = { x ( τ ) | τ ∈ [0 , t ] } represents a temporal trajectory of observable features until time t ≤ T . For ease of description, we will occasionally abuse the notation and use X to denote the entire patient trajectory X T . Let y t ∈ Y be the outcome of our interest which represents the underlying health status of a patient (e.g., diagnosis, adverse outcome, etc.) at time t . We assume the joint distribution p ( X t , y t ) from which the trajectory X t and the outcome y t are drawn. Hereafter, we focus our description on K -class classification tasks, i.e., Y = { 1 , 2 , . . . , K } .

We denote an active sensing policy as π . In the sequential outcome prediction task, for each patient X , policy π arranges a set of follow-up times { t 1 , t 2 , . . . , t I } ⊂ [0 , T ] and determines the subsets of features m i to be measured at each time step t i , where m i is a binary vector. The collected observations of patient trajectory X until time t form a sensing history X π t . It is worth noting that X π t is piece-wise constant over time as it is only updated at discrete follow-ups. At each step, the sensing policy π shall generate a diagnosis of the patient by approximating the conditional p ( y t | X π t ) .

Dataset. For the active sensing task, we assume access to an observational dataset D containing electrical health records (EHRs) of N patients, i.e., D = { X n = ( t i , x ( t i ) , y t i ) L i =1 } N n =1 . The record of each patient consists observations collected from L follow-ups. It is worth noting that the record X n ∈ D is equivalent to a sensing history.

Following the convention in active sensing literature [9, 24], we consider the cost of measurement on each time-varying observable feature to be time-invariant and thus can be represented as a cost vector c , where the j -th element, denoted as ( c ) j , is the cost for measuring the j -th time-varying feature in vector x ∈ X . The central goal of active sensing is then to balance between the outcome estimation accuracy based on sensing history X π t and the accumulated acquisition costs over time.

Challenges. In practice, there are two major challenges that confound the active sensing problem:

Adaptive follow-up intervals. Most existing active sensing policies assume a constant follow-up interval. However, in real-world healthcare settings, the desirable acquisition intervals usually change along with disease progression. Sensing policy with constant decision intervals creates redundant follow-ups and may induce extra costs for feature acquisition.

Failures at the long tail. Vanilla active sensing policies are optimized for the entire population. For diseases like AD, the imbalanced proportion of high-risk and stable patients may lead to over-conservative sensing policies that make no observations. Similarly, sensing policies targeted on high-risk patients may generate biased decision with higher acquisition costs for patients in stable conditions. Both situations lead to failures for samples at the long tail of sensing deficiency distribution.

In the following, we first introduce a generalized sensing policy π with the decomposition of decisions on follow-up interval and feature selection. Then, we formulate the active sensing task into a riskaverse optimization problem, the solution to which addresses the second challenge.

2.2 Decomposing the Active Sensing Policy

Vast research in active sensing has assumed a constant follow-up interval between two consecutive patient visits and has primarily focused on the sequential feature selection problem, determining which feature variables to measure [6, 24]. However, considering the dynamic and complex nature of disease progression, sensing policies with dynamic follow-up intervals are broadly desired in real-world applications. In this paper, we propose a generalized formulation of the sensing policy, i.e., π = ( π m , π ∆ ) , where the acquisition scheduler π ∆ determines the time for next follow-up, the feature selector π m determines which features to measure during the subsequent visit.

Consider the i -th follow-up of patient X at time t i . The next follow-up time t i +1 is firstly determined by acquisition scheduler π ∆ as t i +1 = t i + ∆ i , ∆ i ∼ π ∆ ( X π t i ) . Then, at time t i +1 , the feature selector π m determines which subset of patient covariates to measure using a binary vector m i +1 ∼ π m ( X π t i ) , m i +1 ∈ { 0 , 1 } d . The new observation is calculated as m i +1 ⊙ x ( t i +1 ) , where the j -th element

2.3 Towards Risk-Averse Sensing Policy

Trade-off between accuracy and costs. Consider the i -th follow-up at time t i . The feature selection is determined by m i ∼ π m ( ˜ X π t i -1 ) . The acquisition cost is evaluated as r m ( m i ) = c ⊤ m i , where c is the cost vector. The updated sensing history X π t i = X π t i -1 ∪{ m i ⊙ x ( t i ) } is then used to determine the acquisition interval t i +1 -t i = ∆ i ∼ π ∆ ( X π t i ) . Assume a diagnosis error function r y ( X π t i , ∆ i ) that evaluates the mismatch between p ( y t | X t ) and p ( y t | X π t ) during the interval ∆ i . The trade-off between diagnosis accuracy and measurement costs of the i -th interval is defined as the following reward signal

where λ > 0 is a coefficient chosen to balance the two terms.

Sensing deficiency. We define the sensing deficiency of policy π along a patient trajectory X as the expected cumulative reward as shown in (2).

where discount factor γ ∈ (0 , 1] is used to tackle long trajectories. In this paper, we take γ = 0 . 99 .

Trajectories at the long tail. As mentioned earlier, one major challenge for active sensing in practice is the potential sensing failures for patients at the tail of sensing deficiency distribution. Since the distribution tail of Q π ( X ) changes along with the updates of policy π , in this paper, we propose to optimize the sensing policy over a dynamic subset of patient trajectories located at the long tail. These tail trajectories can be identified through the upper α quantile ρ α , where E X [ I ( Q π ( X ) ≥ ρ α )] = α . Assume the marginal trajectory distribution p ( X ) , we denote with S π α = { X | X ∼ p ( X ) , Q π ( X ) ≥ ρ α } the set of tail trajectories under policy π .

Conditioned on the quantile factor α ∈ [0 , 1] , the sensing deficiency of trajectories in S π α is considered as the value at risk that needs to be improved. Based on the evaluation of CVaR in sensing deficiency distribution, we formulate our active sensing task as a risk-averse optimization problem.

Solutions to problem (3) are considered risk-averse since they reduce the risk of sensing failures in the worst scenarios ( S π α ). Our proposed risk-averse sensing problem has two major distinctions from the class of robust adversarial reinforcement learning (RL) problems (e.g., [18]): i) we directly optimize the value at risk CVaR α ; and ii) the adversarial agent is replaced by the dynamic tail trajectory subset S π α . As such, our proposed problem requires no online interactions with the data generation process and thus can avoid the sensing policy π from being biased toward unrealistic or out-of-distribution trajectories generated by simulators.

3 Method: Risk-Averse Active Sensing

Solution to the risk-averse active sensing problem in (3) given observational data is faced with three unique challenges. First, we only have access to observational data that consist of discrete measurements of (continuous) patient trajectories. Second, we need access to a baseline conditional outcome estimator, f P ( y t | X t ) , to estimate the diagnosis error r y in (1). Third, the solution of (3) heavily relies on the appropriate construction of the subset, S π α , which, by definition, depends on the sensing policy itself.

Handling discrete and sparse observations. Dataset D only contains discrete observations of patient trajectories. However, the adaptive acquisition intervals generated by sensing policy π may require measurements of the underlying patient trajectories at time points not included in D . To solve this problem, we adopt a linear interpolator I as a proxy to obtain estimations of these values. In the meantime, policy π by nature generates sparse sensing history with partial observations of patient trajectories. To properly encode the missingness and adaptive observation intervals in sensing history X π t , we include the neural CDE as the base encoders in our approach. As enlightened in [11], CDE is a continuous-time generalization to recurrent neural network (RNN) and offers a natural and elegant approach to encode the information in sparse and irregular measurements of a patient trajectory.

Specifically, given a discrete sensing history X π t , we first convert it into a continuous trajectory u ( τ ) = I ( τ, X π t ) via the interpolator I . Then, CDE encodes the temporal changes in X π t as the dynamics of a latent variable z ( t ) ∈ R l via

where t 0 ≥ 0 , f ϕ : R l ↦→ R l × d is a map with learnable parameters ϕ , initial state z 0 = z ( t = 0) could be manually specified or learned from data.

Timely and accurate predictions. In this paper, we build CDE-based outcome estimator f P ( t, X ) to approximate the density p ( y t | X t ) , where X is a sensing history of trajectory X . Details about f P can be found in the Appendix. Given a patient with record X ∈ D , to provide timely and accurate predictions about the underlying disease progression, our sensing policy aims to minimize the mismatch between baseline estimate f P ( τ, X ) and sensing history-based prediction f P ( τ, X π τ ) . For this purpose, in time horizon τ ∈ [ t i , t i +1 ) , the diagnosis error function for the i -th follow-ups is computed as the cumulative mismatch :

where D JS is the JS divergence, t i +1 = t i +∆ i .

The distinction from previous works that utilize a single-step mismatch - such as at a pre-specified time interval [24] - is that (5) provides a precise proxy to evaluate how timely and accurate the predictions are by gauging the impact with respect to the acquisition interval ∆ i , which is a desired property. Adverse clinical outcomes of the underlying disease progression are likely to occur when the outcome probability, p ( y t | X t ) , hits a certain threshold, often accompanied by a sudden rise or fall. Making too early or too late detection of such a rise/fall based on the sensing history, i.e., p ( y t | X π t ) , will drastically increase the cumulative mismatch in (5).

Penalty on invalid visits. We define the follow-up at time t i +1 as an invalid visit if no patient covariates are measured, i.e., m i = 0 . Allowing invalid visits potentially yields degenerated active sensing policy and makes the policy decomposition meaningless. To prevent such situations, we

𝒉𝑡𝑖

𝒑𝑡𝑖

𝛼𝑡𝑖

, 𝛽

𝑡 𝑖

𝑓 𝑀

Feature

Selector

𝑓 Δ

Acquisition

Scheduler

𝒎𝑖+1

Beta(𝛼, 𝛽)

Bern(𝒑)

𝑓 𝐶

Critic

Policy

𝝅

ℐ

Interpolator

MLP

∫

CDE

History Encoder

𝑓 𝐻

ℐ

Interpolator

𝒎𝑖+1 ⊙ 𝒙(𝑡𝑖+1)

𝑋𝑡𝑖

𝜋

𝑋𝑡𝑖+1

𝜋

𝒙(𝑡𝑖+1 )

𝑡 𝑖+1

= 𝑡𝑖 +Δ𝑖

Sensing History

Record

𝑋

EHR

RAS

Feature Selection

(𝒊 + 𝟏)

-th follow-up

𝒊

-th follow-up

ℐ

Interpolator

𝑓 𝐸

∫

CDE

Outcome Predictor

𝑓 𝑃

𝑓 𝑌

𝑝 𝑦𝑡𝑖

𝑋𝑡𝑖

𝜋

Diagnosis

̸

extend the acquisition cost function r m as r m ( m ) = c ⊤ m · I ( m = 0 ) + ν · I ( m = 0 ) , where ν ≥ c ⊤ 1 penalizes the invalid visits scheduled by the sensing policy π .

Network structure. Similar to the predictor f P , the neural CDE and interpolator I are also employed in our proposed policy network π to construct an embedding of the discrete sensing history X π t . As illustrated in Figure 2, our sensing policy π (parameterized by θ ) is constructed with the following three networks:

The history encoder , f H , is a neural CDE encoder that maps sensing history X π t into a summary vector h ( τ ) = f H ( τ, I ( X π t )) .

The feature selector , f M , is a sub-policy network that governs the decision about which subset of patient features to be measured. In a follow-up, f M takes a summary vector h ( τ ) as input and outputs a vector p ( τ ) = f M ( h ( τ )) that governs the element-wise Bernoulli distribution to generate a binary feature selection vector m , i.e., m τ ′ ∼ Bern( p ( τ )) for next follow-up at τ ′ .

The acquisition scheduler , f ∆ , is a sub-policy network that determines the time for next follow-up. Particularly, after each follow-up, it takes the summary vector h ( τ ) as input and generates two scalar parameters α, β = f ∆ ( h ( τ )) of a Beta distribution, i.e., ξ ∼ Beta( α, β ) . The realizations of the Beta distribution are then used to generate the time interval ∆ ∈ [∆ min , ∆ max ] via ∆ = ∆ min +(∆ max -∆ min ) · ξ , where ∆ min and ∆ max are the minimum and maximum allowed intervals between two follow-ups, respectively. We set α, β ≥ 1 to ensure that the time interval follows a unimodal distribution.

Policy gradient. Since the diagnostic error function in (5) is non-differentiable, to back-propagate gradients from the objective CVaR α = E X ∈ S π α [ Q π ( X )] through the active sensing process, we apply the advantage-based approach [21, 22] to estimate the policy gradient of π as the following:

where θ is the collection of learnable parameters of policy π . Given the s -th patient in S π α , the cumulative reward R ( X π s,t i ) = ∑ I ι = i γ ι -i r ( X π s,t ι , m ι , ∆ ι ) is an empirical evaluation of the sensing deficiency in (2) for the corresponding sub-trajectory in horizon [ t i , T ] . Following the actor-critic framework, we utilize a baseline sensing deficiency estimation R C ( X π t i ) in (6) to reduce the variance of policy gradient estimation [21].

The baseline estimation R C ( X π t ) is obtained from a critic network f C (parameterized by ψ ). Taking the embedding h ( t ) = f H ( t, I ( X π t )) from the history encoder f H as input. The critic f C is trained

to minimize the following mean squared error (MSE) loss.

Algorithm 1 Risk-averse active sensing

Given the baseline predictor f P , the sensing deficiency Q π ( X ) of policy π is optimized over the subset S π α of tail trajectories as illustrated in Algorithm 1. Specifically, parameters θ of policy π are updated based on the policy gradient ∇ θ CVaR α in (6), and the critic network f C is trained with loss function L C to provide appropriate baselines for the sensing policy π . The collection of patient trajectories in S π α is iteratively updated every M training epochs such that it can stay informative about the sensing deficiency distribution. Note that the learning rates of critic f C and policy π are intentionally set to be different to facilitate convergence.

4 Related Work

We first provide in Table 1 an overview of the distinction between our proposed approach and other related works in active sensing literature. A special class of active sensing task is formulated under the optimal stopping framework. With a set of explicitly defined terminal states, optimal stopping approaches focuses on achieving confident diagnosis of these states while minimizing the time span of the sensing process [1, 6, 15]. The consideration of information acquisition cost was introduced recently into optimal stopping in [9]. Similarly, the trade-off between diagnosis accuracy and costly measurement has been studied in the more general sequential prediction tasks [23, 24]. In this paper, we propose a novel sensing policy decomposition and extend the active sensing analysis into continuous-time settings with adaptive follow-up intervals and highly sparse sensing histories.

Temporal feature selection. As pointed out by Yoon et al. [24], active sensing can be considered as the temporal generalization of feature selection tasks [13, 19]. In each decision step, the sensing policy is expected to select a set of important feature variables to make the prediction. The resulting sensing history marks critical time points and key feature variables for accurate outcome estimation. However, existing active sensing literature primarily focuses on the feature selection perspective and usually assumes a fixed follow-up interval for every patient trajectory [1, 24]. A fixed acquisition

interval can lead to delayed diagnosis or extra costs due to redundant feature measurements. As mentioned earlier, a desirable sensing policy shall achieve flexible follow-up intervals for each patient such that the shifts in underlying disease stages can be properly addressed.

Risk aversion. Conventional active sensing strategies typically consider the diagnosis accuracy to be equally important for all trajectories in a dataset. However, in many healthcare applications, there are needs to prioritize smaller subgroup of high-risk patients for whom timely detection of adverse outcomes is particularly valuable. As a central component of our proposed method, the concept of CVaR [20] enables the selective optimization for patient trajectories in the long tail of sensing deficiency distribution. Optimizing sensing policies over the corresponding tail set S π α yields a risk-averse active sensing strategy π ∗ that achieves guaranteed diagnosis accuracy for high-risk patients. While risk-aware decision-making has been extensively explored in the field of RL [4, 25], to the best of our knowledge, our proposed method is the first risk-averse active sensing algorithm for adaptive feature acquisition in continuous-time settings.

5 Experiment

In the experiments, we evaluate the effectiveness of our proposed risk-averse active sensing approach RAS on both synthetic and real-world healthcare datasets.

Synthetic dataset. We construct a synthetic dataset D S of N = 2000 samples. Every sample X contains L = 20 discrete observations in t ∈ [0 , 2] . Each observation yields a feature vector x = [ x 1 , x 2 , x 3 , x 4 ] ⊤ . We set x 1 ( t ) = min(1 , ( e w ( t -τ ) -w ( t -τ ) -1)) I ( t ≥ τ ) , where w follows a Gaussian mixture distribution of 0 . 8 · N (0 . 3 , 0 . 1 2 ) + 0 . 2 · N (0 . 8 , 0 . 1 2 ) , τ ∼ Exp(1 . 0) . As a proxy of x 1 , we set x 2 ( t ) = w ( t -τ ) I ( t ≥ τ ) . Variables x 3 ( t ) = sin(3 t + ϱ ) and x 4 ( t ) = cos(3 t + ϱ ) , where ϱ ∼ N (0 , 1 2 ) . The outcome y ∈ { 0 , 1 } follows a Bernoulli distribution y t ∼ Bern( p ) , and the likelihood for y t = 1 is calculated as p = e -3 | 1 -x 1 | 2 . The cost vector is set to be c = [1 . 0 , 0 . 1 , 1 . 0 , 1 . 0] ⊤ , which allows x 2 to be measured as a cheap proxy of x 1 for outcome prediction.

ADNI dataset. The Alzheimer's Disease Neuroimaging Initiative 2 (ADNI) dataset includes records on AD progression of N = 1002 patients with regular follow-ups every six months. We consider the active sensing task to predict patient outcomes in the initial L = 12 followups with four biomarkers from PET (FDG and AV45) and MRI (Hippocampus and Entorhinal) imaging, respectively. Based on the disease staging guideline [16, 17], target outcome y t ∈ { Normal , mild cognitive impairment (MCI) , AD } at each time point is determined via the corresponding Clinical Dementia Rating scale Sum of Boxes (CDR-SB) score with cutoff thresholds at 0.5 and 4.5. To reflect the higher cost and harmful radiation exposure in PET scan, measurement costs for PET and MRI biomarkers are set to be 1.0 and 0.5, respectively.

Experiment setup. We first fit the outcome estimator f P on each dataset with 64/16/20 train/validation/test splits and find the optimal estimator with the best prediction accuracy given sparse observation of test samples as inputs. Then, we freeze the parameters of the optimal f P and evaluate the sensing performance of RAS and other baselines on the corresponding data split. We consider four baselines in the experiments: FO ) predictor f P with dense sensing history; ASAC ) [24]; NLL ) adaptive sensing with negative log-likelihood (NLL) as diagnosis error function [12]; AS ) RAS( α = 1 . 0 ) with constant acquisition interval. Further details of the experiment setup and parameter selection results can be found in the Appendix.

The 95% confidence interval (CI) is evaluated with five different random seeds. Best result in each column are highlighted in bold . The marker ‡ indicates p-value p < 0 . 05 .

Timeliness and accuracy. Timely and accurate outcome prediction given sparse observations of patient trajectories is essential for practical applications of active sensing. In the experiments, we report the area under the curve of receiving-operator characteristic (ROC) and area under the curve of precision-recall (PRC) as assessments of the prognostic value of active sensing policies. In the meantime, the timeliness of an active sensing policy π is evaluated with d δ - the mean absolute mismatch in diagnosis time of event p ( y t = 1 | X π t ) ≥ δ compared to ground truth on the test set. The benchmark results of RAS against four baselines on the two datasets are provided in Table 2 and Table 3, respectively.

With a fixed acquisition interval of ∆ = 0 . 2 , FO achieves high predictive accuracy at the expense of highest acquisition cost. Despite having the smallest acquisition cost, ASAC suffers significantly from the low diagnosis accuracy and is unable to achieve timely diagnosis of alert events. In contrast, our method (RAS) has reasonable acquisition costs and best timeliness as illustrated in Table 2. The increase in average cost of RAS compared to the AS baseline could be linked to our focus on the tail set S π α . The increased acquisition costs are necessary to achieve a significantly lower delay in alert event diagnosis for high-risk patients in S π α . The improved timeliness of RAS over FO suggests that the predictor f P might achieve better generalization with sparse sensing history as inputs. Similarly, according to Table 3, our method RAS outperforms most other baselines in timeliness and cost efficiency except for FO. This is within expectation since the outcome estimations from f P are used as the reference for timeliness evaluation, and these estimations are generally close to the predictions made by the FO baseline.

The 95% confidence interval (CI) is evaluated with three different random seeds. The timeliness d δ is measured based on the likelihood of developing AD. Best result in each column are highlighted in bold . The marker ‡ indicates p-value p < 0 . 05 .

0.0

0.2

0.4

0.6

0.8

1.0

1.2

1.4

dδ

=0.5

0

10 1

Cost

Δ=0.5

Δ=1.0

μ

=0.001

μ

=0.005

μ

=0.01

μ

=0.1

λ

=100.0

λ

=200.0

λ

=300.0

λ

=400.0

Method

AS

ASAC

FO

NLL

RAS

(a) Pareto front of sensing policies.

50

100

150

200

250

Q π

(

X

)

0

20

40

60

80

100

120

140

# of samples

AS

RAS(

α

=1.0)

RAS(

α

=0.1)

10 1

10 2

0

25

50

75

100

ρα

=61.59

ρα

=20.01

ρα

=10.40

(b) Distributions of sensing deficiency Q π ( X ) .

Trade-off between timeliness and acquisition costs. The selection of model parameters could be difficult when two or more criteria are involved in the evaluation. Here, we illustrate the sensing performance of different policies on the synthetic dataset in Figure 3(a) and highlight the ones in the Pareto front with gray circles. These policies are considered Pareto optimal since their timeliness

( d δ =0 . 5 ) and average acquisition cost cannot be simultaneously improved by swapping parameters with other policies. Benefitted from the risk-averse training strategy, most sensing policies obtained via RAS are centered around the knee point of the Pareto front, which helps to explain the outstanding cost efficiency of RAS as reported in Table 2.

Reshaping the sensing deficiency distribution. To illustrate the effectiveness of the risk-averse objective in (3), we compare the empirical distribution of sensing deficiency Q π ( X ) of RAS with the ablations of risk-neutral sensing ( α = 1 . 0 ) and AS baseline ( α = 1 . 0 , constant acquisition interval ∆ = 1 . 0 ) on the synthetic dataset. All three models are trained with the same trade-off coefficient λ = 300 . As illustrated in Figure 3(b), RAS is able to effectively optimize the sensing performance for trajectories in the tail set S π α and reduces the upper α quantile of Q π ( X ) to ρ α =0 . 1 = 10 . 40 . Factor α = 1 . 0 completely disables the risk-aversion training strategy in Algorithm 1. Thereby, a clear increase of sensing deficiency (quantile ρ α =0 . 1 grows from 10.40 to 20.01) is observed with the risk-neutral ablation of RAS. Similarly, without adaptive scheduling of acquisition intervals and risk-averse optimization strategies, the AS baseline illustrates the failure of conventional active sensing paradigms at the long tail of Q π ( X ) distribution.

6 Conclusion

In this paper, we introduce a novel risk-averse active sensing approach RAS to address the challenging continuous-time active sensing problem. Through effective decomposition of decisions on feature selection and acquisition intervals, our approach offers valuable insights on both feature importance and timeliness of patient follow-ups. The novel risk-aversion training strategy in RAS enables the prioritization of high-value patients at the long tail of sensing deficiency distribution and provides guarantees on diagnosis accuracy for worst-case scenarios. The effectiveness of our method is evaluated through experiments on synthetic and real-world healthcare datasets, where RAS outperforms all baseline active sensing approaches and achieves accurate and timely outcome diagnosis while maintaining reasonable costs in feature acquisition.

Broader Impacts

The development of an advanced active sensing strategy is essential for precision medicine. Appropriate sensing policies can help clinicians to make tailored follow-up schedules for their patients and effectively utilize the costly and potentially harmful measurements on some important patient characteristics under certain levels of cost pressure. We note that the reduction in acquisition cost is usually associated with degraded diagnosis accuracy and delayed detection of some adverse clinical events, especially for high-risk patients that could benefit from more frequent follow-ups on disease progression. In this paper, we attempt to address this challenge through the risk-averse policy learning strategy in our proposed active sensing approach RAS and achieve improved sensing performance (accuracy and timeliness) on the entire population. Nevertheless, practical applications of active sensing approaches would require careful audits and assessments by human experts to avoid potential negative impacts. The sensing decisions from RAS are only suggestions optimized based on observational data and should not be directly applied without the evaluation of clinicians.

References

S. Ahmad and A. J. Yu. Active sensing as bayes-optimal sequential decision-making. In Proceedings of the Twenty-Ninth Conference on Uncertainty in Artificial Intelligence , pages 12-21, 2013.

A. M. Alaa and M. van der Schaar. Balancing suspense and surprise: Timely decision making with endogenous information acquisition. Advances in Neural Information Processing Systems , 29, 2016.

W. Bao, F. Xie, C. Zuo, Y. Guan, and Y. H. Huang. Pet neuroimaging of alzheimer's disease: radiotracers and their utility in clinical research. Frontiers in Aging Neuroscience , 13:624330, 2021.

L. Brunke, M. Greeff, A. W. Hall, Z. Yuan, S. Zhou, J. Panerati, and A. P. Schoellig. Safe learning in robotics: From learning-based control to safe reinforcement learning. Annual Review of Control, Robotics, and Autonomous Systems , 5:411-444, 2022.

H. Chernoff. Sequential design of experiments. The Annals of Mathematical Statistics , 30(3): 755-770, 1959.

S. Dayanik and A. J. Yu. Reward-rate maximization in sequential identification under a stochastic deadline. SIAM Journal on Control and Optimization , 51(4):2922-2948, 2013.

E. Dolgin. Alzheimer's disease is getting easier to spot. Nature , 559(7715):S10-S10, 2018.

B. Dubois, A. Padovani, P. Scheltens, A. Rossi, and G. Dell'Agnello. Timely diagnosis for alzheimer's disease: a literature review on benefits and challenges. Journal of Alzheimer's disease , 49(3):617-631, 2016.

D. Jarrett and M. van der Schaar. Inverse active sensing: Modeling and understanding timely decision-making. In International Conference on Machine Learning , 2020.

J. Jing, F. Zhang, L. Zhao, J. Xie, J. Chen, R. Zhong, Y. Zhang, and C. Dong. Correlation between brain 18f-av45 and 18f-fdg pet distribution characteristics and cognitive function in patients with mild and moderate alzheimer's disease. Journal of Alzheimer's Disease , 79(3): 1317-1325, 2021.

P. Kidger, J. Morrill, J. Foster, and T. Lyons. Neural controlled differential equations for irregular time series. Advances in Neural Information Processing Systems , 33:6696-6707, 2020.

J. Kossen, C. Cangea, E. Vértes, A. Jaegle, V . Patraucean, I. Ktena, N. Tomasev, and D. Belgrave. Active acquisition for multimodal temporal data: A challenging decision-making task. arXiv preprint arXiv:2211.05039 , 2022.

S. M. Lundberg and S.-I. Lee. A unified approach to interpreting model predictions. Advances in neural information processing systems , 30, 2017.

N. Mastronarde and M. van der Schaar. Joint physical-layer and system-level power management for delay-sensitive wireless communications. IEEE transactions on mobile computing , 12(4): 694-709, 2012.

M. Naghshvar and T. Javidi. Active sequential hypothesis testing. The Annals of Statistics , 41 (6):2703-2738, 2013.

S. E. O'Bryant, S. C. Waring, C. M. Cullum, J. Hall, L. Lacritz, P. J. Massman, P. J. Lupo, J. S. Reisch, R. Doody, T. A. R. Consortium, et al. Staging dementia using clinical dementia rating scale sum of boxes scores: a texas alzheimer's research consortium study. Archives of neurology , 65(8):1091-1095, 2008.

S. E. O'Bryant, L. H. Lacritz, J. Hall, S. C. Waring, W. Chan, Z. G. Khodr, P. J. Massman, V. Hobson, and C. M. Cullum. Validation of the new interpretive guidelines for the clinical dementia rating scale sum of boxes score in the national alzheimer's coordinating center database. Archives of neurology , 67(6):746-749, 2010.

L. Pinto, J. Davidson, R. Sukthankar, and A. Gupta. Robust adversarial reinforcement learning. In International Conference on Machine Learning , pages 2817-2826. PMLR, 2017.

M. T. Ribeiro, S. Singh, and C. Guestrin. " why should i trust you?" explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining , pages 1135-1144, 2016.

R. T. Rockafellar and S. Uryasev. Conditional value-at-risk for general loss distributions. Journal of banking & finance , 26(7):1443-1471, 2002.

J. Schulman, P. Moritz, S. Levine, M. Jordan, and P. Abbeel. High-dimensional continuous control using generalized advantage estimation. arXiv preprint arXiv:1506.02438 , 2015.

R. S. Sutton and A. G. Barto. Reinforcement learning: An introduction . MIT press, 2018.

J. Yoon, W. R. Zame, and M. van der Schaar. Deep sensing: Active sensing using multidirectional recurrent neural networks. In International Conference on Learning Representations , 2018.

J. Yoon, J. Jordon, and M. van der Schaar. Asac: Active sensing using actor-critic models. In Machine Learning for Healthcare Conference , pages 451-473. PMLR, 2019.

S. Zhang, B. Liu, and S. Whiteson. Mean-variance policy iteration for risk-averse reinforcement learning. In Proceedings of the AAAI Conference on Artificial Intelligence , volume 35, pages 10905-10913, 2021.

Appendix

A.1 Notation table

A summary of major notations used in this paper is provided below.

x : Feature variable of the patient.

y : Patient's outcome.

X : The continuous-time patient trajectory.

X π t : The sensing history at time t .

π : Sensing policy.

π m : Feature selector.

π ∆ : Acquisition scheduler.

m :Feature selection mask vector.

Q π ( X ) : Sensing deficiency.

CVaR : Conditional value-at-risk.

α : Tail risk quantile.

T : End time of observation.

I : Number of observations in sensing history.

I : Interpolator.

A.2 Limitation

The risk-averse policy learning strategy proposed in this paper relies on the correct identification of tail-risk patients. Data quality of the EHR datasets has direct impact on our method. Low quality dataset would lead to poor diagnosis accuracy of the baseline predictor f P which compromises the validity and usefulness of RAS in practice. In the meantime, in RAS, patients located at the long tail of sensing deficiency distribution are selected based on empirical evaluations of Q π ( X ) . The selection results could be biased due to potential over-fitting of sensing policies on the training set. To tackle this issue, an 'honest' and unbiased sensing performance evaluation on the validation set is necessary. However, the inference of acquisition costs from validation samples is beyond the main focus of this paper. We leave this challenge for our future work on risk-averse active sensing tasks.

Code Availability. The source code of RAS can be found in the two GitHub repositories listed below:

The van der Schaar lab repo: https://github.com/vanderschaarlab/cvar_ sensing

The author's personal repo: https://github.com/yvchao/cvar_sensing

A.3 Estimation of Time-Varying Outcome

A baseline outcome predictor f P is required in (5) of the manuscript to evaluate the timeliness and accuracy of a sensing policy. Following the model-based approach in [24], we introduce a network f P - which we refer to as the baseline predictor - to estimate the unknown conditional distributions of patient outcomes. Specifically, we construct the predictor f P with the neural CDE [11] model to flexibly handle sensing history with irregular observation intervals compared to the RNN-based counterparts such as [23, 24].

The predictor consists of four components, i.e., f P = f Y ◦ f E ◦ CDE ◦ I . I is a linear interpolator. CDE is a neural CDE that takes the continuous trajectories as inputs. f E is a multi-layer perceptron (MLP) that takes the CDE embedding as input. f Y is an MLP with softmax output layer for categorical outcomes. Utilizing the linear interpolator I 3 , both the discrete EHR record X ∈ D and sparse sensing history X π t can be converted into continuous trajectories.

The solution to the latent CDE provides embeddings of discrete time-series observations and the sensing history as z n,t = f E (CDE ◦ I ( X n,t )) and z π n,t = f E (CDE ◦ I ( X π n,t )) , respectively. The corresponding conditional distributions are then estimated via the MLP as f P ( X n,t ) = f Y ( z n,t ) and f P ( X π n,t ) = f Y ( z π n,t ) , respectively.

The predictor f P is trained to minimize the following loss:

where z n,t l = f E (CDE ◦I ( X n,t l )) , z π 0 n,t l = f E (CDE ◦I ( X π 0 n,t l )) , and ℓ is the K -class cross-entropy based on the observed outcome y n,t l and outputs of the predictor f P . Here, we introduce the second term as a regularization for each patient trajectory with random dropouts. More specifically, we introduce a random acquisition strategy π 0 which randomly collects (sparse) discrete observations from the interpolated trajectories, i.e., I ( X n ) , and constructs auxiliary sensing histories with random dropouts, i.e., X π 0 n,t l . The strategy π 0 randomly select feature variables to keep in the observation history based on the drop rate. This regularization improves the generalization of the predictor by learning to estimate conditional distributions given randomly drawn sensing histories. Here, µ ≥ 0 is a coefficient that controls the strength of regularization.

A.4 Parameter Selection

Outcome predictor. We consider the drop rate p of the auxiliary observation strategy π 0 as the hyperparameter of the outcome predictor. For both the synthetic dataset and ADNI dataset, we select the drop rate of auxiliary strategy π 0 form the set of p ∈ { 0 . 0 , 0 . 3 , 0 . 5 , 0 . 7 } The optimal hyperparameter is selected based on the average accuracy of the outcome predictor on five test sets of random observations generated by an auxiliary observation strategy π 0 with drop rate p = 0 . 7 . The hyperparameter selection result is as follows:

Synthetic dataset : drop rate p = 0 . 7

ADNI dataset : drop rate p = 0 . 7

Sensing policy. Then, we perform hyperparameter selection for the active sensing approaches. Among the trained outcome predictors, we select the one with highest ROC score on its test set as the shared predictor and keep its parameters frozen. The training and test set corresponding to the selected optimal outcome predictor are used for the training and evaluation of the active sensing methods.

In our experiments, the hyperparameters of each active sensing method are selected based on the cost efficiency, i.e., ROC / Cost, obtained on the test set.

For the synthetic data considered in our manuscript, we set the minimum and maximum allowed acquisition intervals as ∆ min = 0 . 2 , ∆ max = 1 . 0 , respectively. ASAC is not affected since it needs to directly work on original EHR records). The hyperparameters of each method are reported as follows:

ASAC : coefficient for acquisition cost µ = 0 . 01 ∈ { 0 . 1 , 0 . 01 , 0 . 005 , 0 . 001 } .

AS : acquisition interval ˜ ∆ = 1 . 0 ∈ { 0 . 2 , 0 . 5 , 1 . 0 } , shared predictor f P with RAS.

NLL : coefficient for diagnostic error λ = 100 ∈ { 100 , 300 } , shared predictor f P with RAS.

RAS : the coefficient for diagnostic error λ = 300 ∈ { 200 , 250 , 280 , 300 , 310 , 320 , 350 , 400 } , discount factor γ = 0 . 99 , tail-risk quantile α = 0 . 1 , penalty for invalid visits ν = 10 .

For the ADNI data considered in our manuscript, we set the minimum and maximum allowed acquisition intervals as ∆ min = 0 . 5 , ∆ max = 1 . 5 , respectively. ASAC is not affected since it needs to directly work on original EHR records).

ASAC : coefficient for acquisition cost µ = 0 . 1 ∈ { 0 . 1 , 0 . 01 , 0 . 005 , 0 . 001 } .

AS : acquisition interval ˜ ∆ = 1 . 5 ∈ { 0 . 5 , 1 . 5 } , shared predictor f P with RAS.

NLL : coefficient for diagnostic error λ = 200 ∈ { 100 , 200 , 300 , 400 } , shared predictor f P with RAS.

RAS : the coefficient for diagnostic error λ = 400 ∈ { 200 , 250 , 300 , 350 , 400 , 450 } , discount factor γ = 0 . 99 , tail-risk quantile α = 0 . 1 , penalty for invalid visits ν = 10 .

All methods are trained with K = 200 iterations in the experiments. For RAS, we set the tail subset update interval M = 10 for Algorithm 1 in the manuscript.

Adjustments in experiment results. We have fixed some minor issues with model convergence when using CDE models. Specifically, the negative feedback is included in the CDE integration as follows:

where diag creates a diagonal matrix from the input vector z . Such negative feedback loop avoids the unlimited growth of CDE integration over time and helps with model convergence. This modification causes some minor changes in the experiment results. However, the updated experiment results are consistent with the previous versions and our conclusion is not affected by these changes.

Additional results. We will release more evaluation on real-world datasets on our GitHub repo.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper studies the problem of balancing timely and accurate outcome predictions with acquisition costs. To this end, a risk averse active sensing approach (RAS) is proposed that determines when to perform feature acquisition as well as which features to acquire. The proposed approach decomposes the policy into an acquisition scheduler that decides when to perform feature acquisition, and a feature selector that decides which features to acquire. A risk-averse training strategy is introduced to focus on high-risk patients. The proposed approach is evaluated on a synthetic dataset and a real-world dataset from Alzheimer's Disease Neuroimaging Initiative.\n\nSTRENGTHS:\n+ A continuous-time risk-averse active sensing approach is proposed to balance timely and accurate outcome predictions with acquisition costs.\n+ The proposed approach optimizes timely and accurate prediction for tail-risk patients.\n+ Experimental results on a synthetic and a real-world dataset are provided that illustrate the performance of the proposed approach compared to 3 baselines.\n\nWEAKNESSES:\n- Even though the problem of active sensing over time (also known as active state tracking) is well-studied, the related work section is very thin on references.\n- Certain parts of the problem and solution description are unclear or left for the appendix.\n- Certain notation definitions are missing.\n- Certain decision choices are not justified.\n- The structure of the proposed solution, as illustrated in Fig. 2, is not justified.\n- Experiments are provided only on two datasets, out of which 1 is synthetic.\n\nQUESTIONS:\n1. The problem of active sensing over time (also known as active state tracking) is well-studied, but the related work section does not include any such references. An example set of references are provided below:\n - Krishnamurthy, V., 2002. Algorithms for optimal scheduling and management of hidden Markov model sensors. IEEE Transactions on Signal Processing, 50(6), pp.1382-1397.\n- Krishnamurthy, V. and Djonin, D.V., 2007. Structured threshold policies for dynamic sensor scheduling—A partially observed Markov decision process approach. IEEE Transactions on Signal Processing, 55(10), pp.4938-4957.\n- Krishnamurthy, V., 2013. How to schedule measurements of a noisy Markov chain in decision making?. IEEE Transactions on Information Theory, 59(7), pp.4440-4461.\n- Zois, D.S. and Mitra, U., 2017. Active State Tracking With Sensing Costs: Analysis of Two-States and Methods for $ n $-States. IEEE Transactions on Signal Processing, 65(11), pp.2828-2843.\n- Molloy, T.L. and Nair, G.N., 2021, June. Active trajectory estimation for partially observed Markov decision processes via conditional entropy. In 2021 European Control Conference (ECC) (pp. 385-391). IEEE.\n- Atia, G.K., Veeravalli, V.V. and Fuemmeler, J.A., 2011. Sensor scheduling for energy-efficient target tracking in sensor networks. IEEE Transactions on Signal Processing, 59(10), pp.4923-4937.\n- Beyer, C., Büttner, M., Unnikrishnan, V., Schleicher, M., Ntoutsi, E. and Spiliopoulou, M., 2020. Active feature acquisition on data streams under feature drift. Annals of Telecommunications, 75, pp.597-611.\n-  Kossen, J., Cangea, C., Vértes, E., Jaegle, A., Patraucean, V., Ktena, I., Tomasev, N. and Belgrave, D., 2022. Active Acquisition for Multimodal Temporal Data: A Challenging Decision-Making Task. arXiv preprint arXiv:2211.05039.\n\nA careful literature review and follow-up discussion is needed to place the current paper within the vast prior work on this area and clarify its novelty. \n\n2. The problem studied in this paper also relates to quickest change detection, so it will be informative to include a discussion how the proposed approach contrasts/relates to this line of work. Some example references are provided:\n - Heydari, J. and Tajer, A., 2017, March. Quickest change detection in structured data with incomplete information. In 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) (pp. 6434-6438). IEEE.\n- Chaudhuri, A., Fellouris, G. and Tajer, A., 2021, July. Sequential change detection of a correlation structure under a sampling constraint. In 2021 IEEE International Symposium on Information Theory (ISIT) (pp. 605-610). IEEE.\n\n3. Certain parts of the problem and solution description are unclear or left for the appendix. For example, the discussion on the training of the predictor, which is necessary to fully understand how the proposed approach works, is left in the appendix. In addition, it should be mentioned early on in Section 2.1 that a subset of features are selected when acquisition happens.\n\n4. Certain notation definitions are missing or are confusing. For instance, D_{JS}(.) in Eq. (1) is not defined (I assume it is the Jensen-Shannon divergence). Furthermore, what is the relationship between T and I?  In Eq. (6), m and \\Delta have 3 indices, not one as before, why? In Algorithm 1, A & B should be inputs, right? How are they selected in practice? What is their effect on the performance of the algorithm?\n\n5. The structure of the proposed solution, as illustrated in Fig. 2, is not justified. Why this structure is adopted? Why these neural networks?\n\n6.  Certain decision choices are not justified. For example, why the beta distribution is adopted for the policy?\n\n7. Experiments are provided only on two datasets, out of which 1 is synthetic. These are not enough to justify the performance of the proposed approach. To make matters worse, three baselines are provided, out of which only 1 is from prior work. The results on real-world data are not impressive, and in fact comparable with the baselines, apart from the feature acquisition reduction. No statistical significance is provided. There is no justification of the synthetic dataset setting. \n\n8. The usage of the terminology \"cost pressure on patient trajectory acquisition\" is unclear. Also, somewhere in the paper, it is mentioned that there is a budget on acquisition costs but I could not locate where this budget gets into the problem formulation.\n\nLIMITATIONS:\nThe authors have addressed the limitations of the proposed approach and the potential negative societal impact of their work.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 5, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2023-06-23"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposed an active sensing method that answers the questions of when and what diagnosis test to conduct to optimize the trade-off between the cost of acquisition and the timeliness and accuracy of the predictive model. Compared with the existing active sensing model that assumes a fixed data collection schedule, it builds flexibility to determine the timing of data collection. The paper demonstrates the utility of the proposed using synthetic data and a real-world dataset.\n\nSTRENGTHS:\nThe proposed method adds extra flexibility in the decision-making policy to decide when (using continuous time active sensing formulation) to conduct a diagnostic test and what test to complete, which is what the existing model can do. This is a well-motivated problem.\n\nWEAKNESSES:\nAs shown in the attachment, Table A3 on the empirical experiment with a real-world dataset, the proposed method shows only marginal improvement in accuracy and timeliness, but the cost has almost doubled, as compared with the benchmark; not sure whether these results suggest the better quality of the model overall. This raises questions whether there is practical utility of the proposed method. \n\nIn addition,  there are several places the presentation/notation is confusing and hard to follow; see below.\nThere is some notation confusion - for example, in Figure 2, the arrow from the topic box only sources from the post-interperation sensing history, not the post-interpolation of observational patient trajectory, it is unclear why the exclusion of trajectory history is justifiable, or it is an error of the drawing? \n\nFigure 1, subplot (a) is extremely small; subplot (b), the vertical dot line is not explained. \n\nThe equation number and figure number at someplace are not clearly identified, leaving the reader to guess, which cause confusing interpretation, e.g. Page 5 bottom part\n\nQUESTIONS:\nIn real-world data set evaluation, it seems to be a three-class classification, while the performance is reported using AUC, etc. which is binary classifier performance metrics, do you weight the multi-class classification in presenting the result? Why it needs to be formulated as classification, is it more relevant if formulating as regression problem instead bin them into discrete categories?\n\nLIMITATIONS:\nauthor could comment on how the bias/quality of the decision in the original dataset may impact the validity of the approach.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 2, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis work studies the problem of active cost-aware feature acquisition assuming time varying feature settings via breaking-down feature selection and prediction decision making as two policies. The problem being considered is generally a difficult problem even with the non time-varying feature settings.\n\nI have a few questions about the real-world applicability, assumptions, and the approach. I would like to hear from the authors (see below).\n\nSTRENGTHS:\nThe problem is well motivated for healthcare domain and is of clear practical value.\n\nThe paper is well-written and clear\n\nI found considering risk and long-tail behavior very interesting and quite important as cost optimization often means dropping precision for minorities which is not acceptable for health domain usecases\n\nWEAKNESSES:\nThe method used to optimize the cost of acquisition i.e., a linear coefficient to balance cost vs. prediction risk is quite simple and does not necessarily lead to stable or near-optimal results. In the real-world settings the balance point might be different for each sample, and it is even more complicated when we take the time-varying assumption into account.\n\n(see other comments)\n\nQUESTIONS:\nI was wondering what was the motivation behind having two policies rather than one with larger action space?\n\nFollowing on the previous question, wouldn’t the two-policy approach result in less optimal solutions compared to the optimization of one joint policy?\n\nIn the real-world settings that we do not have ground-truth values train set (assuming a cost-aware approach or a version of your method was applied at the time of data collection) how would the training process work? Please correct me but my understanding is that we do need fully observed train set for building the baseline predictor.\n\nLIMITATIONS:\nI have no particular concern", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-07"}

comments: SUMMARY:
This paper investigates timely outcome prediction by proposing a novel risk-averse active sensing approach RAS. The proposed RAS decomposes the policy into acquisition scheduler and feature selector to address the composite decision problem of when to conduct the acquisition and which measurements to make. In addition, RAS enables the prioritization of tail-risk patients in the risk-aversion training procedure. The experiments on synthetic and real-world healthcare datasets show its effectiveness.

STRENGTHS:
This paper is studying a very interesting and important problem - active sensing methods for early detection and intervention of adverse events. The proposed risk-averse active sensing method is technically sound. The experimental results show its effectiveness compared to the three baselines.

WEAKNESSES:
Minor comments: 1.there are so many symbols used in the paper writing, it would be helpful to include a notation table in the appendix. 2. The titles of Section 2 and Section 3 are confusing. Probably, Section 2 is Problem Formulation (or General Framework), Section 3 is Methodology. 3. Highlight the best performance in Table 2.

QUESTIONS:
The result shown in Figure 3 shows that both NRAS and RAS can reduce the long tail in the distribution. It seems NRAS is better than RAS for reducing the long tail, right? Why does this happen?

LIMITATIONS:
The authors have discussed broader impact of this work.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2023-07-27