SlowGuess/NeurIPS_2023_OCR huggingface dataset:
- The history encoder,  $f_{H}$ , is a neural CDE encoder that maps sensing history  $X_{t}^{\pi}$  into a summary vector  $\pmb{h}(\tau) = f_{H}(\tau, \mathcal{I}(X_{t}^{\pi}))$ .
- The feature selector,  $f_{M}$ , is a sub-policy network that governs the decision about which subset of patient features to be measured. In a follow-up,  $f_{M}$  takes a summary vector  $\pmb{h}(\tau)$  as input and outputs a vector  $\pmb{p}(\tau) = f_{M}(\pmb{h}(\tau))$  that governs the element-wise Bernoulli distribution to generate a binary feature selection vector  $\pmb{m}$ , i.e.,  $\pmb{m}_{\tau'} \sim \mathrm{Bern}(\pmb{p}(\tau))$  for next follow-up at  $\tau'$ .
- The acquisition scheduler,  $f_{\Delta}$ , is a sub-policy network that determines the time for next follow-up. Particularly, after each follow-up, it takes the summary vector  $h(\tau)$  as input and generates two scalar parameters  $\alpha, \beta = f_{\Delta}(h(\tau))$  of a Beta distribution, i.e.,  $\xi \sim \mathrm{Beta}(\alpha, \beta)$ . The realizations of the Beta distribution are then used to generate the time interval  $\Delta \in [\Delta^{\min}, \Delta^{\max}]$  via  $\Delta = \Delta^{\min} + (\Delta^{\max} - \Delta^{\min}) \cdot \xi$ , where  $\Delta^{\min}$  and  $\Delta^{\max}$  are the minimum and maximum allowed intervals between two follow-ups, respectively. We set  $\alpha, \beta \geq 1$  to ensure that the time interval follows a unimodal distribution.
Policy gradient. Since the diagnostic error function in (5) is non-differentiable, to back-propagate gradients from the objective  $\mathrm{CVaR}_{\alpha} = \mathbb{E}_{\boldsymbol{X}\in S_{\alpha}^{\pi}}[Q^{\pi}(\boldsymbol {X})]$  through the active sensing process, we apply the advantage-based approach [21, 22] to estimate the policy gradient of  $\pi$  as the following:
$$
\nabla_ {\theta} \mathrm {C V a R} _ {\alpha} = \mathbb {E} _ {\pi} \left[ \frac {1}{\left| S _ {\alpha} ^ {\pi} \right|} \sum_ {s = 1} ^ {\left| S _ {\alpha} ^ {\pi} \right|} \sum_ {i = 1} ^ {I} \left(R \left(X _ {s, t _ {i}} ^ {\pi}\right) - R _ {C} \left(X _ {s, t _ {i}} ^ {\pi}\right)\right) \cdot \nabla_ {\theta} \log \pi (\boldsymbol {m} _ {s, i}, \Delta_ {s, i}) \right], \tag {6}
$$
where  $\theta$  is the collection of learnable parameters of policy  $\pi$ . Given the  $s$ -th patient in  $S_{\alpha}^{\pi}$ , the cumulative reward  $R(X_{s,t_i}^{\pi}) = \sum_{\iota = i}^{I}\gamma^{\iota -i}r(X_{s,t_\iota}^{\pi},m_\iota ,\Delta_\iota)$  is an empirical evaluation of the sensing deficiency in (2) for the corresponding sub-trajectory in horizon  $[t_i,T]$ . Following the actor-critic framework, we utilize a baseline sensing deficiency estimation  $R_{C}(X_{t_i}^{\pi})$  in (6) to reduce the variance of policy gradient estimation [21].
The baseline estimation  $R_{C}(X_{t}^{\pi})$  is obtained from a critic network  $f_{C}$  (parameterized by  $\psi$ ). Taking the embedding  $h(t) = f_{H}(t,\mathcal{I}(X_{t}^{\pi}))$  from the history encoder  $f_{H}$  as input. The critic  $f_{C}$  is trained
to minimize the following mean squared error (MSE) loss.
$$
\mathcal {L} _ {C} = \mathbb {E} _ {\pi} \left[ \frac {1}{| S _ {\alpha} ^ {\pi} |} \sum_ {s = 1} ^ {| S _ {\alpha} ^ {\pi} |} \sum_ {i = 0} ^ {I} (R (X _ {n, t _ {i}} ^ {\pi}) - R _ {C} (X _ {n, t _ {i}} ^ {\pi})) ^ {2} \right]. \tag {7}
$$
Algorithm 1 Risk-averse active sensing
Input: dataset  $\mathcal{D}$  , predictor  $f_{P}$  , quantile  $\alpha$  , learning rates  $\mathrm{lr}_{\pi},\mathrm{lr}_c$    
Initialize: learnable parameters  $\theta$  (of  $\pi$  ),  $\psi$  (of  $f_{C}$    
for  $k = 1$  to  $K$  do if  $k - 1$  mod  $M = 0$  then  $\rho_{\alpha}\gets \mathrm{quantile}(Q^{\pi},\mathcal{D},\alpha)$ $S_{\alpha}^{\pi}\gets \{X_{n}|Q^{\pi}(X_{n})\geq \rho_{\alpha},X_{n}\in \mathcal{D}\}$  end if  $\theta \leftarrow \theta -\mathrm{lr}_{\pi}\nabla_{\theta}\mathrm{CVaR}_{\alpha}(S_{\alpha}^{\pi})$ $\psi \gets \psi -\mathrm{lr}_c\nabla_\psi \mathcal{L}_C(S_\alpha^\pi)$    
end for   
Output: risk-averse policy  $\pi$  with parameters  $\theta$
Given the baseline predictor  $f_{P}$ , the sensing deficiency  $Q^{\pi}(\mathbf{X})$  of policy  $\pi$  is optimized over the subset  $S_{\alpha}^{\pi}$  of tail trajectories as illustrated in Algorithm 1. Specifically, parameters  $\theta$  of policy  $\pi$  are updated based on the policy gradient  $\nabla_{\theta}\mathrm{CVaR}_{\alpha}$  in (6), and the critic network  $f_{C}$  is trained with loss function  $\mathcal{L}_C$  to provide appropriate baselines for the sensing policy  $\pi$ . The collection of patient trajectories in  $S_{\alpha}^{\pi}$  is iteratively updated every  $M$  training epochs such that it can stay informative about the sensing deficiency distribution. Note that the learning rates of critic  $f_{C}$  and policy  $\pi$  are intentionally set to be different to facilitate convergence.
We first provide in Table 1 an overview of the distinction between our proposed approach and other related works in active sensing literature. A special class of active sensing task is formulated under the optimal stopping framework. With a set of explicitly defined terminal states, optimal stopping approaches focuses on achieving confident diagnosis of these states while minimizing the time span of the sensing process [1, 6, 15]. The consideration of information acquisition cost was introduced recently into optimal stopping in [9]. Similarly, the trade-off between diagnosis accuracy and costly measurement has been studied in the more general sequential prediction tasks [23, 24]. In this paper, we propose a novel sensing policy decomposition and extend the active sensing analysis into continuous-time settings with adaptive follow-up intervals and highly sparse sensing histories.
Table 1: Comparison of active sensing approaches. Typical active sensing approaches are compared based on the problem formulation, sensing history representation, the modelling of acquisition intervals as well as the following key concerns highlighted in this paper: (1) supports continuous-time patient trajectory. (2) considers cost pressure on features acquisition. (3) achieves decomposition of feature selection and acquisition scheduling. (4) provides worst-case performance guarantees with a risk-averse policy.
Method	Problem Class	History Embedding	Acquisition Interval	(1)	(2)	(3)	(4)
Ahmad and Yu [1]	Optimal Stopping	Bayesian Update	Fixed (Δ = Δ̅)	X	X	X	X
Jarrett and van der Schaar [9]	Optimal Stopping	Bayesian Update	Fixed (Δ = Δ̅)	X	✓	X	X
Yoon et al. [23, 24]	Sequential Prediction	RNN	Fixed (Δ = Δ̅)	X	✓	X	X
RAS (Ours)	Sequential Prediction	Neural CDE	Adaptive (Δ ∼ πΔ̅)	✓	✓	✓	✓
Temporal feature selection. As pointed out by Yoon et al. [24], active sensing can be considered as the temporal generalization of feature selection tasks [13, 19]. In each decision step, the sensing policy is expected to select a set of important feature variables to make the prediction. The resulting sensing history marks critical time points and key feature variables for accurate outcome estimation. However, existing active sensing literature primarily focuses on the feature selection perspective and usually assumes a fixed follow-up interval for every patient trajectory [1, 24]. A fixed acquisition
interval can lead to delayed diagnosis or extra costs due to redundant feature measurements. As mentioned earlier, a desirable sensing policy shall achieve flexible follow-up intervals for each patient such that the shifts in underlying disease stages can be properly addressed.
Risk aversion. Conventional active sensing strategies typically consider the diagnosis accuracy to be equally important for all trajectories in a dataset. However, in many healthcare applications, there are needs to prioritize smaller subgroup of high-risk patients for whom timely detection of adverse outcomes is particularly valuable. As a central component of our proposed method, the concept of CVaR [20] enables the selective optimization for patient trajectories in the long tail of sensing deficiency distribution. Optimizing sensing policies over the corresponding tail set  $S_{\alpha}^{\pi}$  yields a risk-averse active sensing strategy  $\pi^{*}$  that achieves guaranteed diagnosis accuracy for high-risk patients. While risk-aware decision-making has been extensively explored in the field of RL [4, 25], to the best of our knowledge, our proposed method is the first risk-averse active sensing algorithm for adaptive feature acquisition in continuous-time settings.
In the experiments, we evaluate the effectiveness of our proposed risk-averse active sensing approach RAS on both synthetic and real-world healthcare datasets.
Synthetic dataset. We construct a synthetic dataset  $\mathcal{D}_S$  of  $N = 2000$  samples. Every sample  $X$  contains  $L = 20$  discrete observations in  $t\in [0,2]$ . Each observation yields a feature vector  $\pmb{x} = [x_{1},x_{2},x_{3},x_{4}]^{\top}$ . We set  $x_{1}(t) = \min (1,(e^{w(t - \tau)} - w(t - \tau) - 1))\mathbb{I}(t\geq \tau)$ , where  $w$  follows a Gaussian mixture distribution of  $0.8\cdot \mathcal{N}(0.3,0.1^2) + 0.2\cdot \mathcal{N}(0.8,0.1^2)$ ,  $\tau \sim \mathrm{Exp}(1.0)$ . As a proxy of  $x_{1}$ , we set  $x_{2}(t) = w(t - \tau)\mathbb{I}(t\geq \tau)$ . Variables  $x_{3}(t) = \sin (3t + \varrho)$  and  $x_{4}(t) = \cos (3t + \varrho)$ , where  $\varrho \sim \mathcal{N}(0,1^2)$ . The outcome  $y\in \{0,1\}$  follows a Bernoulli distribution  $y_{t}\sim \operatorname {Bern}(p)$ , and the likelihood for  $y_{t} = 1$  is calculated as  $p = e^{-3|1 - x_1|^2}$ . The cost vector is set to be  $c = [1.0,0.1,1.0,1.0]^{\top}$ , which allows  $x_{2}$  to be measured as a cheap proxy of  $x_{1}$  for outcome prediction.
ADNI dataset. The Alzheimer's Disease Neuroimaging Initiative $^2$  (ADNI) dataset includes records on AD progression of  $N = 1002$  patients with regular follow-ups every six months. We consider the active sensing task to predict patient outcomes in the initial  $L = 12$  follow-ups with four biomarkers from PET (FDG and AV45) and MRI (Hippocampus and Entorhinal) imaging, respectively. Based on the disease staging guideline [16, 17], target outcome  $y_{t} \in \{\text{Normal, mild cognitive impairment (MCI), AD}\}$  at each time point is determined via the corresponding Clinical Dementia Rating scale Sum of Boxes (CDR-SB) score with cutoff thresholds at 0.5 and 4.5. To reflect the higher cost and harmful radiation exposure in PET scan, measurement costs for PET and MRI biomarkers are set to be 1.0 and 0.5, respectively.
Experiment setup. We first fit the outcome estimator  $f_{P}$  on each dataset with 64/16/20 train/validation/test splits and find the optimal estimator with the best prediction accuracy given sparse observation of test samples as inputs. Then, we freeze the parameters of the optimal  $f_{P}$  and evaluate the sensing performance of RAS and other baselines on the corresponding data split. We consider four baselines in the experiments: FO) predictor  $f_{P}$  with dense sensing history; ASAC) [24]; NLL) adaptive sensing with negative log-likelihood (NLL) as diagnosis error function [12]; AS)  $\mathrm{RAS}(\alpha = 1.0)$  with constant acquisition interval. Further details of the experiment setup and parameter selection results can be found in the Appendix.
Table 2: Benchmark of sensing performance on the synthetic dataset  ${\mathcal{D}}_{S}$  .
METHOD	ROC	PRC	Cost	dδ=0.3	dδ=0.5	dδ=0.7
FO	0.680±0.000	0.655±0.000‡	31.000±0.000	0.502±0.000	0.349±0.000	0.285±0.000
ASAC	0.605±0.096	0.559±0.080	0.460±1.078‡	1.099±0.664	1.066±0.699	1.052±0.641
AS	0.671±0.001	0.614±0.001	4.501±0.497	0.577±0.029	0.522±0.012	0.479±0.015
NLL	0.636±0.023	0.588±0.016	2.968±0.774	0.993±0.131	0.974±0.141	0.975±0.147
RAS (OURS)	0.680±0.003	0.647±0.006	6.077±0.953	0.325±0.084‡	0.264±0.086‡	0.246±0.071‡
The  $95\%$  confidence interval (CI) is evaluated with five different random seeds. Best result in each column are highlighted in bold. The marker  $\ddagger$  indicates p-value  $p < 0.05$ .
$^{2}$ https://adni.loni.usc.edu
Timeliness and accuracy. Timely and accurate outcome prediction given sparse observations of patient trajectories is essential for practical applications of active sensing. In the experiments, we report the area under the curve of receiving-operator characteristic (ROC) and area under the curve of precision-recall (PRC) as assessments of the prognostic value of active sensing policies. In the meantime, the timeliness of an active sensing policy  $\pi$  is evaluated with  $d_{\delta}$  - the mean absolute mismatch in diagnosis time of event  $p(y_{t} = 1|X_{t}^{\pi})\geq \delta$  compared to ground truth on the test set. The benchmark results of RAS against four baselines on the two datasets are provided in Table 2 and Table 3, respectively.
With a fixed acquisition interval of  $\Delta = 0.2$ , FO achieves high predictive accuracy at the expense of highest acquisition cost. Despite having the smallest acquisition cost, ASAC suffers significantly from the low diagnosis accuracy and is unable to achieve timely diagnosis of alert events. In contrast, our method (RAS) has reasonable acquisition costs and best timeliness as illustrated in Table 2. The increase in average cost of RAS compared to the AS baseline could be linked to our focus on the tail set  $S_{\alpha}^{\pi}$ . The increased acquisition costs are necessary to achieve a significantly lower delay in alert event diagnosis for high-risk patients in  $S_{\alpha}^{\pi}$ . The improved timeliness of RAS over FO suggests that the predictor  $f_{P}$  might achieve better generalization with sparse sensing history as inputs. Similarly, according to Table 3, our method RAS outperforms most other baselines in timeliness and cost efficiency except for FO. This is within expectation since the outcome estimations from  $f_{P}$  are used as the reference for timeliness evaluation, and these estimations are generally close to the predictions made by the FO baseline.
Table 3: Benchmark of active sensing performance on ADNI dataset.
METHOD	ROC	PRC	Cost	dδ=0.1	dδ=0.3	dδ=0.5
FO	0.747±0.000‡	0.577±0.000‡	26.865±0.000	0.141±0.000‡	0.510±0.000‡	0.591±0.000‡
ASAC	0.521±0.160	0.352±0.103	0.043±0.186‡	0.527±0.000	3.008±3.610	3.581±0.000
AS	0.704±0.023	0.519±0.034	3.566±0.854	1.326±0.096	2.314±0.348	2.357±0.375
NLL	0.697±0.018	0.512±0.020	3.986±0.493	1.040±0.149	2.176±0.060	2.739±0.135
RAS (OURS)	0.730±0.007	0.560±0.012	8.614±1.157	0.820±0.096	1.370±0.227	1.192±0.176
The  $95\%$  confidence interval (CI) is evaluated with three different random seeds. The timeliness  $d_{\delta}$  is measured based on the likelihood of developing AD. Best result in each column are highlighted in bold. The marker  $\ddagger$  indicates p-value  $p < 0.05$ .
(a) Pareto front of sensing policies.
(b) Distributions of sensing deficiency  $Q^{\pi}(\mathbf{X})$
Figure 3: The impact of risk-aversion and adaptive acquisition scheduling on active sensing policies. Selection of trade-off coefficients, e.g.,  $\lambda$ , can be performed by searching for the (a) Pareto front. Pareto optimal models belonging to the front are highlighted with circles. Their parameters are annotated in gray texts. Effectiveness of the risk-averse objective in (3) is reflected by the tail size of (b)  $Q^{\pi}(\mathbf{X})$  distribution. The risk-neutral  $(\alpha = 1.0)$  version of RAS has degraded sensing performance for tail trajectories. The significantly worse performance of the AS baseline further highlights the benefit of having adaptive acquisition intervals.
Trade-off between timeliness and acquisition costs. The selection of model parameters could be difficult when two or more criteria are involved in the evaluation. Here, we illustrate the sensing performance of different policies on the synthetic dataset in Figure 3(a) and highlight the ones in the Pareto front with gray circles. These policies are considered Pareto optimal since their timeliness
$(d_{\delta = 0.5})$  and average acquisition cost cannot be simultaneously improved by swapping parameters with other policies. Benefitted from the risk-averse training strategy, most sensing policies obtained via RAS are centered around the knee point of the Pareto front, which helps to explain the outstanding cost efficiency of RAS as reported in Table 2.
Reshaping the sensing deficiency distribution. To illustrate the effectiveness of the risk-averse objective in (3), we compare the empirical distribution of sensing deficiency  $Q^{\pi}(\mathbf{X})$  of RAS with the ablations of risk-neutral sensing  $(\alpha = 1.0)$  and AS baseline  $(\alpha = 1.0$ , constant acquisition interval  $\Delta = 1.0$ ) on the synthetic dataset. All three models are trained with the same trade-off coefficient  $\lambda = 300$ . As illustrated in Figure 3(b), RAS is able to effectively optimize the sensing performance for trajectories in the tail set  $S_{\alpha}^{\pi}$  and reduces the upper  $\alpha$  quantile of  $Q^{\pi}(\mathbf{X})$  to  $\rho_{\alpha=0.1} = 10.40$ . Factor  $\alpha = 1.0$  completely disables the risk-aversion training strategy in Algorithm 1. Thereby, a clear increase of sensing deficiency (quantile  $\rho_{\alpha=0.1}$  grows from 10.40 to 20.01) is observed with the risk-neutral ablation of RAS. Similarly, without adaptive scheduling of acquisition intervals and risk-averse optimization strategies, the AS baseline illustrates the failure of conventional active sensing paradigms at the long tail of  $Q^{\pi}(\mathbf{X})$  distribution.
In this paper, we introduce a novel risk-averse active sensing approach RAS to address the challenging continuous-time active sensing problem. Through effective decomposition of decisions on feature selection and acquisition intervals, our approach offers valuable insights on both feature importance and timeliness of patient follow-ups. The novel risk-aversion training strategy in RAS enables the prioritization of high-value patients at the long tail of sensing deficiency distribution and provides guarantees on diagnosis accuracy for worst-case scenarios. The effectiveness of our method is evaluated through experiments on synthetic and real-world healthcare datasets, where RAS outperforms all baseline active sensing approaches and achieves accurate and timely outcome diagnosis while maintaining reasonable costs in feature acquisition.
The development of an advanced active sensing strategy is essential for precision medicine. Appropriate sensing policies can help clinicians to make tailored follow-up schedules for their patients and effectively utilize the costly and potentially harmful measurements on some important patient characteristics under certain levels of cost pressure. We note that the reduction in acquisition cost is usually associated with degraded diagnosis accuracy and delayed detection of some adverse clinical events, especially for high-risk patients that could benefit from more frequent follow-ups on disease progression. In this paper, we attempt to address this challenge through the risk-averse policy learning strategy in our proposed active sensing approach RAS and achieve improved sensing performance (accuracy and timeliness) on the entire population. Nevertheless, practical applications of active sensing approaches would require careful audits and assessments by human experts to avoid potential negative impacts. The sensing decisions from RAS are only suggestions optimized based on observational data and should not be directly applied without the evaluation of clinicians.
Yuchao Qin was supported by the Cystic Fibrosis Trust. Changhee Lee was supported by the IITP grant funded by the Korea government (MSIT) (No.2021-0-01341, AI Graduate School Program, CAU). This work was supported by Azure sponsorship credits granted by Microsoft's AI for Good Research Lab.
[1] S. Ahmad and A. J. Yu. Active sensing as bayes-optimal sequential decision-making. In Proceedings of the Twenty-Ninth Conference on Uncertainty in Artificial Intelligence, pages 12-21, 2013.
[20] R. T. Rockafellar and S. Uryasev. Conditional value-at-risk for general loss distributions. Journal of banking & finance, 26(7):1443-1471, 2002.
[21] J. Schulman, P. Moritz, S. Levine, M. Jordan, and P. Abbeel. High-dimensional continuous control using generalized advantage estimation. arXiv preprint arXiv:1506.02438, 2015.
[22] R. S. Sutton and A. G. Barto. Reinforcement learning: An introduction. MIT press, 2018.
[23] J. Yoon, W. R. Zame, and M. van der Schaar. Deep sensing: Active sensing using multidirectional recurrent neural networks. In International Conference on Learning Representations, 2018.
[24] J. Yoon, J. Jordon, and M. van der Schaar. Asac: Active sensing using actor-critic models. In Machine Learning for Healthcare Conference, pages 451-473. PMLR, 2019.
[25] S. Zhang, B. Liu, and S. Whiteson. Mean-variance policy iteration for risk-averse reinforcement learning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pages 10905–10913, 2021.
A.1 Notation table
A summary of major notations used in this paper is provided below.
-  $x$  : Feature variable of the patient.
-  $y$  : Patient's outcome.
-  $X$ : The continuous-time patient trajectory.
-  $X_{t}^{\pi}$ : The sensing history at time  $t$ .
-  $\pi$ : Sensing policy.
-  $\pi_{m}$ : Feature selector.
-  $\pi_{\Delta}$ : Acquisition scheduler.
-  $m$  : Feature selection mask vector.
-  $Q^{\pi}(\mathbf{X})$ : Sensing deficiency.
- CVaR: Conditional value-at-risk.
-  $\alpha$ : Tail risk quantile.
-  $T$ : End time of observation.
-  $I$ : Number of observations in sensing history.
-  $\mathcal{I}$ : Interpolator.
The risk-averse policy learning strategy proposed in this paper relies on the correct identification of tail-risk patients. Data quality of the EHR datasets has direct impact on our method. Low quality dataset would lead to poor diagnosis accuracy of the baseline predictor  $f_{P}$  which compromises the validity and usefulness of RAS in practice. In the meantime, in RAS, patients located at the long tail of sensing deficiency distribution are selected based on empirical evaluations of  $Q^{\pi}(\mathbf{X})$ . The selection results could be biased due to potential over-fitting of sensing policies on the training set. To tackle this issue, an "honest" and unbiased sensing performance evaluation on the validation set is necessary. However, the inference of acquisition costs from validation samples is beyond the main focus of this paper. We leave this challenge for our future work on risk-averse active sensing tasks.
Code Availability. The source code of RAS can be found in the two GitHub repositories listed below:
- The van der Schaar lab repo: https://github.com/vanderschaarlab/cvar_sensing
- The author's personal repo: https://github.com/yyvchao/cvar_sensing
A.3 Estimation of Time-Varying Outcome
A baseline outcome predictor  $f_{P}$  is required in (5) of the manuscript to evaluate the timeliness and accuracy of a sensing policy. Following the model-based approach in [24], we introduce a network  $f_{P}$  - which we refer to as the baseline predictor - to estimate the unknown conditional distributions of patient outcomes. Specifically, we construct the predictor  $f_{P}$  with the neural CDE [11] model to flexibly handle sensing history with irregular observation intervals compared to the RNN-based counterparts such as [23, 24].
The predictor consists of four components, i.e.,  $f_{P} = f_{Y} \circ f_{E} \circ \mathrm{CDE} \circ \mathcal{I}$ .  $\mathcal{I}$  is a linear interpolator. CDE is a neural CDE that takes the continuous trajectories as inputs.  $f_{E}$  is a multi-layer perceptron (MLP) that takes the CDE embedding as input.  $f_{Y}$  is an MLP with softmax output layer for categorical outcomes. Utilizing the linear interpolator  $\mathcal{I}^3$ , both the discrete EHR record  $X \in \mathcal{D}$  and sparse sensing history  $X_{t}^{\pi}$  can be converted into continuous trajectories.
3We use Python package torchcde for interpolation.