SlowGuess/NeurIPS_2023_OCR huggingface dataset:
The solution to the latent CDE provides embeddings of discrete time-series observations and the sensing history as  $\mathbf{z}_{n,t} = f_{E}(\mathrm{CDE} \circ \mathcal{I}(X_{n,t}))$  and  $\mathbf{z}_{n,t}^{\pi} = f_{E}(\mathrm{CDE} \circ \mathcal{I}(X_{n,t}^{\pi}))$ , respectively. The corresponding conditional distributions are then estimated via the MLP as  $f_{P}(X_{n,t}) = f_{Y}(\mathbf{z}_{n,t})$  and  $f_{P}(\mathbf{X}_{n,t}^{\pi}) = f_{Y}(\mathbf{z}_{n,t}^{\pi})$ , respectively.
The predictor  $f_{P}$  is trained to minimize the following loss:
$$
\mathcal {L} _ {P} = \frac {1}{N L} \sum_ {n = 1} ^ {N} \sum_ {l = 1} ^ {L} \ell \left(y _ {n, t _ {l}}, f _ {Y} \left(z _ {n, t _ {l}}\right)\right) + \mu \ell \left(y _ {n, t _ {l}}, f _ {Y} \left(z _ {n, t _ {l}} ^ {\pi^ {0}}\right)\right), \tag {8}
$$
where  $z_{n,t_l} = f_E(\mathrm{CDE}\circ \mathcal{I}(X_{n,t_l}))$ ,  $z_{n,t_l}^{\pi^0} = f_E(\mathrm{CDE}\circ \mathcal{I}(X_{n,t_l}^{\pi^0}))$ , and  $\ell$  is the  $K$ -class cross-entropy based on the observed outcome  $y_{n,t_l}$  and outputs of the predictor  $f_P$ . Here, we introduce the second term as a regularization for each patient trajectory with random dropouts. More specifically, we introduce a random acquisition strategy  $\pi^0$  which randomly collects (sparse) discrete observations from the interpolated trajectories, i.e.,  $\mathcal{I}(X_n)$ , and constructs auxiliary sensing histories with random dropouts, i.e.,  $X_{n,t_l}^{\pi^0}$ . The strategy  $\pi^0$  randomly selects feature variables to keep in the observation history based on the drop rate. This regularization improves the generalization of the predictor by learning to estimate conditional distributions given randomly drawn sensing histories. Here,  $\mu \geq 0$  is a coefficient that controls the strength of regularization.
A.4 Parameter Selection
Outcome predictor. We consider the drop rate  $p$  of the auxiliary observation strategy  $\pi^0$  as the hyperparameter of the outcome predictor. For both the synthetic dataset and ADNI dataset, we select the drop rate of auxiliary strategy  $\pi^0$  form the set of  $p \in \{0.0, 0.3, 0.5, 0.7\}$ . The optimal hyperparameter is selected based on the average accuracy of the outcome predictor on five test sets of random observations generated by an auxiliary observation strategy  $\pi^0$  with drop rate  $p = 0.7$ . The hyperparameter selection result is as follows:
- Synthetic dataset: drop rate  $p = 0.7$
- ADNI dataset: drop rate  $p = 0.7$
Sensing policy. Then, we perform hyperparameter selection for the active sensing approaches. Among the trained outcome predictors, we select the one with highest ROC score on its test set as the shared predictor and keep its parameters frozen. The training and test set corresponding to the selected optimal outcome predictor are used for the training and evaluation of the active sensing methods.
In our experiments, the hyperparameters of each active sensing method are selected based on the cost efficiency, i.e., ROC / Cost, obtained on the test set.
For the synthetic data considered in our manuscript, we set the minimum and maximum allowed acquisition intervals as  $\Delta^{\mathrm{min}} = 0.2$ ,  $\Delta^{\mathrm{max}} = 1.0$ , respectively. ASAC is not affected since it needs to directly work on original EHR records). The hyperparameters of each method are reported as follows:
- ASAC: coefficient for acquisition cost  $\mu = 0.01 \in \{0.1, 0.01, 0.005, 0.001\}$ .
- AS: acquisition interval  $\tilde{\Delta} = 1.0 \in \{0.2, 0.5, 1.0\}$ , shared predictor  $f_{P}$  with RAS.
- NLL: coefficient for diagnostic error  $\lambda = 100\in \{100,300\}$ , shared predictor  $f_{P}$  with RAS.
- RAS: the coefficient for diagnostic error  $\lambda = 300 \in \{200, 250, 280, 300, 310, 320, 350, 400\}$ , discount factor  $\gamma = 0.99$ , tail-risk quantile  $\alpha = 0.1$ , penalty for invalid visits  $\nu = 10$ .
For the ADNI data considered in our manuscript, we set the minimum and maximum allowed acquisition intervals as  $\Delta^{\mathrm{min}} = 0.5$ ,  $\Delta^{\mathrm{max}} = 1.5$ , respectively. ASAC is not affected since it needs to directly work on original EHR records).
- ASAC: coefficient for acquisition cost  $\mu = 0.1 \in \{0.1, 0.01, 0.005, 0.001\}$ .
- AS: acquisition interval  $\tilde{\Delta} = 1.5 \in \{0.5, 1.5\}$ , shared predictor  $f_{P}$  with RAS.
- NLL: coefficient for diagnostic error  $\lambda = 200 \in \{100, 200, 300, 400\}$ , shared predictor  $f_{P}$  with RAS.
- RAS: the coefficient for diagnostic error  $\lambda = 400 \in \{200, 250, 300, 350, 400, 450\}$ , discount factor  $\gamma = 0.99$ , tail-risk quantile  $\alpha = 0.1$ , penalty for invalid visits  $\nu = 10$ .
All methods are trained with  $K = 200$  iterations in the experiments. For RAS, we set the tail subset update interval  $M = 10$  for Algorithm 1 in the manuscript.
Adjustments in experiment results. We have fixed some minor issues with model convergence when using CDE models. Specifically, the negative feedback is included in the CDE integration as follows:
$$
\boldsymbol {z} (t) = \boldsymbol {z} \left(t _ {0}\right) + \int_ {t _ {0}} ^ {t} \left(f _ {\phi} (\boldsymbol {z} (\tau)) - 1 0 ^ {- 3} \operatorname {d i a g} (\boldsymbol {z})\right) \mathrm {d} \boldsymbol {u} (\tau), \text {f o r} \tau \in [ t _ {0}, t ], \tag {9}
$$
where diag creates a diagonal matrix from the input vector  $\mathbf{z}$ . Such negative feedback loop avoids the unlimited growth of CDE integration over time and helps with model convergence. This modification causes some minor changes in the experiment results. However, the updated experiment results are consistent with the previous versions and our conclusion is not affected by these changes.
Additional results. We will release more evaluation on real-world datasets on our GitHub repo.