SlowGuess/NeurIPS_2023_OCR huggingface dataset:
For the glucose simulator, we use the meal-glucose data for three patients. This enables individualization between patients through their glucose baseline  $f_{b}(\cdot)$  and the meal response magnitude function  $l(\cdot)$ . We chose to train the glucose simulator on three patients instead of all since (i) the usage of all patients make the posterior sampling slow due to the increase in the simulator size and (ii) three patients are enough to create a realistic individualization behavior where we have three patient groups with similar glycemia:  $(\mathrm{gr}_1, \mathrm{gr}_2, \mathrm{gr}_3)$ . We show 1-day long examples of the outcome simulator fit in Fig. 8. Besides, we show how the glucose baseline  $f_{b}$  and the meal response  $f_{m}$  differs between these three patients in Fig. 10.
(a) Example 1-day long glucose predictions of patient group 1 for the outcome simulator.
(b) Example 1-day long glucose predictions of patient group 2 for the outcome simulator.
(c) Example 1-day long glucose predictions of patient group 3 for the outcome simulator.
Figure 8: Example 1-day long meal-glucose data and outcome simulator fits for three patient patient groups: meals (carbohydrate intake, cyan bars), glucose (black crosses), predicted glucose baseline  $\mathbf{f_b}$  (red dashed line) and predicted glucose progression f (blue dashed line).
C.2.2 Benchmarks
In the semi-synthetic study, we use the benchmark models shown in Table 3. In the following, we describe the model definitions and implementation details of the mediator and outcome models.
Non-interacting mediator model (L15). The non-interacting mediator model [Non-interacting (L15), Lloyd et al., 2015] is a GP-modulated non-homogeneous Poisson process where the intensity  $\lambda (\tau) = f^2 (\tau)$  is the square transformation of a latent intensity  $f\sim \mathcal{GP}$  with a GP prior. The GP model uses inducing point approximations for scalable inference and variational inference is used for learning. We use an implementation of the model in GPflow [Matthews et al., 2017].
(a) Mediator-effect function  $g_{m}^{*}$ .
(b) Outcome-effect function  $g_{o}^{*}$
(c) Carb. intensity  $\lambda (m\mid \tau)$
Figure 9: Comparison of the ground-truth meal simulator intensity components between pre- and post-surgery periods: (a) the mediator-effect function  $g_{m}^{*}$  that models the dependence of a future meal on past meals, (b) the outcome-effect function  $g_{o}^{*}$  that models the dependence of a future meal on past glucose levels, and (c) the carbohydrate intake intensity  $\lambda(m \mid \tau)$ .
(b) Post-surgery glucose baseline  $f_{b}$ .
(a) Pre-surgery glucose baseline  $f_{b}$ .
(c) Pre-surgery meal response  $f_{m}$ .
Figure 10: Comparison of the ground-truth glucose simulator components between pre- and postsurgery periods for three patient groups  $\{\mathrm{gr}_1, \mathrm{gr}_2, \mathrm{gr}_3\}$  corresponding to three real-world patients. For three groups, we show (a) pre-surgery glucose baseline  $f_b$ , (b) post-surgery glucose baseline  $f_b$ , (c) pre-surgery meal response function  $f_m$ , and (d) post-surgery meal response function  $f_m$ .
(d) Post-surgery meal response  $f_{m}$ .
Parametric mediator-response (S17). As a simple benchmark for the outcome model, we add a highly-cited parametric response model [Parametric (S17), Schulam and Saria, 2017]. It has a conditional GP prior with a parametric response function that models the mediator response in the outcome as a constant effect. We use the implementation provided in the supplementary material in Schulam and Saria [2017].
Parametric mediator-response (Z21). Zeng et al. [2021] proposed a longitudinal causal mediation model based on functional principal component analysis, where its response model [Parametric (Z21), Zeng et al., 2021] is a simple linear function with a single slope parameter. The principal components are chosen as a linear combination of a spline basis where the coefficients satisfy the orthogonality constraint. They assume that the treatment intervention only affects the coefficients of the spline basis. In their work, Zeng et al. [2021] consider the mediator as a continuous-valued stochastic process that is observed at the same times with the outcome measurements, which is different than our problem setup where the mediator is a marked point process. Therefore, we combine the outcome model of [Zeng et al., 2021] with a point-process mediator model in the joint benchmark models Z21-1 and Z21-2, and transform the meal sequence into a continuous-valued sequence that is observed at the same times with the outcome measurements. For this, we use the carbohydrate intake values of the
meals in their effective periods and we simply add up the carbohydrate values if two meals are nearby similar to the additive mediator response assumption in our outcome model definition. We use the publicly available code implemented in R. For inference, the method uses Markov chain Monte Carlo (MCMC). We use 20000 samples with a burn-in period of 2000 samples.
Non-parametric model (H22). The non-parametric model is a GP-based joint model that combines an interacting mediator model [Interacting (H22), Hizli et al., 2023] and a non-parametric response model [Non-parametric (H22), Hizli et al., 2023]. The model assumes that the intervention effect is fully-mediated through meals, and hence it can not capture the direct arrows from the treatment to the outcomes  $(A - - Y)$ . Without explicitly modeling these direct effects, the learned outcome model seems to average the baseline and response functions of the pre- and post-surgery periods. We use an implementation of the model in GPflow [Matthews et al., 2017].
(a) 1-day long example real-world data of patient group 2.
(b) 1-day long example semi-synthetic data for patient group 2.
Figure 11: Example semi-synthetic trajectories from the training set. (a) Example 1-day long real-world data for patient group 2. (b) Example 1-day long semi-synthetic data for patient group 2.
C.2.3 Semi-Synthetic Train and Test Trajectories
Using the ground-truth meal-glucose simulator, we sample 1-day long meal-glucose trajectories of 50 synthetic patients for both pre- and post-surgery periods, i.e., for each regime  $a \in \{0,1\}$ . We show some examples of the simulated trajectories in Fig. 11. We train the benchmark models on both periods, and hence obtain a single joint model for each regime  $a \in \{0,1\}$ .
(a) Glucose trajectories  $\mathbf{Y}_{>\bar{t}_a}[\varnothing ,\varnothing ]$  for the pre-surgery regime  $[\varnothing ,\varnothing ]$
(b) Glucose trajectories  $\mathbf{Y}_{>\tilde{t}_a}[\varnothing ,\tilde{t}_a]$  for the hypothetical regime  $[\varnothing ,\tilde{t}_a]$ .
(c) Glucose trajectories  $\mathbf{Y}_{>\tilde{t}_a}[\tilde{t}_a,\tilde{t}_a]$  for the post-surgery regime  $[\tilde{t}_a,\tilde{t}_a]$ .
Figure 12: Example 1-day long interventional (test) meal-glucose trajectories of the oracle model, the proposed model and the benchmark models in the semi-synthetic study for three hypothetical interventional regimes  $[\varnothing ,\varnothing ]$ $[\varnothing ,\tilde{t}_a]$  and  $[\tilde{t}_a,\tilde{t}_a]$ . In all figures, (i) at top panels we show the mealglucose trajectories and (ii) at bottom panels we show the meal time intensities and the point process sampling procedure. (a) The pre-surgery glucose trajectories  $\mathbf{Y}_{>\tilde{t}_a}[\varnothing ,\varnothing ]$  for the pre-surgery regime  $[\varnothing ,\varnothing ]$ . (b) The hypothetical glucose trajectories  $\mathbf{Y}_{>\tilde{t}_a}[\varnothing ,\tilde{t}_a]$  for the hypothetical regime  $[\varnothing ,\tilde{t}_a]$ . (c) The post-surgery glucose trajectories  $\mathbf{Y}_{>\tilde{t}_a}[\tilde{t}_a,\tilde{t}_a]$  for the post-surgery regime  $[\tilde{t}_a,\tilde{t}_a]$ .
Once the models are trained, we sample three 1-day long interventional trajectories  $\mathbf{Y}_{>\tilde{t}_a}[\varnothing ,\varnothing ]$ $Y_{>\tilde{t}_a}[\varnothing ,\tilde{t}_a]$  and  $\mathbf{Y}_{>\tilde{t}_a}[\tilde{t}_a,\tilde{t}_a]$ , corresponding to three hypothetical interventional regimes  $[\varnothing ,\varnothing ]$
$[\varnothing ,\tilde{t}_a]$  and  $[\tilde{t}_a,\tilde{t}_a]$ . Three trajectories are sampled for the benchmarks and our model. These estimated trajectories are compared to the ground-truth trajectories sampled from the ground-truth simulator. To sample meal-glucose trajectories that are comparable w.r.t. their meal times, we fix the noise variables of the point process sampling similar to Hizl et al. [2023]. After the interventional trajectories are sampled, we compute the  $\mathrm{TE}(\tilde{t}_a)$ ,  $\mathrm{NDE}(\tilde{t}_a)$  and  $\mathrm{NIE}(\tilde{t}_a)$  as described in Eqs. (15) to (17). We show a 1-day long example for the three trajectories in Fig. 12. For all three trajectories  $Y_{>\tilde{t}_a}[\emptyset ,\emptyset ]$ $Y_{>\tilde{t}_a}[\emptyset ,\tilde{t}_a]$  and  $Y_{>\tilde{t}_a}[\tilde{t}_a,\tilde{t}_a]$ , we see that our model follows the oracle meal intensity and the glucose trajectory well. Hence, our model also follows the direct, indirect and total effect trajectories well, since these effects are calculated by subtracting the three interventional trajectories.
We can inspect Fig. 12 to understand how the considered benchmark models fit to the synthetic training data. We see that the simple intercept response model of S17 does not capture the meal response curves well, e.g., the constant response (pink dashed line) in Fig. 12c roughly between 5am - 6pm (in the effective meal periods). Similarly, the linear mediator response function of Z21 is unable to capture the complex, non-linear meal response, e.g., simple constant responses in the purple dashed line in Fig. 12c for the time periods [5, 8], [9, 17] and [19, 22]. We see that the non-parametric benchmark model H22 has a glucose baseline and a meal response that is averaged over the pre- and post-surgery periods, e.g., the under-oracle glucose baseline of H22 in Fig. 12a and the above-oracle glucose baseline of H22 in Fig. 12c (brown dashed lines). Similarly, we see the model fit for the non-interacting mediator benchmark model (pink dashed intensity lines in bottom panels of Figs. 12a to 12c), and it does not capture the dynamic, long-range dependence between the past meal-glucose measurements and the future meals.
D.1 Sensitivity Analysis
In this section, we perform sensitivity analysis by creating a hypothetical hidden-confounding scenario in which the causal assumptions do no longer hold. In such a scenario, a hidden confounder that affects both the mediator and the outcome sequences also can be seen as a source of model misspecification. Therefore, our goals are to understand: (i) how sensitive the model predictions are to the causal assumptions, and (ii) how our model behaves under model misspecification.
Setup. To create a hypothetical hidden-confounding scenario, we assume that patients are advised to be more active after the surgery in the observational (training) data. We sample an unobserved binary confounder  $C_i$  for being active during the period of each meal  $i$  with probability  $\beta$ :  $C_i \sim \mathcal{B}\mathcal{E}(\beta)$ . If active in the meal period, a patient both (i) eats more (bias in meal dosage  $m$ :  $m_{\mathrm{active}} = m^2 / 2$ ), and (ii) their glucose peaks less (bias in response to a meal via the response magnitude  $m_{\mathrm{effective}} = m^2 / 4$ ). We vary the amount of confounding by considering  $\beta \in \{0.0, 0.2, 0.4, 0.6, 0.8, 1.0\}$ , where  $\beta = 0$  corresponds to the case of no unobserved confounding (NUC). Under this scenario, the models will learn biased estimates for meal amounts and meal responses.
We consider three different interventional distributions at test time to understand how the models behave under unbiased or biased estimates: (i) same as the observational data where the patients are advised to be active with the same probability  $\beta$  in the post-surgery period, (ii) the patients are advised to be active with the same probability  $\beta$  in the pre-surgery period, and (iii) the patients are not advised at all. The interventional distribution in (i) is the closest to the observational distribution, while the mismatch between the observational distribution and the interventional distributions in (ii) and (iii) increase for larger  $\beta$  values.
Results. For each case, we report the MSE in the natural direct effect  $\mathrm{NDE}(\tilde{t}_a) = \mathbf{Y}_{>\tilde{t}_a}[\tilde{t}_a,\tilde{t}_a] - \mathbf{Y}_{>\tilde{t}_a}[\emptyset ,\tilde{t}_a]$ , the natural indirect effect  $\mathrm{NIE}(\tilde{t}_a) = \mathbf{Y}_{>\tilde{t}_a}[\emptyset ,\tilde{t}_a] - \mathbf{Y}_{\tilde{t}_a}[\emptyset ,\emptyset ]$ , and the total effect  $\mathrm{TE}(\tilde{t}_a) = \mathbf{Y}_{>\tilde{t}_a}[\tilde{t}_a,\tilde{t}_a] - \mathbf{Y}_{\tilde{t}_a}[\emptyset ,\emptyset ]$ . We repeat the experiments 10 times, and report the mean and  $\pm 1$  standard deviation in Figs. 13 to 15.
From Fig. 13, we see that the results are inconsistent when the same hidden confounding is applied to the training and test data. Even though the model makes biased predictions of meal dosages and responses, it is not penalized at test time since the test data has the same bias on average. In Figs. 14 and 15, the hidden confounding between the train and test data is different, i.e., the models are forced to make biased estimates.
In Figs. 14a and 15a, the MSE of  $\mathrm{NDE}(\tilde{t}_a)$  rises noticeably with the increased confounding level  $\beta$ . This is not surprising as the confounding biases both the post-surgery glucose response to meals (which affects  $Y_{> \tilde{t}_a}[\tilde{t}_a,\tilde{t}_a]$ ) and the post-surgery meal dosages (which affects  $Y_{> \tilde{t}_a}[\tilde{t}_a,\tilde{t}_a]$  and  $Y_{> \tilde{t}_a}[\emptyset,\tilde{t}_a]$ ). For our model, the MSE rises more pronouncedly compared to the baselines, because the latter were already misspecified to start with. In Figs. 14b and 15b, the MSE of  $\mathrm{NIE}(\tilde{t}_a)$  changes less with increased confounding compared to  $\mathrm{NDE}(\tilde{t}_a)$ , as the post-surgery meal times are not affected by the confounding, and the dosage bias (which affects only  $Y_{> \tilde{t}_a}[\emptyset,\tilde{t}_a]$ ) affects the performance in  $\mathrm{NIE}(\tilde{t}_a)$  less compared to the response bias in  $\mathrm{NDE}(\tilde{t}_a)$ . In Figs. 14c and 15c, the MSE in  $\mathrm{TE}(\tilde{t}_a)$  increases with confounding. However, this is less pronounced than for  $\mathrm{NDE}(\tilde{t}_a)$ , as the confounding bias only affects  $Y_{> \tilde{t}_a}[\tilde{t}_a,\tilde{t}_a]$ .
(a)  $\mathrm{NDE}(\tilde{t}_a)$
(b)  $\mathrm{NIE}(\tilde{t}_a)$
(c)  $\mathrm{TE}(\tilde{t}_a)$
Figure 13: MSE for NDE, NIE, TE for the confounding scenario (i), where the patients are advised to be active also in the post-surgery period in the test data, i.e., the same hidden confounding in training and test data.
(a)  $\mathrm{NDE}(\tilde{t}_a)$
Figure 14: MSE for NDE, NIE, TE for the confounding scenario (ii), where the patients are advised to be active in the pre-surgery period in the test data, i.e., different hidden confounding in training and test data.
(a)  $\mathrm{NDE}(\tilde{t}_a)$
Figure 15: MSE for NDE, NIE, TE for the confounding scenario (iii), where the patients are not advised to be active in the test data, i.e., hidden confounding in the training data, no hidden confounding in the test data.