SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Marcel Kollovieh $^{2*}$ † Abdul Fatir Ansari $^{1*}$  Michael Bohlke-Schneider $^{1}$
Jasper Zschiegner<sup>1</sup> Hao Wang<sup>1</sup> Yuyang Wang<sup>1</sup>
$^{1}$ AWS AI Labs  $^{2}$ Technical University of Munich
Correspondence to: ansarnd@amazon.de
Diffusion models have achieved state-of-the-art performance in generative modeling tasks across various domains. Prior works on time series diffusion models have primarily focused on developing conditional models tailored to specific forecasting or imputation tasks. In this work, we explore the potential of task-agnostic, unconditional diffusion models for several time series applications. We propose TSDiff, an unconditionally-trained diffusion model for time series. Our proposed self-guidance mechanism enables conditioning TSDiff for downstream tasks during inference, without requiring auxiliary networks or altering the training procedure. We demonstrate the effectiveness of our method on three different time series tasks: forecasting, refinement, and synthetic data generation. First, we show that TSDiff is competitive with several task-specific conditional forecasting methods (predict). Second, we leverage the learned implicit probability density of TSDiff to iteratively refine the predictions of base forecasters with reduced computational overhead over reverse diffusion (refine). Notably, the generative performance of the model remains intact — downstream forecasters trained on synthetic samples from TSDiff outperform forecasters that are trained on samples from other state-of-the-art generative time series models, occasionally even outperforming models trained on real data (synthesize).
Time series forecasting informs key business decisions [27], for example in finance [47], renewable energy [56], and healthcare [7]. Recently, deep learning-based models have been successfully applied to the problem of time series forecasting [46, 31, 5]. Instantiations of these methods use, among other techniques, autoregressive modeling [46, 42], sequence-to-sequence modeling [54], and normalizing flows [43, 8]. These techniques view forecasting as the problem of conditional generative modeling: generate the future, conditioned on the past.
Diffusion models [48, 20] have shown outstanding performance on generative tasks across various domains [10, 28, 30, 36, 44] and have quickly become the framework of choice for generative modeling. Recent studies used conditional diffusion models for time series forecasting and imputation tasks [44, 52, 1, 6]. However, these models are task specific, i.e., their applicability is limited to the specific imputation or forecasting task they have been trained on. Consequently, they also forego the desirable unconditional generative capabilities of diffusion models.
$\dagger$  Work conducted during an internship at Amazon.
Figure 1: An overview of TSDiff's use cases. Predict: By utilizing observation self-guidance, TSDiff can be conditioned during inference to perform predictive tasks such as forecasting (see Sec. 3.1). Refine: Predictions of base forecasters can be improved by leveraging the implicit probability density of TSDiff (see Sec. 3.2). Synthesize: Realistic samples generated by TSDiff can be used to train downstream forecasters achieving good performance on real test data (see Sec. 4.3).
This raises a natural research question: Can we address multiple (even conditional) downstream tasks with an unconditional diffusion model? Specifically, we investigate the usability of task-agnostic unconditional diffusion models for forecasting tasks. We introduce TSDiff, an unconditional diffusion model for time series, and propose two inference schemes to utilize the model for forecasting. Building upon recent work on guided diffusion models [10, 19], we propose a self-guidance mechanism that enables conditioning the model during inference, without requiring auxiliary networks. This makes the unconditional model amenable to arbitrary forecasting (and imputation) tasks that are conditional in nature<sup>3</sup>. We conducted comprehensive experiments demonstrating that our self-guidance approach is competitive against task-specific models on several datasets and across multiple forecasting scenarios, without requiring conditional training. Additionally, we propose a method to iteratively refine predictions of base forecasters with reduced computational overhead compared to reverse diffusion by interpreting the implicit probability density learned by TSDiff as an energy-based prior. Finally, we show that the generative capabilities of TSDiff remain intact. We train multiple downstream forecasters on synthetic samples from TSDiff and show that forecasters trained on samples from TSDiff outperform those trained on samples from variational autoencoders [9] and generative adversarial networks [57], sometimes even outperforming models trained on real samples. To quantify the generative performance, we introduce the Linear Predictive Score (LPS) which we define as the test forecast performance of a linear ridge regression model trained on synthetic samples. TSDiff significantly outperforms competing generative models in terms of the LPS on several benchmark datasets. Fig. 1 highlights the three use cases of TSDiff: predict, refine, and synthesize.
In summary, our key contributions are:
- TSDiff, an unconditionally trained diffusion model for time series and a mechanism to condition TSDiff during inference for arbitrary forecasting tasks (observation self-guidance);
- An iterative scheme to refine predictions from base forecasters by leveraging the implicit probability density learned by TSDiff;
- Experiments on multiple benchmark datasets and forecasting scenarios demonstrating that observation self-guidance is competitive against task-specific conditional baselines;
- Linear Predictive Score, a metric to evaluate the predictive quality of synthetic samples, and experiments demonstrating that TSDiff generates realistic samples that outperform competing generative models in terms of their predictive quality.
The rest of the paper is organized as follows. Sec. 2 introduces the relevant background on denoising diffusion probabilistic models (DDPMs) and diffusion guidance. In Sec. 3, we present TSDiff, an unconditional diffusion model for time series, and propose two inference schemes to utilize the model for forecasting tasks. Sec. 5 discusses the related work on diffusion models for time series and diffusion guidance. Our empirical results are presented in Sec. 4. We conclude with a summary of our findings, the limitations of our proposals, and their potential resolutions in Sec. 6.
Note that in contrast to meta-learning and foundation models literature, we train models per dataset and only vary the inference task, e.g., forecasting with different missing value scenarios.
Diffusion models [48, 20] provide a framework for modeling the data generative process as a discrete-time diffusion process. They are latent variable models of the form  $p_{\theta}(\mathbf{y}) = \int p_{\theta}(\mathbf{y}, \mathbf{x}^{1:T}) d\mathbf{x}^{1:T}$ , where  $\mathbf{y} \sim q(\mathbf{y})$  is the true underlying distribution. The latent variables  $\{\mathbf{x}^1, \ldots, \mathbf{x}^T\}$  are generated by a fixed Markov process with Gaussian transitions, often referred to as the forward process,
$$
q \left(\mathbf {x} ^ {1}, \dots , \mathbf {x} ^ {T} \mid \mathbf {x} ^ {0} = \mathbf {y}\right) = \prod_ {t = 1} ^ {T} q \left(\mathbf {x} ^ {t} \mid \mathbf {x} ^ {t - 1}\right) \quad \text {a n d} \quad q \left(\mathbf {x} ^ {t} \mid \mathbf {x} ^ {t - 1}\right) := \mathcal {N} \left(\sqrt {1 - \beta_ {t}} \mathbf {x} _ {t - 1}, \beta_ {t} \mathbf {I}\right), \tag {1}
$$
where  $\beta_{t}$  is the variance of the additive noise,  $\mathbf{y}$  is the observed datapoint, and  $q(\mathbf{x}^T)\approx \mathcal{N}(\mathbf{0},\mathbf{I})$ . The fixed Gaussian forward process allows direct sampling from  $q(\mathbf{x}^t |\mathbf{y})$
$$
\mathbf {x} ^ {t} = \sqrt {\bar {\alpha} _ {t}} \mathbf {y} + \sqrt {\left(1 - \bar {\alpha} _ {t}\right)} \boldsymbol {\epsilon}, \tag {2}
$$
where  $\alpha_{t} = 1 - \beta_{t}$ ,  $\bar{\alpha}_{t} = \prod_{i=1}^{t} \alpha_{i}$ , and  $\epsilon \sim \mathcal{N}(\mathbf{0},\mathbf{I})$ . On the other hand, the reverse diffusion (generative) process is formulated as,
$$
p _ {\theta} \left(\mathbf {x} ^ {0} = \mathbf {y}, \dots , \mathbf {x} ^ {T}\right) = p \left(\mathbf {x} ^ {T}\right) \prod_ {t = 1} ^ {T} p _ {\theta} \left(\mathbf {x} ^ {t - 1} \mid \mathbf {x} ^ {t}\right) \quad \text {a n d} \quad p _ {\theta} \left(\mathbf {x} ^ {t - 1} \mid \mathbf {x} ^ {t}\right) := \mathcal {N} \left(\mu_ {\theta} \left(\mathbf {x} ^ {t}, t\right), \sigma_ {t} \mathbf {I}\right), \tag {3}
$$
where  $p(\mathbf{x}^T) \sim \mathcal{N}(\mathbf{0}, \mathbf{I})$  and  $\sigma_t = \frac{1 - \bar{\alpha}_{t-1}}{1 - \bar{\alpha}_t} \beta_t$ . The model is trained to approximate the true reverse process  $q(\mathbf{x}^{t-1} | \mathbf{x}^t)$  by maximizing an approximation of the evidence lower bound (ELBO) of the log-likelihood. Specifically,  $\mu_\theta$  is parameterized using a denoising network,  $\epsilon_\theta$ ,
$$
\mu_ {\theta} \left(\mathbf {x} ^ {t}, t\right) = \frac {1}{\sqrt {\alpha_ {t}}} \left(\mathbf {x} ^ {t} - \frac {\beta_ {t}}{\sqrt {1 - \bar {\alpha} _ {t}}} \boldsymbol {\epsilon} _ {\theta} \left(\mathbf {x} ^ {t}, t\right)\right), \tag {4}
$$
which is trained to predict the sampled noise ( $\epsilon$  in Eq. 2) using the simplified objective function [20],
$$
\mathbb {E} _ {\mathbf {y}, \epsilon , t} \left[ \| \epsilon_ {\theta} \left(\mathbf {x} ^ {t}, t\right) - \epsilon \| ^ {2} \right]. \tag {5}
$$
By suitably adjusting the denoising neural network to incorporate the conditioning input, this objective can be employed to train both unconditional and conditional models.
2.2 Diffusion Guidance
Classifier guidance repurposes unconditionally-trained image diffusion models for class-conditional image generation [10]. The key idea constitutes decomposing the class-conditional score function using the Bayes rule,
$$
\nabla_ {\mathbf {x} ^ {t}} \log p \left(\mathbf {x} ^ {t} \mid c\right) = \nabla_ {\mathbf {x} ^ {t}} \log p \left(\mathbf {x} ^ {t}\right) + \nabla_ {\mathbf {x} ^ {t}} \log p (c \mid \mathbf {x} ^ {t}), \tag {6}
$$
and employing an auxiliary classifier to estimate  $\nabla_{\mathbf{x}^t}\log p(c|\mathbf{x}^t)$ . Specifically, the following modified reverse diffusion process (Eq. 3) allows sampling from the class-conditional distribution,
$$
p _ {\theta} \left(\mathbf {x} ^ {t - 1} \mid \mathbf {x} ^ {t}, c\right) = \mathcal {N} \left(\mathbf {x} ^ {t - 1}; \mu_ {\theta} \left(\mathbf {x} ^ {t}, t\right) + s \sigma_ {t} ^ {2} \nabla_ {\mathbf {x} ^ {t}} \log p (c \mid \mathbf {x} ^ {t}), \sigma_ {t} ^ {2} \mathbf {I}\right), \tag {7}
$$
where  $s$  is a scale parameter controlling the strength of the guidance.
3 TSDiff: an Unconditional Diffusion Model for Time Series
In this section, we present our main contributions: TSDiff, an unconditional diffusion model designed for time series, along with two inference schemes that leverage the model for downstream forecasting tasks. We begin by outlining the problem setup and providing a concise overview of our network architecture. Subsequently, we delve into our first scheme — observation self-guidance — which enables conditioning reverse diffusion on arbitrary observed timesteps during inference. Secondly, we present a technique to iteratively refine predictions of arbitrary base forecasters by utilizing the implicit probability density learned by TSDiff as a prior.
Problem Statement. Let  $\mathbf{y} \in \mathbb{R}^L$  be a time series of length  $L$ . Denote obs  $\subset \{1, \dots, L\}$  as the set of observed timesteps and ta as its complement set of target timesteps. Our goal is to recover the complete time series  $\mathbf{y}$ , given the observed subsequence  $\mathbf{y}_{\mathrm{obs}}$  which may or may not be contiguous. Formally, this involves modeling the conditional distribution  $p_{\theta}(\mathbf{y}_{\mathrm{ta}} | \mathbf{y}_{\mathrm{obs}})$ . This general setup subsumes forecasting tasks, with or without missing values, as special cases. We seek to train a single unconditional generative model,  $p_{\theta}(\mathbf{y})$ , and condition it during inference to draw samples from arbitrary distributions of interest,  $p_{\theta}(\mathbf{y}_{\mathrm{ta}} | \mathbf{y}_{\mathrm{obs}})$ .
Generative Model Architecture. We begin with modeling the marginal probability,  $p_{\theta}(\mathbf{y})$ , via a diffusion model, referred to as TSDiff, parameterized by  $\theta$ . The architecture of TSDiff is depicted in Fig. 2 and is based on SSSD [1] which is a modification of DiffWave [28] employing S4 layers [18]. TSDiff is designed to handle univariate sequences of length  $L$ . To incorporate historical information beyond  $L$  timesteps without increasing  $L$ , we append lagged time series along the channel dimension. This results in a noisy input  $\mathbf{x}^t \in \mathbb{R}^{L \times C}$  (see Eq. 2) to the diffusion model, where  $C - 1$  is the number of lags. The S4 layers oper
Figure 2: An overview of observation self-guidance. The predicted noise,  $\pmb{\epsilon}_{\theta}(\mathbf{x}^t,t)$ , first denoises  $\mathbf{x}^t$  unconditionally as  $\hat{\mathbf{x}}^{t - 1}$  and approximates  $\mathbf{y}$  as  $\hat{\mathbf{y}}$ . The reverse diffusion step then guides  $\hat{\mathbf{x}}^{t - 1}$  via the log-likelihood of the observation  $\mathbf{y}_{\mathrm{obs}}$  under a distribution parameterized by  $\hat{\mathbf{y}}$ .
ate on the time dimension whereas the Conv1x1 layers operate on the channel dimension, facilitating information flow along both dimensions. As typical for unconditional diffusion models, the output dimensions of TSDiff match the input dimensions. Note that while we focus on univariate time series in this work, TSDiff can be modified to handle multivariate time series by incorporating additional layers, e.g., a transformer layer, operating across the feature dimensions after the S4 layer.
In the following, we discuss two approaches to condition the generative model,  $p_{\theta}(\mathbf{y})$ , during inference, enabling us to draw samples from  $p_{\theta}(\mathbf{y}_{\mathrm{ta}}|\mathbf{y}_{\mathrm{obs}})$ .
3.1 Observation Self-Guidance
Let  $t \geq 0$  be an arbitrary diffusion step. Applying Bayes' rule, we have,
$$
p _ {\theta} \left(\mathbf {x} ^ {t} \mid \mathbf {y} _ {\text {o b s}}\right) \propto p _ {\theta} \left(\mathbf {y} _ {\text {o b s}} \mid \mathbf {x} ^ {t}\right) p _ {\theta} \left(\mathbf {x} ^ {t}\right), \tag {8}
$$
which yields the following relation between the conditional and marginal score functions,
$$
\nabla_ {\mathbf {x} ^ {t}} \log p _ {\theta} (\mathbf {x} ^ {t} | \mathbf {y} _ {\text {o b s}}) = \nabla_ {\mathbf {x} ^ {t}} \log p _ {\theta} (\mathbf {y} _ {\text {o b s}} | \mathbf {x} ^ {t}) + \nabla_ {\mathbf {x} ^ {t}} \log p _ {\theta} (\mathbf {x} ^ {t}). \tag {9}
$$
Given access to the guidance distribution,  $p_{\theta}(\mathbf{y}_{\mathrm{obs}}|\mathbf{x}^t)$ , we can draw samples from  $p_{\theta}(\mathbf{y}_{\mathrm{ta}}|\mathbf{y}_{\mathrm{obs}})$  using guided reverse diffusion, akin to Eq. (7),
$$
p _ {\theta} \left(\mathbf {x} ^ {t - 1} \mid \mathbf {x} ^ {t}, \mathbf {y} _ {\text {o b s}}\right) = \mathcal {N} \left(\mathbf {x} ^ {t - 1}; \mu_ {\theta} \left(\mathbf {x} ^ {t}, t\right) + s \sigma_ {t} ^ {2} \nabla_ {\mathbf {x} ^ {t}} \log p _ {\theta} \left(\mathbf {y} _ {\text {o b s}} \mid \mathbf {x} ^ {t}\right), \sigma_ {t} ^ {2} \mathbf {I}\right). \tag {10}
$$
The scale parameter  $s$  controls how strongly the observations,  $\mathbf{y}_{\mathrm{obs}}$ , and the corresponding timesteps in the diffused time series,  $\mathbf{y}$ , align. Unlike Dhariwal and Nichol [10], we do not have access to auxiliary guidance networks. In the following, we propose two variants of a self-guidance mechanism that utilizes the same diffusion model to parameterize the guidance distribution. The main intuition behind self-guidance is that a model designed for complete sequences should reasonably approximate partial sequences. A pseudo-code of the observation self-guidance is given in App. A.2.
Mean Square Self-Guidance. We model  $p_{\theta}(\mathbf{y}_{\mathrm{obs}}|\mathbf{x}^t)$  as a multivariate Gaussian distribution,
$$
p _ {\theta} \left(\mathbf {y} _ {\text {o b s}} \mid \mathbf {x} ^ {t}\right) = \mathcal {N} \left(\mathbf {y} _ {\text {o b s}} \mid f _ {\theta} \left(\mathbf {x} ^ {t}, t\right), \mathbf {I}\right), \tag {11}
$$
where  $f_{\theta}$  is a function approximating  $\mathbf{y}$ , given the noisy time series  $\mathbf{x}^t$ . We can reuse the denoising network  $\epsilon_{\theta}$  to estimate  $\mathbf{y}$  as
$$
\hat {\mathbf {y}} = f _ {\theta} \left(\mathbf {x} ^ {t}, t\right) = \frac {\mathbf {x} ^ {t} - \sqrt {\left(1 - \bar {\alpha} _ {t}\right)} \boldsymbol {\epsilon} _ {\theta} \left(\mathbf {x} ^ {t} , t\right)}{\sqrt {\bar {\alpha} _ {t}}}, \tag {12}
$$
which follows by rearranging Eq. (2) with  $\epsilon = \epsilon_{\theta}(\mathbf{x}^t,t)$ , as shown by Song et al. [49]. This one-step denoising serves as a cost-effective approximation of the model for the observed time series and provides the requisite guidance term in the form of the score function,  $\nabla_{\mathbf{x}^t}\log p_\theta (\mathbf{y}_{\mathrm{obs}}|\mathbf{x}^t)$ , which can be computed by automatic differentiation. Consequently, our self-guidance approach requires no auxiliary networks or changes to the training procedure. Applying the logarithm to Eq. (11) and dropping constant terms yields the mean squared error (MSE) loss on the observed part of the time series, hence we named this technique mean square self-guidance.
Quantile Self-Guidance. Probabilistic forecasts are often evaluated using quantile-based metrics such as the continuous ranked probability score (CRPS) [15]. While the MSE only quantifies the average quadratic deviation from the mean, the CRPS takes all quantiles of the distribution into account by integrating the quantile loss (also known as the pinball loss) from 0 to 1. This motivated us to substitute the Gaussian distribution with the asymmetric Laplace distribution that has been studied in the context of Bayesian quantile regression [58]. The probability density function of the asymmetric Laplace distribution is given by,
$$
p _ {\theta} \left(\mathbf {y} _ {\text {o b s}} \mid \mathbf {x} ^ {t}\right) = \frac {1}{Z} \cdot \exp \left(- \frac {1}{b} \max  \left\{\kappa \cdot \left(\mathbf {y} _ {\text {o b s}} - f _ {\theta} \left(\mathbf {x} ^ {t}, t\right)\right), (\kappa - 1) \cdot \left(\mathbf {y} _ {\text {o b s}} - f _ {\theta} \left(\mathbf {x} ^ {t}, t\right)\right) \right\}\right), \tag {13}
$$
where  $Z$  is a normalization constant,  $b > 0$  a scale parameter, and  $\kappa \in (0,1)$  an asymmetry parameter. Setting  $b = 1$ , the log density yields the quantile loss with the score function,
$$
\nabla_ {\mathbf {x} ^ {t}} \log p _ {\theta} (\mathbf {y} _ {\mathrm {o b s}} | \mathbf {x} ^ {t}) = \nabla_ {\mathbf {x} ^ {t}} \max \{\kappa \cdot (\mathbf {y} _ {\mathrm {o b s}} - f _ {\theta} (\mathbf {x} ^ {t}, t)), (\kappa - 1) \cdot (\mathbf {y} _ {\mathrm {o b s}} - f _ {\theta} (\mathbf {x} ^ {t}, t)) \}, \qquad (1 4)
$$
with  $\kappa$  specifying the quantile level. By plugging Eq. (14) into Eq. (9), the reverse diffusion can be guided towards a specific quantile level  $\kappa$ . In practice, we use multiple evenly spaced quantile levels in  $(0,1)$ , based on the number of samples in the forecast. Intuitively, we expect quantile self-guidance to generate more diverse predictions by better representing the cumulative distribution function.
3.2 Prediction Refinement
In the previous section, we discussed a technique enabling the unconditional model to generate predictions by employing diffusion guidance. In this section, we will discuss another approach repurposing the model to refine predictions of base forecasters. Our approach is completely agnostic to the type of base forecaster and only assumes access to forecasts generated by them. The initial forecasts are iteratively refined using the implicit density learned by the diffusion model which serves as a prior. Unlike reverse diffusion which requires sequential sampling of all latent variables, refinement is performed directly in the data space. This provides a trade-off between quality and computational overhead, making it an economical alternative when the number of refinement iterations is less than the number of diffusion steps. Furthermore, in certain industrial forecasting scenarios, one has access to a complex production forecasting system of black-box nature. In these cases, refinement presents a cost-effective solution that enhances forecast accuracy post hoc, without modifying the core forecasting process — a change that could potentially be a lengthy procedure.
In the following, we present two interpretations of refinement as (a) sampling from an energy function, and (b) maximizing the likelihood to find the most likely sequence.
Energy-Based Sampling. Recall that our goal is to draw samples from the distribution  $p(\mathbf{y}_{\mathrm{ta}}|\mathbf{y}_{\mathrm{obs}})$ . Let  $g$  be an arbitrary base forecaster and  $g(\mathbf{y}_{\mathrm{obs}})$  be a sample forecast from  $g$  which serves as an initial guess of a sample from  $p(\mathbf{y}_{\mathrm{ta}}|\mathbf{y}_{\mathrm{obs}})$ . To improve this initial guess, we formulate refinement as the problem of sampling from the regularized energy-based model (EBM),
$$
E _ {\theta} (\mathbf {y}; \tilde {\mathbf {y}}) = - \log p _ {\theta} (\mathbf {y}) + \lambda \mathcal {R} (\mathbf {y}, \tilde {\mathbf {y}}), \tag {15}
$$
where  $\tilde{\mathbf{y}}$  is the time series obtained upon combining  $\mathbf{y}_{\mathrm{obs}}$  and  $g(\mathbf{y}_{\mathrm{obs}})$ , and  $\mathcal{R}$  is a regularizer such as the MSE loss or the quantile loss. We designed the energy function such that low energy is assigned to samples that are likely under the diffusion model,  $p_{\theta}(\mathbf{y})$ , and also close to  $\tilde{\mathbf{y}}$ , ensured by the diffusion log-likelihood and the regularizer in the energy function, respectively. The Lagrange
multiplier,  $\lambda$ , may be tuned to control the strength of regularization; however, we set it to 1 in our experiments for simplicity.
We use overdamped Langevin Monte Carlo (LMC) [50] to sample from this EBM.  $\mathbf{y}_{(0)}$  is initialized to  $\tilde{\mathbf{y}}$  and iteratively refined as,
$$
\mathbf {y} _ {(i + 1)} = \mathbf {y} _ {(i)} - \eta \nabla_ {\mathbf {y} _ {(i)}} E _ {\theta} \left(\mathbf {y} _ {(i)}; \tilde {\mathbf {y}}\right) + \sqrt {2 \eta \gamma} \xi_ {i} \quad \text {a n d} \quad \xi_ {i} \sim \mathcal {N} (\mathbf {0}, \mathbf {I}), \tag {16}
$$
where  $\eta$  and  $\gamma$  are the step size and noise scale, respectively.
Note that in contrast to observation self-guidance, we directly refine the time series in the data space and require an initial forecast from a base forecaster. However, similar to observation self-guidance, this approach does not require any modifications to the training procedure and can be applied to any trained diffusion model. A pseudo-code of the energy-based refinement is provided in App. A.2.
Maximizing the Likelihood. The decomposition in Eq. (15) can also be interpreted as a regularized optimization problem with the goal of finding the most likely time series that satisfies certain constraints on the observed timesteps. Concretely, it translates into,
$$
\underset {\mathbf {y}} {\arg \min } \left[ - \log p _ {\theta} (\mathbf {y}) + \lambda \mathcal {R} (\mathbf {y}, \tilde {\mathbf {y}}) \right], \tag {17}
$$
which can be optimized using gradient descent and is a special case of Eq. (16) with  $\gamma = 0$ . Given the non-convex nature of this objective, convergence to the global optimum is not guaranteed. Therefore, we expect the initial value,  $\mathbf{y}_{(0)}$ , to influence the resulting time series, which can also be observed in the experiment results (see Table 3).