PDETime: Rethinking Long-term Multivariate Time Series Forecasting from the Perspective of Partial Differential Equations

## Abstract

Abstract Recent advancements in deep learning have led to the development of various models for long-term multivariate time-series forecasting (LMTF), many of which have shown promising results. Generally, the focus has been on historical-value-based models, which rely on past observations to predict future series. Notably, a new trend has emerged with time-index-based models, offering a more nuanced understanding of the continuous dynamics underlying time series.
Unlike these two types of models that aggregate the information of spatial domains or temporal domains, in this paper, we consider multivariate time series as spatiotemporal data regularly sampled from a continuous dynamical system, which can be represented by partial differential equations (PDEs), with the spatial domain being fixed. Building on this perspective, we present PDETime,
a novel LMTF model inspired by the principles of Neural PDE solvers, following the encoding-integration-decoding operations. Our extensive experimentation across seven diverse real-world LMTF datasets reveals that PDETime not only adapts effectively to the intrinsic spatiotemporal nature of the data but also sets new benchmarks, achieving state-of-the-art results.

## 1 Introduction

Multivariate time series forecasting plays a pivotal role in diverse applications such as weather prediction (Angryk et al., ), energy consumption (Demirel et al., ), healthcare (Matsubara et al., ), and traffic flow estimation (Li et al., ).
In the field of multivariate time series forecasting, two typical approaches have emerged: historical-value-based (Zhou et al., ; Wu et al., ; Zeng et al., ; Nie et al., ), and time-index-based models (Woo et al., ; Naour et al., ).
Historical-value-based models predict future time steps utilizing historical observations, $\hat{x}_{t+1}=f_{\theta}(x_{t},x_{t-1},...)$, while time-index-based models only use the corresponding time-index features, $\hat{x}_{t+1}=f_{\theta}(t+1)$.
Historical-value-based models have been widely explored due to their simplicity and effectiveness, making them the state-of-the-art models in the field.
However, it is important to acknowledge that multivariate time series data are often discretely sampled from continuous dynamical systems. This characteristic poses a challenge for historical-value-based models, as they tend to capture spurious correlations limited to the sampling frequency (Gong et al., ; Woo et al., ).
Moreover, despite their widespread use and effectiveness, the progress in developing historical-value-based models has encountered bottlenecks (Li et al., ), making it challenging to further improve their performance. This limitation motivates the exploration of alternative approaches that can overcome the shortcomings of historical-value-based models and potentially deliver more robust forecasting models.

In recent years, deep time-index-based models have gradually been studied (Woo et al., ; Naour et al., ). These models intrinsically avoid the aforementioned problem of historical-value-based models, by mapping the time-index features to target predictions in continuous space through implicit neural representations (Tancik et al., ; Sitzmann et al., ) (INRs).
However, time-index-based models are only characterized by time-index coordinates, which makes them ineffective in modeling complex time series data and lacking in exploratory capabilities.

In this work, we propose a novel perspective by treating multivariate time series as spatiotemporal data discretely sampled from a continuous dynamical system, described by partial differential equations (PDEs) with fixed spatial domains as described in Eq 3.

From the PDE perspective, we observe that historical-value-based and time-index-based models can be considered as simplified versions of Eq 4, where either the temporal or spatial information is removed.

By examining Eq 4, we can interpret historical-value-based models as implicit inverse problems. These models infer spatial domain information, such as the position and physical properties of sensors, from historical observations. By utilizing the initial condition and spatial domain information, these models predict future sequences. Mathematically, historical-value-based models can be formulated as:

$$ $\displaystyle\hat{x}_{\tau}=x_{\tau_{0}}+\int_{\tau_{0}}^{\tau}f_{\theta}(x)d\tau$ (1) $$

where $x$ represents the spatial domain information.

Time-index-based models, on the other hand, can be seen as models that consider only the temporal domain. These models solely rely on time-index coordinates for prediction and do not explicitly incorporate spatial information. In this case, the prediction is formulated as:

$$ $\displaystyle\hat{x}_{\tau}=\int_{\tau_{0}}^{\tau}f_{\theta}(t_{\tau})d\tau$ (2) $$

Motivated by the limitations of existing approaches and inspired by neural PDE solvers, we introduce PDETime, a PDE-based model for LMTF. PDETime considers LMTF as an Initial Value Problem, explicitly incorporating both spatial and temporal domains and requiring the specification of an initial condition.

Specifically, PDETime first predicts the partial derivative term $E_{\theta}(x_{his},t_{\tau})=\alpha_{\tau}\approx\frac{\partial u(x,t_{\tau})}
{\partial t_{\tau}}$ utilizing an encoder in latent space. Unlike traditional PDE problems, the spatial domain information of LMT is unknown. Therefore, the encoder utilizes historical observations to estimate the spatial domain information. Subsequently, a numerical solver is employed to calculate the integral term $z_{\tau}=\int_{\tau_{0}}^{\tau}\alpha_{\tau}d\tau$. The proposed solver effectively mitigates the error issue and enhances the stability of the prediction results compared to traditional ODE solvers (Chen et al., ). Finally, PDETime employs a decoder to map the integral term from the latent space to the spatial domains and predict the results as $\hat{x}_{\tau}=x_{\tau_{0}}+D_{\phi}(z_{\tau})$. Similar to time-index-based models, PDETime also utilizes meta-optimization to enhance its ability to extrapolate across the forecast horizon. Additionally, PDETime can be simplified into a historical-value-based or time-index-based model by discarding either the time or spatial domains, respectively.

In short, we summarize the key contributions of this work:

- •
We propose a novel perspective for LMTF by considering it as spatiotemporal data regularly sampled from PDEs along the temporal domains.
- •
We introduce PDETime, a PDE-based model inspired by neural PDE solvers, which tackles LMTF as an Initial Value Problem of PDEs. PDETime incorporates encoding-integration-decoding operations and leverages meta-optimization to extrapolate future series.
- •
We extensively evaluate the proposed model on seven real-world benchmarks across multiple domains under the long-term setting. Our empirical studies demonstrate that PDETime consistently achieves state-of-the-art performance.

## 2 Related Work

Multivariate Time Series Forecasting.
With the progressive breakthrough made in deep learning, deep models have been proposed to tackle various time series forecasting applications. Depending on whether temporal or spatial is utilized, these models are classified into historical-value-based (Zhou et al., , ; Zeng et al., ; Nie et al., ; Zhang & Yan, ; Liu et al., , , ; Wu et al., ), and time-index-based models (Woo et al., ). Historical-value-based models, predicting target time steps utilizing historical observations, have been extensively developed and made significant progress in which a large body of work that tries to apply Transformer to forecast long-term series in recent years (Wen et al., ).
Early works like Informer (Zhou et al., ) and LongTrans (Li et al., ) were focused on designing novel mechanism to reduce the complexity of the original attention mechanism, thus capturing long-term dependency to achieve better performance. Afterwards, efforts were made to extract better temporal features to enhance the performance of the model (Wu et al., ; Zhou et al., ). Recent work (Zeng et al., ) has found that a single linear channel-independent model can outperform complex transformer-based models. Therefore, the very recent channel-independent models like PatchTST (Nie et al., ) and DLinear (Zeng et al., ) have become state-of-the-art.
In contrast, time-index-based models (Woo et al., ; Fons et al., ; Jiang et al., ; Naour et al., ) are a kind of coordinated-based models, mapping coordinates to values, which was represented by INRs. These models have received less attention and their performance still lags behind historical-value-based models.
PDETime, unlike previous works, considers multivariate time series as spatiotemporal data and approaches the prediction target sequences from the perspective of partial differential equations (PDEs).

Implicit Neural Representations. Implicit Neural Representations are the class of works representing signals as a continuous function parameterized by multi-layer perceptions (MLPs) (Tancik et al., ; Sitzmann et al., ) (instead of using the traditional discrete representation). These neural networks have been used to learn differentiable representations of various objects such as images (Henzler et al., ), shapes (Liu et al., , ), and textures (Oechsle et al., ). However, there is limited research on INRs for times series (Fons et al., ; Jiang et al., ; Woo et al., ; Naour et al., ; Jeong & Shin, ). And previous works mainly focused on time series generation and anomaly detection (Fons et al., ; Jeong & Shin, ).
DeepTime (Woo et al., ) is the work designed to learn a set of basis INR functions for forecasting, however, its performance is worse than historical-value-based models. In this work, we use INRs to represent spatial domains and temporal domains.

Neural PDE Solvers. Neural PDE solvers which are used for temporal PDEs, are laying the foundations of what is becoming both a rapidly growing and impactful area of research. These neural PDE solvers fall into two broad categories, neural operator methods and autoregressive methods. The neural operator methods (Kovachki et al., ; Li et al., ; Lu et al., ) treat the mapping from initial conditions to solutions as time $t$ as an input-output mapping learnable via supervised learning. For a given PDE and given initial conditions $u_{0}$, the neural operator $\mathcal{M}$ is trained to satisfy $\mathcal{M}(t,u_{0})=u(t)$. However, these methods are not designed to generalize to dynamics for out-of-distribution $t$. In contrast, the autoregressive methods (Bar-Sinai et al., ; Greenfeld et al., ; Hsieh et al., ; Yin et al., ; Brandstetter et al., ) solve the PDEs iteratively. The solution of autoregressive methods at time $t+\Delta t$ as $u(t+\Delta)=\mathcal{A}(u(t),\Delta t)$. In this work, We consider multivariate time series as data sampled from a continuous dynamical system according to a regular time discretization, which can be described by partial differential equations. For the given initial condition $x_{\tau_{0}}$, we use the numerous solvers (Euler solver) to simulate target time step $x_{\tau}$.

## 3 Method

Figure: Figure 1: The framework of proposed PDETime which consists of an Encoder $E_{\theta}$, a Solver, and a Decoder $D_{\phi}$. Given the initial condition $x_{\tau_{0}}$, PDETime first simulates $\frac{\partial u(x,t_{\tau})}{\partial t_{\tau}}$ at each time step $t_{\tau}$ using the Encoder $E_{\theta}(x_{his},t_{\tau},\tau)$; then uses the Solver to compute $\int_{\tau_{0}}^{\tau}\frac{\partial u(x,t_{\tau})}{\partial t_{\tau}}dt_{\tau}$, which is a numerical solver; finally, the Decoder maps integral term $z_{\tau}$ from latent space to special domains and predict the final results $\hat{x}_{\tau}=x_{\tau_{0}}+D_{\phi}(z_{\tau})$.
Refer to caption: x1.png

### 3.1 Problem Formulation

In contrast to previous works (Zhou et al., ; Woo et al., ), we regard multivariate time series as the spatiotemporal data regularly sampled from partial differential equations along the temporal domain, which has the following form:

$$ $\displaystyle\mathcal{F}(u,\frac{\partial u}{\partial t},\frac{\partial u}{ \partial x^{1}},...,\frac{\partial^{2}u}{\partial t^{2}},\frac{\partial^{2}u}{ \partial x^{2}},...)=0,$ $\displaystyle u(x,t):\Omega\times\mathcal{T}\rightarrow\mathcal{V},$ (3) $$

subject to initial and boundary conditions. Here $u$ is a spatio-temporal dependent and multi-dimensional continuous vector field, $\Omega\in\mathbb{R}^{C}$ and $\mathcal{T}\in\mathbb{R}$ denote the spatial and temporal domains, respectively. For multivariate time series data, we regard channel attributes as spatial domains (e.g., the position and physical properties of sensors).
The value of the spatial domains, $x$, is fixed but unknown (e.g., the position of sensors is fixed but unknown), while the value of the temporal domains is known.
We treat LMTF as the Initial Value Problems, where $u(x_{t},t)\in\mathbb{R}^{C}$ at any time $t$ is required to infer $u(x_{t^{\prime}},t^{\prime})\in\mathbb{R}^{C}$.

Consequently, we forecast the target series by the following formula:

$$ $u(x,t)=u(x,t_{0})+\int_{t_{0}}^{t}\frac{\partial u(x,\tau)}{\partial\tau}d\tau.$ (4) $$

DETime initiates its process with a single initial condition, denoted as $u(x,t_{0})$, and leverages neural networks to project the system’s dynamics forward in time. The procedure unfolds in three distinct steps. Firstly, PDETime generates a latent vector, $\alpha_{\tau}$ of a predefined dimension $d$, utilizing an encoder function, $E_{\theta}:\Omega\times\mathcal{T}\rightarrow\mathbb{R}^{d}$ (denoted as the ENC step). Subsequently, it employs an Euler solver, a numerical method, to approximate the integral term, $z_{t}=\int_{t_{0}}^{t}\alpha_{\tau}d\tau$, effectively capturing the system’s evolution over time (denoted as the SOL step). In the final step, PDETime translates the latent vectors,
$z_{t}$, back into the spatial domain using a decoder,
$D_{\phi}:\mathbb{R}^{d}\rightarrow\mathcal{V}$ to reconstruct the spatial function (denoted as the DEC step). This results in the following model, illustrated in Figure 1,

$$ $\displaystyle(ENC)\,\,\alpha_{\tau}$ $\displaystyle=E_{\theta}(x_{his},t_{\tau},\tau)$ (5) $\displaystyle(SOL)\,\,\,\,z_{\tau}$ $\displaystyle=\int_{\tau_{0}}^{\tau}\alpha_{\tau}d\tau$ (6) $\displaystyle(DEC)\,\,x_{\tau}$ $\displaystyle=D_{\phi}(z_{\tau}).$ (7) $$

We describe the details of the components in Section 3.2.

### 3.2 Components of PDETime

#### 3.2.1 Encoder: α τ = E θ ⁢ ( x h ⁢ i ⁢ s , t τ , τ ) subscript 𝛼 𝜏 subscript 𝐸 𝜃 subscript 𝑥 ℎ 𝑖 𝑠 subscript 𝑡 𝜏 𝜏 \alpha_{\tau}=E_{\theta}(x_{his},t_{\tau},\tau) italic_α start_POSTSUBSCRIPT italic_τ end_POSTSUBSCRIPT = italic_E start_POSTSUBSCRIPT italic_θ end_POSTSUBSCRIPT ( italic_x start_POSTSUBSCRIPT italic_h italic_i italic_s end_POSTSUBSCRIPT , italic_t start_POSTSUBSCRIPT italic_τ end_POSTSUBSCRIPT , italic_τ )

The Encoder computes the latent vector $\alpha_{\tau}$ representing $\frac{\partial u(x,t_{\tau})}{\partial t_{\tau}}$.
In contrast to previous works (Chen et al., ; Yin et al., ) that compute
$\frac{\partial u(x,t_{\tau})}{\partial t_{\tau}}$ using auto-differentiation powered by deep learning frameworks, which often suffers from poor stability due to uncontrolled temporal discretizations,
we propose directly simulating $\frac{\partial u(x,t_{\tau})}{\partial t_{\tau}}$ by a carefully designed neural networks $E_{\theta}(x_{his},t_{\tau},\tau)$.
The overall architecture of $E_{\theta}(x_{his},t_{\tau},\tau)$ is shown in Figure 2. As mentioned in Section 3, since the spatial domains of MTS data are fixed and unknown, we can only utilize the historical data $x_{his}\in\mathbb{R}^{H\times C}$ to assist the neural network in fitting the value of spatial domains $x$.
To represent the high frequency of $\tau$, $x_{his}$, and $t_{\tau}$, we use concatenated Fourier Features (CFF) (Woo et al., ; Tancik et al., ) and SIREN (Sitzmann et al., ) with $k$ layers.

$$ $\displaystyle\tau^{(i)}$ $\displaystyle=\text{GELU}(W_{\tau}^{(i-1)}\tau^{(i-1)}+b_{\tau}^{(i-1)})$ $\displaystyle x_{\tau}^{(i)}$ $\displaystyle=\sin(W_{x}^{(i-1)}x_{\tau}^{(i-1)}+b_{x}^{(i-1)})$ $\displaystyle t_{\tau}^{(i)}$ $\displaystyle=\sin(W_{t}^{(i-1)}t_{\tau}^{(i-1)}+b_{t}^{(i-1)}),\,\,\,i=1,...,k$ (8) $$

where $x_{\tau}^{(0)}\in\mathbb{R}^{H\times C}=x_{his}$, $t_{\tau}^{(0)}\in\mathbb{R}^{t}$ is the temporal feature, and $\tau\in\mathbb{R}$ is the index feature where $\tau_{i}=\frac{i}{H+L}$ for $i=0,1,...,H+L$, $L$ and $H$ are the look-back and horizon length, respectively. We employ CFF to represent $\tau^{(0)}$, i.e. $\tau^{(0)}=[\sin(2\pi\textbf{B}_{1}\tau),\cos(2\pi\textbf{B}_{1}\tau),...,\sin
(2\pi\textbf{B}_{s}\tau),\cos(2\pi\textbf{B}_{s}\tau)]$ where $\textbf{B}_{s}\in\mathbb{R}^{\frac{d}{2}\times c}$ is sampled from $\mathcal{N}(0,2^{s})$.

After representing $\tau\in\mathbb{R}^{d}$, $t_{\tau}\in\mathbb{R}^{t}$, and $x_{\tau}\in\mathbb{R}^{d\times C}$ with INRs, the Encoder aggregates $x_{\tau}$ and $t_{\tau}$ using $\tau$, formulated as $f_{\theta}(\tau,t_{\tau},x_{\tau})$:

$$ $\displaystyle\tau$ $\displaystyle=\tau+\text{Attn}(\tau,x_{\tau})$ $\displaystyle\tau$ $\displaystyle=\tau+\text{Linear}([\tau;t_{\tau}])$ (9) $$

where $[\cdot;\cdot]$ is is the row-wise stacking operation. In this work, we utilize attention mechanism (Vaswani et al., ) and linear mapping to aggregate the information of spatial domains and temporal domains, respectively. However, various networks can be designed to aggregate these two types of information.

Unlike previous works (Chen et al., ; Yin et al., ), we do not directly make $\alpha_{\tau}=\frac{\partial u(x,t_{\tau})}{\partial t_{\tau}}$, but rather make $\int_{\tau_{0}}^{\tau}\alpha_{\tau}d\tau=\int_{\tau_{0}}^{\tau}\frac{\partial u
(x,t_{\tau})}{\partial t_{\tau}}d\tau$ in latent space, without the need for autoregressive calculation of $\alpha_{t}$ which can effectively alleviate the error accumulation problem and make the prediction results more stable (Brandstetter et al., ).

The pseudocode of the Encoder of PDETime is summarized in Appendix A.3.

Figure: Figure 2: The architecture of the Encoder which is used to simulate $\frac{\partial u(x,t_{\tau})}{\partial t_{\tau}}$. In this work, we use INRs to represent $x_{his}$, $t_{\tau}$, and $\tau$, and then use the Attention mechanism and linear mapping to aggregate the information of spatial domains and temporal domains.
Refer to caption: x2.png

#### 3.2.2 Solver: z t = ∫ τ 0 τ α τ subscript 𝑧 𝑡 superscript subscript subscript 𝜏 0 𝜏 subscript 𝛼 𝜏 z_{t}=\int_{\tau_{0}}^{\tau}\alpha_{\tau} italic_z start_POSTSUBSCRIPT italic_t end_POSTSUBSCRIPT = ∫ start_POSTSUBSCRIPT italic_τ start_POSTSUBSCRIPT 0 end_POSTSUBSCRIPT end_POSTSUBSCRIPT start_POSTSUPERSCRIPT italic_τ end_POSTSUPERSCRIPT italic_α start_POSTSUBSCRIPT italic_τ end_POSTSUBSCRIPT

The Solver introduces a numerical solver (Euler Solver) to compute the integral term $z_{\tau}=\int_{\tau_{0}}^{\tau}\alpha_{\tau}d\tau$, which can be approximated as:

$$ $\displaystyle z_{\tau}$ $\displaystyle=\int_{\tau_{0}}^{\tau}\frac{\partial u(x_{\tau},t_{\tau})}{ \partial t_{\tau}}d\tau\approx\sum_{\tau=\tau_{0}}^{\tau}\frac{\partial u(x_{ \tau},t_{\tau})}{\partial t_{\tau}}\ast\Delta\tau$ $\displaystyle\approx\sum_{\tau=\tau_{0}}^{\tau}E_{\theta}(x_{his},t_{\tau}, \tau)\ast\Delta\tau$ (10) $$

where we set $\Delta\tau=1$ for convenience. However, as shown in Figure 3 (a), directly compute $z_{\tau}=\sum_{\tau=\tau_{0}}^{\tau}E_{\theta}(x_{his},t_{\tau},\tau)\ast
\Delta\tau=\sum_{\tau=\tau_{0}}^{\tau}\alpha_{\tau}\ast\Delta\tau$ through Eq 10 can easily lead to error accumulation and gradient problems (Rubanova et al., ; Wu et al., ; Brandstetter et al., ). To address these issues, we propose a modified approach, as illustrated in Figure 3 (b). We divide the sequence into non-overlapping patches of length $S$, where $\frac{H+L}{S}$ patches are obtained. For $\tau(H+L)\mod S=0$, we directly estimate the integral term as $z_{\tau}=f_{\psi}(x_{his},t_{\tau},\tau)$ using a neural network $f_{\psi}$. Otherwise, we use the numerical solver to estimate the integral term with the integration lower limit $\lfloor\frac{\tau\cdot(H+L)}{S}\rfloor\cdot S$. This modification results in the following formula for the numerical solver:

$$ $\displaystyle z_{t}$ $\displaystyle=f_{\psi}(x_{\tau^{\prime}},t_{\tau^{\prime}},\tau^{\prime})+\int _{\tau^{\prime}}^{\tau}f_{\theta}(x_{\tau},t_{\tau},\tau)d\tau,$ $\displaystyle\tau^{\prime}$ $\displaystyle=\lfloor\frac{\tau*(H+L)}{s}\rfloor\ast S$ (11) $$

Furthermore, as shown in Figure 3 (b), Eq 11 breaks the continuity. To address this, we introduce an additional objective function to ensure continuity as much as possible:

$$ $\displaystyle\mathcal{L}_{c}=\mathcal{L}(f_{\psi}(x_{\tau},t_{\tau},\tau),f_{ \psi}(x_{\tau^{\prime}},t_{\tau^{\prime}},\tau^{\prime})+\int_{\tau^{\prime}}^ {\tau}f_{\theta}(x_{\tau},t_{\tau},\tau)d\tau)$ $\displaystyle\mathrm{s.t.}\;\tau\ast(H+L)\mod S=0,\;\tau^{\prime}=\frac{\tau \ast(L+H)-S}{(H+L)}.$ (12) $$

The experimental results have shown that incorporating $\mathcal{L}_{c}$ improves the stability of PDETime.

The pseudocode of the Solver of PDETime is summarized in Appendix A.4.

Figure: Figure 3: The framework of standard and variant of numerical solver proposed in our work.
Refer to caption: x3.png

#### 3.2.3 Decoder: x τ = D ϕ ⁢ ( z τ ) + x τ 0 subscript 𝑥 𝜏 subscript 𝐷 italic-ϕ subscript 𝑧 𝜏 subscript 𝑥 subscript 𝜏 0 x_{\tau}=D_{\phi}(z_{\tau})+x_{\tau_{0}} italic_x start_POSTSUBSCRIPT italic_τ end_POSTSUBSCRIPT = italic_D start_POSTSUBSCRIPT italic_ϕ end_POSTSUBSCRIPT ( italic_z start_POSTSUBSCRIPT italic_τ end_POSTSUBSCRIPT ) + italic_x start_POSTSUBSCRIPT italic_τ start_POSTSUBSCRIPT 0 end_POSTSUBSCRIPT end_POSTSUBSCRIPT

After estimating the integral term in latent space, we utilize a Decoder $D_{\phi}$ to decode $z_{\tau}$ into the spatial function. To tackle the distribution shift problem, we introduce a meta-optimization framework (Woo et al., ; Bertinetto et al., ) to enhance the extrapolation capability of PDETime. Specifically, we split the time series into look-back window $\textbf{X}_{t-L:t}=[x_{t-L},...,x_{t}]\in\mathbb{R}^{L\times C}$ and horizon window $\textbf{Y}_{t+1:t+H}=[x_{t+1},...,x_{t+H}]\in\mathbb{R}^{H\times C}$ pairs. We then use the parameters $\phi$ and $\theta$ to adapt the look-back window and horizon window through a bi-level problem:

$$ $\displaystyle\theta^{\ast}=\arg\min_{\theta}\sum_{\tau=\tau_{t+1}}^{\tau_{t+H} }\mathcal{L}_{p}(D_{\phi}(\int_{\tau_{0}}^{\tau}E_{\theta}(x_{his},t_{\tau}, \tau)),x_{\tau})$ $\displaystyle\phi^{\ast}=\arg\min_{\phi}\sum_{\tau=\tau_{t-L}}^{\tau_{t}} \mathcal{L}_{r}(D_{\phi}(\int_{\tau_{0}}^{\tau}E_{\theta}(x_{his},t_{\tau}, \tau)),x_{\tau})$ (13) $$

where $\mathcal{L}_{r}$ and $\mathcal{L}_{p}$ denote the reconstruction and prediction loss, respectively (which will be described in detail in Section 3.3). During training, PDETime optimizes both $\theta$ and $\phi$; while during testing, it only optimizes $\phi$ to enhance the extrapolation. To ensure speed and efficiency, we employ a single ridge regression for $D_{\phi}$ (Bertinetto et al., ).

### 3.3 Optimization

Different from previous works (Zhou et al., , ; Wu et al., ; Woo et al., ) that directly predict the future time steps with the objective $\mathcal{L}_{p}=\frac{1}{H}\mathcal{L}(x_{\tau},\hat{x}_{\tau})$. As described as Eq 4, we consider the LMTF as the Initial Value Problem in PDE, and the initial condition $x_{\tau_{0}}$is known (we use the latest time step in the historical series as the initial condition). Thus the objective of PDETime is to estimate the integral term $\int_{\tau_{0}}^{\tau}\frac{\partial u(x,t_{\tau})}{\partial t_{\tau}}d\tau$. To achieve this, we define the prediction objective as:

$$ $\displaystyle\mathcal{L}_{p}=\frac{1}{H}\mathcal{L}(D_{\phi}(z_{\tau}),x_{\tau }-x_{\tau_{0}})$ (14) $$

where $z_{\tau}=\int_{\tau_{0}}^{\tau}E_{\theta}(x_{his},t_{\tau},\tau)d\tau$ as described in Section 3.2.1. Additionally, $\mathcal{L}_{r}$ has the same form $\mathcal{L}_{r}=\frac{1}{L}(D_{\phi}(z_{\tau}),x_{\tau}-x_{\tau_{0}})$.

In addition to the prediction objective, we also aim to ensure that the first-order difference of the predicted sequence matches that of the target sequence. Therefore, we introduce the optimization objective:

$$ $\displaystyle\mathcal{L}_{f}=\frac{1}{H}\sum_{\tau=\tau_{t+1}}^{\tau_{t+H}} \mathcal{L}(D_{\phi}(z_{\tau})-D_{\phi}(z_{\tau-1}),x_{\tau}-x_{\tau-1})$ (15) $$

Combining $\mathcal{L}_{p}$, $\mathcal{L}_{c}$, and $\mathcal{L}_{f}$, the training objective is:

$$ $\displaystyle\mathcal{L}_{p}=\mathcal{L}_{p}+\mathcal{L}_{c}+\mathcal{L}_{f}.$ (16) $$

## 4 Experiments

### 4.1 Experimental Settings

Datasets. We extensively include 7 real-world datasets in our experiments, including four ETT datasets (ETTh1, ETTh2, ETTm1, ETTm2) (Zhou et al., ). Electricity, Weather and Traffic (Wu et al., ), covering energy, transportation and weather domains (See Appendix A.1.1 for more details on the datasets). To ensure a fair evaluation, we follow the standard protocol of dividing each dataset into the training, validation and testing subsets according to the chronological order. The split ratio is 6:2:2 for the ETT dataset and 7:1:2 for the others (Zhou et al., ; Wu et al., ). We set the length of the lookback series as 512 for PatchTST, 336 for DLinear, and 96 for other historical-value-based models. The experimental settings of DeepTime remain consistent with the original settings (Woo et al., ).
The prediction length varies in $\{96,192,336,720\}$.

Comparison methods. We carefully choose 10 well-acknowledged models (9 historical-value-based models and 1 time-index-based model) as our benchmarks, including (1) Transformer-based models: Autoformer (Wu et al., ), FEDformer (Zhou et al., ), Stationary (Liu et al., ), Crossformer (Zhang & Yan, ), PatchTST (Nie et al., ), and iTransformer (Liu et al., ); (2) Linear-based models: DLinear (Zeng et al., ) ; (3) CNN-based models: SCINet (Liu et al., ), TimesNet (Wu et al., ); (4) Time-index-based model: DeepTime (Woo et al., ). (See Appendix A.1.2 for details of these baselines)

Implementation Details. Our method is trained with the Smooth L1 (Girshick, ) loss using the ADAM (Kingma & Ba, ) with the initial learning rate selected from $\{10^{-4},10^{-3},5\times 10^{-3}\}$. Batch size is set to 32. All experiments are repeated 3 times with fixed seed 2024, implemented in Pytorch (Paszke et al., ) and conducted on a single NVIDIA RTX 3090 GPUs. Following DeepTime (Woo et al., ), we set the look-back length as $L=\mu\ast H$, where $\mu$ is a multiplier which decides the length of the look-back windows. We search through the values $\mu=[1,3,5,7,9]$, and select the best value based on the validation loss.
We set layers of INRs $k=5$ by default, and select the best results from $N=\{1,2,3,5\}$. We summarize the temporal features used in this work in Appendix A.2.

### 4.2 Main Results and Ablation Study

**Table 1: Full results of the long-term forecasting task. We compare extensive competitive models under different prediction lengths following the setting of PatchTST (). The input sequence length is set to 336 and 512 for DLinear and PatchTST, and 96 for other historical-value-based baselines. *Avg* means the average results from all four prediction lengths.**
| Models | PDETime | iTransformer | PatchTST | Crossformer | DeepTime | TimesNet | DLinear | SCINet | FEDformer | Stationary | Autoformer |  |  |  |  |  |  |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| (Ours) | () | () | () | () | () | () | () | () | () | () |  |  |  |  |  |  |  |  |  |  |  |  |  |
| Metric | MSE | MAE | MSE | MAE | MSE | MAE | MSE | MAE | MSE | MAE | MSE | MAE | MSE | MAE | MSE | MAE | MSE | MAE | MSE | MAE | MSE | MAE |  |
| ETTm1 | 96 | 0.292 | 0.335 | 0.334 | 0.368 | 0.293 | 0.346 | 0.404 | 0.426 | 0.305 | 0.347 | 0.338 | 0.375 | 0.299 | 0.343 | 0.418 | 0.438 | 0.379 | 0.419 | 0.386 | 0.398 | 0.505 | 0.475 |
| 192 | 0.329 | 0.359 | 0.377 | 0.391 | 0.333 | 0.370 | 0.450 | 0.451 | 0.340 | 0.371 | 0.374 | 0.387 | 0.335 | 0.365 | 0.439 | 0.450 | 0.426 | 0.441 | 0.459 | 0.444 | 0.553 | 0.496 |  |
| 336 | 0.346 | 0.374 | 0.426 | 0.420 | 0.369 | 0.392 | 0.532 | 0.515 | 0.362 | 0.387 | 0.410 | 0.411 | 0.369 | 0.386 | 0.490 | 0.485 | 0.445 | 0.459 | 0.495 | 0.464 | 0.621 | 0.537 |  |
| 720 | 0.395 | 0.404 | 0.491 | 0.459 | 0.416 | 0.420 | 0.666 | 0.589 | 0.399 | 0.414 | 0.478 | 0.450 | 0.425 | 0.421 | 0.595 | 0.550 | 0.543 | 0.490 | 0.585 | 0.516 | 0.671 | 0.561 |  |
| Avg | 0.340 | 0.368 | 0.407 | 0.410 | 0.352 | 0.382 | 0.513 | 0.496 | 0.351 | 0.379 | 0.400 | 0.406 | 0.357 | 0.378 | 0.485 | 0.481 | 0.448 | 0.452 | 0.481 | 0.456 | 0.588 | 0.517 |  |
| ETTm2 | 96 | 0.158 | 0.244 | 0.180 | 0.264 | 0.166 | 0.256 | 0.287 | 0.366 | 0.166 | 0.257 | 0.187 | 0.267 | 0.167 | 0.260 | 0.286 | 0.377 | 0.203 | 0.287 | 0.192 | 0.274 | 0.255 | 0.339 |
| 192 | 0.213 | 0.283 | 0.250 | 0.309 | 0.223 | 0.296 | 0.414 | 0.492 | 0.225 | 0.302 | 0.249 | 0.309 | 0.224 | 0.303 | 0.399 | 0.445 | 0.269 | 0.328 | 0.280 | 0.339 | 0.281 | 0.340 |  |
| 336 | 0.262 | 0.318 | 0.311 | 0.348 | 0.274 | 0.329 | 0.597 | 0.542 | 0.277 | 0.336 | 0.321 | 0.351 | 0.281 | 0.342 | 0.637 | 0.591 | 0.325 | 0.366 | 0.334 | 0.361 | 0.339 | 0.372 |  |
| 720 | 0.334 | 0.336 | 0.412 | 0.407 | 0.362 | 0.385 | 1.730 | 1.042 | 0.383 | 0.409 | 0.408 | 0.403 | 0.397 | 0.421 | 0.960 | 0.735 | 0.421 | 0.415 | 0.417 | 0.413 | 0.433 | 0.432 |  |
| Avg | 0.241 | 0.295 | 0.288 | 0.332 | 0.256 | 0.316 | 0.757 | 0.610 | 0.262 | 0.326 | 0.291 | 0.333 | 0.267 | 0.331 | 0.571 | 0.537 | 0.305 | 0.349 | 0.306 | 0.347 | 0.327 | 0.371 |  |
| ETTh1 | 96 | 0.356 | 0.381 | 0.386 | 0.405 | 0.379 | 0.401 | 0.423 | 0.448 | 0.371 | 0.396 | 0.384 | 0.402 | 0.375 | 0.399 | 0.654 | 0.599 | 0.376 | 0.419 | 0.513 | 0.491 | 0.449 | 0.459 |
| 192 | 0.397 | 0.406 | 0.441 | 0.436 | 0.413 | 0.429 | 0.471 | 0.474 | 0.403 | 0.420 | 0.436 | 0.429 | 0.405 | 0.416 | 0.719 | 0.631 | 0.420 | 0.448 | 0.534 | 0.504 | 0.500 | 0.482 |  |
| 336 | 0.420 | 0.419 | 0.487 | 0.458 | 0.435 | 0.436 | 0.570 | 0.546 | 0.433 | 0.436 | 0.491 | 0.469 | 0.439 | 0.443 | 0.778 | 0.659 | 0.459 | 0.465 | 0.588 | 0.535 | 0.521 | 0.496 |  |
| 720 | 0.425 | 0.446 | 0.503 | 0.491 | 0.446 | 0.464 | 0.653 | 0.621 | 0.474 | 0.492 | 0.521 | 0.500 | 0.472 | 0.490 | 0.836 | 0.699 | 0.506 | 0.507 | 0.643 | 0.616 | 0.514 | 0.512 |  |
| Avg | 0.399 | 0.413 | 0.454 | 0.447 | 0.418 | 0.432 | 0.529 | 0.522 | 0.420 | 0.436 | 0.458 | 0.450 | 0.423 | 0.437 | 0.747 | 0.647 | 0.440 | 0.460 | 0.570 | 0.537 | 0.496 | 0.487 |  |
| ETTh2 | 96 | 0.268 | 0.330 | 0.297 | 0.349 | 0.274 | 0.335 | 0.745 | 0.584 | 0.287 | 0.352 | 0.340 | 0.374 | 0.289 | 0.353 | 0.707 | 0.621 | 0.358 | 0.397 | 0.476 | 0.458 | 0.346 | 0.388 |
| 192 | 0.331 | 0.370 | 0.380 | 0.400 | 0.342 | 0.382 | 0.877 | 0.656 | 0.383 | 0.412 | 0.402 | 0.414 | 0.383 | 0.418 | 0.860 | 0.689 | 0.429 | 0.439 | 0.512 | 0.493 | 0.456 | 0.452 |  |
| 336 | 0.358 | 0.395 | 0.428 | 0.432 | 0.365 | 0.404 | 1.043 | 0.731 | 0.523 | 0.501 | 0.452 | 0.452 | 0.448 | 0.465 | 1.000 | 0.744 | 0.496 | 0.487 | 0.552 | 0.551 | 0.482 | 0.486 |  |
| 720 | 0.380 | 0.421 | 0.427 | 0.445 | 0.393 | 0.430 | 1.104 | 0.763 | 0.765 | 0.624 | 0.462 | 0.468 | 0.605 | 0.551 | 1.249 | 0.838 | 0.463 | 0.474 | 0.562 | 0.560 | 0.515 | 0.511 |  |
| Avg | 0.334 | 0.379 | 0.383 | 0.407 | 0.343 | 0.387 | 0.942 | 0.684 | 0.489 | 0.472 | 0.414 | 0.427 | 0.431 | 0.446 | 0.954 | 0.723 | 0.437 | 0.449 | 0.526 | 0.516 | 0.450 | 0.459 |  |
| ECL | 96 | 0.129 | 0.222 | 0.148 | 0.240 | 0.129 | 0.222 | 0.219 | 0.314 | 0.137 | 0.238 | 0.168 | 0.272 | 0.140 | 0.237 | 0.247 | 0.345 | 0.193 | 0.308 | 0.169 | 0.273 | 0.201 | 0.317 |
| 192 | 0.143 | 0.235 | 0.162 | 0.253 | 0.147 | 0.240 | 0.231 | 0.322 | 0.152 | 0.252 | 0.184 | 0.289 | 0.153 | 0.249 | 0.257 | 0.355 | 0.201 | 0.315 | 0.182 | 0.286 | 0.222 | 0.334 |  |
| 336 | 0.152 | 0.248 | 0.178 | 0.269 | 0.163 | 0.259 | 0.246 | 0.337 | 0.166 | 0.268 | 0.198 | 0.300 | 0.169 | 0.267 | 0.269 | 0.369 | 0.214 | 0.329 | 0.200 | 0.304 | 0.231 | 0.338 |  |
| 720 | 0.176 | 0.272 | 0.225 | 0.317 | 0.197 | 0.290 | 0.280 | 0.363 | 0.201 | 0.302 | 0.220 | 0.320 | 0.203 | 0.301 | 0.299 | 0.390 | 0.246 | 0.355 | 0.222 | 0.321 | 0.254 | 0.361 |  |
| Avg | 0.150 | 0.244 | 0.178 | 0.270 | 0.159 | 0.252 | 0.244 | 0.334 | 0.164 | 0.265 | 0.192 | 0.295 | 0.166 | 0.263 | 0.268 | 0.365 | 0.214 | 0.327 | 0.193 | 0.296 | 0.227 | 0.338 |  |
| Traffic | 96 | 0.330 | 0.232 | 0.395 | 0.268 | 0.360 | 0.249 | 0.522 | 0.290 | 0.390 | 0.275 | 0.593 | 0.321 | 0.410 | 0.282 | 0.788 | 0.499 | 0.587 | 0.366 | 0.612 | 0.338 | 0.613 | 0.388 |
| 192 | 0.332 | 0.232 | 0.417 | 0.276 | 0.379 | 0.256 | 0.530 | 0.293 | 0.402 | 0.278 | 0.617 | 0.336 | 0.423 | 0.287 | 0.789 | 0.505 | 0.604 | 0.373 | 0.613 | 0.340 | 0.616 | 0.382 |  |
| 336 | 0.342 | 0.236 | 0.433 | 0.283 | 0.392 | 0.264 | 0.558 | 0.305 | 0.415 | 0.288 | 0.629 | 0.336 | 0.436 | 0.296 | 0.797 | 0.508 | 0.621 | 0.383 | 0.618 | 0.328 | 0.622 | 0.337 |  |
| 720 | 0.365 | 0.244 | 0.467 | 0.302 | 0.432 | 0.286 | 0.589 | 0.328 | 0.449 | 0.307 | 0.640 | 0.350 | 0.466 | 0.315 | 0.841 | 0.523 | 0.626 | 0.382 | 0.653 | 0.355 | 0.660 | 0.408 |  |
| Avg | 0.342 | 0.236 | 0.428 | 0.282 | 0.390 | 0.263 | 0.550 | 0.304 | 0.414 | 0.287 | 0.620 | 0.336 | 0.433 | 0.295 | 0.804 | 0.509 | 0.610 | 0.376 | 0.624 | 0.340 | 0.628 | 0.379 |  |
| Weather | 96 | 0.157 | 0.203 | 0.174 | 0.214 | 0.149 | 0.198 | 0.158 | 0.230 | 0.166 | 0.221 | 0.172 | 0.220 | 0.176 | 0.237 | 0.221 | 0.306 | 0.217 | 0.296 | 0.173 | 0.223 | 0.266 | 0.336 |
| 192 | 0.200 | 0.246 | 0.221 | 0.254 | 0.194 | 0.241 | 0.206 | 0.277 | 0.207 | 0.261 | 0.219 | 0.261 | 0.220 | 0.282 | 0.261 | 0.340 | 0.276 | 0.336 | 0.245 | 0.285 | 0.307 | 0.367 |  |
| 336 | 0.241 | 0.281 | 0.278 | 0.296 | 0.245 | 0.282 | 0.272 | 0.335 | 0.251 | 0.298 | 0.280 | 0.306 | 0.265 | 0.319 | 0.309 | 0.378 | 0.339 | 0.380 | 0.321 | 0.338 | 0.359 | 0.395 |  |
| 720 | 0.291 | 0.324 | 0.358 | 0.349 | 0.314 | 0.334 | 0.398 | 0.418 | 0.301 | 0.338 | 0.365 | 0.359 | 0.323 | 0.362 | 0.377 | 0.427 | 0.403 | 0.428 | 0.414 | 0.410 | 0.419 | 0.428 |  |
| Avg | 0.222 | 0.263 | 0.258 | 0.279 | 0.225 | 0.263 | 0.259 | 0.315 | 0.231 | 0.286 | 0.259 | 0.287 | 0.246 | 0.300 | 0.292 | 0.363 | 0.309 | 0.360 | 0.288 | 0.314 | 0.338 | 0.382 |  |
| $1^{\text{st}}$ Count | 33 | 33 | 0 | 0 | 2 | 2 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |  |

Comprehensive forecasting results are listed in Table 1 with the best in Bold and the second underlined. The lower MSE/MAE indicates the more accurate prediction result. Overall, PDETime achieves the best performance on most settings across seven real-world datasets compared with historical-value-based and time-index-based models. Additionally, experimental results also show that the performance of the proposed PDETime changes quite steadily as the prediction length $H$ increases. For instance, the MSE of PDETime increases from 0.330 to 0.365 on the Traffic dataset, while the MSE of PatchTST increases from 0.360 to 0.432, which is the SOTA historical-value-based model. This phenomenon was observed in other datasets and settings as well, indicating that PDETime retains better long-term robustness, which is meaningful for real-world practical applications.

**Table 2: Ablation study on variants of PDETime. -Temporal refers that removing the temporal domain feature $t_{\tau}$; -Spatial refers that removing the historical observations $x_{his}$; - Initial refers that removing the initial condition $x_{\tau_{0}}$. The best results are highlighted in bold.**
| Dataset | Models | PDETime | -Temporal | -Spatial | -Initial | -Temporal -Spatial | - All |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Metric | MSE | MAE | MSE | MAE | MSE | MAE | MSE | MAE | MSE | MAE | MSE | MAE |  |
| Traffic | 96 | 0.330 | 0.232 | 0.336 | 0.236 | 0.329 | 0.232 | 0.334 | 0.235 | 0.394 | 0.268 | 0.401 | 0.269 |
| 192 | 0.332 | 0.232 | 0.368 | 0.247 | 0.336 | 0.234 | 0.334 | 0.232 | 0.407 | 0.269 | 0.413 | 0.270 |  |
| 336 | 0.342 | 0.236 | 0.378 | 0.251 | 0.344 | 0.236 | 0.343 | 0.236 | 0.419 | 0.273 | 0.426 | 0.272 |  |
| 720 | 0.365 | 0.244 | 0.406 | 0.265 | 0.371 | 0.250 | 0.368 | 0.250 | 0.453 | 0.291 | 0.671 | 0.406 |  |
| Weather | 96 | 0.157 | 0.203 | 0.158 | 0.205 | 0.159 | 0.205 | 0.169 | 0.213 | 0.159 | 0.205 | 0.166 | 0.212 |
| 192 | 0.200 | 0.246 | 0.206 | 0.253 | 0.198 | 0.243 | 0.208 | 0.248 | 0.198 | 0.243 | 0.208 | 0.250 |  |
| 336 | 0.241 | 0.281 | 0.240 | 0.278 | 0.246 | 0.282 | 0.245 | 0.287 | 0.240 | 0.277 | 0.244 | 0.283 |  |
| 720 | 0.291 | 0.324 | 0.292 | 0.323 | 0.290 | 0.322 | 0.300 | 0.337 | 0.294 | 0.327 | 0.299 | 0.337 |  |

We perform ablation studies on the Traffic and Weather datasets to validate the effect of temporal feature $t_{\tau}$, spatial feature $x_{his}$ and initial condition $x_{\tau_{0}}$. The results are presented in Table 2.
1) The initial condition $x_{t_{0}}$ is useful on most settings. As mentioned in Section 3, we treat LMTF as Initial Value Problem, thus the effectiveness of $x_{\tau_{0}}$ validates the correctness of PDETime.
2) The impact of spatial features $x_{his}$ on PDETime is limited. This may be due to the fact that the true spatial domains $x$ are unknown, and we utilize the historical observations $x_{his}$ to simulate $x$. However, this approach may not be effective in capturing the true spatial features.
3) The influence of temporal feature $t_{\tau}$ on PDETime various significantly across different datasets.
Experimental results have shown that $t_{\tau}$ is highly beneficial in the Traffic dataset, but its effect on Weather dataset is limited.
For example, the period of Traffic dataset may be one day or one week, making it easier for PDETime to learn temporal features. On the other hand, the period of Weather dataset may be one year or longer, but the dataset only contains one year of data. As a result, PDETime cannot capture the complete temporal features in this case.

We conduct an additional ablation study on Traffic to evaluate the ability of different INRs to extract features of $x_{his}$, $t_{\tau}$, and $\tau$. In this study, we compared the performance of using the GELU or Tanh activation function instead of sine in SIREN and making $\tau^{(0)}=[\text{GELU}(2\pi\textbf{B}_{1}\tau),\text{GELU}(2\pi\textbf{B}_{1}
\tau),...]$ or $\tau^{(0)}=[\text{Tanh}(2\pi\textbf{B}_{1}\tau),\text{Tanh}(2\pi\textbf{B}_{1}
\tau),...]$.
Table 4 presents the experimental results, we observe that the sine function (periodic functions) can extract features better than other non-decreasing activation functions. This is because the smooth, non-periodic activation functions fail to accurately model high-frequency information (Sitzmann et al., ). Time series data is often periodic, and the periodic nature of the sine function makes it more effective in extracting time series features.

### 4.3 Effects of Hyper-parameters

Figure: Figure 4: Evaluation on hyper-parameter impact. (a) MSE against hyper-parameter layers of INRs $k$ in Forecaster on ETTh1. (b) MSE against hyper-parameter layers of aggregation module $N$ in Forecaster on ETTh1. (c) MSE against hyper-parameter patch length $S$ in Estimator on ETTh1.
Refer to caption: x4.png

We evaluate the effect of four hyper-parameters: look-back window $L$, number of INRs layers $k$, number of aggregation layers $N$, and patch length $S$ on the ETTh1 and ETTh2 datasets. First, we perform a sensitivity on the look-back window $L=\mu\ast H$, where $H$ is the experimental setting.
The results are presented in Table 3.
We observe that the test error decreases as $\mu$ increases, plateauing and even increasing slightly as $\mu$ grows extremely large when the horizon window is small.
However, under a large horizon window, the test error increases as $\mu$ increases.
Next, we evaluate the hyper-parameters $N$ and $k$ on PDETime, as shown in Figure 4 (a) and (b) respectively. We find that the performance of PDETime remains stable when $k\geq 3$. Additionally, the number of aggregation layers $N$ has a limited impact on PDETime.
Furthermore, we investigate the effect of patch length $S$ on PDETime, as illustrated in Figure 4 (c). We varied the patch length from 2 to 48 and evaluate MSE with different horizon windows. As the patch length $S$ increased, the prediction accuracy of PDETime initially improved, reached a peak, and then started to decline. However, the accuracy remains relatively stable throughout. We also extended the patch length to $S=H$. In this case, PDETime performed poorly, indicating that the accumulation of errors has a significant impact on the performance of PDETime.
Overall, these analyses provide insights into the effects of different hyper-parameters on the performance of PDETime and can guide the selection of appropriate settings for achieving optimal results.

Due to space limitations, we put extra experiments on robustness and visualization experiments in Appendix A.5 and Appendix A.6, respectively.

**Table 3: Analysis on the look-back window length, based on the multiplier on horizon length, $L=\mu\ast H$. The best results are highlighted in bold.**
| Dataset | Horizon | 96 | 192 | 336 | 720 |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| $\mu$ | MSE | MAE | MSE | MAE | MSE | MAE | MSE | MAE |  |
| ETTh1 | 1 | 0.378 | 0.386 | 0.415 | 0.411 | 0.421 | 0.420 | 0.425 | 0.446 |
| 3 | 0.359 | 0.382 | 0.394 | 0.404 | 0.427 | 0.421 | 0.443 | 0.460 |  |
| 5 | 0.360 | 0.385 | 0.396 | 0.405 | 0.421 | 0.420 | 0.495 | 0.501 |  |
| 7 | 0.354 | 0.381 | 0.398 | 0.405 | 0.427 | 0.429 | 0.545 | 0.532 |  |
| 9 | 0.356 | 0.381 | 0.397 | 0.406 | 0.446 | 0.440 | 1.220 | 0.882 |  |
| ETTh2 | 1 | 0.288 | 0.335 | 0.357 | 0.381 | 0.380 | 0.404 | 0.380 | 0.421 |
| 3 | 0.276 | 0.331 | 0.339 | 0.374 | 0.358 | 0.395 | 0.422 | 0.456 |  |
| 5 | 0.275 | 0.333 | 0.331 | 0.370 | 0.360 | 0.408 | 0.622 | 0.576 |  |
| 7 | 0.268 | 0.330 | 0.331 | 0.378 | 0.384 | 0.427 | 0.624 | 0.595 |  |
| 9 | 0.272 | 0.331 | 0.331 | 0.378 | 0.412 | 0.451 | 0.797 | 0.689 |  |

**Table 4: Analysis on INRs. PDETime refers to our proposed approach. GELU and Tanh refer to replacing SIREN and CFF with GELU or Tanh activation, respectively. The best results are highlighted in bold.**
| Dataset | Method | PDETime | GELU | Tanh |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- |
| Metric | MSE | MAE | MSE | MAE | MSE | MAE |  |
| Traffic | 96 | 0.330 | 0.232 | 0.332 | 0.237 | 0.338 | 0.233 |
| 192 | 0.332 | 0.232 | 0.338 | 0.241 | 0.339 | 0.235 |  |
| 336 | 0.342 | 0.236 | 0.348 | 0.244 | 0.348 | 0.238 |  |
| 720 | 0.365 | 0.244 | 0.376 | 0.252 | 0.366 | 0.244 |  |

## 5 Conclusion and Future Work

In this paper, we propose a novel LMTS framework PDETime, based on neural PDE solvers, which consists of Encoder, Solver, and Decoder. Specifically, the Encoder simulates the partial differentiation of the original function over time at each time step in latent space in parallel.
The solver is responsible for computing the integral term with improved stability.
Finally, the Decoder maps the integral term from latent space into the spatial function and predicts the target series given the initial condition.
Additionally, we incorporate meta-optimization techniques to enhance the ability of PDETime to extrapolate future series.
Extensive experimental results show that PDETime achieves state-of-the-art performance across forecasting benchmarks on various real-world datasets. We also perform ablation studies to identify the key components contributing to the success of PDETime.

Future Work Firstly, while our proposed neural PDE solver, PDETime, has shown promising results for long-term multivariate time series forecasting, there are other types of neural PDE solvers that could potentially be applied to this task. Exploring these alternative neural PDE solvers and comparing their performance on LMTF could be an interesting future direction.
Furthermore, our ablation studies have revealed that the effectiveness of the historical observations (spatial domain) on PDETime is limited, and the impact of the temporal domain varies significantly across different datasets. Designing effective network structures that better utilize the spatial and temporal domains could be an important area for future research.
Finally, our approach of rethinking long-term multivariate time series forecasting from the perspective of partial differential equations has led to state-of-the-art performance. Exploring other perspectives and frameworks to tackle this task could be a promising direction for future research.

## Broader Impacts

Our research is dedicated to innovating time series forecasting techniques to push the boundaries of time series analysis further. While our primary objective is to enhance predictive accuracy and efficiency, we are also mindful of the broader ethical considerations that accompany technological progress in this area. While immediate societal impacts may not be apparent, we acknowledge the importance of ongoing vigilance regarding the ethical use of these advancements. It is essential to continuously assess and address potential implications to ensure responsible development and application in various sectors.

## Appendix A Appendix

In this section, we present the experimental details of PDETime. The organization of this section is as follows:

- •
Appendix A.1 provides details on the datasets and baselines.
- •
Appendix A.2 provides details of time features used in this work.
- •
Appendix A.3 provides pseudocode of the Encoder of PDETime.
- •
Appendix A.4 provides pseudocode of the Solver of PDETime.
- •
Appendix A.5 presents the results of the robustness experiments.
- •
Appendix A.6 visualizes the prediction results of PDETime on the Traffic dataset.

### A.1 Experimental Details

#### A.1.1 Datasets

we use the most popular multivariate datasets in LMTF, including ETT, Electricity, Traffic and Weather:

- •
The ETT (Zhou et al., ) (Electricity Transformer Temperature) dataset contains two years of data from two separate countries in China with intervals of 1-hour level (ETTh) and 15-minute level (ETTm) collected from electricity transformers. Each time step contains six power load features and oil temperature.
- •
The Electricity (^1^11 [https://archive.ics.uci.edu/ml/datasets/ElectricityLoadDiagrams20112014.](https://archive.ics.uci.edu/ml/datasets/ElectricityLoadDiagrams20112014)) dataset describes 321 clients’ hourly electricity consumption from 2012 to 2014.
- •
The Traffic (^2^22[http://pems.dot.ca.gov.](http://pems.dot.ca.gov)) dataset contains the road occupancy rates from various sensors on San Francisco Bay area freeways, which is provided by California Department of Transportation.
- •
the Weather (^3^33[https://www.bgc-jena.mpg.de/wetter/.](https://www.bgc-jena.mpg.de/wetter/)) dataset contains 21 meteorological indicators collected at around 1,600 landmarks in the United States.

Table 5 presents key characteristics of the seven datasets. The dimensions of each dataset range from 7 to 862, with frequencies ranging from 10 minutes to 7 days. The length of the datasets varies from 966 to 69,680 data points. We split all datasets into training, validation, and test sets in chronological order, using a ratio of 6:2:2 for the ETT dataset and 7:1:2 for the remaining datasets.

**Table 5: Statics of Dataset Characteristics.**
| Datasets | ETTh1 | ETTh2 | ETTm1 | ETTm2 | Electricity | Traffic | Weather |
| --- | --- | --- | --- | --- | --- | --- | --- |
| Dimension | 7 | 7 | 7 | 7 | 321 | 862 | 21 |
| Frequency | 1 hour | 1 hour | 15 min | 15 min | 1 hour | 1 hour | 10 min |
| Length | 17420 | 17420 | 69680 | 69680 | 26304 | 52696 | 17544 |

#### A.1.2 Baselines

We choose SOTA and the most representative LMTF models as our baselines, including historical-value-based and time-index-based models, as follows:

- •
PatchTST (Nie et al., ): the current historical-value-based SOTA models. It utilizes channel-independent and patch techniques and achieves the highest performance by utilizing the native Transformer.
- •
DLinear (Zeng et al., ): a highly insightful work that employs simple linear models and trend decomposition techniques, outperforming all Transformer-based models at the time.
- •
Crossformer (Zhang & Yan, ): similar to PatchTST, it utilizes the patch technique commonly used in the CV domain. However, unlike PatchTST’s independent channel design, it leverages cross-dimension dependency to enhance LMTF performance.
- •
FEDformer (Zhou et al., ): it employs trend decomposition and Fourier transformation techniques to improve the performance of Transformer-based models in LMTF. It was the best-performing Transformer-based model before Dlinear.
- •
Autoformer (Wu et al., ): it combines trend decomposition techniques with an auto-correlation mechanism, inspiring subsequent work such as FEDformer.
- •
Stationary (Liu et al., ): it proposes a De-stationary Attention to alleviate the over-stationarization problem.
- •
iTransformer (Liu et al., ): it different from pervious works that embed multivariate points of each time step as a (temporal) token, it embeds the whole time series of each variate independently into a (variate) token, which is the extreme case of Patching.
- •
TimesNet (Wu et al., ): it transforms the 1D time series into a set of 2D tensors based on multiple periods and uses a parameter-efficient inception block to analyze time series.
- •
SCINet (Liu et al., ): it proposes a recursive downsample-convolve-interact architecture to aggregate multiple resolution features with complex temporal dynamics.
- •
DeepTime (Woo et al., ): it is the first time-index-based model in long-term multivariate time-series forecasting.

### A.2 Temporal Features

Depending on the sampling frequency, the temporal feature $t_{\tau}$ of each dataset is also different. We will introduce the temporal feature of each data set in detail:

- •
ETTm and Weather: day-of-year, month-of-year, day-of-week, hour-of-day, minute-of-hour.
- •
ETTh, Traffic, and Electricity: day-of-year, month-of-year, day-of-week, hour-of-day.

we also normalize these features into [0,1] range.

### A.3 Pseudocode

We provide the pseudo-code of Encoder and Solver in Algorithms 1.

Figure: Algorithm 1 Pseudocode of the aggregation layer of Encoder

### A.4 Pseudocode

We provide the pseudo-code of the Solver in Algorithms 2.

Figure: Algorithm 2 Pseudocode of the numerical Solver

### A.5 Experimental Results of Robustness

The experimental results of the robustness of our algorithm based on Solver and Initial condition are summarized in Table 6. We also test the effectiveness of continuity loss $\mathcal{L}_{c}$ in Table 7. The experimental results in Table 6 demonstrate that PDETime can achieve strong performance on the ETT dataset even when using only the Solver or initial value conditions, without explicitly incorporating spatial and temporal information. This finding suggests that PDETime has the potential to perform well even in scenarios with limited data availability. Additionally, we conducted an analysis on the ETTh1 and ETTh2 datasets to investigate the impact of the loss term $\mathcal{L}_{c}$. Our findings demonstrate that incorporating $\mathcal{L}_{c}$ into PDETime can enhance its robustness.

**Table 6: Analysis on Solver and Initial condition. INRs refers to only using INRs to represent $\tau$; + Initial refers to aggregating initial condition $x_{\tau_{0}}$; +Solver refers to using numerical solvers to compute integral terms in latent space. The best results are highlighted in bold.**
| Dataset | Method | INRs | INRs+Initial | INRs+Solver |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- |
| Metric | MSE | MAE | MSE | MAE | MSE | MAE |  |
| ETTh1 | 96 | 0.371 | 0.396 | 0.364 | 0.384 | 0.358 | 0.381 |
| 192 | 0.403 | 0.420 | 0.402 | 0.409 | 0.398 | 0.407 |  |
| 336 | 0.433 | 0.436 | 0.428 | 0.420 | 0.348 | 0.238 |  |
| 720 | 0.474 | 0.492 | 0.439 | 0.452 | 0.455 | 0.476 |  |
| ETTh2 | 96 | 0.287 | 0.352 | 0.270 | 0.330 | 0.285 | 0.342 |
| 192 | 0.383 | 0.412 | 0.331 | 0.372 | 0.345 | 0.379 |  |
| 336 | 0.523 | 0.501 | 0.373 | 0.405 | 0.357 | 0.399 |  |
| 720 | 0.765 | 0.624 | 0.392 | 0.429 | 0.412 | 0.444 |  |

**Table 7: Analysis on the effectiveness of loss term $\mathcal{L}_{c}$.**
| Dataset | Method | PDETime | PDETime-$\mathcal{L}_{c}$ |  |  |
| --- | --- | --- | --- | --- | --- |
| Metric | MSE | MAE | MSE | MAE |  |
| ETTh1 | 96 | 0.356 | 0.381 | 0.357 | 0.381 |
| 192 | 0.397 | 0.406 | 0.393 | 0.405 |  |
| 336 | 0.420 | 0.419 | 0.422 | 0.420 |  |
| 720 | 0.425 | 0.419 | 0.446 | 0.458 |  |
| ETTh2 | 96 | 0.268 | 0.330 | 0.271 | 0.330 |
| 192 | 0.331 | 0.370 | 0.341 | 0.373 |  |
| 336 | 0.358 | 0.395 | 0.363 | 0.397 |  |
| 720 | 0.380 | 0.421 | 0.396 | 0.434 |  |

### A.6 Visualization

We visualize the prediction results of PDETime on the Traffic dataset. As illustrated in Figure 5, for prediction lengths $H=96,192,336,720$, the prediction curve closely aligns with the ground-truth curves, indicating the outstanding predictive performance of PDETime. Meanwhile, PDETime demonstrates effectiveness in capturing periods of time features.

Figure: Figure 5: Visualization of PDETime’s prediction results on Traffic with different prediction length.
Refer to caption: x5.png

## Peer Reviews

confidence: 3
contribution: 2
initial_rating: 3
presentation: 2
questions: 1. Can the authors clarify the weighting mechanism in Equation 14 for a more transparent understanding of how different losses are balanced?
2. Would it be possible to release the code or provide more detailed implementation details to support reproducibility?
3. How does PDETime perform in terms of runtime and memory usage compared to other baseline models, especially for very long forecasting windows?
soundness: 3
strengths: 1. The application of PDEs in time-series forecasting introduces a unique perspective for capturing continuous dynamical patterns.
2. The authors test PDETime on a wide range of datasets, demonstrating the model’s adaptability.
summary: The paper presents PDETime, a novel approach for long-term multivariate time-series forecasting that models the series as a continuous dynamical system governed by partial differential equations (PDEs). Instead of directly forecasting future values, PDETime predicts the partial derivatives in the latent space, integrating this information over time to generate forecasts.
weaknesses: 1. **Fairness in Comparison**: My most concern is that the experiments may lack fairness due to differences in historical input length (\(H\)) between PDETime and baseline models. While PDETime’s \(H\) is optimized, baseline models use a fixed input length, which can skew results since (1) different input lengths impact the number of samples in the test set for each model, potentially affecting comparability, and (2) input length significantly influences forecasting performance. It’s recommended to either standardize \(H\) across models or optimize it for all baselines.
2. **Code Availability**: The absence of released code reduces the credibility and reproducibility of the results, as reviewers and readers cannot verify the findings independently.
3. **Unclear Loss Weighting**: The weight description for the loss function in Equation 14 is unclear, making it challenging to understand the balance between different loss components.
4. **Lack of Efficiency Analysis**: There is no discussion of PDETime’s computational efficiency, such as runtime or memory consumption, which is essential for understanding its scalability and practicality for long-term forecasting tasks.

---

confidence: 4
contribution: 4
initial_rating: 6
presentation: 3
questions: ## Main Questions Connected to Mentioned Weaknesses

- What is the key difference in your approach compared to the NeuralODE methods mentioned under weaknesses? Please do give an answer which goes beyond "We are coming from PDEs not ODEs", because at the end you only consider deviations with respect to time.
- What is runtime-behavior with respect to the considered baselines?

## Further Questions/Comments
- I was wondering what the forecasting horizon is in Table 1. I had to look at Table 7 in the appendix to see that is is an average over different horizons. I would suggest to put that information into the caption of Table 1.
- What are the results if you replace (9) with always using$ f_{\psi}(z_t)?$
- Please explicitly write down the reconstruction loss L_p
- What are the input time features c^(0) in general/For the particular datasets used?

## Some Small Typos/Comments
- Page 6, line 294: X_{x_0} -> X_{t_0}
- (14): I guess, here somethings is wrong, as L_p occurs on both side of the equation
soundness: 3
strengths: + The idea of capturing the time-dynamics and treating time-series forecasting as an initial-condition problem is a cool idea.
+ The results are promising, PDETime always outperforms all competitors
+ The approach could be potential starting point for a new direction in time-series forecasting
summary: In this paper, the authors propose PDETime, a novel model for time-series
forecasting. The methods novelty lies in the fact, that it does not simply
predict values for a specific time point, but that it generates an encoding
$\alpha_t$ which can be interpreted as a first deviation. Then, the dynamics
of the hidden representation $z_t$ of a time in the interval $[t_0,t] $ are computed
via an integral solver over $\int_{t_0,t} \alpha_t $ and this is then used to decode via $\text{Decoder}(z_t) + x_{t_0}$.
weaknesses: - Please write a clear problem formulation: In Time-Series forecasting, what do
  you have given, what do you want to predict/which objective do you want to
  optimize? What are the domains your inputs and outputs live in. I had to
  somehow guess here sometimes at the beginning, especially it was not clear to
  me at the beginning what $s$ is, and whether it is given or not.
- I really like the PDE motivation, but what you at the end do is simply to
  learn first derivations at time points instead of learning the values
  them self. As your spatial input $s$ is computed in the encoder and you are
  only considering the derivation with respect to time for fixed s, one could
  argue that at the end you are only considering ODEs with respect to time.
  However, then what you do is already established in the literature for
  time-series forecasting with irregular time-points, see [1], [2], [3].

[1] De Brouwer, E., Simm, J., Arany, A., & Moreau, Y. (2019). GRU-ODE-Bayes: Continuous modeling of sporadically-observed time series. Advances in neural information processing systems, 32.

[2]Scholz, Randolf, et al. "Latent Linear ODEs with Neural Kalman Filtering for Irregular Time Series Forecasting." (2023).

[3] Schirmer, Mona, et al. "Modeling irregular time series with continuous recurrent units." International conference on machine learning. PMLR, 2022.
- As your approach is fundamentally different to established transformer models,
  it is important to integrate a runtime-study. What does the performance gain
  over PatchTST cost with respect to efficiency?

---

confidence: 3
contribution: 2
initial_rating: 3
presentation: 1
questions: See weaknesses
soundness: 2
strengths: **S1** Viewing multivariate time series as instances of partial differential equations (PDEs) is an intriguing approach. This perspective is particularly relevant for multivariate time series in which sensors at different locations measure the same thing (e.g., electricity data).

**S2** Authors proposed a new PDE solver which can potentially avoid the problems with Neural-ODE solvers

**S2** The experimental results on various datasets indicate that the approach outperforms a range of baselines in multivariate longterm forecasting. Various ablation studies have been shown in the experiments
summary: The authors propose a novel model, PDETime, for long-term multivariate time series forecasting. PDETime assumes that the multivariate time series is sampled from a continuous dynamical system governed by a partial differential equation (PDE). Unlike existing approaches that rely solely on historical observations or discrete time points, PDETime leverages both observations and time for forecasting.

PDETime encodes the partial derivatives of the system, solves them for future time points in a latent space using the proposed PDE solver, and then decodes the results back to the data space with a decoder. Experiments conducted on seven multivariate datasets with varying forecast horizons show significant improvement in forecasting accuracy compared to baseline models.
weaknesses: **W1. Writing Quality:**

1. The paper contains undefined variables when first introduced. E.g. $\mathbf{X}{his}$, $\mathbf{c}{t}$, and $\tau_t$ in line 8, as well as $\mathbf{W}{\tau}$, $\mathbf{W}c$, $\mathbf{W}x$, $\mathbf{W}$, $\mathbf{b}\tau$, $\mathbf{b}c$, $\mathbf{b}x$, and $\mathbf{b}$. Please define any variable when it is first introduced.
2. Section 3.1 has an incorrect sub-section title. The section is titled "Problem Formulation" but describes the model's overview. Problem formulation typically refers to the specific problem the paper addresses.
3. Some statements are overly vague. For instance, in lines 502-505, "Finally, our approach of rethinking long-term ... promising direction for future research" lacks clarity. Please mention what new perspectives do the authors wish to explore?
4. Please review the order of all tables in the paper; Table 1 should appear before Table 3.
5. Typos:

5.1 line 52: hiders $\rightarrow$ hinders

5.2 line 294: $\textbf{x}{x_0} \rightarrow \textbf{x}{t_0}$

5.3 line 151: $\textbf{u}(t + \Delta) \rightarrow \textbf{u}(t + \Delta t)$

5.4 lines 77-80: "Furthermore, as shown in .... capture temporal dependencies" first sentence doesn't complete the thought

5.5 Eq. 10, $\mathcal{L}($: "(" did not close

5.6 Alg 2, line 4: $t'$ appeared twice

6. There are many other notational issues which require careful attention from authors.
ex. time index have subscript at some places like eq. 2 and superscript in eq. 1

**W2.** Motivation for considering multivariate time series as an instance of PDE is missing. Can authors provide clear motivation behind this approach? Would this assumption hold for multivariate time series data with variables measuring different aspects, such as a physiological dataset where one variable tracks pulse and another tracks blood pressure?

**W3. Model Scope:**

1. Why is the model limited to long-horizon predictions? How does it perform on short horizons?
2. The authors state that existing models rely on either historical observations or time points alone. However, the Informer model uses both local and global time information as embeddings. Can authors clarify this?

**W3. Encoder Architecture:**

1. What is the motivation behind using this particular encoder architecture?
2. The encoder’s design is challenging to understand due to insufficient explanations:
- Clearly define and distinguish between $X^{(k)^i}$ and $\mathbf{X}^{(K)^i}$
- Provide a detailed explanation of the operations in Equation 7
- Explain the rationale behind adding $\mathbf{c}_t^{(k)}$ again after concatenation
- Explain the discrepancy between Alg 1 and Eq 7 (see line 3, Alg 1)

3. Eq. 6; why (6a) and (6b) LHS have subscript $t$ and no information of it on RHS?
4. What is For loop in Alg 1? Is it clearly mentioned in main text? 

**W4. Solver:**

1. The paper does not clearly explain how the proposed solver circumvents the issues associated with the Euler solver. Please provide a comparison between their proposed solver and the Euler/Neural-ODE solver
2. Figure 2b does not effectively illustrate the solver. A clearer figure would help.
3. The PDE solver is claimed to be a novel and key contribution. To substantiate this, could the authors demonstrate the advantages of this solver over existing Neural ODE solvers (e.g., Chen et al., 2018) through a toy example?
4. In line 270, the authors state that Figure 3 illustrates the solver’s advantages over the Euler solver. However, it is difficult to see this comparison, as Figure 3 primarily shows results for hyperparameters $k$, $N$, and $S$.

**W5. Experiments:**

1. Please clarify the input and prediction sequence lengths for the results in Table 1. Comparing Tables 1 and 7, it appears the forecasting horizon is 336 (please correct me if this is incorrect)
2. There is a discrepancy in the third decimal place of Patch TST results between these tables if the forecasting horizon is 336
3. Please inform about Table 7 in paragraph after Section 4.2, otherwise the text is confusing

---

confidence: 3
contribution: 2
initial_rating: 6
presentation: 3
questions: 1. In Section 3.2.2, is the solution generated by the (modified) Euler solver guaranteed to be bounded? Furthermore, as the Euler method is known to be potentially unstable and sensitive to step size, could the authors clarify how they determined the appropriate step size for this implementation? In other words, does it raise any issue by setting $\Delta \mu=1$?

2. Following previous question 1, could authors provide some theoretical or empirical evidence of the stability and boundedness of their modified Euler solver? Additionally, could the author discuss any potential limitations or tradeoffs of using $\Delta \mu=1$ compared to other step sizes.

3. In the Conclusions and Future Works section, the authors note that future developments may enhance PDETime’s ability to handle irregular time series or data with missing values. Could the authors further elaborate on the current limitations of PDETime in this regard?

4. Could the authors provide a detailed analysis of the time and space complexity of PDETime compared to the baseline methods? Given that the method combines aspects of both historical-value- and time-index-based models, does this lead to an increase in complexity compared to other models?
soundness: 2
strengths: 1. The paper presents a unique approach for time-series forecasting by framing the task as an initial value problem governed by PDEs, which provides a theoretically grounded approach for capturing spatiotemporal dependencies.
2. The architecture effectively leverages the latent space to model complex temporal dependencies, mitigating issues of error accumulation that are common in autoregressive methods.
3. Extensive evaluations across seven real-world datasets reveal that PDETime outperforms leading historical-value-based and time-index-based models, including Transformer and CNN models, demonstrating robust, stable performance over varying forecasting lengths.
4. The study includes comprehensive ablation experiments on key components (e.g., Solver, temporal and spatial features) and evaluates the impact of hyper-parameters, enhancing the reproducibility and scientific rigor of the results.
summary: The paper provides a novel approach, PDETime, for long-term multivariate time-series forecasting (LMTF) by modeling time series data as samples from a continuous dynamical system governed by partial differential equations. PDETime introduces an encoding-integration-decoding architecture to predict future values in the latent space by estimating the partial derivative of the system with respect to time. The method employs a neural solver to address the nonstationarity and sampling limitations common in traditional historical-value- and time-index-based models. Finally, the author provides experimental results demonstrating that PDETime achieves state-of-the-art performance across seven benchmark datasets. In my opinion, the paper is organized and easy to follow, but I have some questions.
weaknesses: 1. The use of numerical solvers (e.g., Euler Solver) could introduce sensitivity to hyper-parameter tuning and potential limitations in modeling high-frequency or abrupt temporal changes.
2. The encoding-integration-decoding process, along with meta-optimization, may add computational overhead, especially for large datasets or models with high-dimensional data, limiting scalability.

---

confidence: 5
contribution: 3
initial_rating: 6
presentation: 3
questions: I have several questions about this paper:
1. Why does the author think multivariate time series must have a latent PDE structure? For example, ETTH data does not seem to be governed by any PDE dynamics.
2. The PDETime seems more like ODETime to me, as the author only calculates time derivative instead of spatial derivative. 
3. It is well known that the numerical integration step can be time consuming. I wonder how the model performs with smaller Euler steps in both forecast accuracy and time consumption. 
4. I wonder what is the purpose of utilizing encoder and decoder in this problem along with PDE solver. Does it change the input dimention to facilitate numerical integration?
5. Why does the author utilizes both sin() and GeLU() function? Is their any mathematical intuitions behind this?
soundness: 3
strengths: The key idea of this paper is novel: the author introduces a PDE solver to solve time series instead of using historical data as input, which may be able to capture the inter-series relationship for enhanced accuracy. The paper is also well written with solid experiments.
summary: This paper proposes a novel method for multivariate time series forecast by treating the input continuous dynamical system governed by a latent PDE equation. Instead of predicting future values directly, PDETime predicts the derivative and uses Euler method to integrate the derivative, and applies encoder and decoder for input embedding and yielding the final output. This approach enhances both performance
and stability, especially in scenarios with long-horizon forecast. Extensive experiments on seven diverse real-world LMTF datasets demonstrate that PDETime not only adapts effectively to the intrinsic spatiotemporal characteristics.
weaknesses: The literature review setion seems a little over-compressed, especially the multivariate time series literature. In fact, there is much more literature in the past 2 years that seeks to provide forecast from multiple perspectives. For example, Transformer-based methods, linear interpolation methods (D-linear) and auxiliary series construction (CATS: Enhancing Multivariate Time Series Forecasting by Constructing Auxiliary Time Series as Exogenous Variables, ICML 2024) and so on. I would suggest that the author should enrich their literature on multivariate time series, and compress the neural PDE literature because it is only utilized as a component of the model. 

Also the benchmark methods are not sufficient as there are many papers in 2024 that address LMTF from LLM perspectiuve, including LLM4TS, GPT4TS, CATS and so on. I strongly encourage the author to add these benchmarks into their comparison. 

I would gladly increase my rating if my concerns are addressed.

Meta-review: This paper introduces PDETime, a novel approach to long-term multivariate time-series forecasting using partial differential equations (PDEs) to model the temporal dynamics of latent representations. While the perspective of treating time series as samples from a PDE is intriguing, the paper has critical issues that limit its impact and clarity. The work lacks sufficient empirical rigor, with comparisons to established baselines potentially biased due to differences in input length and insufficient benchmarking against recent advanced methods. While the PDE-based modeling is positioned as novel, the approach seems more akin to ordinary differential equations (ODEs), with limited exploration of spatial derivatives, reducing the distinctiveness of the method. Additionally, concerns about the computational efficiency and scalability of the proposed solver are unaddressed, raising questions about its practicality for real-world use. The presentation has several weaknesses, including unclear problem formulation, inconsistent notation, and an over-reliance on vague claims rather than explicit justifications. While the experimental results appear promising, the lack of code availability and detailed runtime analyses further hinders reproducibility and assessment. These shortcomings, combined with the overextension of the claimed contributions, justify a decision of Reject.

Decision: Reject