Neural Wave Equation for Irregularly Sampled Sequence Data
Authors: Arkaprava Majumdar
M Anand Krishna
P. K. Srijith
Venue: ICLR 2025 Poster (2025)
Keywords: Wave Equation
Neural ODE
Sequence Labelling
Primary area: neurosymbolic & hybrid AI systems (physics-informed, logic & formal reasoning, etc.)
TL;DR: Partial Differential Equations parameterised by a Neural Network (like Neural ODE) can be used to solve sequence modeling problems. We hypothesize why this might be the case and demonstrate that it outpeforms many known continuous RNN models.

Abstract: Sequence labeling problems arise in several real-world applications such as healthcare and robotics. In many such applications, sequence data are irregularly sampled and are of varying complexities. Recently, efforts have been made to develop neural ODE-based architectures to model the evolution of hidden states continuously in time, to address irregularly sampled sequence data. However, they assume a fixed architectural depth and limit their flexibility to adapt to data sets with varying complexities. We propose the neural wave equation, a novel deep learning method inspired by the wave equation, to address this through continuous modeling of depth. Neural Wave Equation models the evolution of hidden states continuously across time as well as depth by using a non-homogeneous wave equation parameterized by a neural network.  Through d'Alembert's analytical solution of the wave equation, we also show that the neural wave equation provides denser connections across the hidden states, allowing for better modeling capability.  We conduct experiments on several sequence labeling problems involving irregularly sampled sequence data and demonstrate the superior performance of the proposed neural wave equation model.

Published as a conference paper at ICLR 2025
NEURAL WAVE EQUATIONS FOR IRREGULARLY SAM- PLED SEQUENCE DATA
Arkaprava Majumdar', MM Anand Krishna', and P.K. Srijith'
Indian Institute of Technology, Hyderabad a #124mtech020022 ttt.thhh.. ac.in sri jithhcsee . iith. ac.in
cs22mtech
ABSTRACT
Sequence labeling problems arise in several real-world applications such as health- care and robotics. many such applications, sequence data are irregularly sampled and are of varying complexities. Recently, efforts have been made to develop neural ODE-based architectures to model the evolution of hidden states continuously time, to address irregularly sampled sequence data. However, they assume a fixed architectural depth and limit their flcxibility to adapt to data sets with varying complexities. We propose the neural wave equation, novel deep learning method inspired by the wave equation, to address this through continuous modeling of depth. Neural Wave Equation models the evolution of hidden states continuously across time as well as depth by using a non-homogeneous wave equation parame- terized by neural network. Through d' Alembert's 's analytical solution of the wave equation, we also show that the neural wave equation provides denser connections across the hidden states, allowing for better modeling capability. We conduct experiments on several sequence labeling problems involving irregularly sampled sequence data and demonstrate the superior performance of the proposed neural wave equation model.
INTRODUCTION
Sequence data arise in several real-world applications like health care, robotic systems, and speech recognition. Models such as Recurrent Neural Networks (RNNs) Rumelhart et al.) (1986); Hochreiter & Schmidhuber [197)); Cho et al. (2014); De Brouwer et al. (2019); SSchsster & Paliwal (997))
their variants have proven to be highly effective processing such sequential data. Traditionally, RNNs perceived discrete approximations of underlying dynamical systems, concept well documented in the literature ichi Funahashi & Nakamura (193)); [Biler-jones et al. (2002). However, RNNs face significant challenges in effectively addressing sequence labeling problems that arise in applications such as healthcare, social media, and business, which involve irregularly sampled or partially observed sequence data Rubanova al. (2019). There have been efforts in the community to develop deep learning models that allow continuous transformation of the hidden representation. Neural Ordinary Differential Equations continuous transformation of the input-output map. Neural ODEs combine neural networks with ordinary differential equations to achieve this, resulting in an architecture similar to Resnets( (He) et al. (2016)) This continuous modeling allows for flexible, adaptive representations that capture hierarchical relationships without relying on fixed depth.
al. (2018) implicitly model depth by treating it as
Recognizing the limitations of non-uniform data sampling, there has been paradigm shift toward developing sequence models inspired by Neural ODEs that emulate the continuous evolution of hidden states over time. ODE-RNN(Rubanoo) over time using a NODE, where hidden representations are continuously transformed taking into account the time gaps between observations, leading to better hidden state representations. Variants of ODE-RNN like the RR Brouwer et al. (2019)) and ODE-LSTM(Lechne & Hasani (2020)) were consequently proposed for irregular time series data.
et al. (2019)) modeled hidden state transformations

---

Published as a conference paper at ICLR 2025
69.0
022.1
NG20
10.20
112.00
A- RNN Variant
B ODE-RNN Variant
C- Neural Wave Equation
Figure I: Architectural comparison between discrete depth discrete time, discrete depth continuous time and continuous depth continuous time model
In several real-world problems with irregular observations such as social media post-classification, input data could also exhibit varying complexities, and fixed discrete transformations on the depth dimension would become a limitation. A shallower network may not be able to capture the complexity of the data properly, while a deeper network may overfit the data. ODE- RNN or their variants perform a discrete transformation of the hidden state along depth using neural network transformation, limiting their flexibility to adapt to complex data sets and require exhaustive model selection.
Unifying the principles of Neural ODE and ODE-RNN naturally leads to Partial Differential Equa- tions(Farlow (1993)). which model hidden states continuously over both time and depth. PDEs provide a principled framework for capturing multidimensional dependencies, enabling adaptive representations for hierarchical and temporal complexities.
The continuous depth recurrent neural differential equation Anumasa et al. (2023) ) (CDR-NDE) proposed the application of partial differential equation to model the evolution of hidden states continuously over time and depth. The authors the CDR-NDE paper use the non-homogenous heat equation with the source function being neural network to model the hidden states continuously over time and architectural depth. Though heat equation-base based PDEs are useful for modeling continuous evolution, we find them to have certain limitations that restrict their effectiveness for sequence data. Intuitively, the diffusive nature of the heat equation implies that the initial information is often smoothed out and lost.
We propose the neural wave equation, a wave equation-based neural differential equation, which can provide an effective and natural way to model sequence data. The wave equation can implicitly consider the dependency with neighboring hidden states over window and addresses the limitations exhibited by the heat equation in sequence modeling. The dependency with some particular hidden states does not arise naturally in the heat equation and hence, needs to be supplemented through the source terms. These dependencies are captured in the wave equation directly. The propagating nature of the wave equation makes it more robust to loss of initial information. Further motivation comes from the existence of an analytical solution for wave equations, and this helps understanding the effectiveness of wave equations in modeling sequence data. A schematic representation of the hidden state evolution in neural wave equations compared to other popular approaches is provided in Figure TT Neural wave equations allow for continuous evolution of hidden states across time and depth while capturing more dependencies. We develop neural wave equation models with several parameterizations of the source function through a neural network. Neural wave equations can be solved based on existing solvers for PDE and efficient training techniques based on adjoint methods can be used learn the parameters. Our experiments, conducted diverse datasets such person activity recognition, Walker2d kinematic simulatior[Lechner & Hasani (2020), sepsis (PhysioNet R0e9rReyna et al. (2019) and stance classification Derezynski al. (2017 7) demonstrate the superior performance of neural wave equation models over existing baselines for sequence labeling problems. In summary,
1. We propose neural wave equations a non- non-homogeneous wave equation with its source function parameterized by neural network, for sequence labeling problems.
2. We speculate on the potential benefits of using non-homogenous partial differential equations for sequence modeling over continuous time RNN models with discrete depth by looking at their analytical solutions.
3. We empirically demonstrate the effectiveness of the neural wave equations on several sequence labeling tasks with irregularly sampled data.
2

---

Published as a conference paper at ICLR 2025
2 RELATED WORK
Several variants of RNNs were introduced in the past to deal with irregularly sampled sequence data or sequence data with missing observations. GRU-D and RNN-D Weinan (2017) models make use of a decay rate for predicting the missing values. CT-LSTMMei & Eisner (2017) combines both LSTM and continuous time neural Hawkes process to model the continuous transformation of hidden states. The latest works in this direction make us of the framework of the neural ordinary differential equation Chen et al. (2018) to model the continuous evolution of hidden states over time.
ODE-RNN(Rubandn et al.] 2 and their variants such as GRU-ODHDe Brouwer et al. 2019) and ODE-LSTM(Lechner & Hasani (2020)), use a NODE-based formulation to model the hidden state transformation across the irregular observation times, and then an MLP to map it to the corresponding output. Neural Controlled Differential equation (Kidger et al. (2020) calculates a continuous path over the sequence data using cubic splines and subsequently constructs the evolution of a hidden state in continuous time from it. After a hidden state is calculated, the output from the hidden cell obtained by passing the hidden state vector through an MLP. Variants of Neural CDE such as attentive Neural CDEghin et al. (2024) and attentive co-evolving Neural CDEChin et al. (2021)) attempt to combine the attention mechanism with NODE by using two NeuralCDEs. Contiformer (Chen et al. (023)) introduced a continuous time attention mechanism in transformers (Vaswani et al. (017)) to model irregularly sampled time-series data. The other promising direction in sequence modeling tasks is the structured state-space models (Gu et al. (2022) which focuses on discretizing differential equation with an alternate RNN and CNN view.
Continuous depth recurrent neural differential equations (CDR-NDE) Anumasa et al. 2023) proposed the use of partial differential equations, in particular heat equation, to model the evolution of hidden states over both the temporal and depth dimensions. This overcame the limitations of the discrete depth modeling of the prior approaches for irregularly sampled sequence data. There exists hardly any work on modeling deep learning architectures using PDEs. However, there exists a line of research that aims to use neural networks to solve partial differential equations known as physics- neural networks (PINNS) or Neural PDEs Zubov et al. (2021) Brandstetter et al. (2021 Hu et al.)
(2020); Raissi et al. (2019).
Hughes et al Hughes et al. (2019) draw a similarity between homogeneous wave equation and RNN from a computational physics perspective. In contrast to earlier efforts, the paper focuses on studying the effectiveness of PDEs in developing adaptable deep-learning architectures for modeling. addressing the irregularly sampled sequence data. In particular, study and propose neural wave equations as an effective solution to solve such sequence labeling problems.
3 BACKGROUND
3.1 PROBLEM SETTING
We assume a sequence data set consisting of multiple irregularly sampled sequences of length K. Let the elements in the sequence be represented as {f1, X1, V1). (f2, X2, (2,,),,,., (tx, XK, μK)} where t; E R+ is the observation time stamp and 2; E R is D-dimensional input observed at time t,. The corresponding output y/ can be class label for classification task or a real value for regression task. Our model considers the sequence of observed inputs X2, Xk] and their corresponding times [4, 12. lx] as input and aims to predict the output sequence [y1, W/......... .. ØK] considering the dependencies among the input elements in the sequence and their observation times. Our aim is to learn function f.., 0) which predict the output sequence given the input sequence and observation times so that it exhibits good generalization performance on the unseen sequences.
3.2 IMPLICIT DEPTH DEEP LEARNING
The depth of neural network is important hyperparameter that determines the model's capacity to learn complex patterns. However, a deeper model may overfit the input data does not show complex patterns. Implicit layer depth techniques offer novel approach to structuring deep learning models by adaptively determining the effective depth during training or inference. Models such as Neural ODEs bypass the constraints of predefined layer depth by defining continuous transformation of
3

---

Published as a conference paper at ICLR 2025
the input with respect to depth via a differential equation. A neural ordinary differential equation (NODE) offers a continuous approach to deep learning by approximating the imputto-ourph map using a learnable neural network and differential equation of the following form.
dh(t) (o())(), dt
(1)
where (to) ho is the initial condition which is the input to the model or some transformation of the imput, and fe is a learnable neural network with parameters 9. The differential equation is then solved with the help of adaptive step-size solver, which automatically adjusts the step size with varying input complexity. This alleviates the need for tuning the depth of a network manually.
3.3 RECURRENT NEURAL ODE
Recurrent Neural ODEs (ODE-RNN Rubanova et al. (2019) adapt the neural ODE approach to model irregularly sampled sequence data. In the ODEERNN framework, the dynamics of the system are modeled using a combination of RNN cell and ordinary differential equations, which describe how the state of the system changes continuously over time. Unlike the neural ODE, ODE-RN RNN-NNN models the hidden state evolution continuously the temporal dimension while following discrete modeling in depth. Assuming, h is the hidden state obtained at the previous timestamp f at some discrete depth d. h first passed through neural ODE obtain the hidden representation ie at time la-
h ODDSDDDE (1x-1,14))
where ODESolve calls numerical solver solve ODE as referred in Equation The hidden state hg at time 4 is then ob- tained by passing ig and previous layer hidden representation he-1 to RNN- cell, ng o (W + Wrig + bz), where Wi, WT. by are parameters of the RNN-cell. The final output y4 at time ty is obtained by repeating the hidden repre- sentation computation over a user-defined discrete depth in RNN-ODE. ODE-RNN provides a better modeling capability, es- pecially for irregularly sampled sequence data. However, their capability is limited by the modeling of depth discretely. In Fig. [ we see that in ODE-RNN, depth modeled explicitly by stacking multiple layers of sequential ODE-RNN cells, sim- ilar to traditional RNN architectures. Neural Wave Equation, there is no explicit concept depth as seen traditional neu- ral networks with stacked layers. Instead, the transformations of the hidden states are governed by partial differential equation, and the solver performs a number of small transformations which implicitly define depth (See Section In RNN and ODE-RNN one has to perform exhaustive model selection over depth achieve good performance. We demonstrate this in Figure 2 by comparing the performance of ODE-RNN and LSTM for varying depths. We also compare them against the proposed neural wave equation which models depth implicitly (red line).
MSE Comparisen of LSIM and ODE-NNN with Noural Weve Equation
60
Figure 2: Discrete depth models such as ODE-RNN and LSTM require model selection over depth to obtain the right model. Neural Wave Equation (red line) achieves this by implicitly and continuously modeling the depth. The dataset used in ETTHI and the task predict 24 time steps in the future while considering the previous 96 timesteps.
3.4 WAVE EQUATION
In this section, we aim to introduce some ideas and terminologies related to the wave equation which forms the basis of our work. Wave Equation is partial differential equation of 2nd order in two variables and intuitively, can model more complex dynamics than 1st order PDEs. describes the propagation of mechanistic waves, such sound, light, and water waves through medium. The
4

---

Published as a conference paper at ICLR 2025
propagative nature of the wave equation prevents the loss of initial information opposed to the heat equation. The homogenous 1D wave equation formulated as
ōul2z t) c² P (u(ztt)
òt
0z2
with the initial value condition u(2,0) = f(z), un (2, = au (z.0) 0) g(2 and C is a constant assume g(z)
denoting the speed of the wave in the medium. For simplicity, का work. The solution of the wave equation, u(2, 1) gives the displacement of a wave at any given point 2 over time t. It mathematically models how waveforms evolve over time and space in a continuous manner. The analytical solution of wave equations is given by d' AlAembert's 's formula (Sooolev as t) ((=-<t) However,
throughout our
often do not know the explicit form of the initial value function fz), and instead we know only the values of f()) at certain points. In such scenarios, we may use numerical method to solve the equation via discretization. One such popular method is the finite difference methodd(FMM) Abdulkadir et al. (2015). Under the FDM scheme, the wave equation can be discretized
U, 2ugtt U2
Uu+t22t 21x,t + Us-Aa.t A
Ug. is the value of the function u(2, t) at position z and time We can solve the wave equation using the above discretization scheme by using numerical methods for solving ODEs. Higher-order
methods like RK-4 are often preferred for higher precision during solving a numerical FDM scheme.
Wave dynamics are better characterized by non-homogeneous wave equation Farlow (9933) which is written as
8uu(z,t) Pulzz ((2,)) O=2
012
where the function F(2,t) 1) is called a source. It is physically interpreted as an external force that is acting on each point. The source is function of time and space as well, which means that the external force acting over each data point may vary over time.
NEURAL WAVE EQUATIONS
Sequence modeling problems require predicting the sequence of outputs while capturing dependencies across the elements in the input sequence. We observe that wave equation offers us a natural way to accomplish this as they are capable of capturing the dependencies and interactions among the system states. We propose to model the evolution of hidden states as a wave equation with the source function parameterized as neural network. The inputs in the sequence at different observation times are considered as the initial displacement values associated at the various spatial locations in the equation. The evolution of wave equation over time implicitly models the depth and number of hidden layer transformations. The proposed neural wave equation captures the dependencies among the hidden states and models their evolution continuously in both the temporal dimension and depth dimension.
Considering the FDM discretization Abdulkadir al. (201)) for the wave equation in Equation 4 We rewrite it to represent the hidden state evolution with the point representing the hidden state at some point time the sequence data and time representing the evolution of the hidden state at some point in depth d.
h,d+ = 2hr, - -h,
[litaadd - 2hy + by- A..d]
(6)
Here, lt,a represents the hidden states corresponding data point time / and at depth d.
From the FDM expression, we can clearly see the interaction between the hidden states fe,d, he- and he,d-ad while calculating h,,4+ A drawback with directly adopt- ing a homogenous wave equation in a deep-learning setting is the absence of a learnable function which may be required if the sequence modeling task is complex. To account for we also add leamable neural network to the FDM scheme as
A2 A
hr,+41 22i Me.d- +
2he + he-
+ Eo, (1o-- Arids he,ds let Dr (e.d-aa)
5

---

Published as a conference paper at ICLR 2025
where Fe, (.) is a learnable neural network with parameters 8,. This leads to the proposed neural wave equation model, which is a non-homogeneous wave equation with a source function modeled as the neural network. The neural network Fs, (-) helps in capturing the non-linear dependencies with neighboring hidden states. The proposed neural wave equation is given as
Phi,d Od
Ohh 062
Fo, (he,d, Ae-d De+A. de,d-
(8)
The initial value condition for the neural wave equation, f(4+) = in(t; 0) for some time t; is generated
from the corresponding input z; in the input sequence. For simplicity, we assume @h(t.0) 0. We can get a better understanding of the interaction among the hidden states by studying the analytical solution for the non-homogeneour wave equation[Sbooley
dd
h(t, d) f(t + cd) + f(t - cd)
2c 10 1
d (teld-r)
Fo, (ha,7, he- hos,,,h--s, ho- A,,, hg. Ar ddsd
2
(9)
where f(t) h(t, 0) and Fo, (.) is the source function. We can see that to compute the hidden state h(t,d) at any time and depth d. it considers the source function values of hidden states in a neighborhood and from the previous depths such as h[( A,z, d - Aa), h(t A 1, d Aa)..., h(t Acz, d Aa), h(t Aes, d 2&2), h(t Acst1, d 2A2), h(t + Aes, d - 2 and so on where takes the value of the current depth ie Ad, d 2 ........ A detailed explanation of boundary condition is provided in [A.12] We discuss the implications of the analytical solution of the PDFs general and wave equation in particular in more detail in the following sections.
4.1 SOURCE FUNCTIONS
We model the source function terms using neural network. We use combination of GRU-Cell and MLPs model the source terms. The source terms serve two main purposes. It adds non-linearity to the hidden state interactions in the wave equation and provides more control over the flow of information. Secondly, since we are solving a I-dimensional wave equation with vector-valued hidden states, the source terms enable the mixing of information across the latent dimension of the hidden state. We experiment with several source term formulations of varying numbers of parameters. We experiment with GRU-cells because it provides gating mechanism to control the flow of information into the concerned hidden state from its neighboring states.
Single GRU Model the source term as Fe, (he- d. d.,d with only hi-a.. and bi,d-A, passed through GRU-Cell.
Single MLP: Models source term Fe, (he-sa.ds hi,d, het All the terms are concate- nated together and passed through an MLP layer.
Double Gating Models source term as Fo, (hr- Ar.d: hr he htdt-a)): The h and hy,d-A, are concatenated together and passed through a MLP. The output the MLP is then passed into a GRU-Cell with hidden state h h and into another GRU-Cell with hidden state h and the results are added.
MLP+GRU Models source term as Fo, (h+- d,, he,d A, (e.d-A,) The terms h4- -.. hy,d; d; h a, h4 A, concatenated and passed into an MLP. The output of the MLP is then passed into GRU-Cell with hidden state dr As.
4.2 FORWARD PASS
The forward pass in our neural wave equation model consists of 3 stages:
1. An MLP layer that takes the input element in the sequence x; at time t and projects them into latent space to obtain the hidden state initial value hy he_0 MLPap... (2t)). Let the initial hidden state values associated with the elements in the sequence be h
2. The second stage uses ODESolver to solve the proposed neural wave equation and obtain the final hidden states, h,p = ODESolve(hh, (h.teeehh,,, o [0, D), ODEFunc). ODESolve is 2nd-order adaptive step size ode solver and ODEFunc the function that is responsible

---

Published as a conference paper at ICLR 2025
for calculating the following discretization term.
Shid
Det 2he,a h4 A,, (.)
(10)
dd
A
The ODESolve and ODEFunc used as black box solve the neural wave equation using the method of lines. calculates he for all values t at once for a particular d and moves forward in depth until it reaches D.
3. The third stage uses an MLP layer that takes the output of the ODESolver and projects it onto the required output space. The output y/ associated with an input zy and time is obtained as
M ML Opore (ht.D).
The architecture of the neural wave equation, forward pass and hidden layer interactions can be understood from Figure 3 The FDM method attempts to approximate the analytical solution to high degree of precision using a particular class of numerical solvers called adaptive step-size solvers Andersson et al. (2015). We used the adaptive step size solver based on Dopri45. It uses the RK-4 and RK-5 as the lower and higher-order solutions respectively. The pseudocode for the algorithm is provided in Appendix [A.10] The model parameters, including wave speed MLP parameters OMIP = (0pre post) and source function parameters 0g are learned using the loss function computed over the output observations in sequence and over all the sequences. For obtaining the gradients, we use an adjoint sensitivity method developed for PDEs, which works by converting the wave equation to system of linear 1st-order equations Choon et al. (2019) Lewis jet al. (006).
4.3 DISCUSSION
In a normal RNN architecture, the evolution of the hidden state dynamics is as follows: he, F(W,hy-1,d + Wa-1ha.d-1). So, the hidden state at point (t,, d) depends only on h and he.d-1- In the wave equation, the presence of the inte- gral over the source term from 0 to d ensures that each he. is modeled as a function of several hid-
den states. The trainable parameter с determines the number of the hidden states with depth less than d that contributes to the evolution of hi.d-
Sequence
Hid + Del.)
Laget
Most of the other works that combine neural ODE architecture with RNN use the neural ODE to predict the flow of hidden states over a con- tinuous time (Rubanova et al. (2019); Kidger et al. (2020)) However, they are still discrete in the depth direction. CDR-NDE (DR-NDaEAnnuaaa
with 90-40 and orear
10- Del.4)
hit- Dei.4e
d-Dd,
Mudi Deta
Inital Hidden Scane
(eDel
Anumasa al. (2023) addresses this by using a PDE based on heat equation.
Shukow
During our investigation, we note that the rea- son PDEs can be used to model sequence data lies in their analytical solution. The analyti- cal solution of hi,a where the evolution is gov- erned by a PDE will often incorporate term
Figure 3: Neural wave equation architecture con- sists of a shallow MLP over input, PDE solver, and a shallow MLP to produce the output.
like Joe (T()) J, rd)ddr. This implies that particular hidden state at arbitrary depth is affected directly by all the values of hidden states at a lower depth. Consider the last term in Equation [ which provides d' Alembert's solution for the wave equation. It considers all the source terms from the previous depths at each time point to compute the hidden state at the current depth. This is also true in the case of the Heat Equation. The analytical solution of the heat equation is given by separation of variable Widder (976))
00
an (0) exp (h) + f cxp (xhnltt)) (T))
ltt,, d) = Σ
(11)
where (t) = sin (nxt) and qm (d)= 1et F(t, d) (t) The presence of the negative exponential
term in the solution of the heat equation means that the effect of the hidden states located at lower
7

---

Published as a conference paper at ICLR 2025
depths is diminished while calculating the hidden states located higher depths. The wave equation does not suffer from this problem as can be observed from its analytical solution.
5 EXPERIMENTS
The performance of the proposed Neural wave models was assessed through experiments on datasets containing irregular sequence data such person activity recognition Kelly (2000), walker2d-v2 kinematic simulatior classification of social media posts Desczynscc et al. 2017 These models are benchmarked against baselines specifically developed for handling irregular sequence data. The experimented configuration includes setting the hidden state dimension 64 for all source functions, keeping a minibatch size of 256, use of the Adam optimizer, a learning rate of 5 x 10-3 and 200 training epochs. These configurations follow the guidelines as established in Lechner layer is a single layer with hidden dimension 64. The last MLP layer is also single layer with hidden dimension equal to the output size. We use the Tsit5 from the package torchdyn Poli et al. as our adaptive solver, which is an efficient reimplementation of the Dopri45 by the Julia Computing group Rackauckas & Nie (2017). The information about the step size and the ODESolvers that have been used for all our models and baselines is mentioned in Table 4 in Appendix A.13 Model training conducted on an Nvidia Tesla V-100 32GB GPU and an L4 GPU. In our evaluation, we measured the efficacy of our newly developed model against set of established baselines. These include GRU-ODE, CT-GRU, CT-RNN, GRU-D, Phased-LSTM, ODE-LSTM, bidirectional-RNN, RNN decay, Hawk-LSTM, Augmented LSTM, ODE-RNN, Neural CDE and CDR NDE models.
& Hasani (2020) PhysioNet sepsis prediction, and stance
Hasani (2020). The first MLP
5.1 RECOGNIZING PERSON ACTIVITY FROM IRREGULARLY SAMPLED SEMEISERIES
The dataset consists of sensor readings from four sensors attached to five individuals (ankle, chest, and belt) performing five activities. The objective to utilize this sensor data to categorize the performed activities, Initially containing II actívities, it was refined to 7 classes recommended by |Rubanova et al. (2019). Each recording step includes 7 values, indicating active classes and representing sensor data. Data is segmented into overlapping 32-step intervals with a 16-step overlap, yielding 7.769 training and 1,942 testing sequences. We evaluate our model against established baselines for irregularly sampled activity recognition Markelle Kelly) (2000). The neural CDE model achieves 75. 16% + 0.71 accuracy after 40 epochs. While GRU-based models perform best among baselines, they are surpassed by Neural Wave Equation variants. On average, the solver makes 32 function calls in the person dataset and 26 in the walker dataset.
In Table [ Column presents the test accuracy for all models trained on the person-activity recognition dataset. Notably our Neural Wave model Double Gating variant outperforms all the established baseline models with the highest test accuracy.
5.2 WALKER2D-V2 KINEMATIC SIMULATION.
The Walker2D dataset Lechner & Hasani 2020 is derived from simulations in the Walker2d-v2 OpenAl Gym environment, powered by the MuJoCo physics engine. The underlying motion of the walker is governed by continuous-time physical dynamics, simulating kinematic systems evolving smoothly over time. The training set was compiled through rollouts in the Walker2d-v2 environment under deterministic policy pre-trained via Proximal Policy Optimization, albeit employing non- recurrent policy framework. To achieve irregular sampling, 10% of the timesteps were omitted. The wereeeeeeeeeeee
data partitioned into 9,684 training sequences, 1,937 for testing, and 1.272 for validation.
We tested the performance of our model on irregularly sampled Column 3 of Table [ delineates the efficacy various models on the Walker2d dataset. Our proposed Neural Wave- Single MLP model outperforms all the basclines. We were unable to run the Neural CDE modelKidger et al. (2020) due the long duration required to complete one epoch. We suspect that the construction of the continuous path with cubic splines is bottleneck in the Neural CDE model, as increasing the sequence length and dimension of input features significantly slows it down. Even in the Person's activity dataset, the neural CDE model took 300 sec compared to 18-30 seconds by that of neural wave equation or 30 50 sees of CDR-NDE models. Computational complexity is discussed in detail in A.7
8

---

Published as a conference paper at ICLR 2025
Table 1: Column 2 outlines the test accuracy (mean standard deviation) of each model trained on the dataset titled Person Activity Recognition Markelle Markelle Kelly (2000). In Column 3, the Mean-square error (mean standard deviation) for the test data from models trained on the Walker2d dataset Lechner & Hasani 2020) is detailed. Column 4 and 5 show the AUC Performance of Models for Unseen Events in Stance ClassficationnLeon Derczynski Kochkina 2019). For all the datasets,
every model is trained for 5 times with 5 different seeds.
Model
Person Activity
Walker2d Test MSE
Sydneysiege AUC1
Charliehebdo AUC 1
Test-Accuracy f
Discrete Time Discrete Depth
RNN- MNNN-DecayPn Weinan 2017
78.74 +: 3,65
1.44 = 0.01
0.62 1 0.00
0.63 1 0.01
Bidirectional RaaeeeiiooonnNNNNNNNNNNN Schuster & Paliwal 1997
82.86 = 1.17
1.09 - 0.01
0.61 + 0.00
0.64 - 0.02
GRU-D Che et al.
2016
82.52 + 0.86
1.14 + 0.01
0.63 + 0.00
0.65 + 0.01
Phased-LS LST Nei al. 2016
83.34 = 0.59
1.10 = 0.01
0.58 = 0.01
0.60 = 0.01
Continuous Time Discrete Depth
CT-RNNiici ichi Funahashi & Nakamura
1993
82.32 = 0.83
1.25 = 0.03
0.56 1 0.00
0.61 1 0.01
ODE-RNN Rubanova al. (0199)
75.03 =: 1.87
1.88 - 0.05
0.55 + 0.00
0.57 0.03
ODE-LSTM[Lachnee & Hasani 2020
83.77 + 0.58
0.91 + 0.02
0.56 + 0.00
0.59 + 0.00
CT-GRU Mozer et al. 2017
83,93 + 0.86
1.22 1 0.01
0.63 1 0.01
0.65 1 0.02
GRU-ODE De Brouwer ct al.
2019
82.80 + 0.61
1.08 = 0.01
0.56 + 0.00
0.61 + 0.00
CT-LSTM Lechner & Hasami
2020
83.42 + 0.69
1.03 - 0.02
0.62 + 0.00
0.65 + 0.01
Continuous Time Continuous
Depth
CDR-NDE Anumasa et al.
2023
87.54 :: 0.34
0.97 : 0.04
0.57 E 0.02
0.55 : 0.01
CDR-NDE heat (Buler)
88.24 i 0.31
0.54 1 0.01
0.62 1 0.01
0.58 1 0.02
CDR-NDE-heat (Dopri5)
88.60 = 0.26
0.49 + 0.01
0.62 0.01
0.57 + 0.01
Neural Wave - Single GRU
88.52 + 0.34
0.49 - 0.01
0.60
0.63 1 0.02
Neural Wave - Single MLP
90.58 = 0.58
0.11 0.01
0.09
0.61 - 0.01
Neural Wave - Double Gating
93.62 1 0.38
0.16 1 0.01
0.61 0.01
0.62 1 0.02
Neural Wave - MLP+GRU
92.06 1 0.34
0.12 1 0.01
0.60 0.01
0.63 1 0.04
5.3 SEPSIS PREDICTION USING PHYSIONET 2019 DATA
We analyze a dataset initially used in the PhysioNet 2019 challenge Reyna et al. 20199 Goldberger al. (2000), focusing on sepsis prediction.
This dataset contains 40,335 sequences
Table 2: Test AUC (mean + standard deviation over five runs) for sepsis prediction on the PhysioNet.
of variable lengths, documenting pa- tient admissions in an intensive care unit (ICU), and includes five static features, such as patient age, well thirty- four dynamic features like Heart Rate, Blood pressure, etc. The measurements are taken at hourly intervals. A signifi- cant portion of the data is missing, with only 10.3% of the values being observed. Our analysis focuses hours of a patiends stay, addressing the binary classification task of predicting sepsis development throughout their en-
Model
Test AUC
GRU-ODE De Brouwer et al.
(0019
0.852 + 0,010
GRU-A/
0.878 0.006
GRU-D Che et al. (2016)
0.871 + 0.022
ODE-RNN ODE-RNN Rubanova of al. 2019
0,874 + 0.016
Neural CDE Kidger al. (2020
0,880 + 0.006
CDR-NDE-heat Anumasa et al. (0233)
0.880 + 0.000
the initial 72
Neural Wave Single GRU
0.885 + 0.003
Neural Wave Single MLP
0.885 + 0.001
Neural Wave Double Gating
0.890 + 0.006
Neural Wave MLP++GUU
0.889 + 0.003
tire stay. We divided our data into train,
validation and test split of 70%, 15 % and 15% respectively. We compared Neural Wave's per- formance against GRU-ODE, GRU-D, ODE-RNN, Neural CDH et al. (2020). CDR-NDE and GRU-AI, variant of GRU. that incorporates the time difference between observations as an additional input. We conduct experiments with various models considering the observational intensity. Observational intensity refers to the frequency of data observations, which can indicate the level of attention or concern, such as more frequent measurements for patients considered at higher risk ( more details mentioned in Section 3.5 and 3.6.). Table 2 iilllsstrats the findings, where we use AUC for evaluation due to the dataseds imbalance.
For the ODE-RNN, GRU-D, and GRU-A/ models, observational intensity is given by appending an tbserved/not-oobeeeee mask to the input at each observation. The Neural CDE, GRU-ODE and Neural Wave model use continuous, per-channel intensity as explained in Section 3.6 in Kidger

---

Published as a conference paper at ICLR 2025
et al. (0200. The results demonstrate that the proposed model, Neural Wave's Double gating provides the best performance, while other neural wave equation models are also equally competitive.
5.4 STANCE CLASSIHICATION
In practical scenarios, particularly on social media platforms like Twitter, tweets associated with specific event are posted at varying times, and the intervals between these tweets are not uniform. We evaluate the models based on their ability to classify the stance of social media posts, specifically using the Twitter datase(Leen Derczynski & Kochkina (2019), which includes rumors associated with eight events. Each event comprises a collection of tweets labeled as Support, Query, Deny, or Comment. To create a sequence data point, we randomly selected 10 tweets and then sorted them by observation time in the ascending order. To evaluate our models, we considered an unseen event prediction setup, where the model performance is evaluated the sequences from unseen event. Here, the model is trained on the sequences formed from data from all the events except test event and tested the sequences formed from the unseen test event. We selected two events: Sydneysiege and CharlieHebddo as the test event for our experiments.
In Table 1 Columns 4 and 5 showease the test AUC for unseen events for all models. We could not run the Neural CDE model due to lengthy epoch times, likely slowed by the cubic spline path construction, particularly as the input feature dimensions and sequence lengths increased. Despite the high dimensionality (316) of text embeddings in our stance classification data,neural wave equation models demonstrated robust performance. The stance classification task was intentionally chosen to check the performance of our model on high dimensional discrete system. Even if our model does not beat some of the baselines, the Neural Wave Model MLP + GRU remains competitive hence showing the robustness of our model.
5.5 ABLATION STUDIES
We conducted experiments with models considering a homogenous PDE with no source terms to understand the effect of source functions. The homogeneous neural wave equation model, without a source term, achieved a test accuracy of 51.73 % + 0.16 on Person Activity, a test MSE of 0.99 + 0.003 on Walker2D, and test AUC of 0.857 = 0.001 on Physionet Sepsis. We observe that the learnable source function helps to capture the dependencies present in the complex sequence modeling tasks more effectively and improves performance. From Table [ we observe that the performance of the single GRU neural wave equation is lower than the rest. This is attributed to the fact that in the single GRU model, the source function captures nonlinear dependency between two neighboring hidden states whereas in the rest of the models, the nonlinear dependency is captured between all the four neighboring hidden states present in the FDM discretization the wave equation. We study the memory consumption of our models on the PhysioNet data. Our models consume more memory, ranging from 1807MB to 21 2137MB, compared to the memory-efficient Neural CDE, which consumes up to 244MB. But, order of magnitude faster than neural CDE and are scalable to large sequence lengths. More details on time and memory complexity can be found in A.7.)
saw in the Person activity data, neural wave equations an
CONCLUSION, LIMITATIONS AND FUTURE WORK
In this work, we propose neural wave equation, a sequence model based on the non-homogeneous wave equation. We show that a non-homogeneous wave equation with a learnable source function a good fit for sequence modeling tasks involving irregularly sampled data. We establish that the analytical solution of a non homoooogggnsuus wave equation presents a way to implicitly model denser connections between hidden states. We empirically demonstrate this by comparing our model against several baselines and outperforming them in several real-world data sets. Neural wave equations have reasonable computational speed, however this comes at the cost of memory consumption. Studying the benefits of using partial differential equations for sequence modeling with theoretical rigor and finding the correct balance between memory and speed is left for future work.
10

---

Published as a conference paper at ICLR 2025
REFERENCES
Yahya Ali Abdulkadir et al. Comparison of finite difference schemes for the wave equation based on dispersion. Journal of Applied Mathematics and Physics, 3(11))1544, 2015.
Christian Andersson, Claus Führer, and Johan Akesson. Assimulo: A unified framework for [ODE) solvers. Mathematics and Computers in Simulation, 116(0):26 43, 2015, ISSN 0378-4754, doi: http:/dxx.oii.ogg/ 10. 10.11166..mm.. wat.mmmm00....0.mm
Srinivas Anumasa, Geetakrishnasai Gunapati, and P. K. Srijith. Continuous depth recurrent neu- ral differential equations. In Machine Learning and Knowledge Discovery in Databases: Re- search Track: European Conference, ECML PKDD 2023, Turin, Italy, September 18-22, 2023, Proceedings, Part 11, pp. 223-238, Berlin, Heidelberg, 2023. Springer- Verlag. ISBN 978-3- 031-43414-3. doi: 10.1007/978-3-000 3-031-43411--- 14. URL https: http:/o....g.....01 1007/ 978 3 031 001--44411 43415 4
Coryn Bailer-jones, David Mackay, and Philip Withers. A recurrent neural network for modelling dynamical systems. Network: Computation in Neural Systems, 9, 08 2002. doi: 10.1088/
0954-898X_94 104X_99440000
Johannes Brandstetter, Daniel E Worralll and Max Welling. Message passing neural pde solvers, In International Conference on Learning Representations, 2021.
Zhengping Che, Sanjay Purushotham, Kyunghyun Cho, David A. Sontag, and Yan Liu, Recurrent neural networks for multivariate time series with missing values. CoRR, 0bs//100601865. 2016.
URL
org/ abs /1e.//666.... 01865
Ricky T. Q. Chen, Yulia Rubanova, Jesse Bettencourt, and David K Duvenaud. Neural ordinary differential equations. S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, and
R. Garnett (eds.), Advances in Neural Information Processing Systems, volume 31. Curran Asso- ciates, Inc., 2018. BRLLhtttps///rroeeeeeiie neuri ps wttes.cc/ppeeee cc/paper files/paper) 2018 2005/111//55550b1eee622222663333
Paper.pdE
Yuqi Chen, Kan Ren, Yansen Wang, Yuchen Fang, Weiwei Sun, and Dongsheng Li. Contiformer: Continuuus-time transformer for irregular time series modeling. In Thirty-sevnnth Conference on Neural Information Processing Systems, 2023. U https: httpss//opppevieeeee wereenreview..ee/foorrree
id Id-YJD24F2AZu
Kyunghyun Cho, Bart van Merriënboer, Caglar Gulcehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. Learning phrase representations using RNN decoder-decoder for statistical machine translation. In Alessandro Moschitti, Bo Pang, and Walter Daelemans (eds.), Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 1724-1734, Doha, Qatar, October 2014. Association for Computational Linguistics. doi: 10.1115/11/114-111 1179. URL httpps///bellatthoooooo..///////
Loy Kak Choon et al. On numerical methods for second-order nonlinear ordinary differential equations (odes): A reduction system of first-order odes. Universiti Malaysia Terengganu Journal of Undergraduate Research, 1(4):1-8, 2019.
Edward De Brouwer, Jaak Simm, Adam Arany, and Yves Moreau. Gru-ode-bart bayes: Continuous modeling of sporadically-observe. time series. Advances neural information processing systems, 32, 2019.
Leon Derezynski, Kalina Bontcheva, Maria Liakata, Rob Procter, Geraldine Wong Sak Hoi, and Arkaitz Zubiaga. CamEval-2017 Task 8: RumourEval: Determining rumour veracity and support for rumours. In Steven Bethard, Marine Carpuat, Marianna Apidianaki, Saif M. Mohammad, Daniel Cer, and David Jurgens (eds.), Proceedings of the I Ith International Workshop on Semantic Evalua- tion (SemEval-2011 20117) pp. 69-76, Vancouver, Canada, August 2017. Association for Computational Linguistics. doi: 10.18653/v11 10/653//1/111720000/ 1/017-2006. BRLLhtpps:00 httttp::/accaattiin www.llannhooooy..roooooo -006 http0//////tttpp////aaaaaoooooo
Lawrence C. Evans. Partial differential equations. American Mathematical Society, Providence, R.I., 2010. ISBN 3780821849743 0821849743.

---

Published as a conference paper at ICLR 2025
Stanley J Farlow. Partial differential equations for scientists and engineers. Courier Corporation, 1993.
Ary Goldberger, Luís Amaral, Leon Glass, Jeffrey Hausdorff, Plamen Ivanov, Roger Mark, Joseph Mietus, George Moody, Chung- Thung-Kang Peng. and H. Stanley. Physiobank, physiotoolkit, and physionet: : Components of a new research resource for complex physiologic signals. Circulation, 101:E215-20, 07 2000. doi: 10.1161/01.CIR 1II. 101.23.e215.
Albert Gu, Karan Goel, and Christopher Re. Efficiently modeling long sequences with structured
state spaces. International Conference on Learning Representations, 2022. URL https?
//openreview netriieweee//eeeeeemm ottprr/wweee.ee.eoorraaaadaaaaa /ttp://ummmid-yLLoooi-.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2016.
Sepp Hochreiter and Jürgen Schmidhuber. Long short-term memory. Neural compulation, 9(8): 1735-1780, 1997.
Yihao Hu, Tong Zhao, Shixin Xú, Lizhen Lin, and Zhiliang Xu. Neural-pde: a rnn based neural network for solving time dependent pdes. Commun. Inf. Syst., 22:223-245, 2020. URL https
/empi.smmant semant cacho lar wrt/Corpgs
Tyler W. Hughes, Ian A. D. Williamson, Momchil Minkov, and Shanhui Fan. Wave physics as an analog recurrent neural network. Science Advances, 5(12))caa66946, 2019. doi: 12.1/26//.. 694
aay6946. http///tt///////.oooooooor/o//o/
Ken ichi Funahashi and Yuichi Nakamura. Approximation of dynamical systems by continuous time recurrent neural networks. Neural Networks, 6(6):801-806, 1993. ISSN 0893-6080. doi: hitps:
//doi.org/10. http:///w......00000000000001010
URL http:///wwww.scceeceeeeee... com/ com/
Sheo Jhin, Heęjoo Shin, Sujie Kim, Seoyoung Hong, Minju Jo, Solhee Park, Noscong Park, Se- ungheom Lee, Hwiyoung Maeng. and Seungmin Jeon. Attentive neural controlled differential equations for time-series classification and forecasting. Knowledge and Information Systems, doi: 00.1007/101115-03--09977 URL htttps://doi.rgg/
66(3))188511115, 2024.
1007/s10115 023-019999000 01977-5
Sheo Yon Jhin, Minju Jo, Taeyong Kong, Jinsung Jeon, and Noseong Park. ACE-NODE: attentive co-evolving neural ordinary differential equations. CoRR, abs/2105. 14953,,, 2021. URL https
Patrick Kidger, James Morrill, James Foster, and Terry Lyons. Neural controlled differential equations for inregular time series. In H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin (eds.), Advances in Neural Information Processing Systems, volume 33, pp. 6696-6707. Curran Asso- ciates, Inc., 2020. URL http:::/ppoeeeinsss.euui... ccppppr fi (alle/pppprr/
Mathias Lechner and Ramin Hasani. Learning long-term dependencies in eeggularly-sampllee time series. arXiv preprint arXiv:2006.044: 08 2020.
Arkaitz Zubiaga Ahmet Aker Kalina Bontcheva Maria Liakata Leon Derczynski, Genevieve Gorrell and Elena Kochkina. Rumoureval 2019 data. Figshare, 2019. URL/tttss://EIghhaae com/
articles/Ruul htt.llee/e/moorraaaaaaaa 2019 2011__daaa//8455800000
John M. Lewis, S. Lakshmivarahan, and Sudarshan Dhall. First-order adjoint method: nonlinear dynamics, pp. 401-421. Encyclopedia of Mathematics and its Applications. Cambridge University Press, 2006.
Kolby Nottingham Markelle Kelly, Rachel Longjohn. The uci machine learning repository, hupps//archivv.ic.uui... 2000.
Hongyuan Mei and Jason M Eisner. The neural hawkes process: A neurally self celf-modulatin multivariate point process. Advances in neural information processing systems, 30, 2017.
12

---

Published as a conference paper at ICLR 2025
Michael C Mozer, Denis Kazakov, and Robert V Lindsey. Discrete event, continuous time arXiv preprint arxiv: 2017.
Daniel Neil, Michael Pfeiffer, and Shih-Chii Liu. Phased Istm: Accelerating recurrent network training for long or event-based sequences. In D. Lee, M. Sugiyama, U. Luxburg. Guyon, and R. Garnett (eds.), Advances in Neural Information Processing Systems, volume 29, Curran Asso- files/paper)
ciates, Inc., 2016. URL /tpppppss/pppooeeisssssssouuuupp
Michael Poli, Stefano Massaroli, Atsushi Yamashita, Hajime Asama, Jinkyoo Park, and Stefano Ermon. Torchdyn: Implicit models and neural numerical methods pytorch.
Christopher Rackauckas and Qing Nie. Differentialequations..] performant and feature-rich ecosystem for solving differential equations in julia. Journal of Open Research Software, 5(1):15, 2017.
M. Raissi, P. Perdikaris, and G.E. Karniadakis. Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. Journal of Computational Physics, 378:686-707, 2019. ISSN 0021-9991. doi: https://
1oo..rg//1..1011//jpp..20000 10.045. URL https: //wp:::://w.././////// /wwww www.sceciiieeeneeeoeoecc.ee rect. com/ science/ atticlle/p11/000219999
Matthew Reyna, Christopher Josef, Russell Jeter, Supreeth Shashikumar, M Brandon Westover, Shamim Nemati, Gari Clifford, and Ashish Sharma. Early prediction of sepsis from clinical data: The physionet/coomputing in cardiology challenge 2019. Critical Care Medicine, 48:1, 12 2019. doi: 10.1007/CCC.0000000000000
Yulia Rubanova, Ricky T. Q. Chen, and David K Duvenaud. Latent ordinary differential equations for irreuulall--sampled time series. In H. Wallach, H. Larochelle, A. Beygelzimer, F. d'Alché-BBB Bucc E. Fox, and R. Garnett (eds.), Advances in Neural Information Processing Systems, volume 32, Cur- ran Associates, Inc., 2019. URL https: http::///////ooooccceccoooeoe/ge neg neur http://s.nnnnrsp.eeeeaeaeeeeeeee cc/paper forrrffllee files/
David E. Rumelhart, Geoffrey E. Hinton, and Ronald J. Williams. Learning representations by pock-propagating errors. Nature, 323 (6088))333 536, Oct 1986. ISSN 1476-4687. doi: 10.1038/
323533a0. URL httph:t////. /ttp:://do...o orw. 000.11033322333
M. Schuster and K.K. Paliwal. Bidirectional recurrent neural networks. IEEE Transactions on Signal Processing, 45(11):2673-26 2681, 1997. doi: 10.1109/78.6500 6650999....
S.L Sobolev. Partial Differential Equation in Mathematical Physics. Dover Publications.
Alexander Strauss. Partial Differential Equations: An Introduction. John Wiley Sons.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, É ukasz Kaiser, and Illia Polosukhin. Attention is all you need. In I. Guyon, U. Von Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett (eds.), Ad- vances in Neural Information Processing Systems, volume 30. Curran Associates, Inc.. 2017. URL https: htttt::://proceedinng file/ fteee355ee24333 4354 7dee91 9 2448855a-Paapes .pff]
files//paerr/2177
E Weinan. A proposal on machine learning via dynamical systems. Communications in Mathematics and Staristics, 5:1-11, 02 2017. doi: 10.10077/40334-007-000--
David Vernon Widder. The heat equation, volume 67. Academic Press, 1976.
Kirill Zubov, Zoe McCarthy, Yingbo Ma. Francesco Calisto, Valerio Pagliarino, Simone Azeglio,
Luca Bottero, Emmanuel Luján, Valentin Sulzer, Ashutosh Bharambe, al. Neuralpde: Au- tomating physics-informen neural networks (pinns) with error approximations. arXiv e-prints, pp. arXiv:2107, 2021.
13

---

Published as a conference paper at ICLR 2025
A APPENDIX / SUPPLEMENTAL MATERIAL
Most of the derivations regarding wave and heat equation can be found in more detail in Sobolev
Strauss and Evans (2010)
A.1 SOLUTION OF WAVE EQUATION
Wet Uzz + 21) where
u(0, x) fz) and u,(0 g(z)
We break it into two separate problems.
Utt = 2n where
v(0, x) f(x) and (0, g(z)
and
Wxz = c² Uax + F(t, x) where
w0, x) O and w. (0, x)) = 0
In such a case, the sum of the solutions of the above equations will give us the solution of the wave equation. To solve the first part Introduce new variables and n):
{ = x - dt
n = x + cl
(12)
Then, the partial derivatives transform follows:
ag 0
On
0
+
+
(13)
Ox dx 0
0z dn
05
On'
0
0 0
On 0
0
0
+
(14)
Ot
Ot
0
Ot 0q
0E
On
The second derivatives are:
02
0
2
82
82
+ ÿ2
Ox2
=
00
+
On
=
0
+2
dgdn 0m22
(15)
02
0
0
2
82
8² 82
(16)
012
a
8n
dçan
0mp
Substitute into the Wave Equation
Substitute these into the wave equation:
2 02u
8v
8 v
182v
82
8v
2
2
(17)
DEan
7m
Simplify this to:
22u
0 - 4c2
(18)
agdm
This implies:
82
agan 0.
(19)
The solution of the above equation can be found by integrating twice. So, we know that the solution will be of the form
v(x = A(x - B(x
-Bx +c)
(20)
Now, A(x) fz())) उे See g(s) and B(x) =f((x))) + t So 9())ds solves the first part of the
wave equation. The solution then can be written down as
v(x, t) f(r at) f(r ct)
tot
g(s)ds
(21)
2
2c x-ot
To solve the second part, let consider another initial value formulation of the equation.
14

---

Published as a conference paper at ICLR 2025
Tu c'raz where r(T, 0 and (=(T,;;;) F(r,
In this case, w(t,z) x) = So r(T, a; d) solves the second part of the wave equation. Using the Leibnitz rule for differentiation under integral sign, we write,
wy = r(t, v; + Lo relt, = So rı(t,, x; r)
Whr - (x, r F(t, + So rea (1((,
and we have
Ugn = So (z, T)) a So (t(
putting the values Waz and Wer in the equation, we get
W wr xt,
(22)
By D' Alemberds formula, the solution of this initial value problem is
1 x=+((--r)
r(1,,;T) = 2c z-c(t-r)
F(r,n)dn
(23)
and
w(t, x) ("
(
2--(((-)) F(T,n))d
(24)
Adding the solutions of both the parts, we get the solutions wave equations as
u((,2)) f(x et) + f(x ct)
r+
1 /" (xtct->)
g())ds +
2c. Jo (x-c(t-))
1(r,
(25)
x--t
A.2 FDM DISCRETIZATION OF WAVE EQUATION
8hyyd
8
(26)
dd
Ōt²
Discretization of the ID Wave equation is as follows:
(h(r.d-Ad) 2h h1ad-aa) (hr-ared 2hr + h14 0
(27)
A
A?
(M(r,4sa))
(2 h4d - hadd-aa)
(he- 2he,d hetand)
(28)
A3
A3
A?
A2 2hr + hrdd
hi,d+ 2hr hed Du
(29)
A
A.3 SOLUTION OF HEAT EQUATION
kuzz + Qx, f) where u(0,t) = 0, u(L, = u(x, 0) = f(x)
Using separation of variables (assuming that the solution is of the form u(x,() X())T(!) leads to an eigenvalue problem
6" Xo 0, ((0) 0, (L)
The eigenfunctions and eigenvalues are given by
(x = sin D52
An = (Z))
L
15

---

Published as a conference paper at ICLR 2025
Leds assume that the solution is off the form
u(2,t)= Σ
an (t())n(z))
(30)
n=1l
We write
20 f(x) u(x, = Σ
an (0)
(31)
20
((x,t) = - 2o (t)
(32)
The coefficients of the above equations are solved using the Fourier series. We expand
(x t) Σi d' (z), Usn (z, -i an (t)))
Inserting into the heat equation we get,
Ur = kuzz + Q(x,t) ΣS (t) -k Ei an (z) + Σn (x)
(t)()) + kt) (t)
(33)
Solving the above ODE, we get
(x(T))xphh))) a, (t) exp a0) + /' (T( exp
(34)
-hhnt + 1' 9n (u(T)e (halt)-) dr
(t) = (0)e kdat
(35)
So, we write the solution follows
= Σ Jan (0)e khat + (( ( -hha(l-r)dddonnn
u(x, 1) = E an (n)))(()))
(36)
A.4 SOLVING A 2ND ORDER EQUATION AS A SYSTEM OF 1ST ORDER EQUATIONS
In this case, since the entire sequence fed once as input, we know the values of yee So, we can write the 2nd-order wave equation system of 1st-order odes.
Y =
y y/
(37)
Yı = 1 1 Y
0
à yeu F(x,t)
(38)
A.5 EXAMPLE OF SOLUTION PROBLEM IN SOLVER
Vew y(0, t) = v(L = 0, y(x, 0) = vo(2,)) =(x)
Du
By D' A'Ambeert's 's formula, we know the solution is
v(z,t) f(x al) + f(x + cl +
1 rta
g(s)ds
(39)
2c xz-a
In the question, f(x) (T) and g(x) sin(TT) So. the solution is
v(x,t)= t) sin t) + sint(x + t) + 2 x-t
1 z+2
sin( )dz = (co(s))) sin(Tt)
(40)
2
16

---

Published as a conference paper at ICLR 2025
Analytical Solution
Numerical Solutieniterchdym))
BmOr
075
0.75
0.1
04
025
0.25
Postion b4)
0.5
D.2
0.50
0.00
0.4
-0.50
-0.50
-0.5
000
9.0
-1.20
60
00
-1.00
40
3.0
8.0
-1.00
Lime
Time
lime
Figure 4: The leftmost figure is analytical solution over a grid, middle figure numerical solution, the last figure the error between the analytical and numerical solution. We notice that the error between analytical and numerical solution is of the order 1e which was used as the relative error tolerance.
A.6 FAST IMPLEMENTATION
ODESolvers are not suited to handle a sequence of vector data. The initial value condition in neural wave equation, h is N x M matrix where N is the sequence length and M is the number of input features, While training with batches. B being the batch size, the initial value condition becomes
N M matrix. To efficiently utilize GPUs during training, we collapse the batch and the sequence length into a single dimension resulting in (BN) x M matrix. During training, the input sequence a tuple (batchsizeX scquencelength Xninut features). However, neural ODE solvers can't It handle such data directly. One way to overcome this problem is to loop over batch. However, not GPU efficient. The elegant solution is to collapse batch size and sequence length into a single dimension and convert the 3d input array into a 2d array. The input matrix looks like-
[X] X2, Xx, X2
X7 is the input vector corresponding to ith batch at time sequence j. We append this matrix the start and the end by repeating the first and last time sequence of every batch. For example if assume the batch size to be 3, and sequence length to be 3, we will have the following matrix
[1 X2,
Let this matrix be called h. Note that shifting this matrix lets us calculate the finite difference terms easily. For example
X1
X
X1
X
X
X
AAYTKT X
X
X
X
X
4
26[b: -b] Al::
26]
2
+
X
(41)
X
X
X
X
We can then append the necessary boundary conditions to the calculated matrix and pass it into the solver again. However, the shifting of matrix for calculating finite difference and source terms must be done carefully so that the batches does not mix amongst themselves.
17

---

Published as a conference paper at ICLR 2025
A.7 COMPUTATIONAL COMPLEXITY
We also analysed the computational complexity of neural wave equation with other baselines. Our implementation indeed more memory but is much faster compared the exisiting methods.
Model
Memory (in MB)
Speed (epoch/s)
CTRNN
321
259
ODE-LSTM
348
82.39
CTGRU
662
11.45
GRUODE
161
11.93
Average
373
90.445
BIRNN
162
30.66
GRUD
48
13.22
PHASED
38
9.4
Average
82.67
17.76
Neural Wave
1972
8.66
Neural Wave with checkpointing
400
12
Table 3: Computational Complexity Analysis of Neural Wave Equation with other models
It is important to emphasize that the observed speed-memory tradeoff arises from implementation technique rather than being an inherent property of the model. Specifically, in most RNN variants with ODE solvers, it is necessary to loop the sequence dimension because ODE solvers typically cannot handle 3-dimensional data directly:To To address this limitation, we collapsed the batch and sequence dimensions into a single dimension. This approach enabled to utilize the GPU more efficiently, significantly improving speed by eliminating the need for looping. However, this opti- mization leads higher peak memory allocation, as the entire collapsed batch-sequence matrix must fit in memory during computation. If memory constraints arise, gradient checkpointing can reduce the neural wave equation's memory usage by 80%, albeit with 50% increase in training time due to recomputation overhead.
A.8 LOSS FUNCTION AND TRAINING
The loss function used to train the model depends on the problem. We use cross-entropy loss for classification problems and mean-squared error for regression problems. The loss is function of the MLP parameters OMLP- = (Opre: postt and source function parameters 0g. Since we use the neural ODE framework during training, we prefer the adjoint sensitivity method over the traditional backpropagation as it offers memory efficiency Chen et al. (2018) Choon et al. (2019). Even though mention ODESolvers solve the wave equation, there
practical problems
may face 8d in the
during training. The ODESolvers normally have an ODEFunc argument which is @h(t.d)
dd. wave equation. However, the wave equation is 2nd-order PDE, and hence @l(1.d) is not known.
required to convert the wave equation to a system of linear 1st-order equations. We define H (d) = [he,d- 8h..d and consider a first-order system H = G(l.. (d), d). Note that here, G is a function of both h1 and 0h.. dd
Ad
Following this, we define the adjoint state as a(d) aH( and an
ODE which satisfies.
dH
dd - @G (d).
(42)
0H(d)
we find (d) by making an extra call to the ODESolver with db
aH(D) as the initial condition.
A.9 COMPARISON
D' D'AEMBEERT SOLUTION OF THE WAVE EQUATION
The wave equation with a source term is given by
1 (f cd) + f(t ed)) + (
p p(tcll-s)
h(t, d) where O),, f(t) and F(.) is the source term.
2c Jrol- F(s,
(43)
2
18

---

Published as a conference paper at ICLR 2025
COMPARISON WITH RNN TYPE ARCHITECTURES
In normal RNN architecture, the evolution of the hidden state dynamics is as follows:
h, = F(W, Wa-1hh,a-1)
(44)
So, the hidden state at point (t, d) depends only ht- and dt.d- 1. In the wave equation, the presence of the integral over the source term from to d ensures that each hed is modeled as function of several hidden states. The trainable parameter e determines the number of the hidden states with depth < d that contributes to the evolution of h..d-
COMPARISON WITH HEAT EQUATION
has been shown that heat equation can also be used to model sequence data. The discretization of the heat equation is:
Ad
1(t, d - Aa)
A, [hyt [lyt- A.,d) 2h(e,d) + hie-s.,d + h((ed)
(45)
Straight up comparing with the discretization of the wave equation, we notice that the (() 85) term is absent in the heat equation. The analytical solution of the heat equation is:
h(t.d) = Σ (0) exp (kknn)) + S0" (x()))xpp khh((t--)) expl-kdn(tt dr dr)
(46)
where on sin (nxt)
T Au 1o F(,, d))
Esponeriis niall Decay for Dillerent Values of and
n-1 ked
n-1,
n-1
0.-
0.0
Figure 5; The rate of exponential decay in heat equation corresponding to different depth and heat diffusivity.
The presence of the negative exponential term the solution the heat equation means that the effect of the hidden states located at lower depths is diminished while calculating the hidden states located at higher depths. The wave equation does not have this problem. In the case of the depth of the model being shallow, heat and wave equations show similar performance.
A. NEURAL WAVE EQUATION ALGORITHM
The algorithm for neural wave equation is provided. The architecture diagram of the encoder-decoder version also provided.
19

---

Published as a conference paper at ICLR 2025
Algorithm 1 Neural Wave Equation
function PRE-NN(input data)
Compute the initial condition using a neural network
h(t, 0) <- Pinpui data)
return (t, 0)
end function
function ODEFuNC(source function, current state)
Compute the second-order time derivative:
second derivative 4- Finite Difference Formula + source function
Store the current state as it will be required to calculate the FDM for next state. return second derivative
end function
function NEURAL Wiviiiiitill condition)
Initialize second-order ODE solver
Solve the wave equation using the solver return the solution at final time end function
function Post-NN(solution at D)
Apply neural network for post-processing output MLP(solution at D)
return output
end function
A.11 ADAPTIVE STEP SIZE SOLVERS
(t) = ft, v(t)), v(a) y/a
(47)
The exact solution nth point is Yn, and the numerical approximation is yn. Approximation of yn using RK-4 method yields
RK4 = Fn. + +(4 Un
(48)
while RK-5 yields,
I RK5 = + 0(4,) RK4 yn
(49)
RK5 1
(50)
Given relative error tolerance, we can calculate the required step size A, by solving:
A
(51)
tol
A
tol
A
(52)
A.12 BOUNDARY CONDITIONS
For numerically solving partial differential equation ODE, need boundary conditions initial value conditions. Since, we are using numerical solver, we also need a list of initial value conditions. Here, we aim discuss in detail how the initial value conditions can be initialized in case of the neural wave equation. First let us take a look at the update equation again. For simplification, we look the discretization equation of homogenous wave equation.
hrd+ = 2h h4
[hets.d 2ha + h4 A.-A,.d]
We have sequence of data at different time points to begin with. 1 corresponds these data. (please note that instead of the raw data, we often pass them through a MLP to get the initial values hy This is mainly reduce increase the dimension of the data.) Now, let how hg,0+ gets calculated. The update equation will read as follows
A A
hyo+a, 2h,0 h4,0-1. +
[h144..0 2hz h4-A..o
The h4 A. value is not available to us and we can use a value our choice as the boundary condition. Please note that this is same specifying the partial derivative of h wrt t' at point 0. If
20

---

Published as a conference paper at ICLR 2025
write custom implementation of solver, initializing boundary values like dirichlet, neumann or robins. However, our implementation, we used the torchdyn solver which takes the derivative at time 0 as O. Now, look at the second set of points where we need boundary conditions. Let's take look at the update rule for ho,d+A,
follow one of the common scheme for
A A
ha Mo.++A. 2ha,d ho.d
+ (ho+ 2ha + ho ho-A..d)
Here, A is again not known to us and we need to tackle it just like the above case. Again note that this is same as specifying the derivative wrt at 0 and use 0 in our implementation. However, one can come up with custom boundary condition according to the problem in hand.
Visualization of the boundary conditions
3.0
2.5
NO.
(P)
1.0
0.5
16-101 h10.0)
h11.0
-6.5
-1.0
Figure 6: Visualization the boundary conditions for solving the PDE. The red cells represent the boundary conditions that must be defined to compute the values the blue cells (ho, and ho.2). These boundary conditions correspond to unknown values, such as 1-1,1, h-1,0. and ho.- which are set based on the problem or solver design (e.g.. Dirichlet or Neumann boundary conditions). The black cells denote the values that are either given (e.g., initial conditions) or computed as part of the numerical solution process.
A.13 EXPERIMENTS
Table 4 outlines the numerical methods selected for each model. The Neural Wave model employs the Dopri5/Tsit5 method, setting the absolute and relative tolerance levels to le 33 A scheduled learning rate decay strategy is implemented, with decay coefficient 0.1, activated at the 100th epoch.
Table ODE solvers used for different RNODE models. For the Neural Wave model using Dopri5, the absolute and relative tolerance values are le and 1e-3 respectively.
Model
ODE-Solver
Time-step Ratio
CT-RNN iihi Funahashi & Nakamura 1993)
4-th order Kunge-Kutta
1/3
ODE-RNN Rubanova et al.
(2019)
4-th order Runge-Kutta
1/3
GRU-ODEDe Brouwer et (2019)
Explicit Euler
1/4
ODE-LSTM Lechner & Hasam
(202
Explicit Euler
1/4
Neural-CDE Kidger el 22020
Dopri5
CDR-NDE Anumasa et al. (2023)
Explicit Euler
1/2
CDR-NDE heat Anumasa al. 2023
Dopri5
Neural Wave
Dopri5/Tsit5
WALKER V2 KINEMATICS
The output is a 17-dimensional vector at each time point, we visualize the comparison between the ground truth and the predicted values across several randomly selected time points over test samples.
21

---

Published as a conference paper at ICLR 2025
Seperee
Se-an of Septine
yay Comple Cepprneat
Vare Du Searce)
(ratuee you
Com shil Capureall
Comoraal Creaett
Comple IIE Sourme
Figure 7: Comparison between ground truth and predicted position of the observation space of the walker2d kinematics model.
STANCE CLASSIFICATION
Table 5: Test AUC Performance for Models on Seen Events
Model
AUC (Sydneysiege event)
AUC (Charliehebdo event)
CT-RNN
0.57 = 0.00
0.63 0.01
ODE-RNN
0.55 #.01
0.59 # 0.02
ODE-LSTM
0,56 656
0.61 = 0.01
CT-GRU
0.64 0.01
0.67 0.01
RNN-Decay
0.63
0.67 # 0.02
Bidirectional-NNN
0.01
0.67 0.00
GRU-D
0.01
0.69 = 0.01
Phased-LSTM
0.01
0.64
GRU-ODE
0.56
0.63 0.01
CT-LSTM
0.01
0.66
Augmeente-LLSTM
0.64
0.68
CDR-NDE
0.01
0.00
CDR-NDE--eat(Eule)
0.01
0.01
CDR-NDE--hea((Dppii)
0.01
0.65 0.01
Neural Wave Single GRU
0.00
0.62 = 0.01
Neural Wave Single MLP
0.60
0.65 = 0.01
Neural Wave Double Gating
0.60 = 0.01
0.60 0.01
Neural Wave MLP+GRU
0.61 + 0.02
0.64 = 0.01
SENSITIVITY OF WAVE SPEED PARAMETER
We conducted a small study the homogeneous model excluding the neural net- work, as it likely compensates for suboptimal initial values of e on the Walker dataset.
This allowed us to isolate the impact of e without the corrective influence of trainable network. The results A.13) show that test loss varies with c. For the model where e is trainable, the MSE is 0.99. This suggests that the flexibility of trainable mitigates sensitivity. improving overall robustness.
c
Test Loss
0.1
2.728
0.5
2.429
1.281
3
2.329
6.243
22


Reviews:
[{"review_id": "d7jhhD1jYS", "reviewer": "Reviewer_uqft", "summary": "This paper introduces the neural wave equation, a novel approach to handling irregularly sampled sequence data by modeling hidden state evolution using wave equations. The authors propose using a non-homogeneous wave equation with a neural network-parameterized source function to capture sequence dependencies. The method allows continuous modeling across both time and depth dimensions, addressing limitations of existing approaches that use discrete depth transformations. The authors demonstrate the effectiveness of their approach through experiments on various sequence labeling tasks, including person activity recognition, Walker2d kinematic simulation, sepsis prediction, and stance classification, showing competitive or superior performance compared to existing baselines.", "questions": "- How sensitive is the model to the choice of wave speed parameter c?\n- Could you elaborate on how boundary conditions are handled in practice?\n- How does the computational complexity compare to existing methods?\n- Have you explored any techniques to reduce memory consumption?\n- How does the model perform with very long sequences?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "- Strong theoretical foundation with clear mathematical derivations\n- Comprehensive experimental evaluation across diverse datasets\n- Thoughtful comparison with existing approaches, especially heat equation-based methods\n- Detailed ablation studies showing the importance of different components", "weaknesses": "- The motivation for choosing wave equations could be stronger\n- Memory consumption issues are noted but not thoroughly addressed\n- The connection between theoretical advantages and empirical improvements could be clearer\n- Limited discussion of computational complexity trade-offs\n- Some implementation details about boundary conditions could be more explicit", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "dK0a7VpvSj", "reviewer": "Reviewer_HEKw", "summary": "This paper introduces a novel architecture for latent space modeling and prediction using the Wave equation. Motivated by the homogeneous and non-homogeneous wave equations, the authors proposed neural wave equation to model the hidden layer transformations. In the experiments, the authors tested and showed superior performances on person activity recognition, walker2d-v2, sepsis, and social media posts datasets.", "questions": "1. Can the neural wave equation be applied only to irregularly sampled sequence data, or could it also work in more general sequence modeling settings? What is the reason for only putting this framework into the irregularly sampled sequence data setting? \n\n2. The reviewer suggests improving the quality of Figure 1. Some fonts are too small to read, so please use PDF or SVG formats to enhance the visual clarity. Additionally, could the authors consider redesigning the figure? The current design makes it difficult to distinguish between straight and curved lines. While it makes sense that curved lines represent continuity, there may be a clearer way to convey this distinction.\n\n3. Practically, is this algorithm fast to run? \n\n4. What will be the limitation for incorporating this structure of wave equation. What happens if the real latent dynamics follows another PDE that cannot be modeled as a wave equation? \n\n5. What is the reason that in some experiments, the performance is suboptimal compared to baseline methods? \n\n6. In page 6, Line 302, please consider changing the '-' to \":\". \n\n7. The algorithm description of neural wave equation would require a significant rewrite. The current version is hard to be understood. \n\n8. The appendix provides an introduction of the solution of wave equation. The reviewer believes that the authors should provide proper citations here and it is very important.", "limitations": "", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 2, "contribution": 4, "strengths": "This paper proposed a very innovative idea which utilizes the wave equation to model the latent space. From the reviewer's perspective, this is good since this kind of structure could model physical/natural phenomenon(s) better. And the model is still generalizable from the non-homogeneous formulation with F(z,t). \n\nExperimentally, neural wave equations also dominated the baseline methods in most of the metrics. \n\nIn general, this paper introduces a very innovative method which requires further study, and could have huge potential impact to the field.", "weaknesses": "There are several weaknesses of this manuscript. \n\n1. The presentation of this manuscript is not optimal. For example, the Figure 1 requires some imagination to understand the design to distinguish the three methods. \n\n2. The experiments are nice, but the neural wave equation might not perform as well as the baseline methods for some data. This could potentially be because the form of the wave equation acts as a strong prior that might not fit the data. It would be nice to understand further the limitation of this framework. The ablation study could potentially be extended.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "B3u2Ys0PRQ", "reviewer": "Reviewer_Fbp1", "summary": "This paper addresses the limitation of fixed architectural depth in modeling irregularly sampled sequence data. A neural wave equation method is proposed to 1) continuously model the hidden states across time and depth and 2) allow denser connections across hidden states. Empirical results show improved performance compared to baseline methods.", "questions": "Please check the questions listed in the Weaknesses section.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "1. The problem of fixed assumption in architectural depth is important. \n2. The method of using the wave equation to continuously model the time and depth is creative. Developing the neural network based on the analytical solution to the wave equation allows more dependencies in hidden states.\n3. Experiments were performed on different scenarios to show the effectiveness of the proposed method.", "weaknesses": "1. The introduction and related work need to be better structured. For instance, what is the concept of hidden state depth in real-world problems? Could the authors elaborate on why PDE-based models are a better model for continuous modeling on depth? What is the benefit of the wave equation over the other PDEs? The authors could provide more details in both sections for a smoother transition from the problem setting to the proposed method.\n2. Section 3 and 4 could be better presented. For example, instead of presenting Figure 2, the authors could provide a comparison between ODE-RNN and the proposed method in terms of how they model in depth. In fact, Figure 2 should be in an ablation study and needs a better explanation of how the depth affects the three comparison models. Also, using $t’$ for depth is confusing given that $t$ is used for time. For Figure 3, the authors should explain the blue arrows with the description in Section 4.2.\n3. The experiments should be performed and analyzed in more detail. For each experiment, could the authors highlight the purpose, such as how the depth of hidden states or the number of missing data affects the model performance? What is the depth of each group of data? Could the authors group the comparison baselines as the categories in Figure 1? Table 1 could be separated by the experiments. Also, it would be better to have visualizations about the predicted sequences vs the ground truth to better understand the improvement in metrics.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)