C OO P REDICT : C OOPERATIVE D IFFERENTIAL G AMES FOR T IME S ERIES P REDICTION
Anonymous authors
Paper under double-blind review
A BSTRACT
Modeling time series dynamics with neural differential equations has become a major line of research that opened new ways to handle various real-world scenarios ( e .
g ., missing observations, irregular times).
Despite the progress, most existing methods still face challenges in providing an explainable rationale on temporal association, which tells how past observations affect future states.
To tackle this challenge, we introduce novel multi-agent based neural stochastic differential equations and analyze the time series prediction through the lens of cooperative differential games .
Our framework provides an explainable method that can reveal the underlying temporal relevance of the data and fully utilizes this information to systemically solve the prediction problem.
We develop the gradient descent based deep neural fictitious play to approximate the Nash equilibrium and theoretical results assure the convergence.
Throughout the experiments on various datasets, we demonstrate the superiority of our framework over all the benchmarks in modeling time series prediction by capitalizing on the underlying temporal dynamics without any inductive bias.
An ablation study shows that neural agents of the proposed framework learn intrinsic temporal relevance to predict accurate time series.
I NTRODUCTION
The key challenge of time series prediction is understanding how past observations contribute to the future and building a probabilistic model that intrinsically captures the temporal correlation.
A model is required to differentiate the relative importance of past observations on the future time series such that the redundant information is well-suppressed during inference.
To address this challenge, a series of previous works followed the philosophy of defining information redundancy by adopting a concept known as temporal decay (Che et al., 2018; Mei & Eisner, 2017).
Such an inductive bias relies on the belief that the influence of past observations exponentially decreases over time.
However, due to the strong assumption, any method built upon the fixed inductive bias may fail to properly capture different temporal dynamics in various real-world scenarios.
A promising direction to tackle this challenge is to learn the temporal correlation structure from data.
One candidate for learning the implicit relation is recurrent neural networks (RNNs) which subsequently encode the past observations into a latent space with a gating mechanism.
Due to the implicitness of latent encoding, the model provides ambiguous explanations on how observations are related to the prediction.
Furthermore, conventional RNNs are incapable of capturing the irregularly sampled time series, which is common in many real-world applications.
Recently, remarkable advances have been made to model underlying continuous temporal dynamics utilizing neural differential equations (Chen et al., 2018;
Li et al., 2020; Kidger et al., 2020) (NDEs).
For instance, a stream of research (Rubanova et al., 2019; Deng et al., 2021; Schirmer et al., 2022) has focused on overcoming the initial condition problem of conventional differential equation models by encoding temporal dynamics into the latent space.
A set of past observations are incorporated into latent representations and fully utilized for time series prediction.
However, owing to the inexplicable relation between temporal states in the latent representations, these methods are still incapable of showing the explicit rationale that can reveal the impact of observations on the future.
In this paper, we propose a novel framework, which we call CooPredict , built upon a game-theoretic formalism to model temporal dynamics of time series data.
More specifically, we extend the conventional differential equation (DE) to the multi - agent counterpart for decomposing the observational
[Figure]
time series.
Each agent individually encodes the impact of a partial observation into an underlying stochastic trajectory and interacts with each other to extract meaningful information to predict the future.
We formulate the collaboration among agents from the view of cooperative differential games (Leitmann, 1974; Staatz, 1983; Sexton, 1986) to integrate the individual information and adaptively balance the importance of each agent.
As a result of the differential game, cooperative agents achieve the Nash equilibrium , and agree to suppress redundant observations and to highlight the contribution of important observations for accurate prediction.
To solve the differential game, we propose a novel deep learning-based algorithm to conduct the fictitious play .
During the process, our method applies the gradient descent scheme to solve decoupled optimal control problems in a tractable and parallel way.
Furthermore, we provide theoretical foundations upon the Feynman-Kac formalism for solving optimal control problems and theoretical analysis to guarantee the convergence of the proposed algorithm.
Unlike the existing methods, our framework explicitly provides a clear explanation of how the past is related to the future in time series.
We validate our method through experiments on multiple synthetic and real-world datasets that cover various types of temporal dynamics.
Our method outperforms state-of-the-art DE-based methods by capturing the underlying temporal dynamics without any inductive bias via a novel game-theoretic framework.
M ULTI - AGENT C OOPERATIVE P REDICTION
Problem Setup.
Consider a general time series forecasting set-up where each instance of a time series is defined on the entire time interval [0 , T ] comprising of observable past ( i .
e ., O ) and target future intervals ( i .
e ., T ).
We assume that the irregularly sampled observations are collected at time stamps
O = { s 1 , . . .
, s I } where s i ∈
O for i ∈ { 1 , . .
, I } .
Then, we focus on building a probabilistic model that can predict future values at any time t ∈ T based on the set of past observations in O .
2. M ULTI - AGENT N EURAL SDE S
Suppose we have a set of I number of past observations for a given time series, i .
e ., { y s i } s i ∈O .
The primary object of our interest is the set of stochastic processes
X α t =
[ X 1 , α t , . . .
, X I, α t ] parameterized by multiple control agents α =
[ α 1 , . .
, α I ] , which is defined as a collection of solutions to the following multi-agent stochastic differential equations (MaSDEs)
(Øksendal & Sulem, 2007) 1 :
[Equation]
MaSDEs
Here, { W i t } are independent d -dimensional Wiener processes, and b and σ are drift and diffusion functions, respectively.
The control agent α i : [0 , T ] × R d × Θ → R d , which we refer to as a neural agent , is modeled as a neural network α i :=
α i ( t, X i, α t ; θ i ) parameterized by θ i .
Each neural agent takes input a spatio-temporal tuple ( t, X i, α t ) and produces infinitesimally successive outputs
( t + dt, X i, α t + dt ) .
We define a neural agent’s decision as a continuous stochastic trajectory over the future interval T , i .
e ., { X i, α t } t ∈ T .
By setting the past observation y s i as the initial condition , the decision of a neural agent can be considered as an information propagator that can explicitly represent the impact of the corresponding past observation via the conditional expression p ( X t ∈ T | y s ∈ O ) ; see
Figure 1-(a) for a pictorial illustration.
Temporal Aggregation.
Since each neural agent can only represent the individual impact of partial information, we aggregate the individual decisions made by each neural agent to capitalize on the temporal dynamics available from the entire set of past observations.
More specifically, we introduce the predictor Λ α t that produces the future value at time t as a weighted sum of the individual decisions as the following:
[Equation]
Here, we introduce the decision attention
A α which provides the relative importance of each decision, i .
e .,
A α : T × R d × Θ → ∆ I where ∆ I := { a ∈ [0 , 1] I | 1 ⊤ a = 1 } is a I -simplex and A α =
[ A α 1 1 , . .
A α
I
I ] .
Given the decision attention, the predictor Λ α t produces a temporally aggregated decision from the decisions made by the individual agents.
The purpose of decision attention is to differentiate the associated impact of stochastic trajectories on predicting the future values in a data-driven fashion.
It is worth highlighting that the proposed predictor becomes equivalent to the temporal average suggested by (Park et al., 2022) when the decision attention assigns uniform weights irrespective of spatio-temporal variable, i .
e .,
A α i
i
= 1 /I .
Stochastic Optimal Control .
To train neural agents in MaSDEs, we adopt the stochastic optimal control (Yeung & Petrosjan, 2006; Carmona & Delarue, 2018) as our central methodology to formally define an objective functional J i .
Specifically, each neural agent is given a behavioral rule as follows:
[Equation]
where h i (cid:16) t, X i, α t , X ( − i ) , α t , [ α i , α
[Equation]
−
[Equation] [Equation]
=
[Equation]
and Ψ i ( X i, α T ) =
1 2 || X
i, α T || 2 .
Objective Functional
Here, X ( − i ) , α s ∈ R d ( I − 1) and α ( − i )
∈
A are the sets of other agents’ decisions and actions, respectively.
The cost function h i is designed to minimize the distance between the prediction ( i .
e ., Λ α t ) and the target time series ( i .
e ., y t ).
This encourages each neural agent to make contributions in the temporal aggregation under the goal of accurate time series prediction.
The terminal cost function Ψ
i regularizes decisions at the terminal state and plays a central role in the Feynman-Kac formalism to solve the cooperative game.
We will further elaborate the details in Section 3.
Given the environment ( i .
e ., the entire interacting objects), each neural agent changes its action to find a value function V i that defines the optimal state of actions:
[Equation]
]) , (4) where the optimal control problem in (4) takes arbitrary actions α ( − i ) made by other agents.
To solve (4), we propose a gradient descent-based method motivated from the fictitious play, which we will further elaborate in Section 3.
2. C OOPERATIVE D IFFERENTIAL G AMES
If the past observation y s is highly associated with the future event y t ( i .
e ., p ( y t ∈ T | y s ∈ O ) ↑ ) , it is reasonable to pay high attention ( i .
e .,
A α ( t, · ) ↑ ) to capitalize on the observation y s .
Unfortunately, determining the optimal attention is a daunting task as the temporal correlation is intrinsically datadependent and not given a priori in general.
Hence, the method requires a unified structure that (i) can systemically extract information from data to adjust the decision attention
A α based on the relative importance of past observations and (ii) can provide a framework on the coalition of the interacting agents for the proper coordination.
To address the above requirements, we propose a novel framework, which we call CooPredict, to formalize the time series prediction problem as a cooperative differential games between multiple agents.
Cooperative differential games have been extensively studied in many disciplines (Perelman et al., 2011; Jørgensen et al., 2010; Yeung & Petrosyan, 2012) to investigate the group rationality of agents sharing the same goal.
Inspired by these works, we design a cooperative differential game that provides cooperative prediction by achieving optimally balanced decision attention among neural agents whose shared goal is an accurate prediction.
The critical point is that the neural agent supporting a non-informative observation voluntarily sacrifices its own cost ( i .
e ., V i ↑ ) by decreasing the influence on temporal aggregation
so that the impact of other informative observations ( i .
e .
, V i ̸ =
j ↓ ) can be further emphasized.
Such a cooperative behavior is mutually advantageous to the interacting agents as it ultimately achieves the performance improvement.
Figure 1-(c) shows the conceptual illustration of the proposed cooperative game where the homogeneous decision attentions are gradually transformed into the equilibrium state achieved by enthusiastic agents following the group rationality.
2. C OOPERATIVE P REDICTION VIA N ASH E QUILIBRIUM
For a mathematical derivation of the optimal decision states, we introduce an important type of equilibrium states called Nash Equilibrium as follows:
Definition 1.
(Markovian ϵ -Nash Equilibrium)
Let us consider a set of closed-feedback type
Markovian controller parameterized by neural networks α ∗ := { α ∗ i } i ∈{ 1 ,...,I } that induce value functions.
For the arbitrary actions β taken in action set A , we say that neural agents are in the ϵ -Nash Equilibrium if the following inequality is satisfied:
[Equation]
In the state of equilibrium, any non-optimal action β produces worse cost (with a margin ϵ :
=
{ ϵ i } i ∈ I ) than taking the optimal actions α ∗ .
Here, the marginal constant ϵ represents a degree of capacity for neural agents to achieve the perfect equilibrium.
The inequality in (5) describes an optimal state of interacting agents having no incentive to change their action.
While each neural agent mutually finds the best response to the environment made by the optimal colleagues, the entire cost is comprehensively minimized and a more accurate prediction can be obtained.
Further, the following relation implies that the equilibrium state achieved in (5) is indeed a result of cooperation:
[Equation]
where J co is the objective functional that describes the coalition cost (Yeung & Petrosjan, 2006) of the cooperative neural agents.
The inequality in (6) can be easily shown by the algebraic property of cost functions as the following:
[Equation]
Fictitious Play.
In order to achieve the Nash equilibrium for multi-agents, the previous interest on considering separate and individual objectives in (4) shifts to an interacting scenario.
From this new perspective, we transform the original value function that only considers insensitive feedback into a new one that can consider optimal colleagues’ feedback driven by their contribution to the prediction:
[Equation]
Since finding the Nash equilibrium is PPAD hard (Daskalakis et al., 2009; Goldberg, 2011), it is intractable to directly compute the equilibrium owing to a large number of agents considered in time series prediction problems of our experiments ( i .
e .
, I ≈ 50 ).
Thus, for a tractable computation, we instead rely on an alternative dynamical formulation called fictitious play (Cardaliaguet & Hadikhanloo, 2017).
The fictitious play is an iterative procedure that decouples interacting agents and separately solves the stochastic optimal control.
The central idea is first to share public information about the entire system and then to decouple value functions by solving individual Hamiltonian-Jacobi-Bellman equations (HJBEs) in Section 3.
D EEP N EURAL F ICTITIOUS P LAY
In this section, we suggest the novel deep learning framework for the fictitious play.
To this goal, we start by introducing the set of adjoint equations called the forward-backward stochastic differential equations (FBSDEs) (Carmona, 2016) as a probabilistic framework to tackle optimal control problems of obtaining value functions.
Theorem 1.
(Non-linear Feynman Kac theorem Pham (2015)).
Let us consider closed-feedback type
Markovian controls α t .
We define the triplet ( X α t , Y α t , Z α t ) that constitutes the system of SDEs:
[Equation]
FBSDEs
where H i is the stochastic Hamiltonian system.
Then, auxiliary adjoint variables Y i, α t and Z i, α t can be reformulated as follows:
[Equation]
where we denote F i, α t
[Equation]
To solve the optimal control problem, the existing method (Han & Hu, 2020) considers quadratic linear forms in the forward dynamics and its corresponding global convex Hamiltonian system ( i .
e ., ∇ α
i H i = 0) .
Despite the numerical advantage in obtaining an optimal control, the restricted form of linear SDEs limits the usage of their method on high-level applications such as time series prediction.
This shortcoming motivates us to utilize neural networks when modeling control agents for enough expressivity.
A critical issue, here, is to approximate the solution to non-convex type
HJBEs that support the neural networks structure.
To tackle this issue, we reformulate the equivalent adjoint problem to find value functions:
[Equation]
Instead of searching the analytic solution to HJBEs ( i .
e .,
A i ≈ ker ∇ α
i H i ) , the newly suggested adjoint equation in (11) enables us to apply the gradient descent scheme to find optimal actions.
In this case, our method searches the parameter space to obtain the optimal admissible action ( i .
e .,
A i ≈ α i ( · , · ; ker ∇ θ H
i
[ α i ( θ i )]))
meaning that we focus on the local Nash equilibrium.
Deep Neural Fictitious Play.
Let us define an entire set of action profiles as α m at stage m ∈ { 1 , . . .
, M } .
Then, our method iteratively conducts the following two-step optimization:
• Information Distribution.
At the beginning of the m -th stage, each neural agent publicly share the information about the entire system ( α m − 1 , X α ,m − 1 t ) from the ( m − 1) -th stage.
In this case, we can define the decoupled value function for individual neural agents as follows:
[Equation] [Equation]
where X ¯
α m − 1 t indicates the set of decisions driven by action profiles ¯ α m − 1 of the ( m − 1) -th stage.
In this step, each neural agent recognizes response of colleagues towards the environment and makes its relative decision.
Note that each agent solves a minimization problem according to the fixed environment of the previous stage.
• Decoupled Gradient Descent.
After defining the decoupled problems in (12), each neural agent solves its individual adjoint problem in (11) and updates its parameters as
[Equation]
where γ m is a learning rate of the optimizer at the m -th stage.
In (13), gradient descent is applied to minimize the separate cost J i,m while fixing the parameters of colleagues ¯ θ m ( − i ) .
After solving (13) for I neural agents in a parallel way, we collect the entire action profiles and update the public information for the next stage, i .
e ., α m +
i ←
α
m
i ( ·
[Equation] [Equation]
θ +
[Equation]
Convergence of the Fictitious Play.
In the previous contents, we suggest an alternative way to approximate the Nash equilibrium via the gradient descent scheme.
In what follows, we will show the existence of an action set A related to the marginal constant ϵ :=
{ ϵ i } i ∈{ 1 ,...,I } in the inequality of (5).
Proposition 1.
(informal)
There exists a action set α ( ϵ ) ∈
A where the Markovian ϵ -Nash equilibrium in (5) can be achieved.
Moreover, the obtained optimal actions α m +
i i ∈{ 1 , ··· ,I } ∈
α ( ϵ ) from deep neural fictitious play preserves the stochastic optimality and the solution to HJBEs 2 :
[Equation]
where H is the stochastic Hamiltonian system, F t is an adjoint variable related to σ and b , and
Hess V t denotes the Hessian of function V t .
Proposition 1 reveals that the proposed fictitious play preserves the stochastic optimality during the two-step optimization by showing the existence of local solutions to the decoupled Hamiltonian systems.
The crucial point is that the local action set is related to the capacity of neural agents and neural networks with a large capacity assures small marginal values ϵ for accurate prediction.
Corollary 1.
( Convergence of Predictor )
There exist a constant C that is proportional to I , T and
Lipschitz constants L b , L α for { b i , α i } i ∈{ 1 ,...,I } such that the following relation holds:
[Equation]
where Σ := sup t ∈ T E (cid:2)(cid:13)(cid:13) σ T σ ( t, · ) (cid:13)(cid:13) F (cid:3) is the maximal bound with respect to quadratic variations of neural agents.
Corollary 1 shows that the convergence of the temporal aggregation made by parameterized neural agents is highly dependent on the dynamics of the gradient descent during the fictitious play.
If one reformulates the proposed cooperative game into the zero-sum type counterpart, the opposed direction of gradients induced by conflict agents can cause the divergent learning dynamics and unpleasant results.
In other words, the proposed gradient descent based fictitious play fits to the cooperative scenario where neural agents pursue a shared goal.
Appendix A.3 provides the numerical analysis on the behavior of neural agents under the cooperation/competition to elucidate how the proposed fictitious play operates in different game scenarios.
R ELATED W ORK
Neural DEs for Time Series.
In the pioneering works (Dupont et al., 2019; Rubanova et al., 2019), the general framework of encoding the complex time series into the latent space was first introduced to improve the representational power of conventional Neural ODE.
Further, latent SDEs (Li et al., 2020) was proposed to enrich conventional deterministic models by considering a stochastic component ( e . g .,
Wiener process).
Neural RDE (Morrill et al., 2021) exploited the representation of log-signatures of successive time series.
CLPF (Deng et al., 2021) combined two distinctive ideas, continuous SDE and normalizing flow, to model continuous latent flows.
CRU (Schirmer et al., 2022) extended Kalman filters to model-based continuous-discrete filters and showed the relation to neural SDEs.
Neural
Laplace (Holt et al., 2022) showed novel interpretation on time series modeling by introducing the
Laplace representation that generalizes conventional DEs in the frequency domain.
CSDE-TP (Park et al., 2022) suggested a different perspective by adopting a control-theoretic interpretation of time series prediction tasks to obtain the optimal paths driven by neural agents.
The notable distinction is that our proposed framework generalizes the equiconcentration of temporal attention in CSDE-TP to enhance the aggregated information by balancing the relative importance.
Inductive Bias on Temporal States.
In probabilistic modeling for time series analysis, existing methods assumed a strong inductive bias on temporal states.
The conditional future state p ( T | O ) given past observations { y s i } s i ∈
O is related to the temporal difference between T and O .
For example, GRU-D (Che et al., 2018) suggested a temporal decay module in the recurrent model that can exponentially decrease the influence of past observations.
Another strand of works (Mei & Eisner, [Table] 2017; Zuo et al., 2020; Chen et al., 2020) directly parameterized temporal point processes ( e .
g .,
Hawkes process) that intrinsically assume the temporal decay with exponential intensity kernels.
More recently, Neural CDE (Kidger et al., 2020) applied the cubic spline that combines information of temporally adjacent observations to enhance the representational power of the law data.
To the best of our knowledge, we propose the first DE-based framework that fully utilizes the temporal correlation without making any inductive bias on the temporal states.
E XPERIMENTS
Benchmarks.
We compared CooPredict against a broad range of DE based continuous dynamical models.
The dynamic models include GRU-D (Che et al., 2018), Latent ODE (Rubanova et al., 2019), Latent SDE (Li et al., 2020), NJ-ODE (Herrera et al., 2021), Res-Flow (Biloš et al., 2021),
GRU-Flow (Biloš et al., 2021), CRU (Schirmer et al., 2022), Neural Laplace (Holt et al., 2022), and
CSDE-TP (Park et al., 2022).
We implemented the benchmarks using open-source codes published by authors except for Latent SDE whose decoder architecture was replaced from Neural ODE to SDE (Li et al., 2020).
Further details on hyper-parameters and implementation can be found in Appendix A.6.
Experimental Settings.
We evaluated the time series prediction performance of CooPredict and the benchmarks on multiple real-world datasets: BAQD (Zhang et al., 2017), Speech (Warden, 2018), and Physionet (Silva et al., 2012).
More detailed descriptions of these datasets are provided in
Appendix A.6.2.
We split each time series in the interval [0 , T ] into two sub-intervals: the first 80% as the observation interval, i .
e ., O = [0 , 0 .
8 T ] , and the remaining 20% as the prediction interval, i .
e ., T = [0 .
8 T, T ] .
We split time series samples into two halves as training/evaluation sets in
Physionet.
For BAQD and Speech datasets, we divided time series samples into 80/20 training/testing splits for training and evaluation, respectively.
For fair comparisons, we normalized data features by utilizing a min-max normalization to ensure all data features lie within a unit cube [0 , 1] d .
We used temporally averaged mean square errors (MSEs) and negative log-likelihoods (NLLs) as the performance metrics.
To evaluate the performance of the benchmarks, we followed an identical protocol suggested by (Rubanova et al., 2019).
5. T IME S ERIES P REDICTION
Table 1 reports the performance comparison of our method and benchmarks.
As can be seen in the table, our model significantly outperforms all the benchmarks for the evaluated datasets by a large margin.
Notably, most of the existing methods based on latent encoding fail to make accurate predictions.
We argue that the reason for such performance degradation is the absence of the calibration process of training data.
More specifically, in prior works, the past observations are indiscriminately encoded in the latent space without awareness of temporal uncertainty of irregular time series.
For that reason, the model is vulnerable to the temporally noisy environment during testing, which may cause a large generalization gap.
In contrast, neural agents in our method actively make cooperative decisions to calibrate the vicious effect of unseen data.
If the agent detects harmful signals, they voluntarily restrain the decision to follow the group rationality ( i .
e ., accurate prediction).
Eventually, the cooperative group robustly suppresses the non-informative temporal dynamics in the inference.
Qualitative analysis on the robustness to out-of-distribution noises is provided in the following subsection.
When compared to the baseline model, i .
e ., CSDE-TP, the key distinction of our method is the ability to discriminate the importance of past observations.
More specifically, the baseline model, i .
e .,
CSDE-TP, disregards the effect of temporal association in the model design and simply leverages uniform attention to predict future time series.
Contrarily, our method delicately captures the intrinsic temporal association by learning from data such that the cooperative group adaptively filters out the impact of redundant and noisy signals.
This results in a significant performance improvement over
CSDE-TP ( ≈ 20%) .
We further investigate how informative the learned temporal associations is in the next subsection.
5. A BLATION S TUDY
In this subsection, we provide ablation studies that model four distinct scenarios each of which mimics different types of temporal dynamics in real-world time series: decay, impulse, delay, and noisy observation.
For each scenario, we investigate the advantages of the proposed cooperative prediction and the expressivity of decision attention learned without any induction bias.
(A) Temporal Decay.
Figure 2 shows that our decision attention infers temporal decay .
The agent α ∗voluntarily decreases its own attention so that the observation at the farthest past merely contributes to the inferred prediction.
Such a phenomenon can be clearly observed at the prediction times proximate to the beginning of the prediction interval showing that the rational group assigns high attention to s 35 .
Contrarily, the decision attention becomes less distinguishable when predicting the distant future.
This is remarkable since the cooperative agents reach the inductive bias ( i .
e ., p ( y t ∈ T | y s ∈ O )
∝ e −| t − s | ) a posteriori without any prior knowledge.
Furthermore, the performance is drastically worsened after excluding the agent α ∗ 35 (green) in the decision-making whereas no meaningful performance drop can be observed after excluding agent α ∗ 1 (red), clearly showing that the decision X 1 , α ∗ t is less informative than others.
(B) Temporal Impulse.
Modeling physical phenomenon with point processes ( e .
g .
, Hawkes process) is common in many domains including finance for a high-frequency market (Bacry et al., 2015) and geology for earthquake (Ogata, 1998).
In this experiment, we utilized Gaussian-impulse noises to mimic such discrete noisy events occurring at random times 3 .
Figure 3 illustrates the results on the proposed decision attention of neural agents.
Given impulse signals in the observation interval O , neural agents accurately restore random peaks.
Interestingly, neural agents { α 5 , α 17 , α 25 } that encode the past impulse signals focus on the times at which impulse noise occurs.
In other words, neural agents emulate stochastic peaks by learning intrinsic temporal correlations ( i .
e ., p ( y t ∈ T | y s ∈ O ) , t, s ∼
Pois ) between the random occurrence times of the past and the future impulse signals.
[Table]
(C) Temporal Delay.
In this experiment, we consider a dynamical system based on Mackey–Glass DDE (Mackey & Glass, 1977).
The simulated trajectory is highly sensitive to given initial conditions, which influence the future values in a time-delayed manner.
Thus, capturing the long-term temporal dependency is the key for accurate time series prediction.
In Table 2, we compare our method to benchmarks reported in (Holt et al., 2022) using the same experimental setting.
The result shows that our method outperforms all the evaluated benchmarks and effectively captures the delayed effect.
(D) Robustness to OOD Noise.
To show the robustness of our method to the out-of-distribution noise unseen during the training time, we set an experiment suggested in (Deng et al., 2021).
During the test time, we generate a set of time stamps from a Poisson process { τ } ∼ Pois ( λ ) ∈ O .
Then, the original values at the sampled time stamps are replaced by a constant value in the range of [0 . 0 , 1 . 0] .
We set the intensity level λ for the Poisson process to 7 .
in all experiments.
Figure 4 shows that our method provides the most robust performance that outperforms the prior works, while other methods often fail as the noise level increases.
This is because our method can compensate for the information loss by relying on the decisions of the remaining agents.
That is, when an observation is regarded as a noisy signal, our decision attention simply filters out the impact of that observation when predicting the future values.
Contrarily, existing methods ( e . g ., Latent ODE) are incapable of dealing with the unseen noises since there is no mechanism that can calibrate the impact of those noisy observations.
Figure 4 highlights that the existing methods lose the expressivity due to increased noise levels while our method maintains superior performance even under high noise levels.
Learning Temporal Relevance from Data .
Based on the investigations of the four scenarios, we conclude that the proposed neural agents learn different types of temporal dynamics from the data by cooperatively pursuing the group goal.
The rational group filters out the influence of observations that are redundant for predicting future states.
This encourages neural agents to focus more on the informative signals for accurate predictions by capitalizing on the data statistics.
C ONCLUSION AND F UTURE W ORK
In this paper, we proposed a novel framework for time series prediction as an application of cooperative differential games.
The new formulation is built upon the multi-agent dynamics called MaSDEs where each individual agent encodes the partial information from past observations.
Under the shared goal, neural agents collaborate to achieve the Nash equilibrium which describes the optimal action states that provide the most accurate future prediction.
Theoretical analysis shows the convergence of the proposed novel fictitious play highlighting the effectiveness of the proposed cooperative game.
Future Work.
Beyond the basic form of objective functional, the behavioral rule for neural agents can be delicately redesigned for a specified goal of interest.
For example, one may extend the proposed framework to the cooperative mean-field game (Carmona & Delarue, 2018) where infinitely many agents are considered.
In this direction, the system can provide mathematical formalism to understand the individual influences of an infinitely large number of observations.
This direction may open a new way to understand the long-term dependency problem of modern recurrent networks.
Further, we hope this can bring the application of recent studies on mean-field fictitious play (Perrin et al., 2020;
Guo et al., 2019; Min & Hu, 2021) to time series analysis.
A A PPENDIX A. M ATHEMATICAL B ACKGROUND
In this section
[Equation]
we briefly recall some basic elements of stochastic optimal controls needed in the paper.
We start by introducing the backward stochastic differential equation used in Section 3.
In particular, the adjoint variables ( Y i, α t , Z i, α t ) are derived from the classical theory of non-linear
Feynman-Kac formula (Carmona & Delarue, 2018):
[Equation]
where the stochastic Hamiltonian system with uncontrolled volatility for i -th agent is defined as follows:
[Equation]
For simplicity
[Equation]
we define the auxiliary function F having following form:
[Equation]
Next
[Equation]
we introduce the diriftless stochastic differential equation:
[Equation]
Then, the Girsanov’s theorem gives the following Radon-Nikodym derivative:
[Equation]
where we denote M − T :
=
[ M − 1 ] T for any inevitable matrix M .
Note that the regularity conditions (H1) are essential to assure the existence of the Doléans-Dade exponential ε which defines the stochastic exponential for the (local) martingale dM
[Equation]
The second equality holds since M i
=
and the quadratic variation is calculated as [ M i , M i ] t = (cid:82) t
| F i, α | 2 ds < ∞ .
Then, by definition, the transformed representation W it in following equation is also a Wiener process with respect to the probability measure P .
[Equation]
Identically, the above relation can be rewritten as following differential form:
[Equation]
Notably, we can restore the proposed controlled neural SDE from driftless SDE in (A.4) by the relation between W it and B it shown in (A.8).
[Equation]
This relation gives the explicit form of adjoint dynamics as follows:
Y i, α t :
[Equation]
In the last equality, the Brownian motion is changed from B is to W is , and the generic form of BSDE for tuple ( P , W it ) is presented.
Since the third term in last line (cid:82) Z i, α s · dW is is a martingale with respect to the measure P , we can identify the conditional expectation with original objective functional
J i in (3) as follows:
[Equation]
where ˜ F t is the augmented filtration according to the Brownian motion B it .
A. P ROOFS
Assumptions.
Throughout the appendix, we make following assumptions for the proof.
(H1)
σ − T ( t, x, α ( · , · ; θ ))
b ( t, x, α ( · , · ; θ )) is uniformly bounded, twice differentiable, Lipschitz continuous.
(H2)
The k -th derivatives of neural agents with respect to both spatial and temporal variables are Lipschitz continuous to the parameter space,
[Equation]
for all i ∈ { 1 , . . .
, I } , 0 ≤ k ≤ 2 , ∀ θ, ˜ θ ∈ Θ .
(H3)
The expectation of Frobenius norm of adjoint variable Z t ∈
Sym I + 4 is bounded, E ∥
Z t ∥ F <D 2 .
Lemma 1.
(Grönwall’s Inequality (Jacod & Shiryaev, 2013))
The left inequality induces the inequality in right-hand side:
[Equation]
A.2. P ROOF OF P ROPOSITION
The first step to showing the convergence of neural agents towards Nash equilibrium is to obtain the deviation of the adjoint variable Y t during the fictitious play.
For this, we first introduce the HJBE of decoupled SDE system.
In particular, the proposed system trains neural agents to solve the individual decoupled HJBE at each stage m :
[Equation]
where the cost functional at the next stage ( i .
e ., m + 1) is related to the optimal actions in previous stage that minimizes the decoupled Hamiltonian in (A.14).
Then, one of our interest is to investigate the deviation of Hamiltonian system:
[Equation]
where the deviation shows the difference between Hamiltonian systems that are derived by neural agents lying in optimal and sub-optimal regions, respectively.
Similarly
[Equation]
we define the deviations for both adjoint variables:
[Equation]
Following by the notations in (A.16) and the equality (A.10), the deviation for the adjoint variable ( i .
e ., δY i,mt ) at stage m can be represented as the following Itô’s differential:
[Equation]
Next, we evaluate the squared norm of δ Y ( · ) t to investigate the convergence of objective functional
[Equation]
[Table]
representation A t
[Equation]
where µ A t and σ A t are the terms corresponding to bounded variations and local martingales of process A t , respectively.
By taking the expectation on both sides of (A.18)
[Equation]
we obtain the following result:
[Equation]
Owing to the characteristic of backward SDE ( i .
e .
, Y m t ) , the integral sign is reversed in the second term in (A.20).
The expectation with respect to the local martingale term vanishes as set of Wiener processes B t are related to Q .
Since BSDE imposes the terminal constraint Y mT =
Ψ m , Y ∗ T =
Ψ ∗ , the following result is given by Lipschitzness of Ψ as:
[Equation]
This shows the inequality in (A.20).
By rearranging the relation and applying Hölder’s inequality we have
[Equation]
Finally
[Equation]
we apply Grönwall’s inequality in Lemma 1 to above inequality, we obtain
[Equation]
Now, our next step is to bound the right-hand side of (A.23).
We start by estimating an upper bound of the following time averaged mean-squared Hamiltonian (A.1):
[Equation]
For this
[Equation]
we rearrange the Hamiltonian deviation δ H mt by inserting two terms δ F mt · δ Z mt and δ h mt .
In this case, Hamiltonian deviation is rewritten as
[Equation]
Here, we define the Lipschitz constants of three different objects as follows:
[Equation] [Equation]
By the Lipschitz continuity defined above (A.26)
[Equation]
one can obtain the squared norm of the Hamiltonian deviation:
[Equation]
Consider we have a proper M = C 1 D 1 D 2 ∨ 2( C 1 D 2 ∨ C 3 ) ∨
I ( C 1 D 2 ∨ C 3 ) from the definitions of pre-determined constants C 1 , C 2 , C 3 , D 1 and D 2 .
Then, the above inequality can be rewritten in a compact form:
[Equation]
Note that the equality (cid:13)(cid:13)(cid:13) [ β i , α m ( − i ) ] − α m, ∗ (cid:13)(cid:13)(cid:13) =
(cid:13)(cid:13) β i − α m, ∗ i (cid:13)(cid:13) holds for an arbitrary action β ∈ A .
Finally, we conclude the result (A.28) by showing following inequalities:
[Equation]
In previous contents, the detailed convergent states of adjoint variable Y ∗ t are not specified.
To continue our discussion from (A.28), we develop a gradient descent-based update rule for training neural agents.
For any neural parameters θ ∈ Θ , we assume that the i -th agent’s action θ i → α i ( · , · ; θ i ) ≜
α i ( θ i ) lies in the compact subset
A ∈ A ( ϵ ) ∈
L 2 ([0 , T ] × R d ) ( i .
e ., (cid:82) ||
α i ( t, x ; θ i ) ||
2 dtdp t ( x ) dx < ∞ ) for some ϵ > 0 .
Then, we introduce the following notations:
[Equation]
where the auxiliary notation u ∈ R + is an indicator for gradient descent steps.
Similarly, we define the notations ( Y i,m,k , Y m,k ) and ( H i,m,k , H m,k ) .
Let us define the operator B : N + → L 2 ( T × R d ) as B [ θ mi ( k )]
:= α m,
k +1 i , and if k > K , then we denote α
m +1 i :
= α m +1 ,k =
i .
By Lipschitz continuity of neural agents, we have
[Equation]
where ˜ L = max i L i .
In the fourth inequality, the probability measure for the integration is switched from Q to P with the Radon-Nikodym d P /d Q .
Since the cost function is uniformly bounded with the vanishing derivative of terminal cost ( i
[Equation]
e ., ∂ θ
i g i
( X T ) = 0) , the expectation is well-defined.
It is worth noting that the filtration F t contains the information of past observations { y s i } as we impose constraints to satisfy X i,αs i =
y s i almost surely for neural agents.
As a next step
[Equation]
we evaluate the upper bound of mean-squared evaluation as follows:
[Equation] [Equation]
Let us assume that there exist representations ˆ b
[Equation]
ˆ α such that b ( t, X t , α i ) =
ˆ b ( t, X t )
α i ( t, X t ; θ i ) and
A i ( t, X t ; θ i ) =
ˆ A ( t ) α
i ( t, X t ; θ i ) .
This brings the evaluations of the following two norm bounds:
[Equation]
A.
[Equation]
Similarly
[Equation]
For simplicity
[Equation]
suppose that there exists a constant M 2 such that each function ˆ b , ˆ A ,
∂ t ˆ A has expectation norm bound:
[Equation]
The inequality in (A.32) together with evaluations in (A.33), (A.34) yields:
[Equation]
This directly gives the L 2 -bound of the operator B during the fictitious play over stages according to the cost function shown in the right-hand side:
[Equation]
Let us say that the gradient descent optimizes the neural parameters to achieve small enough values for cost function for k ≥ K .
[Equation]
Following by the definition of constant K , the L 2 deviation of neural agent can be bounded as follows:
[Equation]
If we denote α ∗ i = α m ∗ i i , m ∗ i ∈ N + , the following relation holds by triangle inequalities:
[Equation]
As υ i,m ∨ sup m υ i,m = sup m υ i,m for all m ≤ m ∗
, the expectation of Hamiltonian deviation can be bounded as a summation of two terms in (A.28):
[Equation]
where υ m = { υ i,m +1 } and m ∗ = { m ∗ i } .
By inserting above inequality into (A.23) and replacing m + 1 → m , we obtain
[Equation] [Equation]
Next, we evaluate the upper-bound of mean-squared decision deviations in (
[Equation] [Equation]
[Equation]
By denoting the deviation of parameters as κ m = ∥
δ θ m ∥ 2 , we select small enough values
υ i,m that is related to the marginal constants
ϵ i,m :
[Equation]
If the neural agent have large enough capacity to minimize the via gradient descent
[Equation]
these values assures minimal upper bound of the inequality (A.23).
Finally, the non-linear Feynman-Kac theorem, which is related to the equation (A.10), directly gives the :
[Equation]
Since J i ( α ∗ ) <
J i ( α m ) , our next step is to show the inequality between cost functionals given the action α mi obtained from the fictitious play and the the other arbitrary action β
i ̸
= α mi :
[Equation]
Then, by triangle inequality, we obtain inequality that shows the ϵ -Nash equilibrium:
[Equation]
The final step of the proof is to show the stochastic optimality according to the defined marginal constants ϵ .
Let θ mi ( k ) :
N + → Θ be the trajectory of the neural parameters of the i -th neural agents at learning iteration k .
Recall the fact that α mi = α m − 1 ,K i :=
B [ θ mi ( K − 1)] .
Let us denote that
Y mt | θ is the adjoint variable given neural parameters θ .
We define the closed metric balls { B kδ mi } k ∈ N + centered at θ mi ( k ) with the radius 0 < δ mi such that
[Equation]
local minimum of E
[Equation]
Next, we consider the sub-sequence { θ m i (¯ k ) } ¯ k ∈ ¯ N ⊆ { θ m i ( k ) } k ∈ N , which defines the strictlydecreasing sequence { E Y mt | θ m ( k ) } ¯ k ∈ ¯ N an ordered index set ¯ N .
Then, the admissible set
A i is defined as follows:
[Equation]
where ¯ K is a maximal element in ¯ N .
Intuitively saying, the admissible set
A
i is a collection of local metric balls centered at neural parameters of i -th agent updated by gradient descent.
To define the optimal actions of neural agents, let us consider an arbitrary local convex set β ∈ C ⊂
A where A is the admissible action set for multi-agents defined as follows:
[Equation]
By the convexity
[Equation]
there exist I pairs { ( ω i , β i ) } i ∈{ 1 ,...,I } such that the following equality holds:
[Equation]
and β i ( · ) ̸ = 0 .
Then, the Gâteaux derivative (Carmona, 2016, Theorem 4.12) of adjoint variable Y mt is derived as follows:
[Equation]
The first inequality is trivial due to the definition of admissible set
A i .
Since ∥ β ∥ 2 is nonzero, one can deduce that ∇ α H | α m + ωβ ≥ 0 .
In other words, the action profiles α m are optimal actions and any arbitrary actions β
i ∈ C i are eventually non-optimal.
Hence, we can conclude from the above result that the inequality in (A.46) holds in the admissible action set.
Finally, we conclude this proof by showing the stochastic optimality is preserved during the fictitious play by the definition of A that induces the following relation:
[Equation]
where Σ = σ T σ .
The stochastic optimality is obtained by the Hamiltonian equation (A.53) for every stages .
A.2. P ROOF OF C OROLLARY
In the proof of the Proposition
[Equation]
we have shown the existence of the local action set that assures the convergence of the fictitious play.
In this proof
[Equation]
we show the convergence of predictor Λ α t according to the action set assuring the local Nash equilibrium.
We start by calculating the norm deviation for the Itô’s differential of the predictor ( i
[Equation]
e ., δ Λ α t ) .
[Equation]
We followed by the rules for denoting the object deviation in the previous proof to denote object deviations combined with the linear operators ( ∂ t , ∇ , ∇ 2 , Tr ) on C 2 ( R d ) .
From the last equality of (A.54), we estimate the upper bounds of each term.
[Equation]
The result directly follows from By the assumption in (A.35), the following evaluation is trivial:
[Equation]
Now
[Equation]
we estimate the two terms ( b ) , ( c ) that contain spatial and temporal derivatives of decision attention function in following bracket:
[Equation]
where the basic property of vector gradient ∇ ( v · u ) = ( ∇ v ) · u + v · ( ∇ u ) for all v, u ∈ R d , ∇ v, ∇
u ∈ R d × d is used in the estimation of (c).
For the last two terms, i .
e ., ( d ) , ( e ) , we used the fact that the deviation of the matrix Σ is not defined because our MaSDEs assume uncontrollable volatility.
  
[Equation]
A.
[Equation]
Note that the first line in (A.58) can be obtained by the trace inequality ( i .
e .,
Tr [ AB ] ≤ ∥
A ∥ F ∥ B ∥ F ) .
By collecting the evaluated terms above, we conclude the proof by showing the following relation:
[Equation]
A. N ON - COOPERATIVE G AME
Empirical Study for the Non-cooperation.
This section is devoted to answer the following important question: Why do we consider a cooperative scenario rather than a non-cooperative one?
This question is crucial as the shared goal could have been achieved in a highly competitive and noncooperative environment (Ye et al., 2018).
For an intuitive explanation, we introduce a specific scenario where the neural agent becomes adversarial and determines that interfering other agent’s objectives is the best possible strategy to achieve its own goal.
Followed by the definition, the behavioral rules for the adversarial agent can be summarized by the following equations:
[Equation]
The second inequality shows that the adversarial agent (here, the j -th agent) can easily ruin other agent’s goal although the victim (here, the i -th agent) produces its best response α ∗ i to the adversarial environment.
Eventually, one can expect that the competitive group will fail to accomplish the original goal ( i .
e ., the accurate prediction).
To formalize the aformentioned scenario by estimating the total amount of inefficiency derived from the non-cooperation, we formulate the non-cooperative game ( e .
g ., min-max) by converting the cooperative behavior into an adversarial one:
[Equation]
With the adversarial cost ˆ h j , selfish agents maximize the influence on temporal aggregation by interfering with opponents.
As the symmetric relation in (7) fails to be satisfied, the proposed framework with cost function in (A.61) is regarded as a non-cooperative game.
To elucidate the importance of cooperation in our framework of time series prediction, we conduct an experiment on Mackey-Glass dataset by setting a single adversarial agent α 1 .
The red and blue learning curves in Figure A.1 illustrate the coalition costs during the fictitious play over 20 stages produced by competitive and cooperative groups, respectively.
As one might expect, the cooperative group shows the stable learning dynamics until it converges to the agreement on the accurate prediction.
This shows that the proposed fictitious play developed in
Section 3 enjoys the desirable characteristic for the accurate time series prediction.
Contrarily, the non-cooperative group under the competition fails to achieve a shared goal and shows that the adversarial behavior is fatal to the proposed framework regardless of the number of adversaries.
In the optimization perspective, the primal reason for the failure is the instability induced by the naive approach for the gradient-descent based min-max type optimization.
As a remedy, one may restrict the admissible action sets by considering an additional constraint on neural agents such as PL condition (Nouiehed et al., 2019) to develop a sophisticated algorithm for convergent result.
Yet, it is an open question whether this theoretical perspective can lead to the accurate time series prediction. [Equation]
UMMARY OF RELEVANT CONCEPTS
The most relevant strand of research about the multi-agent system in the machine learning community is multi-agent reinforcement learning (MARL) (Foerster et al., 2016). The relevant concepts between MARL and CooPredict are summarized in Table A.1.
However, MARL and CooPredict differ in that MARL usually assumes model-free in discrete time, while CooPredict assumes model-based in continuous time.
[Table]
{ y s i }
[Table]
i ∈
O
[Table]
{ y t } t ∈ T
[Table]
α =
[ α 1 , . .
, α I ] , α i := α i ( · , · ; θ i )
[Table]
α i ( t, X i, α t ; θ i )
[Table]
X α t =
[ X 1 , α t , . . .
, X I, α t ] := MaSDEs ( { y s i } s i ∈
O , α )
[Table]
−|| Λ t − y t
||
[Table]
J i ( t, X α t , α )
[Table]
V i
[Table]
t
[Table]
J
[Table]
J
[Table]
[Table]
[Table]
A. P SEUDO -C ODE OF C OO P REDICT
We provide pseudo code for the training procedure of CooPredict via the deep neural fictitious play described in Section 3 in Algorithm 1 and the inference procedure of CooPredict in Algorithm
[Table]
Input
[Table]
Time
[Table]
series data y
=
( { y s i } s i ∈
O , { y t } t ∈ T )
[Table]
{+Neural Parameters θ m =
+}
=
[ θ 01 , . .
{+. , θ 
+}
I ]
{+Publicly share the given environment α m =
+}
= [ α 01 ( · , · ,
[Equation]
{+, α 
+}
I
[Equation] · , · , θ 0 I )] for m = 1 to M ( i . e ., The total number of stages ) do Simulate set of decisions with action profiles α m − 1 at previous stage with equation (1).
for all i ∈ [1 , · · · , I ] in parallel do
Fix the environment of the previous stage except agent i .
[Equation]
Solve the decoupled adjoint problem for agent i in equation (13).
[Equation]
dt
[Equation]
Collect the updated neural parameters θ m =
[ θ m 1 , . .
. , θ mI ] end for
Update the total environment for the next stage α m ← α m − 1 ( · , · , θ m ) .
end for Return Optimal action profiles α ∗ =
[ α ∗ 1 , . .
, α ∗ I ] .
Algorithm
Input: Held-out time-series data ˆ y =
( { ˆ y s i } s i ∈
O , { ˆ y t } t ∈ T ) , Optimal action profiles α ∗ =
[ α ∗ 1 , . .
, α ∗ I ]
Simulate optimal decisions over the future interval T given observations { ˆ y s i } s i ∈
O .
[ X 1 , α ∗ t , . . .
X I, α ∗ t ] t ∈ T = MaSDEs ( { ˆ y s i } s i ∈ O , α ∗ )
Aggregate individual decisions with decision attention A α ∗ =
[Equation]
[Table]
A. I MPLEMENTATION D ETAILS
Evaluation .
In all experiments with real-world datasets, we train each model for 500 epochs using the Adam optimizer with a learning rate of 10 − 3 and batch size of 128 .
We reported the MSE and (Gaussian) NLL as suggested in (Rubanova et al., 2019).
The performance is evaluated on the predicted parts of the test dataset.
For the evaluation of our method, we evaluate MSEs between the predictor Λ α ∗ t and a test data ˆ y t ∼ ˆ ν t as follows:
[Equation]
where the predictor is conditioned by the past observations { ˆ y s i }
s i ∈
O of the testing time series.
As neural agents collaborate to minimize the temporally averaged utilities ( i .
e .,
J i ≈ (cid:82) T E || Λ s − y s
|| 2 ds ) via cooperative differential game, the goal of the proposed cooperative game becomes identical to forecast the time series ˆ y t ∈ T given observations.
A.6. N ETWORK A RCHITECTURE
In this section, we briefly introduce the network architecture of multi-agent neural SDEs as shown in
Figure A.2.
Given the initial condition { y ( · ) } s i ∈
O , each neural agent takes the spatio-temporal variable ( t, X i, α t ) and produces infinitesimally transformed outputs to propagate its own stochastic trajectory.
First, the temporal variable t is embedded into inhomogeneous and non-linear representation as t ′ =
( t, sin ( t ) , cos ( t )) .
Note that we adopt the temporal privacy function suggested in (Park et al., 2022) for further temporal encoding.
After the non-linear time embedding, the intermediate representations ( t ′ , X i, α t ) are fed into the novel module referred to as agent identification layer (AIL), which is defined as follows:
[Equation]
The AIL is motivated by the adaptive instance normalization (Huang & Belongie, 2017) that adaptively transforms the statistics of intermediate feature representations according to user-guided information ( i .
e ., index for neural agent).
This encourages each neural agent to propagate the individual stochastic trajectory and ensures the separated representations.
The outputs from AIL are fed into two subsequent linear layers with LipSwish (Chen et al., 2019) and produce values for drift b i and decision attention functions
A α i i .
In the experiments with real-world dataset, we identically set the dimension of hidden layers of each neural agent and the drift network as 128 following the baseline model (Park et al., 2022). We set the number of hidden layers as 2 for neural agents and 1 for the drift network. For the attention network, we used a relatively small dimension ( i . e
., , 36) with 2 hidden layers across all the experiments.
For the simulation of the stochastic trajectory given the architecture, we discretize sampled times by applying the Euler-Maruyama scheme.
A.6. D ATASET
PhysioNet Challenge 2012 , (Silva et al., 2012), contains 8000 multivariate clinical time series obtained from the intensive care unit (ICU).
Each time series has various clinical features of the patient’s first 48 hours after admission to ICU.
We processed the dataset to hourly time series and eliminated static features so that each time series has a length of 48 with 35 time varying features ( e .
g ., Albumin, Heart-rate, etc.,).
We used half
Speech Commands , (Warden, 2018), consists of one-second audio records of various spoken words such as “Yes”, “No”, “Up”, and “Down”.
Since there are more than 100,000 record samples, we sub-sampled the dataset on two conflicting classes ( i .
e ., “Right” and “Left”).
As a result,time series records were selected.
We pre-processed these time series by computing Mel-frequency cepstrum coefficients so that each time series has a length of 54 with 65 channels.
We used 80% of selected data as training dataset and the remaining parts as the test dataset.
Beijing Air-Quality , (Zhang et al., 2017; Dua & Graff, 2017), consists of multi-year air quality recordings across different locations in Beijing.
Each sample contains 6-dimensional time series features of PM 2 . 5 , PM 10 , SO 2 , NO 2 , CO, and O 3 , which are recorded per hour.
We segmented data per two days so that each sample has a length of 48.
We combine recordings of 12 different locations into a single dataset and randomly split 80% of data into the training dataset and 20% of data into the test dataset.
Gaussian-Impulse Noise.
To generate the Gaussian-impulse noise, we first sample the random time-stamp { t 1 , t 2 , · · · } from a homogeneous possion process with intensity level of 8 . 0 .
With this time-stamp, we generate the data Y t over the time interval [0 , 48] as follows:
[Equation]
where U t ∼ Unif [0 , 1] , Z t ∼ N (0 , σ 2 ) and the standard deviation of Gaussian noise is set to 0 .
Similar to other datasets, we set the observable and the prediction intervals to contain the first 80% and the last 20% of total points, respectively.
Delayed Differential Equation.
In this experiment, we simulated deterministic trajectories of delayed differential equations (DDEs) as following: Mackey-Glass DDE:
[Equation]
where we set hyper-parameters as τ = 10 , n = 10 , β = 0 .
25 , γ = 0 .
Among the entire set of trajectories, we randomly split 80% of data into training dataset, and remaining 10% and 10% into validation and test dataset, respectively.
We follow the identical experiment setting as suggested in (Holt et al., 2022) where the entire interval is split into two halves, i .
e ., O = [0 , 10] and T =
A. C OMPARISONS WITH N ON -DE B ASED M ODEL
[Table]
The most significant advantage of using
DE-based methods over non-DE based methods is the capability of handling irregularly sampled or partially observed time-series data which is common in real-world settings such as medicine or business.
In such settings, non-DEbased counterparts are incapable of applying without careful consideration of pre-processing steps such as discretization and interpolation.
Therefore, we compared CooPredict with the state-of-the-art Transformerbased time-series forecasting methods on the Speech dataset.
We chose the Speech dataset since it comprises complete time-series observations with regular time intervals which can be favorable for applying non-DE-based methods and can avoid potential bias due to design choices of pre-processing steps.
As can be seen in Table A.2, CooPredict outperforms its non-DE-based counterparts implying that our proposed method can be a promising tool for general time-series modeling under both regular and irregular settings.
A. A DDITIONAL Q UALITATIVE R ESULTS
In this subsection, we illustrate the results of an experiment in Section 5.2 on Mackey-Glass DDE and provide additional qualitative results on BAQD across overall features of different instances to show the effectiveness of the proposed method on time series prediction.
Figure A.3 shows the ground truth trajectories and inferred predictions/attentions results for various sampled trajectories.
Notably, the future trajectory is highly sensitive to initial states, so capturing the delayed effect of the distant past on the future is required for successful prediction in this dataset.
As can be seen in Figure A.3, our method can adaptively capture the delayed effect and provide accurate predictions regardless of different histories.
Compared to our method, the CSDE-TP provides worsened performance.
In light of this, we can interpret that adjusting the relative impact of given observations is more effective for generating accurate future predictions than treating them equally.
Figure A.4 shows prediction results for all features and learned attentions on the BAQD.
We can observe that our method exhibits attentions similar (but not strictly restricted) to temporal decay assumption.
As can be seen in Figure A.4, the learned decision attention varies depending on the given observation.
We claim that neural agents have a variety of cooperation patterns to generate accurate prediction.
That is, our method flexibly extracts complex and heterogeneous temporal correlations beyond simple temporal decay assumed by the conventional methods.