L earning S tochastic B ehaviour from A ggregate D ata
Anonymous authors
Paper under double-blind review
A bstract
Learning nonlinear dynamics from aggregate data is a challenging problem since the full trajectory of each individual is not available, namely, the individual observed at one time point may not be observed at next time point, or the identity of individual is unavailable due to technical limitations, experimental cost and / or privacy issues. This is in sharp contrast to learning dynamics with trajectory data, on which the majority of existing methods are based.
In this paper, we present a complementary strategy designed for aggregate data.
Our method uses weak form of Fokker Planck Equation(FPE) to describe density evolution of data in a sampling form, which is combined with Wasserstein generative adversarial network(WGAN) in training process.
In such a sample-based framework we are able to study nonlinear dynamics from data without solving the partial di ﬀ erential equation(PDE).
The model can also handle high dimensional cases with the help of neural networks.
We demonstrate our approach in the context of a series of synthetic and real-world data sets.
I ntroduction
Aggregate data refers to the data sets that full trajectory of each individual is not available, meaning that there is no known individual level correspondence. Typical examples include data sets collected for DNA evolution, social gathering, density transport in control problems, bird migration during which it is impossible to identify individual bird, and many more.
In those applications, some observed individuals at one time point may be unobserved at next time spot, or when the individual identities are block or unavailable due to various technical and ethical reasons. Rather than inferring the exact information for each individual, the main objective of learning dynamics in aggregate data is to recover and predict the evolution of distribution of all individuals together.
Trajectory data , in contrast, is a kind of data that we are able to acquire the information of each individual all the time, although some studies considered the case that some individual trajectories are partially missing. However, the identities of those individuals, whenever they are observable, is always assumed available.
For example, stock price, weather, customer behaviors and most training data sets for computer vision and natural language processing.
There are many existing models to learn dynamics of full-trajectory data. Popular ones include
Hidden Markov Model (HMM)(Alshamaa et al., 2019; Eddy, 1996), Kalman Filter (KF)(Farahi & Yazdi, 2020; Harvey, 1990; Kalman, 1960) and Particle Filter (PF) (Santos et al., 2019; Djuric et al., 2003).
These models and their variants (Deriche et al., 2020; Fang et al., 2019; Hefny et al., 2015;
Langford et al., 2009) require full trajectories of each individual, which may not be directly applicable to the aggregate data as we mentioned earlier.
On the other side, only a few methods are focused on aggregated data in the recent learning literature.
In the work of Hashimoto et al.
(2016), a stochastic di ﬀ erential equation(SDE) was adopted to capture the dynamics of particles directly on observations, in particular the drift coe ﬃ cient is parameterized by a recurrent neural network.
Furthermore, Wang et al.
(2018) improved traditional HMM model by using an SDE to describe the evolving process of hidden states.
To be the best of our knowledge, there is no method directly learning the evolution of the density of objects from aggregate data yet. We propose to learn the dynamics of density through the weak form of Fokker Planck Equation (FPE), which is a parabolic partial di ﬀ erential equation (PDE) governing many dynamical systems subject to random noise perturbations, including the typical SDE models in existing studies.
Our learning is accomplished by minimizing the Wasserstein distance between predicted distribution given by FPE and the empirical distribution from data samples. Meanwhile we utilize neural networks to handle higher dimensional cases.
More importantly, by leveraging the framework of Wasserstein Generative Adversarial Network (WGAN) (Arjovsky et al., 2017), our model is capable of approximating the distribution of samples at di ﬀ erent time points without solving the SDE or FPE.
More speciﬁcally, we treat the transport coe ﬃ cient, the goal of learning, in the FPE as a generator, and the test functions in the weak form of FPE as adversarial. In other words, our method can also be regarded as a data-driven method to estimate transport coe ﬃ cient in FPE, which corresponds to the drift coe ﬃ cients in SDEs.
We would like to mention that several methods of solving SDE and FPE (Weinan et al., 2017; Beck et al., 2018; Li et al., 2019) adopt opposite ways to our method, they utilize neural networks to estimate the distribution P ( x , t ) with given drift and di ﬀ usion terms.
Additionally, though we treat di ﬀ usion term as a constant in our model, it is straightforward to generalize it to be a neural network as well, which can be an extension of this work.
In conclusion, our contributions are:
• We design a formulation that uses the weak form of FPE to recover nonlinear dynamics from aggregate observations.
• We combine our weak form of FPE with WGAN so that we do not need to solve the FPE.
Finally, we empirically demonstrate the e ﬀ ectiveness of our framework via some synthetic and real-world data sets.
P roposed M ethod 2.1 F okker P lanck E quation for the density evolution
We assume the individuals evolve in a pattern as shown in Figure 1. An example satisﬁes such process is the following stochastic di ﬀ erential equation(SDE) also known as Itô process:
[Equation]
where the distribution of X represents the observations along time, d X represents a tiny change of X along with time interval dt , g
( X , t ) is drift coe ﬃ cient that drives the dynamics of the SDE, σ is the di ﬀ usion constant, d W is a stochastic process and N (0 , 1) is a standard Gaussian noise.
Given the above SDE model, the density of the solution X is governed by the Fokker Planck Equation(FPE).
Lemma 1.
Suppose φ ( x ) : R d (cid:55)→ R n has a continuous partial time derivative and continuous second partial space derivatives, then the density of φ that satisﬁes Equation 1 leads to the Fokker Planck
Equation:
[Equation]
As a linear evolution PDE, FPE describes the evolution of density function of the stochastic process driven by a SDE.
Due to this reason, FPE plays a crucial role in stochastic calculus, statistical physics and modeling (Nelson, 1985; Qi & Majda, 2016; Risken, 1989).
Its importance is also getting more attention among statistic and machine learning community(Liu & Wang, 2016; Pavon et al., 2018;
Rezende & Mohamed, 2015).
In this paper, we utilize the weak form of FPE as a basis to study hidden dynamics of the whole group data without solving FPE.
Our task can be described as: assume that the individuals evolve with the process indicated by Figure 1, which can be simulated by Equation 1.
Then given observations x t along time axis, we aim to recover the transport coe ﬃ cient g ( x , t ) in FPE.
For simplicity we treat g ( x , t ) as a function uncorrelated to time t , namely, g ( x , t ) =
g ( x ).
Notice that though evolving process of individuals can be simulated by Equation 1, in reality since we lose identity information of individuals, the observed data become aggregate data, thus we need a new way other than traditional methods to study individuals’ distribution.
2.2 W eak F orm of F okker P lanck E quation
Given FPE stated in Lemma 1, if we multiply a test function f ∈ H 10 ( Ω ; R ) on both sides of the FPE, then the integration on both sides leads to the weak form of FPE:
[Equation]
where H 10 ( Ω ; R ) denote the Sobolev space.
The ﬁrst advantage of weak solution is that the strict solution of a PDE requires su ﬃ cient regularity of the problem thus may not exist in the classical sense, however the weak solution may exist and provide more advantages of solving PDEs(Zang et al., 2019).
Specially, if f ( x ) is a density function, the integral form then helps us investigate density in a samples level view, thus machine learning methods could be applied.
Suppose the observed samples at time points t m − 1 and t m are following true densities ˆ p ( x , t m − 1 ) and ˆ p ( x , t m ), we utilize following method to approximate density ˜ p ( x , t m ):
[Equation]
where x is the solution satisﬁes Equation 1 and p ( x , τ ) is the true density of x at time τ .
Then it is natural to compare the di ﬀ erence between two distributions ˜ p ( x , t m ) and ˆ p ( x , t m ).
Applying
Kantorovich-Rubinstein duality form(Villani, 2008), the Wasserstein distance between ˜ p and ˆ p is:
[Equation]
for some f whose gradient norm should be less or equal to 1.
Notice that x g is generated by Equation 1 from the initial data x 0 .
If we describe this process with G ( x 0 ), representing the Euler increment form of Equation 1 starting with x 0 :
[Equation]
[Equation] then Equation 4 can be rewritten as a framework of Wasserstein generative adversarial network (WGAN)(Arjovsky et al., 2017; Goodfellow et al., 2014): [Equation]
[Equation]
where we treat f as a discriminator and G as a generating process.
Inspired by Equation 3, the way we deal with E x g ∼ ˜ p
( x , t m )
[ f ( x g )] is:
[Equation]
∂τ
[Equation]
In above equation, the second term on the right is actually a weak form of FPE and f ( x ) is a test function.
We plug in Equation 3 and use integration by parts to solve this term, refer the proof of
Theorem 1 to see more details.
Theorem 1.
Suppose f satisﬁes (cid:107)∇ f (cid:107) ≤ 1 and generated data evolves only one step from x t m − 1 by
Equation 5, we deﬁne an operator:
[Equation]
then the Wasserstein distance between real data and generated data at time point t m is:
[Equation]
By Theorem 1 we are able to compute the Wasserstein distance between true and generated distributions at a single time point.
Di ﬀ erent with normal form of Wasserstein distance shown in Equation 4, we have two extra terms at the end.
2.3 W asserstein D istance on T ime S eries
The next question is that in real cases, it is not realistic to observe the data on two consecutive time points, especially when ∆ t is small.
To make our model more ﬂexible, we extend our formulation to followings so that we are able to plug in data that is observed at arbitrary time points.
Theorem 2.
Suppose f satisﬁes (cid:107)∇ f (cid:107) ≤ 1 and generated data evolves multiple steps starting with arbitrary data x t m 0 by Equation 5.
We adopt the same operator F deﬁned in Theorem 1, then the
Wasserstein distance between real data and generated data at time point t m n is:
[Equation]
Objective Function: Base on Theorem 2, ﬁnally we obtain objective function by minimizing the accumulated Wasserstein distance on time-series observations:
[Equation]
We call our algorithm Fokker Planck Process(FPP) which is shown in Algorithm 1.
We also provide an error analysis in Appendix.
Require:
Initialize f θ n , g ω 1: for # training iterations do 2: for k steps do 3: for observed time t m s , t m n in [ t m 0 : T ] do 4:
Generate x ( k ) t ms , x ( k ) t mn by g ω 5:
Sample real { x ( k ) t mn } from p ( x t mn ) 6: end for 7:
For each f n compute: 8:
[Equation]
Update f n by ∇ θ
[Equation]
10: end for 11:
Update g ω by ∇ ω (cid:80) f n ( −
[Equation]
F n ) 12: end for
E xperiments
In this section, we evaluate our model on various synthetic and realistic data sets.
Baselines: We compare our model with two recently proposed methods.
One model(NN) adopts recurrent neural network(RNN) to learn dynamics directly from observations of aggregate data(Hashimoto et al., 2016).
The other one model(LEGEND) learns dynamics in a HMM framework(Wang et al., 2018).
The baselines in our experiments are two typical representatives that have state-of-the-art performance on learning aggregate data.
Furthermore, though we simulate the evolving process of the data as a SDE, which is on the same track with NN, as mentioned before, NN trains its RNN via optimizing Sinkhorn distance(Cuturi, 2013), our model starts with a view of weak form of PDE, focuses more on WGAN framework and easier computation.
3.1 S ynthetic D ata
We ﬁrst evaluate our model on three synthetic data sets which are generated by three artiﬁcial dynamics: Synthetic-1 , Synthetic-2 and Synthetic-3 .
Experiment Setup:
In all synthetic data experiments, we set the drift coe ﬃ cient g and the discriminator f as two simple fully-connected networks.
The g network has one hidden layer and the f network has three hidden layers.
Each layer has 32 nodes for both g and f .
The only one activation function we choose is Tanh.
Notice that since we need to calculate ∂ 2 f ∂ x 2 , the activation function of f must be twice di ﬀ erentiable to avoid loss of weight gradient.
In terms of training process, we use the Adam optimizer (Kingma & Ba, 2014) with learning rate 10 − 4 .
Furthermore, we use spectral normalization to realize (cid:107)∇ f (cid:107) ≤ 1(Miyato et al., 2018).
We initialize the weights with Xavier initialization(Glorot &
Bengio, 2010) and train our model by Algorithm 1.
Synthetic-1:
[Equation]
Synthetic-2:
[Equation]
Here we need to further state more information about G :
[Equation]

[Equation]
Synthetic-3 (Van der Pol oscillator(Li, 2018)):
[Equation]
The data size at each time point is N = 2000, we treat 1200 data points as the training set and the other 800 data points as the test set.
In Synthetic-1, this is a two dimensional case following simple linear dynamics, where A = [(4 , 0) , (0 , 1)] , b = [ − 3 , − 3] T .
We let ∆ t = 0 .
01, σ = 1, Σ 0 =
We utilize true x 0 , x 20 and x 200 in training process and predict the distributions of x 50 and x 500 .
In Synthetic-2, this is a two dimensional case following complex nonlinear dynamics.
We let ∆ t = 0 .
01, σ = σ 1 =
σ 2 = 1, µ 1 = [15 , 15] T and µ 2 = [ − 15 , − 15] T .
We utilize true x 0 , x 3 and x 6 in training process and predict x 2 , x 4 and x 10 .
In Synthetic-3, ∆ t = 0 .
01, σ = 1, we utilize true x 3 , x 7 and x 20 in training process then predict the distributions of x 10 , x 30 and x 50 .
We consider population evolution with three di ﬀ erent dimensions:
d = 2, d = 6 and d = 10.
In each high dimensional case (d = 6, d = 10), to be convenience, we set every two dimensions follow the dynamics of low dimensional case(d = 2) in each data set.
Results: We ﬁrst show the capability of our model for learning hidden dynamics of low-dimensional (d = 2) data.
As visualized in Figure 2, the generated data(blue) covers all areas of ground truth(red), which demonstrates that our model is able to precisely learn the dynamics and correctly predict the future distribution of data.
We then evaluate three models using Wasserstein distance as error metric for both low-dimensional (d = 2) and high-dimensional (d = 6, 10) data.
As reported in Appendix
Table 1, our model achieves lower Wasserstein error than the two baseline models in all cases. [Figure]
3.2 R ealistic D ata – RNA S equence of S ingle C ell
In this section, we evaluate our model on a biology data set called Single-cell RNA-seq which is typically used for learning the di ﬀ usion process where embryonic stem cells di ﬀ erentiate into mature cells (Klein et al., 2015).
The cell population begins to di ﬀ erentiate from embryonic stem cells after the removal of LIF (leukemia inhibitory factor) at day 0 (D0).
Single-cell RNA-seq observations are sampled at day 0 (D0), day 2 (D2), day 4 (D4) and day 7 (D7).
At each time point, the expression of 24,175 genes of several hundreds cells are measured (933, 303, 683 and 798 cells on D0, D2, D4 and
D7 respectively), notice that there is only whole group’s distribution but no trajectory information of each cell on di ﬀ erent days.
We focus on the data of cell di ﬀ erentiation for the two chosen makers, i.e., Mt1 and Mt2.
In ﬁrst task we treat gene expression level at D0, D4 and D7 as training data to learn the hidden dynamics and predict the distribution of gene expression level at D2.
In second task we train the model by gene expression level at D0, D2 and D4, then predict the distribution of gene expression level at D7.
Experiment Setup: We set both f and g as fully connected three-hidden-layers neural networks, each layer has 64 nodes.
The only activation function we choose is Tanh.
The other setups of neural networks and training process are the same with the ones we use in Synthetic data.
Notice that in realistic cases, ∆ t and T / ∆ t become hyperparameters, here we choose ∆ t = 0 .
05, T / ∆ t = 35, which means the data evolves 10 ∆ t from D0 to D2 , then 10 ∆ t from D2 to D4 and ﬁnally 15 ∆ t from D4 to D7.
For preprocessing, we apply standard normalization procedures (Hicks et al., 2015) to correct batch e ﬀ ects and use non-negative matrix factorization to impute missing expression levels(Hashimoto et al., 2016; Wang et al., 2018).
Results: As shown in Table 1 in Appendix, when compared to other baselines, our model achieves lower Wasserstein error on both Mt1 and Mt2 data, which proves that our model is capable of learning the hidden dynamics of the two studied gene expressions.
In Figure 3 (a) to (d), we visualized the [Figure] predicted distributions of the two genes.
The distributions of Mt1 and Mt2 predicted by our model (curves in blue) are closer to the true distributions (curves in red) on both D2 and D7.
Furthermore, our model precisely indicates the correlations between Mt1 and Mt2, as shown in Figure 3 (e) and (f), which also demonstrates the e ﬀ ectiveness of our model since closer to the true correlation represents better performance.
In Figure 3 (g) and (h), we see that with simpler structure, the training process of our model is easier with least computation time.
3.3 R ealistic D ata – D aily T rading V olume
In this section we would like to demonstrate the performance of our model in ﬁnancial area.
Trading volume is the total quantity of shares or contracts traded for speciﬁed securities such as stocks, bonds, options contracts, future contracts and all types of commodities.
It can be measured on any type of security traded during a trading day or a speciﬁed time period.
In our case, daily volume of trade is measured on stocks.
Predicting traded volume is an essential component in ﬁnancial research since the traded volume, as a basic component or input of other ﬁnancial algorithms, tells investors the market’s activity and liquidity.
The data set we use is the historical traded volume of the stock "JPM".
The data covers period from January 2018 to January 2020 and is obtained from Bloomberg.
Each day from 14:30 to 20:55, we have 1 observation every 5 minutes, totally 78 observations everyday.
Our task is described as follows: given ﬁrst two years data, we use the traded volume at 14:30, 14:40, 15:05, 15:20 and 16:20 as training data, namely, x 0 , x 2 , x 7 , x 10 , x 22 to train our model, then for next 100 days we predict traded volume at 14:35, 15:15, 15:35 and 16:15, namely, x 1 , x 9 , x 13 , x 21 .
One of baselines we choose is classical rolling means(RM) method, which predicts intraday volume of a particular time interval by the average volume traded in the same interval over the past days.
The other one baseline is a kalman ﬁlter based model (Chen et al., 2016) that outperforms all available models in predicting intrady trading volume.
Experiment Setup: Following similar setup as we did for RNA data set, we utilize the same structures for neural networks here.
For hyperparameters we set ∆ t = 0 .
02, T / ∆ t = 22, which means it takes one single ∆ t from x t to x t + 1 .
For preprocessing, we rescale data by taking natural logarithm of trading volume, which is a common way in trading volume research.
We conduct experiments on two groups to show advantages of our method, for ﬁrst group we train our model on complete data set, in this case the data has full trajectory; for second group we manually delete some trajectories of the data, for instance, we randomly kick out some samples of x 0 , x 2 , x 7 , x 10 , x 22 then follow the same procedures of training and prediction.
Results: We present prediction results in Figure 4.
As shown in ﬁrst four ﬁgures, with full trajectory, prediction made by RM is almost a straight line, the prediction value bouncing up and down within a very small range, thus this model cannot capture the volume movements, namely, regularities existing in the time series; prediction made by the kalman ﬁlter based model captures the regularities better than RM model, but it fails to deal with noise component existing in the time series, thus some predictions are out of a reasonable range.
Traded volume predicted by our model is closer to the real case, moreover, our model captures regularities meanwhile gives stable predictions.
Furthermore, without full trajectory, kalman ﬁlter based model fails to be applied here and RM model still fails to capture the regularities, we randomly drop half of the training samples and display predictions made by our model in last four ﬁgures of Figure 4, we see our model still works well.
C onclusion
In this paper, we formulate a novel method to recover the hidden dynamics from aggregate data.
In particular, our work shows one can simulate the evolving process of aggregate data as an Itô process, in order to investigate aggregate data, we derive a new model that employs the weak form of FPE as well as the framework of WGAN.
Furthermore, in Appendix we prove the theoretical guarantees of the error bound of our model.
Finally we demonstrate our model through experiments on three synthetic data sets and two real-world data sets. A A ppendix [Table]
B E rror A nalysis
In this section
[Table]
we provide an error analysis of our model.
Suppose the hidden dynamics is driven by g r ( x ), the dynamics that we learn from data is g f ( x ), then original Itô process, Euler process and
Fokker Planck Process are as follows:
[Equation]
Estimating the error between original Itô process and its Euler form can be very complex, hence we cite the conclusion from (Milstein & Tretyakov, 2013) and focus more on the error between original form and our model.
Lemma B.1.
With the same initial X t 0 = x t 0 = x 0 , if there is a global Lipschitz constant K which satisﬁes:
[Equation]
then after n steps, the expectation error between Itô process x t n and Euler forward process x rt n is:
[Equation]
Lemma B.1 illustrates that the expectation error between original Itô process and its Euler form is not related to total steps n but time step ∆ t .
Theorem B.1.
With the same initial x 0 , suppose the generalization error of neural network g is ε and existence of global Lipschitz constant K:
[Equation]
then after n steps with step size ∆ t = T / n, the expectation error between Itô process x t n and approximated forward process x ft n is bounded by:
[Equation]
Theorem B.1 implies that besides time step size ∆ t , our expectation error interacts with three factors, generalization error, Lipschitz constant of g ( x ) and total time length.
In our experiments, we ﬁnd the best way to decrease the expectation error is reducing the value of K and n .
[Equation] C.1 P roof of L emma
Lemma C.1.
Suppose φ ( x ) : R d (cid:55)→ R n have a continuous partial time derivative and continuous second partial space derivatives, then the density of φ that satisﬁes Equation 1 leads to the Fokker
Planck Equation:
[Equation]
Proof.
This result has been widely proved, here we just give a simple sketch of proof.
Suppose φ ( x , t ) is a solution of Equation 1, by Itô formula we have:
[Equation]
Let ∂φ∂ t = 0, then we take expectation on both sides:
[Equation]
Use integration by parts on right side and consider p ( ∞ , t ) = 0, we have:
[Equation]
Finally we have 1-dimensional form of Fokker Planck Equation:
[Equation]
We extend 1-dimensional form to multi-dimensional form to get proof done.
[Equation]
C.2 P roof of T heorem
Theorem C
[Equation]
Suppose f satisﬁes (cid:107)∇ f (cid:107) ≤ 1 and generated data evolves only one step from x t m − 1 by (2), deﬁne an operator:
[Equation]
then the weak form of Wasserstein distance between real data and generated data at time point t m can be approximated by:
[Equation]
Proof.
Suppose x ( k ) t m and x ( k ) t m − 1 are our observed samples at t m and t m − 1 respectively, then expectations could be approximated by observed samples and generated data:
[Equation]
[Table]
[Equation] [Equation]
Then for the second term I above
[Equation]
it is di ﬃ cult to calculate directly
[Equation]
but we can use integration by parts to rewrite I as:
[Equation]
To approximate the integral from t m − 1 to t m
[Equation]
we adopt an average method between two endpoints
[Equation]
then we could rewrite the expectation in Equation (13) as:
[Equation]
Put (13), (14) and (15) together to ﬁnish the proof.
[Equation]
C.3 P roof of T heorem
Theorem C.2.
Suppose f satisﬁes (cid:107)∇ f (cid:107) ≤ 1 and generated data evolves multiple steps from any initial data x t m 0 by (2), deﬁne the operator:
[Equation]
 then for arbitrary n, the weak form of Wasserstein distance between real data and generated data at time point t m n can be approximated by:
[Equation]
Proof.
Given initial x t m 0 and an updated g
[Equation]
we generate x t m
[Equation]
x t m 2 , x t m 3 ... x t mn sequentially by Equation 5.
Then the expectations can be rewritten as:
[Equation]
which is:
[Equation]

[Equation]
N
[Equation]

[Equation]

[Equation]

[Equation]
N
[Equation]

[Equation]
Finally it comes to:
[Equation]
C.4 P roof of E rror A nalysis
Proof.
The proof process of Lemma B.1 is quite long and out of the scope of this paper, for more details please see ﬁrst two chapters in reference book (Milstein & Tretyakov, 2013).
While for
Theorem B
[Equation]
1, with initial X and ﬁrst one-step iteration:
[Equation]
Then we have:
[Equation] [Equation]
Follow the pattern we have:
[Equation]
Which leads to:
[Equation]
Now let S = (cid:80) n − 1 i =
(1 + K ∆ t )
i , then consider followings:
[Equation]
[Equation]
Finally we have:
[Equation]