Dynamic Time Lag Regression: Predicting What & When
Authors: Mandar Chandorkar
Cyril Furtlehner
Bala Poduval
Enrico Camporeale
Michele Sebag
Venue:  (2020)
Keywords: Dynamic Time-Lag Regression
Time Delay
Regression
Time Series

Abstract: This paper tackles a new regression problem, called Dynamic Time-Lag Regression (DTLR), where a cause signal drives an effect signal with an unknown time delay.
The motivating application, pertaining to space weather modelling, aims to predict the near-Earth solar wind speed based on estimates of the Sun's  coronal magnetic field. 
DTLR differs from mainstream regression and from sequence-to-sequence learning in two respects: firstly, no ground truth (e.g., pairs of associated sub-sequences) is available; secondly, the cause signal contains much information irrelevant to the effect signal (the solar magnetic field governs the solar wind propagation in the heliosphere, of which the Earth's magnetosphere is but a minuscule region). 

A Bayesian approach is presented to tackle the specifics of the DTLR problem, with theoretical justifications based on linear stability analysis. A proof of concept on synthetic problems is presented. Finally, the empirical results on the solar wind modelling task improve on the state of the art in solar wind forecasting.

Published as a conference paper at ICLR 2020
DYNAMIC TIME LAG REGRESSION: WHAT & WHEN
PREDICTING
Mandar Chandorkar
Cyril Furtlehner PARIA-Saclayy
Bala Poduval
Centrum Wiskunde en Informatica Amsterdam 1098XG
University of New Hampshire Durham, NH 03824
Enrico Camporeale
Michèle Sebag
CIRES, University of Colorado Boulder, CO
CNRS - Univ. Paris-Saclay
ABSTRACT
This paper tackles new regression problem, called Dynamic Time-Lag Regression (DTLR), where
signal drives effect signal with unknown time delay. The motivating application, pertaining to space weather modelling, aims to predict the near- Earth solar wind speed based estimates of the Sun's 's coronal magnetic field. DTLR differs from mainstream regression and from sequence-to-sequee learning in two respects: firstly, ground truth (e.g.. pairs of associated sub- sequences) is available; secondly, the cause signal contains much information irrelevant to the effect signal (the solar magnetic field governs the solar wind prop- agation in the heliosphere, of which the Earth's 's magnetosphere is but a minuscule region).
A Bayesian approach is presented to tackle the specifics of the DTLR problem, with theoretical justifications based on linear stability analysis. A proof of concept on synthetic problems is presented. Finally, the empirical results on the solar wind modelling task improve on the state of the art solar wind forecasting.
INTRODUCTION
significant body of work in machine learning concerns the modeling of spatio-tempon phenomena (Shi and Yeung 2018: Rangapuram et al. 2018), including the causal analysis of time series PPteer let 2011)), with applications ranging from markets (Pennacchioli al. 2014] to bioinformatics (Brouard et al. 2016) to climate (Nooteboom et al. (2088).
This paper focuses on the problem of modeling the temporal dependency between spatio-temporal phenomena, where the latter one is caused by the former one (Granger] 1969 Runge 2018 with non-stationary time delay.
The motivating application domain is that of space weather. The sun, a perennial source of charged energetic particles, at the origin of geomagnetic phenomena within the sun-carth system. Specifi- cally, the sun ejects charged particles into the surrounding space in all directions and some of these particle clouds, a.k.a. solar wind, reach the Earth's vicinity. High speed solar wind is a major threat for the modern world, causing severe damages e.g., satellites, telecommunication infrastructures, under sea pipelines, among others]
A key prediction task thus is to forecast the speed of the solar wind in the vicinity of the Earth (Munteanu al. 22133; [Haaland et al., 2010 Reiss et al., 2019), sufficiently early to emit an alarm and be able to prevent the damage the best possible extent. Formally the goal is to model the dependency between heliospheric observations (available at light speed), referred to as cause series, and the solar wind speed series recorded at the Lagrangian point L1 (a point on the Sun-Earth line 1.5 million kilometers away from the Earth), referred to as effect series. The key difficulty is that the
'The adverse impact of space weather is estimated to cost 200 400 million USD per year, but can sporadically lead to much larger losses.

---

Published as a conference paper at ICLR 2020
time lag between input and its effect, the solar wind recorded at L1, varies from circa 2 to 5 days depending on, among many factors, the initial direction of emitted particles and their energy. Would the lag be constant, the solar wind prediction problem would boil down to mainstream regression problem. The challenge here is to predict, from the solar image xxt) at time / the value y(t + At) of the solar wind speed reaching the earth at time t lag At depend on zz)).
Át where both the value y(t At) and the time
Related work. Indeed, the modeling of dependencies among time series has been intensively tackled (see e.g., Zhou and Sornette (2006); Runge (0188), When considering varying time lag, many approaches rely on dynamic time warping (DTW) (Sakoe and Chiba 1978). For instance, DTW is used in Gaskell et al. (2015), taking a Bayesian approach to achieve the temporal alignment of both series under some restricting assumptions (considering slowly varying time lags and linear relationships between the cause and effect time series). More generally, the use of DTW in time series analysis relies on simplifying assumptions on the cause and effect series (same dimensionality and structure) and builds upon available cost matrices for the temporal alignment.
Also related is peuueecc-to-sqqeene learning (Sutskever al., 20144) primarily aimed to machine translation. While Seq2Seq modelling relaxes some former assumptions (such as the fixed or comparable sizes of the source and target series), still relies on the known segmentation of the source series into disjoint units (the sentences), each one being mapped into a large fixed-size vector using an LSTM; and this vector exploited by another LSTM to extract the output sequence. Attention-baan based mechanisms Graves (2013); [aahdanau et al. (2015) alleviate the need to encode the full source sentence into a fixed-size vector, by learning the aligmment and allowing the model to search for the parts of the source sentence relevant to predict target part. More advanced attention mechanisms (Kim et al. 2017 Vaswani et al. 2017) refine the way the source information is leveraged to produce target part. But to our best knowledge, the end-to-end learning of the requence-to-sequeee modelling relies on the segmentation of the source and target series, and the definition of associated pairs of segments (e.g. the sentences).
Our claim is that the regression problem of predicting both what the effect is and when the effect is observed, called Dynamic TimeLag Regression (DTLR). constitutes a new ML problem: With respect to the modeling of dependencies among time series, it involves stochastic dependencies of arbitrary complexity; the relationship between the cause and the effect series can be non-linear (the what model). Furthermore, the time lag phenomenon (the when model) can be non smooth (as opposed to e.g. /Zhou and Sornette (2066))
With respect to eqquence-to-seqeene the model cannot rely on pairs of associated units (the sentences), adversely affecting the alignment learning.
translation, a main difference is that the end-to-end training of
Lastly, and most importantly, in the considered DTLR problem, even if the cause series has high information content, only small portion of is relevant to the prediction of the effect series. On one hand, the cause series might be high dimensional (images) whereas the effect series is scalar; on the other hand, the cause series governs the solar wind speed in the whole heliosphere and not just in near-Earth space. In addition to avoiding typically one or two orders of magnitude expansion of an already large input signal dimension, inserting the time-lag inference explicitly in the model can also potentially improve its interpretability.
Organization of the paper. The Bayesian approach proposed to tackle the specifies of the DTLR regression problem is described in section the associated learning equations are discussed, followed by stability analysis and proof of consistency (section 3).. The algorithm is detailed in section The experimental setting used to validate the approach is presented in section 5 the empirical validation on toy problems and on the real-world problem are discussed in section
2
PROBABILISTIC DYNAMICALLY DELAYED REGRESSION
2.1 POSITION OF THE PROBLEM
Given two time series, the cause series x(t) (x(t) E X C R) and the observed effect series y(t), the sought model consists of a mapping f() which maps each input pattern x(t) to an output v(o(t)). and mapping s() which determines the time delay ((t) between the input and output patterns:
2

---

Published as a conference paper at ICLR 2020
v(o(1) = s(x(1) p(1) ++ g((0]
with
f:X
and g: g:X-R*
where I+ represents the continuous temporal domain. The input signal x(t) is possibly high dimensional and contains the hidden cause to the effect y(t) E I: y(t) is assumed to be scalar in the remainder of the paper. Function g X It represents the time delay between inputs and outputs. Vectors are written using bold fonts.
As said, Eqs] 1 define a regression problem that differs from standard regression in two ways: Firstly, the time lag xt) is non-stationary as it depends on x(t). Secondly, g[x(t)] is unknown, i.c. it is not recorded explicitly the training data.
For the sake of the model identifiability and computational stability, the time warping function o(t) t + g[x(t)] is assumed to be sufficiently regular w.r.t. t. Formally, o(.) is assumed to
Assumption.
be continuous
2.2 PROBABILISTIC DYNAMIC TIME-LAG REGRESSION
For practical reasons, cause and effect series are sampled at constant rate, and thereafter noted x and y/ with in N. Accordingly. mapping g maps x, onto a finite set 7 possible time lags. with Atmax an integer interval defined from domain knowledge. The unavoidable error due the discretization of the continuous time lag is mitigated along a probabilistic model, associating to each cause x. set of predictors (xx = (x), E T) and a probability distribution p(x) on T estimating the probability of delay of the effeets of x. Overall, the DTLR solution is sought probability distribution conditioned
T = {ft
GaussiannFeentteeed centered on the predictors (x). where the mixture weights are defined from p(x). More formally, letting y: denote the vector of random variables (yeti, T):
cause mixture of Gaussian
P -
Σ (TEE
( E(T)
(3)
with Σ = (iag(ax(T))) the diagonal matrix of variance parameters attached to each time-lag i E T. Two simplifications are made for the sake of the analysis. Firstly, the stochastic time lag modelled
as the vector (T:). E T of binary latent variables, where indicates whether x drives Yt+i (T: 1)
or not (T, 0). The assumption that every cause has single effect is modelled by imposing: :
Σ T =1 SE
(4)
From constraint (4). probability distribution p(x) thus is sought as the vector (xx(x) for in T, summing to 1, such that (x) stands for the probability the effect of x, = x to occur with delay i. The second simplifying assumption is that the variance (r( of predictor ÿ; does not depend on by setting:
o.(T) =-( Σ asjjT
with o² default variance and 0æj 2 matrix of non-negative real parameters. This particular formulation supports the tractable analysis of the posterior probability of T; (in supplementary
2 For some authors (Zhou and Sornette 2006 monotonicity of d(.) is additionally required and enforced using constraints:
d(t.) S 6(t2), Vt, I tz.
This assumption is not retained as one may achieve a similar effect by using regularization based smoothness penalties.
Note that pre-processing can be used if needed to map non-Gaussian data onto Gaussian data. Note however that the cause-effect correspondence might be many-to-one, with effect depending on several causes.
3

---

Published as a conference paper at ICLR 2020
material). The fact that influence Y; through predictor (x) when т 0 reflects an indirect influence due to the auto-correlation of the y series. This influence comes with a higher variance, enforeed by making aij decreasing function of j - j More generally, a large value of Ozii compared to Ozj for # j corresponds to a small auto-correlation time the effect series.
2.3 LEARNING CRITERION
The joint distribution is classically learned by maximizing the log likelihood of the data, which can here be expressed in closed form. Let us denote respectively the dataset and parameters as {(x, y) Fatt
and (ý, p. 2,,,a)) a). From Eq.
3 the conditional probability q: (x(x.))
def P( 1|x, y) reads:
yx(x,y) ( y P: (x) exp
a3 رز (x)x))
log(1 + +af+))
(5)
2
JET
with normalization constant
yxx, - (x)(x) exp
a3i (x) +
Σ JET
log(1 az;))
20
RET
JET
The log-like likelikelhoodd then reads (intermediate calculations supplementary material, appendix A):
C[{ [x((xx (xx 0 -
Edata
1 ΣS 20 (x))) log (z)
Σ iET
(6)
where Edata denotes averaging over the dataset. For notational simplicity, the time index is omitted the following and the empirical averaging on the data noted Edota- The peper-parameters a and matrix of the model are obtained by optimizing L:
Estara [V: U (x)) 2 Oi (X)
o²
Edata ( (x, y)
In addition the optimal y and p reads:
Edata (i(11 + E aigg((xx)))
(x(x)
Edata + EjEr (x, (x,
(x) Edora (xx y))
where the above conditional empirical averaging operates as an averaging over samples close to x.
These are self-consistent equations, since (x. y) depends the parameters o² and agj- and p. The proposed algorithm detailed in section section|4iimplement implements the saddle point method defined from Eqs (589) 9 alternatively, hyper- arper-parametess and aij updated from Eq. (7) based on the current 9; and pi; and predictors Ú; and mixture weights Pú: are updated according to Eqs (8) and (9 respectively.
THEORETICAL ANALYSIS
The proposed DTLR approach is shown to be consistent and analyzed the simple case where a is diagonal matrix (aij = ad:z).
3.1 LOSS FUNCTION AND RELATED OPTIMAL PREDICTOR
Let us assume that the hyper-paramettee of the model have been identified together with predictors 9. (x) and weights (x). These are leveraged to achieve the prediction of the effect series. For any given input x, the sought eventual predictor is expressed as (i)(x, [(x)) where i(x) is the predicted time lag and (x) the predicted value. The associated L2 loss is:
(2 1) Edata (Vios x(x)) P].
(10)
Then it comes:
4

---

Published as a conference paper at ICLR 2020
With same notations as in Eq. predictor (v",I*) is given by
Proposition 3.1.
13 with aij -ddij, 0, the optimal composite
y (x) (x)
with
(x) arg
Proof. In supplementary material, Appendix C
3.2 LINEAR STABILITY ANALYSIS
The saddle point (Eqs 7
5 g 9 admits among others degenerate solution, corresponding to
(x) = 1/[7], ogy = for all pairs (i,5), with o² = o- Informally the model converges toward
this degenerate trivial solution when there not enough information to build specialized predictors d;.
Let us denote Ay² (ys- (x)) the square made by predictor ù for and
1
o Edtata Σ Axp(x))
NET
the average of MSE over the set of the predictors Di, İET.
Let us investigate the conditions under which the degenerate solution may appear, by computing the Hessian of the log-likelihood and its eigenvalues. Under the simplifying assumption
Oij ad.y
the model involves 2|7] functional parameters y and p and two hyper-parameters a and r = o² o²/0 After the computation of the Hessian (in supplementary material, Appendix B) the system involves three key statistical quantities, two global ones:
C [i
Edata
Σ TET
4:(x, y)
(11)
0
C21ql = Edota Σ
q((xx)) A1 Σ ()
(12)
ET
JET
and local veetor of components
u, [x ql = %
Eduta yx
y))( JET
Up to a constant, C1 represents the covariance between the latent variables { and the normalized predictor errors. C1 smaller than one indicates a positive correlation between the latent variables and small errors; the smaller the better. For the degenerate solution, i.e. q = qo uniform, [i = 1 and C2[9o] represents the default variability among the prediction errors. u,(x, xx ql informally measures the quality of predictor i relatively to the other ones at x. More precisely, a negative value of u, |x. q] indicates that iú is doing better than average in the neighborhood of
At a saddle point the parameters are given by:
o2
T - Ci
T 1- Ci
and a T - C1 q
T
The predictors y are decoupled from the rest whenever they are centered, which we assume. So the analysis can focus the other parameters.
If p fixed a saddle point is stable iff
C2(ql < 20Hial 0 1
(T]
In particular, the degenerate solution is unstable if
1
C2(qo] 2(1
IT
Note that for (x) iid centered with variance of and relative kurtosis (conditionally to x) one has C2 = (2 + (1- 1/(T'). Therefore, whenever 2y, fluctuates and the relative kurtosis is non-negative, the degenerate solution is unstable and will thus be avoided.
5

---

Published as a conference paper at ICLR 2020
If is allowed to evolve (after Eq. 9 the degenerate trivial solution becomes unstable as soon C2 [olq is non-zero, due to the fact that the gradient points in the opposite direction to u(x) (with dp(x) - -C 4o (x)),, thus rewarding the predictors with lowest errors by increasing their weights.
The system is then driven toward other solutions, among which the localized solutions of the form:
(x) 6s,((x)
with an input dependent index [(x) E As shown (in supplementary material, Appendix C the maximum likelihood localized solution also minimizes the loss function (Eq. T0). The stability of such localized solutions and the existence of other (non-localized) solutions is left for further work.
4 THE DTLR ALGORITHM
The DTLR algorithm learns both regression models fxx) and p(x) from series x, and yt, using alternate optimization of the model parameters and the model hyper-parameters and o² after Eqs 75... The model search space is that neural nets, parameterized by their weight vector 0. The inner optimization loop updates 0 using mini-batch based stochastic gradient descent. At the end of each epoch, after all minibatches have been considered, the outer optimization loop computes prper-parameters a and o² on the whole data.
Initialization of and
it < 0:
while < max do
while epoch do
0
Optimize(L(0,a, o²)))
end
2
[T| C7
0
IT
1-C.
IT|-1 Ci
end
Result: Model parameters = (y. p). hyper-parameters a, o² Algorithm 1: DTLR algorithm
The algorithm code is available in supplementary material and will be made public after the reviewing period. The initialization of parammeerssssrss a and a is settled using preliminary experiments (same setting for all considered problems: a U0..55, 2); o² U(10-5 5,5)..
The neural architecture implements predictors y(x) and weights p(x) on the top of a same feature extractor from input In the experiments, the architecture of the feature extractor is 2-hidden layer fully connected network. On the top of the feature extractor are the single layer y and p models, each with (T) output neurons, with [T] the size of the chosen domain for the time lag.
EXPERIMENTAL SETTING
The goal of experimental validation threefold. first issue regards the accuracy of DTLR, measured from the mean absolute еттоr (MAE), root mean square eгror (RMSE) and Pearson correlation of learned DTLR model (y* (x:), (x)). DTLR compared to natural baseline defined as regressor with constant time lag. Vx (x+), with A being the average of all possible time lags in T. The predictions of DTLR and the baseline are compared with the ground truth value of the effect series. However the predicted time lag (xx) can only be assessed if the ground truth time-lag relationship is known. In order to do so, three synthetic problems of increasing difficulty are defined below.
Secondly, the stability and non-degeneracy of the learned model assessed from the statistical quantities on and C₁ (section (.22) compared to the degenerate solution (x = 1/[T]. For C₁ < 1, the model accurately specializes the found predictors P..
Lastly, and most importantly, DTLR assessed on the solar wind prediction problem, and compared to the best state of the art in space weather.

---

Published as a conference paper at ICLR 2020
Synthetic Problems. Four synthetic problems of increasing difficulty generated using Stochas- tic Langevin Dynamics. In all problems, the cause signal X2 E R16 and the effect signal y/ are generated as follows (with 1 0.02, s² 0.7):
Xx+1 (1 -1)xx +N(00,
(13)
22 k] [a|xxll22 +
(14)
(x+)(X.) = f(v)),
(15)
with time-lag mapping (xx) ranges in time interval with width 20 (except for problem I where [T] 15). The complexity of the synthetic problems governed by the amplitude and time-lag functions f and (more in appendix, T)ble
Problem
Sie.)
((x))
Other
k=10,e=0
II
100/0m
k=1,
III IV
(v %h +2ad-v)/a
g (/xppeec)///ttsoppe/////
k=10,c=10
Solar Wind Speed Prediction. The challenge of predicting solar wind speed from heliospheric data is due to the non-stationary propagation time of the solar plasma through the interplanetary medium. For the sake of fair comparison with the best state of the art Reiss et al. (2019). the same experimental setting is used. The cause series x; includes the solar magnetic (flxx tube expansion. FTE: the rate at which the magnetic flux tubes expand between the photosphere and reference height in the corona at 2.5 solar radii) and the coronal magnetic field strength estimates produced by the Current Sheet Source Surface model (Zhao and Hoeksema) 1995: Poduval and Zhao, 2014] Poduval 2016), exploiting the hourly magnetogram data recorded by the Global Oscillation Network Group from 2008 to 2016. The effect series, the hourly solar wind data is available from the OMNI data base from the Space Physics Data Facility After domain knowledge, the time-lag ranges from 2 to 5 days, segmented in six-hour segments (thus T 12). For the i-th segment, the "ground truth" solar wind y; is set to its median value over the hours.
DTLR is validated using nine fold cross-validation (Table in appendix), where each fold is continuous period corresponding to a solar rotation
6 EMPIRICAL VALIDATION
Table summarizes the DTLR performance on the synthetic and solar wind problems (detailed results provided in the appendix).
Table 1:
DTLR performance: accuracy (MAE and RMSE, the lower the better; Pearson, the higher the better) and stability on and C₁ (the lower the better). For each indicator, is reported the DTTR value (9-fold CV), the baseline value and the time-lag emror.
Problem
M.A.E
R.M.S.E
Pearson Corr.
F
C₁
8.82 / 21.79/ 0.021
12.35 28.79/..77...
0.98 / 00.77 /-
29.8
0.14
II
10.15 / 27.40/ 0.4
13.70 23.70/5..11/2 35.11/0.66
0.95 / 0.73 0.70
26.83
0.16
III IV
/.17/11.01. 0.17
4.63 14.991 0.42
0.98 / 0.79 0.84
11.84
0.09
3.88 12.28 0.34
5.33 15.89 0.64
0.98 /0.79/ 0.81
12.18
0.13
Solar Wind
56.35 / 66.45/-
74.20 84.537
0.6 0.417-
76.46
0.89
6.1 SYNTHETIC PROBLEMS
On the easy Problem DTLR predicts the correct time lag for 97.93% of the samples. The higher value of do in problems and II compared to the other problems is explained from the higher variance the effect series y(t).
httpps: / cmm web. fs nasa.ggao.
"The Sun completes rotation (or Carrington rotation) in approximately 27 days.
7

---

Published as a conference paper at ICLR 2020
On Problem II, DTLR accurately learns the inverse relationship between Nt. (xx)) and yt average. The time lag is overestimated in the regions with low time lag (with high velocity), which is blamed on the low sample density in this region, due to the data generation process. Interestingly, Problems III and IV are better handled by DTLR, despite a more complex dynamic time lag relationship. In both latter cases however, the model tends under-estimate the time lag in the high time lag region and conversely to over-estimate it in the low time lag region.
6.2 THE SOLAR WIND PROBLEM
DTLR finds an operational solar wind model (Table I. though the significantly higher difficulty of the solar wind problem is witnessed by the C₁ value close to the degenerate value 1. The detailed comparison with the state of the art Reiss al. (2019) (Fig. I Left) shows that DTLR improves on the current best state of the art (on all variants including ensemble approaches, and noting that median models are notoriously hard beat). (Fig. Right) shows the good correlation between the predicted solar wind and the measured solar wind.
Model
M.A.E
R.M.S.E
WS
74.09
85.27
600
DCHB
83.83
103.43
WSA
68.54
82.62
Ensemble Median (WS)
71.52
83.36
(empp)
500
Ensemble Median (DCHB)
78.27
100.04
Ensemble Median (WSA)
62.24
74.86
Persistence (4 days)
130.48
161.99
400-
Persistence (27 days)
66.54
78.86
Fixed Lag Baseline
67.33
80.39
DTLR
60.19
72.64
300
300
400
500
650
(a) Comparative assessment the Solar Wind problem compared to the state of the art Reiss et 2019 Table I)
(km/s)
(b) Scatter Chart (9 fold CV)
Figure I: DTLR the solar wind problem. Left: comparative quantitative assessment w.r.t. the state of the art (Carrington rotation 2077). Right: qualitative assessment of the prediction.
DISCUSSION AND PERSPECTIVES
The contribution of the paper is twofold. A new ML setting, Dynamic Time Lag Regression has been defined, aimed at the modelling of varying time-lag dependency between time series. The introduction of this new setting is motivated by important scientific and practical problem from the domain of space weather, an open problem for over two decades.
Secondly, Bayesian formalization has been proposed tackle the DTLR problem, relying saddle point optimization process. A closed form analysis of the training procedure stability under simplifying assumptions has been conducted, yielding a practical alternate optimization formulation, implemented in the DTLR algorithm. This algorithm has been successfully validated on synthetic and real-world problems, although some bias toward the mean has been detected in some cases.
On the methodological side, this work opens short term perspective (handling the bias) and longer term perspective, extending the proposed nested inference procedure and integrating the model selection step within the inference architecture. The challenge is to provide the algorithm with the means of assessing online the stability and/or the degeneracy of the learning trajectory.
Regarding the motivating solar wind prediction application, next step consists of enriching the data sources and the description of the cause series x,, typically by directly using the solar images. Another perspective is to consider other applications of the general DTLR setting, e.g. considering fine-grained modelling of diffusion phenomena.
Thhe predicted values, every 6 hours, are interpolated for comparison with the hourly measured solar wind.
8

---

Published as a conference paper at ICLR 2020
REFERENCES
Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio Neural machine translation by jointly learning to align and translate. In Proc. ICLR 2015. 2015.
Céline Brouard, Huibin Shen, Kai Dührkop, Florence d' Alché-Buc, Sebastian Böcker, and Juho Rousu. Fast metabolite identification with input output kernel regression. Bioinformatics, 32 (12):28-36, 2016. doi: 10.1099/hoonfommaiis/ooott
bloinformatics//wwee
forecasting with time-varying lags. Journal of Forecasting, 35(1):70-85, 2015. Paul Gaskell, Frank McGroarty, and Thanassis Tiropanis. Signal diffusion mapping: Optimal
C. J. Granger. Investigating causal relations by econometric models and cross-spectral methods. Econometrica, 37(3):224-38, 1969.
Alex Graves, Generating sequences with recurrent neural networks. CoRR, 0bs//338.082 2013. URL http:::/aaxii..rrg/// haapr///ab//1100 0850
S. Haaland, C. Munteanu, and B. Mailyan. Solar wind propagation delay; Comment minimum variance analysis-based propagation of the solar wind observations: Application to real-time global magnetohydrodynamic simulations by A. Pulkkinen and L. Raststatter. Space Weather, 8(6), 2010.
Yoon Kim, Carl Denton, Luong Hoang, and Alexander M. Rush. Structured attention networks. In Proc. ICLR 2017. 2017.
C. Munteanu, S. Haaland, B. Mailyan, M. Echim, and K. Mursula. Propagation delay solar wind discontinuities: Comparing different methods and evaluating the effect of wavelet denoising. Journal of Geophysical Research: Space Physics, 1iaiss,,11888 118(7):3985 3994, 2013.
P. D. Nooteboom, Q Y. Feng. C. López, E. Hernández- and H. A. Dijkstra. Using network theory and machine learning to predict niño. Earth System Dynamics, 9(3):969 983, 2018. doi: 10.5 1..119//sdd-9999-200 2018/]
URL https: //wp://wwwww...... earth syst www.m.aentt/.../mmmmmmm
D. Odstrčil. Modeling 3-d solar wind structure. Advances in Space Research, 3244):497-566, 2003. ISSN 0273-1177 doi: http://bb.nn///llo/ss02221111110 URL (tLLtttp:/// sciencedirect. httppp/cceeccttcocccc/eeeeerrrap httpt////..//........///////////
Heliosphere Heliosp
at Solar Maximum.
D. Odstrčil and V. J. Pizzo. Three-dimensional propagation of coronal mass ejections (cmes) in structured solar wind flow: 1. cme launched within the streamer belt. Journal of Geophysical Research:
Space Physics, 104(A1):483 492, 1999a. doi: 10. 10.1100911981119000010 URL https
//aqupubs on ineeiinararry wi wiley cwt.doo/abs//cocoooctt.occc. htt.:////.....0011111100.0000000
D. Odstrëil and V. J. Pizzo. Three-dimensional propagation of coronal mass ejections (cmes) in structured solar wind flow: 2. cme launched adjacent to the streamer belt. Journal of Geophysical Research: Space Physics, (04(A1):493-503, 1999b. doi: 10. 1029/ 9981A900038. URL https
D. Odströil, Smith, and M. Dryer. Distortion of the heliospheric plasma sheet by inter- planetary shocks. Geophysical Research Letters, 23(18)2521-2524 1996. doi: 1029/ 96GL00159. URL httpeeeeeaaaaaaalllaaaaooeo////a httpa/////....11//999/0000000000 1029/ 96GL00159.
//agupubs. onwinennnnnnnnennennennnnennnnnn libraryy.rrry. wi ley. wtttly/oooooo////oooo////
onl www.onniinnnarrrrroooiaaoaaaaaae ibrrar.rr....r www.wwwee....... wwwwwwwwwow.......o www.///o/////aaeeeeeeeeee
D. Odstrčil, P. Riley, and X. P. Zhao. Numerical simulation of the 12 may 1997 interplanetary event. Journal of Geophysical Research: Space Physics, 109(A2), 2004. doi: 1029/ 2003JA010135. URL httl:tttt:///agpppaa htgpppbb///uuuu....pp /ttps////aaattt///uuuuussssssss www.:uus....innnniinnnnnni cattaaarrywillle.... com/ doim//di//// abs 0129/2003JAA10113
Diego Pennacchioli. Michele Coscia, Salvatore Rinzivillo, Fosca Giannotti, and Dino Pedreschi. The retail market as complex system. EPJ Data Sci., 3(1):33, 2014.

---

Published as a conference paper at ICLR 2020
Jonas Peters, Dominik Janzing, and Bernhard Schölkopf. Elements of Causal Inference Foundations and Learning Algorithms. MIT Press, 2017.
B. Poduval.
Controlling Influence of Magnetic Field on Solar Wind Outflow: An Investigation Using Current Sheet Source Surface Model. The Astrophysical Joumaal 827(1):L6, aug 2016. doi: 10. 01577220441-82258877/1888 http/////e..e//aaaeee..e........ http://pp..eeee...oooooorrrraaaa
B. Poduval and X. P. Zhao. Validating Solar Wind Prediction Using the Current Sheet Source Surface Model. The Astrophysical Journal, 782(2):L22, jan 2014. doi: 10.1088/2041-8205/ 782/2/122. URL htt.:::s//ggssllee....o....o http:///wppppceeee.....o.rrrrrra 10t.:///.rr.a.......111100000000 782/2 / L22/222//aet meta.
Syama Sundar Rangapuram, Matthias W. Seeger, Jan Gasthaus, Lorenzo Stella, Yuyang Wang, and Tim Januschowski. Deep state space models for time series forecasting. In NeurIPS 2018, pages 7796-7805, 2018.
Martin A. Reiss, Peter J. MacNeice, Leila M. Mays, Charles N. Arge, Christian Möstl, Ljubomir Nikolic, and Tanja Amerstorfer. Forecasting the ambient solar wind with numerical models. i. on the implementation of an operational framework. The Astrophysical Journal Supplement Series, 240(2):35, 2019.
J. Runge. Causal network reconstruction from time series: From theoretical assumptions to practical estimation. Chaos: An Interdisciplinarry Journal of Nonlinear Science, 28(7):075310, 2018. doi: 10.1063/1.5025500 URL /t 50550 5025050
H. Sakoe and S. Chiba. Dynamic programming algorithm optimization for spoken word recognition.
IEEE Transactions on Acoustics, Speech, and Signal Processing, 26(1):43-49, 1978.
Xingjian Shi and Dit-Yan Yan Yeung. Machine learning for spatiotemporal sequence forecasting: A
survey, ArXiv, abs/ 808.06865, 2018.
llya Sutskever, Oriol Vinyals, and Quoc Le. Sequence to sequence learning with neural networks. In Z. Ghahramani, M. Welling, C. Cortes, N. D. Lawrence, and K. Q. Weinberger, editors, Advances in Neural Information Processing Systems 27, pages 3104-31 11143112222 12. Curran Associates, Inc., 2014.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, E. ukasz Kaiser, and Illia Polosukhin. Attention is you need. I. Guyon, U. V. Luxburg. S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, editors, Advances in Neural Information Processing Systems 30, pages 5998-60900 Curran Associates, Inc., 2017.
Y. M. Wang and Jr. Sheeley, N. R. Solar Wind Speed and Coronal Flux-Tube Expansion. The Astrophysical Journal, 355:726, Jun 1990. doi:
Zhao and JT Hoeksema. Predicting the heliospheric magnetic field using the current sheet-source surface model. Advances in Space Research, (6(9):181-188 1995.
Wei-Xing Zhou and Didier Sornette. Non-parametric determination real-time lag structure between two time series: The optimal thermal causal path method with applications economic data. Journal of Macroeconomics, 28(1):195 224, 2006.
APPENDIX A LOG LIKELIHOOD OF THE LATENT MODEL (3)
A.1 DIRECT COMPUTATION
Due to the single effect constraint (4) the mixture model (3) be expressed simply
P(y|x) = Σ
(x
II JET
I aj e - (1+a0) (=()))
2no²
1T
+ aj e 202 a (n (oo=)))))))
= Σ (x)
II
exp
Σ
(y 0, (x))))
2n
20
E
JET
JET
10

---

Published as a conference paper at ICLR 2020
Let 0 def (ý.P. G, (,aaa a)) denote the parameters of the model and consider the probability that predictor 9: is the good one conditionally to pair of observation (I, y):
y) P(r: = y)r,y)
( y y10) Pr (z(()) exp
1
Σ
a3ı - 0){()))
+
1
Σ
log(1 + azi)
2
2
JET
JET
with
y(x Ed((1) exp
aji (1j - (x))
+
Σ
log(1 1 + ag:))
20
2
SE
JET
JET
This gives immediately
-7]10(((())
Edata
Σ SET
20 (yi (x())) - log -log(Z(x; (2xx,
A.2 LARGE DEVIATION ARGUMENT
Even though the log likelihood can be obtained by direct summation, for sake of generality we show how this can result from a large deviation principle. Assume that the number of learning samples tends infinity, and so that in small volume du drdy around given joint configuration (æ, y). the number of data Nz.y becomes large. Restricting the likelihood to this subset of the data yields the following:
Na C II
p(r(m) (x)
1 Σ ET
(y: (x()))
Ihe v (T()))
exp
o;(((m))2
m=1 (T(m))
Upon introducing the relative frequencies:
Now
(xx
1
(m)
satisfying
Σ =) =1
Na.
m=1
ET
the sum over the (() is replaced by sum over these new variables, with the summand obeying large deviation principle
Luy Σ
exp (eNe-NN
9
where the rate function reads
1+ ΣjET Ugjli
Fz, [q] [T] lo) + 1 ( (xxz))
log(1 azj) + qi log 4i
20
2 JET
p.
FET
Taking the saddle point for yield as a function of (x, y) expression (7). Inserting this into F and taking the average over the data set yields the log likelihood (5) with opposite sign:
C2( y) |
-Edata Fee.y [q(
A.3 SADDLE POINT EQUATIONS
Now we turn to the self-consistent equations relating the parameters 0 of the model at a saddle point of the log likelihood function. First, the optimization of the predictors ý yields:
ac
Edotaa (w-- (x(x)) + Σ (x
(0.())
JET
Then the optimization of p gives:
0C
(x.5)) \(x) x
Edota
(z:
pa(x))
Edata [2((:,,))/3] -xxx)
(x(x)
11

---

Published as a conference paper at ICLR 2020
with A(x) Lagrange multiplier to insure that Σ, P2(x) -1 This gives
1
pa(x)=
Edata (x)
1(x)
Hence
Ed((z)
=1
Vx
((x)
iET
in order to fulfiil the normalization constraint, yielding finally expression (9),
Finally the optimization of reads:
Oc
1 Edata (x,
1
Edora [(vi (z
dazj
2(1 ass))
20
APPENDIX B PROOF OF PROPOSITION 3.1
Given (xx) candidate index function associate the point-like measure
pi(x) dg,((z)
Written in terms of the loss function reads
(z( Eay (p(())(
Under (3) (with ouj = (x65s)) the loss is equal to
C2(0.P) Exg Σ p((()) (((0) -(a)) p((x) ao² -
) +0²
The minimization w.r.t. 9 yields
0(x) Σ (x)))))))).
(16)
iET
In turn, as a function of the loss being convex combination, its minimization yields
(x(x)= 8 8z,((x):
(17)
0(x))² - (.(x)) ag²
I(x) = arg min ((=)) sET
(18)
1 a
Combining these equations 16
18
we get
T(x)
arg max (f:(z)), iET
which concludes the proof.
APPENDIX C STABILITY ANALYSIS
The analysis is restricted for simplicity the case aú adzj. The log likelihood function of 0²/ // and B = a/r after inserting the optimal q(z,y) reads in that case
C(r,8) = - [T]
[T] 2r
log(r)
log(1 + r() + Edata log(Z) - Mxx) Σ (x(x))
with
B Z Σ (x(r)
Ax)(
20
12

---

Published as a conference paper at ICLR 2020
and where (2x) is Lagrange multiplier which has been added to impose the normalization of The gradient reads
0C 1
Br²
[ -r)+ 1 Br
Ör
2T2
Oc
2(1 + rr) Ci(g],
03
Oc
1
Edata [vi (1(x))) (1 xz) [)]
(z.
0C
Edata y)1c2 A(x),
(x)
(z(
with
C1 = 3
Edata Σ
Eg(t,))Aut
iET
This leads the following relation the saddle point:
T C1
TT
[T|
1 Ci
[T Ci [
Edata vi(1 + (2gi(2,
(x(x)
Edata [ + (ci(1,)) 27
px(x) = Edata
Let us now compute the Hessian. It is easy to see that the block corresponding to the predictors ý decouples from the rest as soon as these predictors are centered.
Denoting
Edata Ea(T,
- (1)r))) )
C2lql =
00
iET
1=1
we have
0C 1 /
ITI
-|T
+
T (G/1q -1) 3
0r 2/
0L 1
ciia
Or03
21
(Calal - 2c91al)
0B
LC
Edata (1, y) v)g, {z) (x(x) p0 (x)
(p:(x) (x)
Or
82C
us|2, ql 2p.(x)
030p((x)
13

---

Published as a conference paper at ICLR 2020
where
Ue [x. ql def
() Edtta ()
of
JET
There two blocks in this Hessian, the corresponding and and the corresponding to derivatives with respect to p,. The stability of the first one depends on the sign C2iqi 2C qq] for [7] large while the second block is always stable being average of the exterior product of the vector (q) (I, /)/i (z),..., 7 (/rr((()))))))))))) by itself. At the degenerate point a = 0, r = 1, P; = 1/[T| the Hessian simplifies as follows. Denote
dn drei + dBe2 + f dx (d((())e42(())
iET
a given vector of perturbations, decomposed onto a set unit tangent vectors, {er and 02) being respectively associated to and 3, while (2) associated to (xx for all E T and X. Denote
Σ
drw (x)
SE
v(x) Σ (x)
EE
with
C2
data
(1)(1))
1
Σ
(r)(
iToo
[T]
IET
JET
u [o]]
Edata A - oz1o].
With these notations the Hessian reads:
C2 1 (- + eie ezo + ( -
-f dov(
H =
uej - ezu'
In fact we are interested in the eigenvalues of H in the subspace of deformations which conserve the norm p. i.e. orthogonal to v(r), thereby given by
77 1/e 1722 + 73...
In this subspace the Hessian reads
1
0
C2
H
1
-
M2
2
0
-M/T|Ca
where M is the number of data points, resulting from the fact that
M
Σ
dru,
[x]]
Edata
Σ SET
- (z)
of
MC2,
because Edata (-1z) as function of is actually point-wise function on the data. [u|2 > 0 or if [u] and [T] 1) there is at least positive eigenvalue. Let A be such cigenvalue. After eliminating dr and d3 from the eigenvalue equations in dr,, the deformation along this mode verifies
dn x Aei - A(T + AAe2 M[T[([+++)@) which corresponds to increasing r and a while decreasing for each r the p. having the highest mean relative error [x]x).
14

---

Published as a conference paper at ICLR 2020
Concerning solutions for which
= ;
is concentrated on some index [(x), the analysis is more complex. In that case C2lp] = 0 and Ci[p] O. The (r, 3) sector has 2 negative eigenvalues, while the p block (-) covariance matrix, so it has as well negative eigenvalues. The coupling between these two blocks could however principle generate in some cases some instabilities.
Still, the log likelihood of such solutions reads
L =
[T|
log(o") + 2
log(1 aa
20 Edata
Σ TET
Ayy((z)
a Edata AUP(EE) (z))
2
20
so we get the following optimal solution
a²
Edtata
Σ
(1))
[Ti
NET
Edata
(x)
1
1+
x²
T(x)
arg min Edata TET
Aut (x)()))
15

---

Published as a conference paper at ICLR 2020
Source Surface
r=Rss
Cusp Surface
r= Rcc
Solar Surface
r= Rs
HCS
Figure 2: Geometry of the CSSS Model (Zhao and Hoeksema 1995 The spherical surface with radius Rep is the cusp surface, representing the locus of cusp points of helmet streamers and the location of the inner corona. The source surface (radius R.,) is free to be placed anywhere outside the cusp surface. Figure reproduced from Zhao and Hoeksema (1955).
APPENDIX D EXPERIMENTS: ADDITIONAL DETAILS
Here we provide some additional details and context to the experimental validation of the DTLR methodology described in section 5. Table 2ablepprone provides some information about the datasets used in the synthetic and solar wind prediction problem Sections D.1.1 and D.1.2 give additional plots for evaluating the experimental results.
For the solar wind prediction task, the solar wind data was mapped into standardized Gaussian space using a quantile-quantile and inverse probit mapping. Nine fold cross-validation was performed using splits specified in table 3 To compare the DTLR results with the state of the art solar wind forecasting, we used results from Reiss et al. (2019) Table 1). Since Reiss et al.] (2019) compared the various forecasting methods only one solar rotation (first row of table comparing these results with DTLR can be considered as preliminary examination. Nevertheless, the results presented in table Ta show encouraging signs for the competitiveness and usefulness of the DTLR method.
is rell-established that the solar wind speed observed near the Earth's 's orbit (at the L1 Lagrange point about million kilometers upstream solar wind) bears an inverse correlation with the rate (FTE) at which the magnetic flux tubes expand between the photosphere and reference height called the source surface, typically placed 2.5 solar radii in many coronal models (Wang and Sheeley, 1990). That is, regions of the Sun with smaller FTE give rise to fast wind and regions with larger values of FTE emanate slow solar wind. This inverse correlation forms the basis of the current solar wind prediction techniques (Poduval and Zhao 2014 Poduval (2016) including the WSA-ENLIIL model (aang and Sheeley, [990] Odstrëil al. 1996, Odstrëil and Pizzo 1999a b) Odstrëil 2003 Odstrëil
al. 2004) of NOAA/SWPC. Mathematically, FTE is defined as:
R3 R
B, (phot) B,sss)
FTE =
(19)
where, B, (phot) and B, (ss) are the radial component of magnetic fields at the photosphere and the source surface, and Rg and Res are the respective radii. The coronal magnetic field computed using CSSS model, the geometry of which is shown in figure 2 by extrapolating the photospheric magnetic field measured by various ground based observatories and spacecraft. For the present work,
'In the solar wind problem, the training and test data sizes correspond
vals-valio split
16

---

Published as a conference paper at ICLR 2020
used the photospheric field measured using the Global Oscillation Network Group (GONG),
network of ground based solar observatories over different places on the globe.
Table 2: Synthetic and World Problems
Problem
train
# test
T
I
10, 000
2,000
10
15
II
10, 000
2,000
10
20
III
10, 000
2,000
10
20
IV
10,000
2,000
10
20
Solar Wind 77,367
2,205
374
12
Table 3: Cross validation splits used to evaluate
DTLR on the solar wind forecasting task
Split ld
Carrington Rotation
Start
End
2077
2008/1 1/20 07:00:04
2008/12/17 14:38:34
2 2090
2009/1 1/09 20:33:43
2009/12/07 04:03:59
3 2104
2010/1 1/26 17:32:44
2010/12/24 01:15:56
4
2117
201 1/16 07:04:41
2011/12/13 14:39:28
5
2130
2012/1 1/04 20:39:43
2012/12/02
04:06:23
6
2143
2013/10025 10:17:52
2013/11/21 17:36:35
7
2157
2014/1 1/11 07:09:56
2014/12/08 14:41:02
8
2171
2015/1 1/28 04:09:27
2015/12/25 11:53:33
2184 9
2016/11/16 17:41:04
2016/12/14 01:16:43
17

---

Published as a conference paper at ICLR 2020
D.1 SUPPLEMENTARY PLOTS
D.1.1 SYNTHETIC PROBLEMS
Predicted Time Leg
Predkted Outout
300-
2+
200
Lag
Actual Output
(b) Problem II, Goodness of fit, Time lag
(a) Problem II, Goodness of fit, Output (xx
T(t)
Type prodicted actual
1.2
6-
Timel Lag
dersity
3-
200
300
400
Output
Error: Time Lag
(d) Problem II. Output Time Lag Rela-
(c) Problem II, Error of time lag prediction
tionship
Figure 3: Problem II, Results
18

---

Published as a conference paper at ICLR 2020
275
250
Predicted Time Lag
Prediceed Ouguut
225
6-
200
175
4-
150
150
175
200
Actual Output
Actual Time Lag
(a) Problem III, Goodness of fit, Output
(b) Problem III, Goodness of fit, Time lag r(t)
(x)
Type
predicted
actual
6.
Time Lag
density
0.5
0.0
150
135
200
225
750
275
Output
Lag
(d) Problem III, Output vs Time Lag Rela-
(c) Problem III, Error of time lag prediction
tionship
Figure 4: Problem III, Results
19

---

Published as a conference paper at ICLR 2020
150-
Predicted Time Lag
Prediceed Ouguut
5.0
50
150
Actual Output
Actual Time Log
(a) Problem IV, Goodness of fit, Output
(b) Problem IV, Goodness of fit, Time lag r(t)
(x)
Type
predictes
actual
Time Lag
density
0.5
0.0
50
160
150
Output
Lag
(d) Problem IV. Output vs Time Lag Rela-
(c) Problem IV, Error of time lag prediction
tionship
Figure 5: Problem IV, Results
20

---

Published as a conference paper at ICLR 2020
D.1.2 SOLAR WIND PREDICTION
700
600
500
kevv
lenbs
500
400
400
300
300
200
400
600
200
400
600
(hours)
(hours)
(a) Hourly forecasts for period 3008-11-20 07:00 to 2008-12-17 14:00
(b) Hourly forecasts for period 2016-11 1016-11-16 1-16 17:00 2016 12- 01:00
Figure 6: Solar Wind Prediction: reconstructed time series predictions
APPENDIX E NEURAL NETWORK ARCHITECTURE DETAILS
Table 4: Network Architecture Details
Problem
#
Hidden layers
Layer sizes
Activations
40, 40
[ReLU, Sigmoid]
II
2
40, 40
[ReLU, Sigmoid]
III
[40, 40
[ReLU, Sigmoid]
IV
2
[60, 40
[ReLU, Sigmoid]
Solar Wind
50, 50
[ReLU, Sigmoid]
connected
fully
G(rr), f(x))
Soft max
layers
Figure 7: Architecture of the neural network specified by the number of units (n" (n", n/t n1y 2(T() in each layer.
21


Reviews:
[{"review_id": "rkxgO3WwcB", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The paper is motivated by a specific application about the prediction of solar wind. It is a major threat to the modern world. Authors propose a new regression model called Dynamic Time-Lag Regression which is a special case for spatio-temporal data. For solving this regression problem, the authors use a standard Bayesian approach to fit the model. The analysis of saddle point in Section 3.2 is quite impressive and meaningful. Correspondingly, the practical DTLR algorithm iteratively updates the parameters and hyper-parameters. In experiments, it reveals that the proposed method improves the performance for this particular problem significantly. The application solved by authors is very important, and it is improved by the proposed method. However, the novelty of the work is not presented clearly.\n\nMain argument\n1.\tIt is not clear about the difference between the proposed regression model and others. In my knowledge, it is a quite normal model for spatio-temporal data.\n2.\tThe Bayesian approach and its practical algorithm are very common solutions. What is the highlighted content besides the linear stability analysis?\n\nFor the experiments, the following should be addressed.\n1.\tRMSE in Section 5usually stands for root mean square error. What does relative MSE mean?\n2.\tAuthors said, “After domain knowledge, the time-lag ranges from 2 to 5 days, segmented in six-hour segments.” Does the setting of segmentation have a significant effect?\n3.\tIt would be beneficial to empirically demonstrate the evidence for supporting the theory of avoiding saddle point by synthetic regression.\n\n", "extra_scores": {"experience_assessment": "I have read many papers in this area.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory."}, "extra_text": {}}, {"review_id": "rJg8LdshcH", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "\nThis paper proposes a bayesian method, Dynamic Time Lag Regression (DTLR), to predict effect-data y from a cause x. \nWhile both, cause and effect, are time series (x = x(t) and y = y(t)), the cause is assumed to have influence on the effect via a time lag, y(x(t), t) = y(phi(t)) = y(t + g(x(t))). The novelity of the problem posed is that the time lag g(x(t)) is allowed to be non-stationary and supposed to be unknown. The authors outlie a a Bayesian approach. The likelihood of the cause-effect data is conidered used to optimize the regressors for a given, finite set of time lags and the probability of these time lags. This discussion is very brief and detailled derivations are given in the appendix. An accompanying stability analysis is performed. In the implementation a neural network-based approach is used, where in turns the parameters of the model are optimised by a typical gradient descent algorithm and afterwards the hyperparemeters are set to values determined by the parameters. The theoretical deviation is accompanied by practical expirements on synthetically generated data and (real) space weather data. The performance of the model is compared to the one of a prediction with a fixed time lag, which it in fact outperforms. On the space weather (solar wind) data the model is shown to outperform several state-of-the-art models. A codebase for an implementation done in the Scala language is given.\n\nThe paper presented provides a meaningful contribution to the field of time series analysis and Bayesian methods. The novelty of the problem imposed as well as the accompanying solution is outlined very clearly. The problem is well justified and the motivation is clearly given.\n\nI recommend to accept it, even though a few details could have been described more precisely.\n\n1. The definition of the time lag g(x(t)) could have been made more clearly, providing a few examples at this point could help to clarify these definitions here.\n2. For the synthetic data generation a justification would be helpfull in order to motivate the choice of the functions f and v. \n3. A chart with sample time series data (y) against the time would be helpful to better visualize the problem.\n ", "extra_scores": {"experience_assessment": "I have published one or two papers in this area.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper thoroughly."}, "extra_text": {}}, {"review_id": "Syx3u5oKFB", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The topic is out of my scope. I talked to the PC about it, and the PC asked me to continue reviewing this paper. So I tried to understand this paper, and I found it is difficult for me. I am not sure if the writing is not good or for some other reasons. I read Section 2.1, and I thought the following description would discuss how to solve \\phi(t). Unfortunately, I did not find where \\phi(t) is mentioned in Section 2.2. Maybe, I misunderstood Section 2.2. \n\nI do not know how to rate this paper. To avoid killing a good paper, I rated this paper as Weak Accept, and leave the decision to other reviewers and the AC.", "extra_scores": {"experience_assessment": "I do not know much about this area.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I did not assess the derivations or theory.", "review_assessment:_checking_correctness_of_experiments": "I did not assess the experiments.", "review_assessment:_thoroughness_in_paper_reading": "N/A"}, "extra_text": {}}, {"review_id": "HyleWB2CYH", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The paper presents a new sequence to sequence regression problem in which every element of the input sequence contributes to the output sequence after an unknown delay. The setup is motivated by a problem in space weather that seeks to predict the solar wind from solar observations. \nA mixture of Gaussian-based model is proposed and a closed-form update for the maximum likelihood problem is derived in the paper. After providing extensive analysis for the alternating optimization of the parameters, the proposed model is tested in synthetic and real-world experiments, where it achieves state-of-the-art in solar wind prediction.\n\nThe approach of the paper to the problem is principled and it provides extensive theoretical and experimental results for further insight. Unfortunately, one of the main equations (equation 3) is difficult to parse due to its unclear notation and moving forward from that early point, I could not trust my interpretation of the presented model (to be more precise, use of subscript “t” for both x and y on the LHS, the summation index, and the probability space of p all use notations that I found confusing and problematic.)\n\nAs a separate note, early on, the paper argues against existing sequences to sequence models for this task due to their limitation in working with individual sentences. However, the paper proceeds to make a modeling assumption of the same sort which bounds the time lag. In light of this, I do think that transformer type models can be adapted to this setting, and in fact, an input dependent attention mechanism is ideal for combining the prediction of time-lag and the raw response in the output sequence. The expectation is for this type of model to perform well in the proposed setting. Could you please comment?  \n", "extra_scores": {"experience_assessment": "I do not know much about this area.", "review_assessment:_thoroughness_in_paper_reading": "I made a quick assessment of this paper.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I did not assess the derivations or theory."}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper proposes a Bayesian approach for time-series regression when the explanatory time-series influences the response time-series with a time lag. The time lag is unknown and allowed to be non-stationary process. Reviewers have appreciated the significance of the problem and novelty of the proposed method, and also highlighted the importance of the application domain considered by the paper. 
Decision: Accept (Poster)