Venue: ICLR 2020 Conference

Reviews:

final_score: rating: 6: Weak Accept
review_content: the paper attempts to infer Granger causality between nonlinearly interacting stochastic processes from their time series measurements. instead of using MLP/LSTM etc to to model time series measurement, the paper proposed to use component-wise time series prediction model with Statistical Recurrent Units to model the measurements. they consider a low-dimensional version of SRU, which they call economy-SRU. in particular, they use group-wise regularizing to accompany the particular structure of the model to aid interpretability. they compared the performance with existing models with MLP/LSTM and show some gains in a few examples (but not all.) the proposal is interesting, but the experiment section might need further strengthening. currently, the experimental results do not immediately pop out as showing eSRU particularly useful.
review_title: Official Blind Review #2

final_score: rating: 6: Weak Accept
review_content: In this paper the authors propose using Statistical Recurrent Units to predict the network for Granger causality. They motivate this choice by the high representation power of SRUs for multivariate time series, the good performance they usually enjoy and as a way to alleviate the vanishing gradient problem. More importantly the particular form of the SRUs gives a very simple predictor and therefore explanability for Granger causality: the authors propose to simply mark serie $i$ as Granger caused by $j$ if the $j$th column of the input mixing matrix of the $g_i$ is non-zero.
In order to force whole columns of the input matrix to be negative, the authors use a group regularization on the columns of the input matrix $W_{\text{in}}$. The resulting problem is then optimized by proximal gradient descent.
The second contribution of the authors is eSRU, a smaller (in terms of parameters) variant of SRU in order to prevent overfitting.
First the authors introduce a dimensionality reduction layer before the SRU units by using fixed non-trainable random projections. Then the authors propose adding a regularization term for the output mixing matrix, which represents the bulks of the parameters of the model.
The eSRU and SRU causal models presented are then compared to the previous state of the art on several datasets, both synthetic and real, where they manage to reach a new state of the art.
Regarding the two improvements in eSRU proposed by the authors, I have several questions:
First, regarding the random projections, have the authors tried learning the projections ? While it increases the risk of overfitting it is possible, with enough regularization, that it may help learn good representations. Secondly, as I understand it a single projection is drawn at random for every component. While we know from Johnson-Lindenstrauss's Lemma that this will in average be a good strategy, how stable is the model to spurious projections ?
Regarding the scarification of the output matrix: the number of parameters is effectively reduced but the computational requirements are still the same. Did the authors try any explicit methods, maybe matrix factorization, to exploit the very specific sparse structure of $W_o$ and reduce the number of operations and parameters ?
Finally, while the results are undeniable, just observing the non-zero columns of $W_\text{in}$ is as I understand it a mostly ad hoc rule. It would have been interesting to empirically verify its validity by contrasting it with the results given by following equation 2.
Overall the paper presents an interesting model for inferring Granger causality. The authors clearly present their two main contributions and verify them using varied datasets.
The authors also made a clear and appreciated effort toward reproducibility by including all hyperparameters and implementation details as well as the code, including those of competing techniques.
The inclusion of the literature review in the appendix is also greatly appreciated.
review_title: Official Blind Review #1

final_score: rating: 6: Weak Accept
review_content: The paper proposed to use SRU for inferring nonlinear granger causality. It also provided two extensions of SRU with regularization to alleviate the issue of overfitting.

SRU was proposed in a previous paper. This paper extended this algorithm for inferring granger causality through applying group sparse regularization, which is a pretty smart design to me. Another major innovation comes from the two modifications to combat the possible overfitting when using very fine-grained scale parameters in SRU.

The paper was well-written in general. I can follow the paper without problem.
The effectiveness of the algorithm depends on the sparse regularization, which lacks theoretical guarantee for non-convex optimization. Based on the experimental results, it doesn't seem a big problem though. But I'd like to see some analysis of the actual sparsity of the weight with the proposed group sparse regularization.

The experiments seem convincing to me and I really appreciate the authors provided the tuned hyper-parameters in the appendix.

review_title: Official Blind Review #3

Meta-review: The authors propose a modification of the statistical recurrent unit for modelling mutliple time series and show that it can be very useful in practice for identifying granger causality when the time series are non-linearly related. The contributions are primarily conceptual and empirical. The reviewers agree that this is a useful contribution in the causality literature.
Decision: Accept (Poster)