Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: The manuscript proposed a time series embedding through structured state spaces + dynamic graph structure learning + GNN model to learn the representations of multi-variate time series data. The model was evaluated on three datasets (two graph-level classification tasks and one node-level forecasting task). Experiment results indicate superior performance of the proposed model over commonly-applied methods. 
strength_and_weaknesses: Strength: Capturing and utilizing dynamic graph structures over time is an important yet largely overlooked topic in graph-based representation learning. Model performance is also quite good. 

Weakness: Most importantly, the author did not provide sufficient details for the reader to understand how “Dynamic Graph Structure Learning” is performed (section 4.3). Specifically, the author mentioned that “each signal forms a graph,” which is confusing: does the statement indicate each variate forms a single graph; or does the multi-variate signal at a specific time point/short resolution form a single graph? If it is the latter case, it is unclear how the adjacency matrix W is learned. Furthermore, it is unclear how the KNN graph is constructed, especially if the length of the “short resolution” is small or, in the extreme case, a single time point. Secondly, it is unclear how the singles on multiple graphs W^(1)…W^(nd) were fed to the GNN layer. Based on the current description in 4.4 and Figure 1, it seems that the GNN layer is accepting signals defined on a single graph rather than multiple graphs. In addition, the ablation study performance shown in Table 4 indicates only a marginal benefit of using the dynamic graph structure. 

clarity,_quality,_novelty_and_reproducibility: Clarity: there are major points in the manuscript that need further improvement of clarity or providing more details, see my comments above.

Quality: overall quality of the manuscript is acceptable, with extensive experiments and ablation studies to support the author’s claims.  

Reproducibility: the manuscript did not provide any source code, even though the datasets used for the experiments are public. Combined with the clarity issue in its methodology section, the proposed model and the experiments would be very difficult to reproduce. 

summary_of_the_review: An interesting model for multivariate time series analysis by leveraging dynamic graph structure yet lacks sufficient algorithm details to understand the reproduce.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This paper proposed a method for long-range spatial-temporal multivariate signal forecasting, by combining S4 and graph structure learning to automatically learn the dynamic graph structures.
strength_and_weaknesses: Strength:
1. The motivation is clear and straightforward.

Weaknesses:
1. This paper proposed to taking the output of S4 as the input of GSL to overcome the limitation of GSL on capturing long-range dependencies. Two concerns: (1) Is only taking the output of S4 as input of GSL enough? More interactions are expected between S4 and GSL to guarantee long-range structure learning. (2) As the authors claimed "long range", how the performance of GRAPHS4MER varies for different "ranges" are not clear. Does it robust? It should be discussed in the experiment.
2. In 4.3, the authors first explained why they learn a graph for a time interval, then they claimed "our GSL layer learns a unique graph for each data point rather than a universal graph for all data points." Interval or a data point? It is confusing.
3. The interval r and the threshold k are two important hyperparameters. But the authors did not discuss how the performance varies to different r and k in the experiment.
clarity,_quality,_novelty_and_reproducibility: The technical contribution is not significant. There is no surprise to combine GSL and GNN for multivariate time series forecasting. The writing is clear and easy to follow. The authors only provided the range of hyperparameter tuning, the specific setting for the reported results is not provided, which may hinder the reproducibility.
summary_of_the_review: While the motivation and writing are clear, some concept descriptions and experiment settings need improvement, and the technical contribution is not significant.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: The paper combines the Structured State Space Model (S4) (to learn time behaviors of multivariate series) and Graph Structual Learning (by coding as an attention layer in inductive settings, and a learnable embedding for transductive settings, regularized with a k-NN graph and smoothness of (representation) of the series of the learnt graphs). This leads to a general proposed architecture which is used in three examples where one needs  to follow both some specific time dynamics (encoded in S4) and  a network structure (encoded in the proposed GSL).  
strength_and_weaknesses: Strength

1- the global architecture is sound with some novelties in the combination of elements.

2- numerical experiments are well conducted, on relevant and diverse tasks and datasets 

3- even if there is no new theoretical content, there are many interesting methodological insights in Sections 3 and 4 explaining the rationales for the proposed method.

4- Excepted as mentioned below (about 4.4 and lack of details on the full pipeline), the article is well written and clear.

Weaknesses

1- The work combines S4 model (from Gu et al, 2022) and ideas of Graph Structural Learning (present in many works in the literature of these past years), yet it does not really offer a major new insight. For that, it is more an incremental work. 

2- The work is not precise enough in the description of the exact model / architecture proposed. Section 4.4 sends back ton Figure 1 yet the figure is not, for me, evocative enough. I would prefer to have a details model (with clearly precised input and trainable parameters) in 4.4.

3- It seems that there is some limitation in that the work assumes that the graph comes necessarily from the multivariate series X, while in many settings data consists of separate graphs and multivariate series. And it is a question in itself to fathom or asses if the graph is well related to X. Here, it seems that one assumes to whe have a pipeline going from $X$ to $h$ (embedding from S4) then $W$ (learned graph). Maybe I didn't catch one point in eq. (8): is the learning of $E$ a way to incoporate pre-existing information on $G$ ? Or is $G$ always unknown and to be derived from $X$ ? 

4- There is not that much variations or explorations of the parameters. 

Some (more minor) points to improve are:

5- The S4 models could be presented with either more insight (how to choose $L$ ? Where goes $D*$ in (2) and  (3) ?), or taken as given by the original work of Gu and al. yet with the explanation that the model of eq. (4) has a rationale coming from state  space model of time series. The current presentation is too much in-between.

6- The pipeline from $X$ to the GNN layers should be clarified

7-  Some of the dimensions are not clear. For instance, are the T time slots cut in intervals of size r ? Or is there some superposition in sliding windows ?

8- As questioned above: is it opossible to have the situation where the input is $X$ and $G$ ?

9- For eq. (9), please use a different notation for $W$ on the left and $W$ on the right. (Also, the weights in eq (7) could use another letter than W... or Use A for the learned weighted adjacency matrix if you prefer to keep W for weights).

10- The effect of GSLk, as discussed in 5.2 on page 8 (and Table 4) doesn't appear to be major. Why that ? For DOD-H, why not simply use as baseline the K-NN graphs from the S4 embedding (without learning another graph) as comparison. This ablation study, remplacing GSP by pre-defined graph, looks to reduce the significance of the work so more comments and discussion is expected. 

11- the graphs of figure 2 are hard to interpret because it is difficult, from the plotted time-series, to assess the state of the seizure in each clip. As it stands currently, it would be clearer to compare the obtained graphs to what is obtained in other studies (and remove the time series to save space).  


clarity,_quality,_novelty_and_reproducibility: I am stated many weaknesses that have to be revised yet, if these things are corrected, I stand quite positive about the general quality and clarity of the work.

For me, originality is in the proposed methodological insights, and the fact that the global pipleline is tested in three different situations (with comparison to SOTA) which are relevant, difficult and interesting.

The work appears to be well reproducicle, with a code well documented (even if I had no time to run it actually, only read it through). 
summary_of_the_review: My apppreciation is that the work has good potentials yet that there are several things that the authors should revise or anwer to. 
My recommendation is that, as it stands, the work is "marginally below the acceptance threshold" but I would be happy to engage discussion and see improvement in the presentation of the work to have a more positive evaluation.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: The authors present a GNN architecture combining Structured State Spaces model and graph structure learning for spatiotemporal modeling of multivariate signals. 

The proposed model has two major advantages: (1) it leverages S4 to capture long-range temporal dependencies in signals and (2) it is able to dynamically learn the underlying graph structures in the data without a predefined graph.
strength_and_weaknesses: Strength 

*The authors have carefully designed a model for mining space-time relations.

*The authors evaluated the model performance on several datasets.

Weaknesses

*The innovation of the paper is insufficient. The proposed model is just a simple combination of some existing technologies. For example, the S4 and the attention mechanism in transformer are both existing technologies.

*The idea of graph structure learning exists widely in many literatures. What is the difference between the proposed method and the existing graph structure learning[1-2]?

*Explainability oriented visualization is not comprehensive. For example, there seems to be no visualization of sleep stages and traffic data. In addition, there are many edges in Figure 2 (a). But this stage does not seem to be the spread stage of epilepsy.

[1] A unified structure learning framework for graph attention networks. Neurocomputing.2022
[2] GraphSleepNet: Adaptive Spatial-Temporal Graph Convolutional Networks for Sleep Stage Classification. IJCAI.2020



clarity,_quality,_novelty_and_reproducibility: Clarity: Good

Quality: Fair

Novelty: Poor

Reproducibility: Fair

summary_of_the_review: The performance of the model is evaluated on several datasets. The model performs better than multiple baseline methods. However, the noveltyof the manuscript is very limited, which causes my concern.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: This paper proposes GRAPHS4MER, a spatiotemporal model that combines S4 with graph structure learning to model long-range time series data. The model is tested on three benchmark datasets to show better performance than the state-of-the-art models.

The task of modeling long-range spatiotemporal dependency is interesting, difficult and important for time series analysis. The empirical results on three  benchmark datasets are supportive for the effectiveness of the proposed method.

However, as all the reviewers pointed out, the proposed method is a combination of the previous S4 model and a graph-based learning technique, which is rather an incremental contribution. The model design and experiment do not provide sufficient new insights for state-space modeling and graph-based learning. The long-range modeling has been provided by the S4 model, which can't be counted as a major contribution of this paper. Key questions such as how to quantify "long range" for different types of data and how much the long-range modeling can affect graph-based learning are not discussed (reviewer HRnf). Also, the discussion of existing works on dynamic graph modeling is not enough (reviewer FDcr). Other concerns include insufficient discussion on how to combine GSL with an existing (reviewer dUxW).

Although the reviewer improved the presentation of the paper and addressed some of the reviewer's questions, the major concerns still remain. Therefore, we recommend rejection. 

justification_for_why_not_higher_score: A major concern of the paper is on the innovation, as S4 model and dynamic graph modeling are already established in the community. Although the paper provides some improved results on benchmark datasets, the technical contribution is rather limited. The paper tackles long-range dependency problems in time series modeling, but it lacks insightful analysis and in-depth discussions on how long-range are characterized for different time series data, and the analysis on the interaction between long-range modeling and graph-based learning.
Decision: Reject