Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: The paper proposes a method to identify anomalous subsequences in a long time series using a graph neural network. The core idea is to use two different graph representations for fixed length subsequences from the time series, one which captures the semantic distance (based on different distance measures) and the second which captures the temporal distance (both time adjacency and periodicity). The method allows for capturing similarity between subsequences at different length scales. The two graph neural networks are then combined within a single learning framework to learn representations for the subsequences, which are then used for anomaly detection. 

The method is compared to a few other methods and shown to outperform them on a selection of data sets.
strength_and_weaknesses: Strengths:
- The problem is important and has real-world applications.
- The idea of not relying on a single window length is good and could make the method applicable in more settings.

Weaknesses:
- The paper is not very well-written and it is hard to understand how all the steps fit together. While the overview figure is useful, it does not provide the complete picture.
- The paper seems to be putting together several individual components to solve the problem but is not necessarily pushing the boundary significantly. While I agree that the results are better than others, the comparison is not comprehensive enough to show that this will generalize well. I can see some novelty in using two different representations (semantic and temporal), but the authors do not discuss the impact of using both representations.
- The experimental results are not comprehensive. I would have liked to see comparisons with the vast literature on discord detection which have been shown to be very effective in such problems, and for many of the data sets used in this paper.
- While I commend the relaxation of the need to specify a window length, it is unclear what is impact of the choice of the initial window length on the overall performance
clarity,_quality,_novelty_and_reproducibility: I think the paper can be improved in terms of clarity and organization. The authors do a good job of explaining individual components, but the poor organization means that it is hard to understand the overall algorithm. The paper has limited originality. As I mentioned earlier, there is some innovation in using two types of graph representation. Otherwise the authors have strung up different components without much discussion on the justification for each of those design choices.
summary_of_the_review: Overall, the paper is targeting an important problem. There is some marginal benefit shown on a set of data sets over a selection of methods. But the results are not convincing enough to show that the approach is generally applicable. There are some questions on the novelty.
review_title: null

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: This paper presents a method for anomaly detection in time series. The proposed method constructs two graphs of subsequences of the input time series, by encoding subsequence distance and periodic patterns, with multiple choices of length scales of subsequences. The model uses GNN and length embeddings to learn to weigh different scales. The aim is to capture resolve the problem of selecting subsequence lengths. The experimental results demonstrate the method outperforms some baseline methods.
strength_and_weaknesses: Strength
1. The proposed method considers two versions of graphs to encode the relationship between subsequences, a distance based graph and periodic pattern based graph.
2. The method introduces a subsequence length selection mechanism which is trained together with other parts of the model so that the length selection problem is mitigated to some extent.
3. The experimental results demonstrate the effectiveness of the proposed method in terms of improvements over several baseline methods.

Weakness
1. The overall novelty of the proposed method is limited. Using graph and multi-scale subsequence length of anomaly detection in time series has been studied before (e.g., A deep neural network for unsupervised anomaly detection and diagnosis in multivariate time series data. AAAI, 2019). Learning to select lengths is not significant innovation as it can be done by adding some attention based mechanisms. The paper briefly discussed these existing methods in the appendix, without sufficient details. Comparison may be necessary. The related work section is important, and should be moved to the main paper.
2. The proposed method may be limited by the initial subsequence length selection, which was assumed to be large, without a clear selection guidelines and impacts on performance.
3. It is not clear what the dimension $d_{e}$ is in $E_{j,i}^{(se)}$. It seems to be 1 for representing the semantic proximity.
4. The temporal graph only encodes periodic patterns, which may ignore other temporal dependency patterns.
5. The meaning of $Z_{i,0}, ..., Z_{i,P}$ are not clear. The question is why to aggregate information for all lengths before $l$, i.e. , $R_{i, :l}$, instead of using $l$ directly, i.e., $R_{i,l}$.
6. The intuition behind the length selection method Eq 4 and Eq 5 should be better elaborated. The current description is unclear because $Z_{i,P}$ represents aggregation for all lengths less than $l$, instead of using $l$ directly.
7. The motivation to use GNN to encode context information for each subsequence representation is also unclear. To detect anomalies, the method aims to find the difference between a subsequence and its context using Eq 9. GNN seems do the contrary. It aims to mitigate the difference between a subsequence representation and its context.
8. The authors may want to discuss if the model trained with injected anomalies is biased to detect anomaly patterns follows the injection. This could be further investigated in the experiments. Also Eq. 11 does not consider unbalanced classes (normal and abnormal).
9. The experiments could be further improved by including more evaluation metrics, and baseline methods. There are more subsequence based anomaly detection methods, as discussed in Anomaly Detection in Time Series: A Comprehensive Evaluation, VLDB 2022. Some of them could be evaluated in the experiments.

clarity,_quality,_novelty_and_reproducibility: See Strength and Weakness.
summary_of_the_review: The paper introduces a graph based subsequence anomaly detection in time series. The overall novelty of the proposed method is limited. The method design and experiments remain to be further justified and improved.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: The paper proposes GraphSAD, a subsequence time-series anomaly detection method that fuses discord-based anomaly detection solutions with graph neural network (GNN) /deep neural network (DNN) architectures. The core contribution of the paper is the construction of two types of graphs, semantic and temporal, to capture the distances and temporal dependencies in the data.  In addition, the proposed method proposes a solution for selecting the appropriate length for subsequences, a core parameter for that family of methods. Experimental results demonstrate the benefits of the proposed solution against several baselines on multiple datasets.
strength_and_weaknesses: Strengths:

1. Important and timely problem for time series
2. Well-written paper and easy to follow
3. Practical ideas fused appropriately into a single method

Weaknesses

1. Lack of novelty, mainly fusion of ideas 
2. Missing several baselines
3. Missing several appropriate datasets and evaluation measures

Comments:

- The ideas are reasonable and appropriately merged into a single DNN/GNN framework. Unfortunately, there is a lack of technical depth and novelty. If we abstract the contributions, the core idea is to construct two types of graphs, and both such kinds of graphs exist/are well known (no novelty here). The remaining parts are mainly plug-ins of existing solutions. There is some interesting concepts in choosing lengths, but really similar solutions could be applied to every solution (e.g., run the method with different parameters, select on that maximizes anomaly score or using some voting etc.)

- Core very recent works are missing. There has been new works published well in-advance before the ICLR submission about new measures for evaluating subsequence anomaly detection methods as well as new benchmarks with 2000+ time series. Unfortunately, the paper does not consider the more recent advances in the area, and some methods are outdated. This is a fast pacing field, unfortunately.

"Tsb-uad: an end-to-end benchmark suite for univariate time-series anomaly detection." Proceedings of the VLDB Endowment 15.8 (2022): 1697-1711.

"Volume under the surface: a new accuracy evaluation measure for time-series anomaly detection." Proceedings of the VLDB Endowment 15.11 (2022): 2774-2787.
clarity,_quality,_novelty_and_reproducibility: Well-written paper; code is provided to assist with the reproducibility of results.

Novelty is somewhat low - nice engineering effort to put existing "practical knowledge" together, but the evaluation is lacking (no appropriate measures, no appropriate datasets, not all recent state-of-the-art classifiers). 
summary_of_the_review: I believe the paper is on the correct track to getting accepted eventually in a top venue. However, I have significant concerns about the evaluation of the work: missing recent comprehensive benchmarks in the area and missing the appropriate methods to evaluate subsequence methods. For the baselines, similar tricks for selecting the length should be used to ensure fairness.

From the ablation study, we see that the most important "idea" is the temporal graph. Semantic graph lacks behind, which contradicts the entire concept of using GNNs in the first place. Temporal information seems sufficient and tons of methods exist (and not being evaluated) to capture these cases. Check references for appropriate evaluation on new benchmarks and new evaluation measures, using the appropriate baselines.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: The paper addresses the problem of time-series subsequence anomaly detection, and this is executed through converting the time-series to a graph, and then performing anomaly detection on this graph representation. The problem is motivated by the fact that time-series anomalies can be subsequences, while most existing work focuses on point anomalies.

The method starts with splitting the time-series into multiple subsequences, followed by construction of graph priors - semantic priors and temporal priors. Semantic graphs is based on distances measures and incorporates multi-scale lengths. Temporal graph also uses multiple heuristics to incorporate temporal information. These graphs are followed by a multi-scale feature encoder and a length selection module to aid representation learning for the time-series. Auto encoding regularization and Laplacian regularization is used to stabilize the training. Experiments are performed on both synthetic and real world datasets, and GraphSAD achieves competitive performance - even performs well for point anomaly detection which is not the focus of the algorithm. 
strength_and_weaknesses: Strengths: Fairly well written and motivated well, appears technically novel from an application perspective, experiments seem convincing
Weakness: The key distinction from so many of the competing baselines is not argued well (e.g. Series2graph, THOC), which makes it slightly unclear what the new principle of this paper is compared to the literature, as these papers also try to address the same problem from a certain angle (e.g. subsequence, multi-scale). 
clarity,_quality,_novelty_and_reproducibility: Well written

Overall quality of work is good, experiments are convincing

The authors have released an anonymous link for code, indicating reproductibility
summary_of_the_review: Overall the paper reads well and is a well done work (though not something I'd consider as breakthrough). I'll be happy to raise score after author's justify the key technical novelty compared the baselines addressing the same problem.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: This paper studied an important problem motivated by real-world applications: identifying anomalous subsequences in a long time series using a graph neural network. The referees raised many concerns about the writing, limited novelty, contributions, insufficient experiments, and limitation by the initial subsequence length selection. After rebuttals, one referee raised their score, but it seems there is still insufficient enthusiasm from the referees to accept the current version of this paper at the 2023 ICLR.
justification_for_why_not_higher_score: After rebuttals, one referee raised their score, but it seems there is still insufficient enthusiasm from the referees to accept the current version of this paper.
Decision: Reject