Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: The authors introduce a deep learning framework to analyze time series (identification of different states, detection of state transition, etc.). The framework presented in the paper is based on an ensemble network composed of LSTMs (each corresponding to a state) and a 1D CNN that allow to choose the best LSTM or state. The proposed technique has been applied for forecasting and clustering tasks on various time series data.
strength_and_weaknesses: Strengths:
1.	Use of different databases and applications (though does not see the clinical usefulness of the biomedical application, would have been interesting to assess on a classification problem (such as sepsis prediction for ICU data)
2.	The authors performed an ablation to highlight the added-value of each part of their technique.
3.	Source code is provided for better reproducibility.
Weaknesses:

1.	The authors could have given more insight on the practical usefulness of their framework, for instance with the interpretation of clinical states with the ICP waveforms. Is the method clinically relevant given the low silhouette scores obtained? Could be interesting to visualize ECG forecasting. What more does SpaRTEn bring compared to the other methods, as it is hard to tell with sMAPE values only. Are the predicted ECGs still interpretable (clear delineation of P, QRS, T waves …)? I don’ t understand what it means to predict 20 ECG samples (what is the sampling frequency? Given the typical ECG sampling frequency, 20 samples would correspond to less than 100ms, so I fail to understand the clinical usefulness of predicting such a short window of ECG samples
2.	The description of the proposed method lacks clarity. I would suggest present the methodology of the technique in a more structure way, clearly explaining how he training process is performed.
3.	I fail to understand the contrastive learning approach of the method. Do the authors suggest that the spatial representation is time independent? Could the authors compare their approach to a BYOL approach?  On top of that, one of the characteristics of biomedical times-series lies in their non-stationarity nature which would imply that the assumption of stable spatial representation does not hold. Could the author comment on that?

clarity,_quality,_novelty_and_reproducibility: The overall paper description lacks clarity, which makes it really hard for me to understand the proposed method.
The authors should describe how their proposed approach is different from other contrastive learning techniques, such as BYOL or for biomedical applications such as CLOCS. 
Moreover, proposed the proposed metrics for evaluation of the technique are very hard to interpret, and does not allow to judge the “clinical” usefulness of their approach.

summary_of_the_review: The presentation of the proposed method lacks clarity, which makes it hard to understand their approach, how it differs from state-of-the-art. The results do not allow me to judge the usefulness of the proposed technique.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: Spatial Resolved Temporal Networks (SpaRTeN) are described in this paper as a novel, composite deep learning model for online, unsupervised representation learning in a spatially constrained latent space. In the proposed approach, two distinct components are simultaneously learned: a recurrent neural network ensemble that captures states in high-frequency time series, as well as a spatial block that is able to spatially resolve state changes based on the predictions generated by the other network. Throughout, it is an interesting paper, and the proposed method yields several practical applications for a wide range of machine learning problems. Major concerns and minor comments are presented in the review section.
strength_and_weaknesses: Please refer to the review section.

clarity,_quality,_novelty_and_reproducibility: Please refer to the review section.

summary_of_the_review: a) The empirical studies do not provide convincing evidence. The proposed method can be applied to time series in a variety of applications, as indicated in the abstract. Consequently, it would be beneficial if authors benchmarked the proposed technique against several real-world datasets.

b) The performance of the proposed method should be compared with similar state-of-the-art approaches. That would be interesting to compare the proposed method with some recent studies in Transformers, Energy-Based approaches, and Reinforcement Learnings.

c) The figures and tables in this paper are ambiguous. As an example, what are the metrics in Table 1 (mean column) and Table 2? Furthermore, Figure 2 is blurry and difficult to read.

d) In this paper, the notations are confusing. In regular papers, scalers are denoted by small letters; vectors are defined with small letters (highlighted in bold), and matrices are denoted by capital letters using bold. In this paper, they are a lot of conflicts. It is so hard to trace what is a set, a matrix, or even a distribution.

f) There are some minor linguistic and typo problems in this paper.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: In this paper, the authors propose a new time series forecasting model for univariate high-frequency time series. There two major components/blocks of the model: one block contains ensemble of base predictors/LSTM for time series forecasting and the second spatial block is trained using contrastive learning to find the best base predictor or LSTM. The whole model works like the minimax framework. Experiments on online forecasting and zero-shot clustering show efficacy of the proposed method.

strength_and_weaknesses: Strength
The whole framework looks novel and incorporates recent advances in ML like contrastive learning for time series forecasting. 

Weakness
Motivations and rationale behind the proposed model is not well explained. It is not very clear how these components interact and why all the components work in a synergetic way.
clarity,_quality,_novelty_and_reproducibility: Clarity needs improvement to make the whole framework easy to follow and understand. 
Novelty looks pretty good. However, I only see empirical support and there are no theoretical guarantees. 
Reproducibility is feasible with the submitted code.

summary_of_the_review: 1. Regarding contrastive loss, we know that it works fine for images. Here the motivation and rationale of employing the distance-weighted contrastive loss DWCL for univariate time series forecasting is not clear to me. How are similar and dissimilar pairs selected in the training process?
2. What is the benefit of using a two-dimensional discrete state-space in the S-block? For the purpose of selecting the best base predictor, one-dimensional state-space should be enough. Also in the training of the S-block, cross-entropy loss function is used which actually works fine with the one-dimensional state-space.
3. In Sec. 3.3, why does ensemble weight sharing help for time series forecasting? If we wanna a diverse ensemble which can cover a broader state-space, should we limit the use of ensemble weight sharing which may lead to the mode collapse (common challenge in GAN).
4. The S-block actually reminds me the attention mechanism which can also be used to select the optimal state/predictor. It would be interesting to see discussion on comparison of S-block and attention. For example, what are the advantages of S-block over attention and gating in the temporal fusion transformer (TFT).
5. In practice, individual LSTM has limited forecasting power compared with state-of-the-art (SOTA) algorithms for time series forecasting. More analysis on how ensemble of LSTMs could achieve better prediction performance will help reader to understand the contribution and novelty of this work.
6. Evaluations are limited and only done on two datasets which makes it less convincing. More experiments and comparisons with SOTA methods are encouraged
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: The paper proposes a "composite deep learning model for online, unsupervised representation learning through a spatially constrained latent space", using two blocks, a block that encodes states and a spatial block that learns the state transitions. The method is tested on forecasting and clustering. The strengths of the paper, as identified by the reviewers, include the extensiveness of the datasets on which the tests were performed and the ablation studies -- the authors have augmented their experiments following the review period. The weakness of this paper was its clarity, various aspects of the paper: the notation, motivating the use of the contrastive loss, connections between the blocks and other mechanisms such as attention, the clinical importance of the application. The authors have given extensive responses to the issues raised concerning clarity, which would result in considerable modifications to the paper. Unfortunately, the reviewers have not read the responses, so it is impossible to determine whether the issues were addressed. Given that all 3 reviewers were initially unconvinced, and the extensiveness of the changes results in a paper that is very different from the initial submission (and thus warrants a new review cycle), I have no choice but to reject this paper.
justification_for_why_not_higher_score: The initial version of the paper has several drawbacks: lack of clarity, insufficient experiments, less than convincing case study. The revision have addressed the second aspect, in my opinion, but it's impossible to determine if the other two were addressed without the reviewers reading the updated version. 

It is unfortunately all-too-common for reviewers to not respond to author comments, especially when all of them opted for rejection based on the first read, and the only thing the authors can really do about it is attempt to provide an initial version of the paper that is in good enough shape to obtain at least borderline scores and thus warrant further discussion. The reviewers are a randomly selected sample of the target audience for the paper, so if neither of them indicated an interest in accepting the paper, then ICLR might not be the venue for it.
Decision: Reject