Question: The authors mention that TS2Vec achieves 2nd best performance because it adopts subsequence by random cropping. How do the authors know this is the reason? No causal conclusions can be drawn from the presented Tables.
Evidence: Forecasting is a critical task in time series analysis. Deep learning architectures used in the literature include Recurrent Neural Networks (RNNs), Convolutional Neural Networks (CNNs), Transformers, and Graph Neural Networks (GNNs). N-BEATS deeply stacks fully-connected layers with backward and forward residual links for univariate times series forecasting. TCN utilizes a deep CNN architecture with dilated causal convolutions. Considering both long-term dependencies and short-term trends in multivariate time series, LSTnet combines both CNNs and RNNS in a unified model. LogTrans brings the Transformer model to time series forecasting with causal convolution in its attention mechanism. Informer further proposes a sparse self-attention mechanism to reduce the time complexity and memory usage. StemGNN is a GNN based model that considers the intra-temporal and inter-series correlations simultaneously. Unlike these works, we aim to learn general representations for time series data that can not only be used for forecasting but also other tasks, such as classification. Besides, the proposed framework is compatible with various architectures as encoders.
Time series forecasting aims to predict the future L y time stamps, with the last L x observations. We follow to train a linear model regularized with the L2 norm penalty to make predictions. The output has dimension L y in the univariate case and L y × F for the multivariate case, where F is the feature dimension.
Datasets and Baselines. Four benchmark datasets for time series forecasting are adopted, including ETTh1, ETTh2, ETTm1, and the Electricity dataset. These datasets are used in both univariate and multivariate settings. We compare unsupervised In-foTS to the SOTA baselines, including TS2Vec, Informer, StemGNN, TCN, LogTrans, LSTnet, and N-BEATS. Among these methods, N-BEATS is merely designed for the univariate and StemGNN is for multivariate only. We refer to Performance. As shown in Tabel 1 and Tabel 4, comparison in both univariate and multivariate settings indicates that InfoTS consistently matches or outperforms the leading baselines. Some results of StemGNN are unavailable due to the out-of-memory issue. Specifically, we have the following observations. TS2Vec, another contrastive learning method with data augmentations, achieves the second-best performance in most cases. The consistent improvement of TS2Vec  over other baselines indicates the effectiveness of contrastive learning for time series representations learning. However, such universal data augmentations may not be the most informative ones to generate positive pairs. Comparing to TS2Vec, InfoTS decreases the average MSE by 11.4%, and the average MAE by 8.6% in the univariate setting. In the multivariate setting, the MSE and MAE decrease by 4.6% and 2.3%, respectively. The reason is that InfoTS can adaptively select the most suitable augmentations in a data-driven manner with high variety and high fidelity. Encoders trained with such informative augmentations learn representations with higher quality.
Performance. As shown in Tabel 1 and Tabel 4, comparison in both univariate and multivariate settings indicates that InfoTS consistently matches or outperforms the leading baselines. Some results of StemGNN are unavailable due to the out-of-memory issue (Yue et al., 2021). Specifically, we have the following observations. TS2Vec, another contrastive learning method with data augmentations, achieves the second-best performance in most cases. The consistent improvement of TS2Vec over other baselines indicates the effectiveness of contrastive learning for time series representations learning. However, such universal data augmentations may not be the most informative ones to generate positive pairs. Comparing to TS2Vec, InfoTS decreases the average MSE by 11.4%, and the average MAE by 8.6% in the univariate setting. In the multivariate setting, the MSE and MAE decrease by 4.6% and 2.3%, respectively. The reason is that InfoTS can adaptively select the most suitable augmentations in a data-driven manner with high variety and high fidelity. Encoders trained with such informative augmentations learn representations with higher quality.
Answer: The most important components of TS2Vec are random cropping and contrastive learning. That's why we draw the causal conclusions in the initial version. However, for more rigorous statements, we have revised this part accordingly. Thanks a lot for the comments.