Title: Learning Without Augmenting: Unsupervised Time Series Representation Learning via Frame Projections
Venue: NeurIPS 2025
Accepted: yes
Keywords: Unsupervised learning

contrastive learning

time series
Abstract: Self-supervised learning (SSL) has emerged as a powerful paradigm for learning representations without labeled data. Most SSL approaches rely on strong, well-established, handcrafted data augmentations to generate diverse views for representation learning. However, designing such augmentations requires domain-specific knowledge and implicitly imposes representational invariances on the model, which can limit generalization. In this work, we propose an unsupervised representation learning method that replaces augmentations by generating views using orthonormal bases and overcomplete frames. We show that embeddings learned from orthonormal and overcomplete spaces reside on distinct manifolds, shaped by the geometric biases introduced by representing samples in different spaces. By jointly leveraging the complementary geometry of these distinct manifolds, our approach achieves superior performance without artificially increasing data diversity through strong augmentations. We demonstrate the effectiveness of our method on nine datasets across five temporal sequence tasks, where signal-specific characteristics make data augmentations particularly challenging. Without relying on augmentation-induced diversity, our method achieves performance gains of up to 15--20\% over existing self-supervised approaches. Source code: \url{https://github.com/eth-siplab/Learning-with-FrameProjections}


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors propose to use projections of the data into Fourier space and Gabor wavelet space as 'views' for self supervised learning, in-place of traditional augmentations. They show empirically that this method improves performance slightly across a range of time-series datasets and compared with a diverse set of SSL benchmarks.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths:**\n- The proposed problem is important and well motivated. Indeed, data-specific augmentations are one of the big limitations of modern SSL methods, especially when applying them to data domains outside of images or text. Therefore, methods to alleviate this need for hand-crafted augmentations are highly valuable. Additionally, the overall insight that different frames can be used in place of these augmentations is an interesting and valuable one.\n- The experimental results are strong, showing broadly improved performance on many tasks, and increased performance at a range of model sizes. \n- The authors perform a set of ablations to validate their modeling choices, which is a welcome addition.\n\n\n**Weaknesses**\n- The theoretical understanding provided to justify the method is quite limited.\n- The experimental evaluation is focused on time-series modeling, while the most prominent SSL methods are used for vision and language -- yet the authors make no mention of these domains in the text. The authors should at least acknowledge this fact, and describe why or if their method does not work on such domains, or if they do not know, rather than ignoring this application. \n- It appears the authors re-trained all baseline methods from scratch in their reported results since the datasets they used are not the traditional SSL benchmarks the methods were developed on. It is not clear that in this process the baselines were tuned properly. There are many possible parameter settings, and with so many baselines, tuning would appear challenging. For Barlow Twins for example, the authors report using the hyperparameter lambda to 0.005 without any tuning. These details can have a significant effect on performance. If the authors could find any published results from prior work on these datasets that would be beneficial to increase confidence in their results.\n- The performance standard deviation on many experiments seem quite large relative to the differences in model mean performance, yet the authors have no statistical significance indications. \n- The figures 3 and 4 are missing error bars despite the tables having them.\n- The related work section is missing some related work which also studies self-superivsed learning without augmentations. One particularly related method is \"Homomorphic Self-Supervised Learning\" (Keller, Suau, Zappella, 2023 TMLR).\n\nQUESTIONS:\n- The `t` domain is not defined prior to equation (2). \n- The authors note that SSL often results in models which are overly reliant on a subsset of features, or an undesirably invariant to a subset of features. How does using othonormal basis fix this other than the fact that the specific invariances may be harder to intuit now? \n- Can the authors justify the use of the term 'manifold'? It is not immediately clear that these representation spaces are manifolds in the formal sense. \n- A more extensive coverage of prior work specifically related to 'augmentation-free SSL', and an improved theoretical understanding of the proposed approach would be beneficial. For example, if the authors can formalize or empirically measure the types of invariances that their proposed projects would induce, this would greatly increase the impact of the work. \n- The proposed approach seems amenable to image SSL representation learning as well. Have the authors experimented with this?\n- The claim of 'up to 20-30%' performance increase seems a bit misleading given the difference in performance on many datasets is relatively small. I would encourage the authors to make the claim more quantitative and informative of the actual distribution of results.\n\nLIMITATIONS:\nThe authors should mention if or why there method was not evaluated in image settings.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-26"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper addresses the task of time series self-supervised learning (SSL). The authors propose a novel augmentation-free SSL method. Specifically, the method eliminates the need for traditional data augmentation by leveraging Fourier and Gabor wavelet transforms as additional views alongside the original input. In addition, the authors introduce a mapping function from the time domain to these transformed views during training to facilitate more efficient inference. The proposed method is evaluated on several time series classification tasks and datasets, where it outperforms existing augmentation-based SSL methods.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths\n- The authors effectively highlight the challenge of designing hand-crafted augmentations, which makes the proposed augmentation-free approach highly relevant.\n- The paper is well-written, with particularly clear definitions of notations and methodology. The approach is easy to follow and understand.\n- To the best of my knowledge, this is the first work that utilizes Fourier and Gabor wavelet transforms for time series self-supervised learning (SSL), contributing new methods to the field.\n- Comprehensive Experimentation: The authors conduct an extensive set of experiments across a variety of tasks and datasets. The comparison between larger backbones and multiple views for the baseline methods (L198-L201) is insightful and provides a fair evaluation of the proposed method.\n\nWeakness\n- Some relevant works are missing: \n  - The paper does not discuss more recent work on time series contrastive self-supervised learning (SSL), such as Timesurl [1], which shows to outperform some of the included baselines. \n  - Other relevant research on augmentation-free SSL methods, such as [2] and [3], is not discussed or compared. \n- The mapping function is an important part of the work, but the effect of its accuracy is not discussed. \n- The paper lacks a discussion on the applicablity of the proposed method on ftime-series forecasting tasks.\n- The paper focuses evaluation on linear probing only without finetuning experiments, which is another popular way for testing representation quality.\n\n[1] Liu, J. and Chen, S., 2024, March. \"Timesurl: Self-supervised contrastive learning for universal time series representation learning.\" In Proceedings of the AAAI Conference on Artificial Intelligence (Vol. 38, No. 12, pp. 13918-13926).\n[2]Verma, V., Luong, T., Kawaguchi, K., Pham, H. and Le, Q., 2021, July. Towards domain-agnostic contrastive learning. In International Conference on Machine Learning (pp. 10530-10541). PMLR.\n[3] Sui, Y., Wu, T., Cresswell, J.C., Wu, G., Stein, G., Huang, X.S., Zhang, X. and Volkovs, M., 2023. Self-supervised representation learning from random data projectors. arXiv preprint arXiv:2310.07756.\n\nQUESTIONS:\n- Could you discuss relevant works that are not covered in the paper?\n- What is the effect of the learned mapping function’s accuracy on the overall performance?\n- Does the proposed SSL method work on forecasting tasks?\n- Would it be possible to include fine-tuning experimental results as well?\n\nLIMITATIONS:\nN/A", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 4, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces a self-supervised learning framework that learns from three invertible transformations of each time-series example: raw signal, full FFT coefficients, and Gabor-wavelet frame, each encoded separately. Each pair of domains are applied with InfoNCE loss. Two tiny MLP mappers predict the frequency-domain embeddings from the time-domain embedding. In inference only the time encoder and the mappers are needed. On 9 ECG / EEG/ PPG / IMU datasets, the method surpasses SimCLR-style and other time-series-specific baselines by 30 %.\n\nSTRENGTHS AND WEAKNESSES:\nThe strengths are 1. The information-preserving design removes the need for hand-crafted augmentations. 2. The “coordinate consistency as (self) supervision” is novel and could potentially generalize to vision or audio.\n\n\n\nWeaknesses are \n1. Supervised references for EEG / ECG are outdated. Recent foundation models such as LaBraM, EEGPT, and CBraMod report strong numbers; comparison or discussion is required.\n\nJiang et al., 2024, “LaBraM: Large Brain Model for Learning Generic Representations with Tremendous EEG Data”.\nYue et al., 2024, “EEGPT: Unleashing the Potential of EEG Generalist Foundation Model by Autoregressive Pre-training”.\nWang et al., 2024, “CBraMod: A Criss-Cross Brain Foundation Model for EEG Decoding”.\n\n2. The paper misses the preprocessing time of the FFT + Gabor transforms. A simple wall-clock benchmark on the largest dataset—ideally side-by-side with the run-time of the common augmentation pipeline used by SimCLR-style baselines—would make the compute overhead clear.\n\nQUESTIONS:\n- Can you add frozen-encoder baselines from LaBraM, EEGPT, and CBraMod, or explain why their checkpoints cannot run on your datasets?\n- Mapper ablation: what happens when each 500-parameter MLP is replaced by a linear layer?\n- Have you tried joint training in which mappers and encoders are optimised together? If so, what are the results?\n- Can the authors explain intuitively why this works? For instance, in a toy example where one view is cartesian coordinates and the other is polar, a polar-view encoder could trivially apply the known mapping from cartesian to polar. By analogy, why doesn’t your protocol collapse into each encoder simply inverting the other view? A concise, low-dimension explanation would help readers grasp where the learning signal comes from. Given that a theoretical explanation is not provided, I would hope to see an intuitive explanation.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-03"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper notes that most SSL approaches require handcrafted augmentations, which need to be optimized for specific downstream tasks and are not necessarily generalizable. To address this they propose a novel approach where they do instance discrimination after a projection step. They use several representations of their data and train an encoder for each domain. I was actually a bit confused after that by Figure 1 and its description, but my understanding is that after multiple mappings they have a separate embedding for each branch. They then compute instancewise discrimination loss between the branches over a batch for neighboring points. Finally, they show that their method outperforms several well known classic SSL techniques on a number of benchmarks.\n\nSTRENGTHS AND WEAKNESSES:\n# Strengths\n- very clear. I am less familiar with frames and manifolds, so I appreciated how you provided a lot of intuition. I appreciated how e.g. proposition 2.1 had intuition before the statement of the proposition.\n- mathematically elegant solution to an important problem, previously mostly solved by heuristics. Further, it’s tied well to the applications.\n- One sees a large improvement over baselines.\n\n# Weaknesses\n- The baselines are more classic than SOTA and largely come from a few years ago. This makes the method potentially look better than it is. I’m forgiving a bit of this because the technique is elegant, but really, you should use harder baselines.\n- the datasets are also a bit older, but since there are a good number of them that is ok.\n\nOverall, I see this as worth publishing as it is an elegant, interesting method that performs at least reasonably against more classic baselines. Future work could be trying to push these sorts of ideas towards SOTA on competitive modern benchmarks.\n\nQUESTIONS:\n# Questions\n- You mentioned in your related work several related approaches and fields, but you actually didn’t talk about how frame projections are used, if at all, in machine learning or contrastive learning. It would be good to mention it and contextualize how large the niche is where you’re novel (is it SSL views, something bigger)?\n- Could you add a little bit more intuition right before or after prop 2.1 involving a concrete example for what this means in e.g. a specific type of data type or representation? Not sure if this is possible.\n- Despite most of the paper reading quite clearly, I found figure 1 a bit difficult to parse.\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 4, "IMPACT": 2, "ORIGINALITY": 4, "DATE": "2025-07-03"}