Title: AdaTS: Learning Adaptive Time Series Representations via Dynamic Soft Contrasts
Venue: NeurIPS 2025
Accepted: yes
Keywords: Time Series

Representation Learning

Contrastive Learning

Self-Supervised Learning

Adaptive Learning

Dynamic Temporal Modeling
Abstract: Learning robust representations from unlabeled time series is crucial, and contrastive learning offers a promising avenue. However, existing contrastive learning approaches for time series often struggle with defining meaningful similarities, tending to overlook inherent physical correlations and diverse, sequence-varying non-stationarity. This limits their representational quality and real-world adaptability. To address these limitations, we introduce AdaTS, a novel adaptive soft contrastive learning strategy. AdaTS offers a compute-efficient solution centered on dynamic instance-wise and temporal assignments to enhance time series representations, specifically by: (i) leveraging Time-Frequency Coherence for robust physics-guided similarity measurement; (ii) preserving relative instance similarities through ordinal consistency learning; and (iii) dynamically adapting to sequence-specific non-stationarity with dynamic temporal assignments. AdaTS is designed as a pluggable module to standard contrastive frameworks, achieving up to 13.7% accuracy improvements across diverse time series datasets and three state-of-the-art contrastive frameworks while enhancing robustness against label scarcity. The code will be publicly available upon acceptance.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces AdaTS, an adaptive soft contrastive learning method for time-series data. AdaTS introduces three novel components: a physics-guided similarity measure using Time-Frequency Coherence (TFC), which improves the robustness of time-series comparisons; an ordinal consistency learning strategy that preserves the relative ordering of similarities between instances, making it resilient to noisy data; and a dynamic temporal assignment mechanism that adapts to the varying dynamics of different sequences. AdaTS demonstrates improvements in the performance of self-supervised contrastive learning across various time-series datasets.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n1. AdaTS leverages Time-Frequency Coherence to capture physical correlations in time series, addressing a critical gap in prior work that overlooked underlying signal physics. By analyzing harmonic structures in the frequency domain, TFC enables robust similarity quantification that is both computationally efficient and noise-resilient.\n2. The dynamic temporal assignment mechanism in AdaTS adapts to sequence-specific non-stationarity by measuring each sequence’s temporal unpredictability. The ordinal consistency loss further reinforces relative similarity ordering, making representations more robust to noise and context-dependency.\n3. AdaTS is designed as a pluggable module, seamlessly integrating with three state-of-the-art contrastive frameworks. The extensive evaluations across 6 datasets demonstrate accuracy improvements, with rigorous ablation studies validating the contribution of each component. The low computational overhead highlights its practical applicability.\n\nWeaknesses:\n1. While TFC is effective for many signals, its reliance on short-time Fourier transform (STFT) assumes local stationarity, which may struggle with highly non-stationary data (e.g., abrupt transients). The framework could benefit from more adaptive time-frequency transforms (e.g., wavelets) to handle such cases, as acknowledged in the future work.\n2. In datasets with extreme dynamics (e.g., sudden environmental shifts), the dynamic temporal assignment’s reliance on average dissimilarity might fail to capture instantaneous changes. Incorporating real-time anomaly detection or event-driven weighting could enhance adaptability in such cases. \n3. The ordinal consistency loss may suffer from \"oversampling\" if similar instances are frequently sampled, potentially shrinking embedding distances. Although not observed in experiments, this issue requires mitigation strategies, such as adaptive sampling weights, to prevent representation collapse.\n4. While the method is effective, the additional complexity introduced by dynamic temporal assignments and the physics-guided TFC measure may require careful tuning, which could be an obstacle for practitioners seeking simplicity.\n\nQUESTIONS:\n1. The discussion briefly mentions an \"oversampling\" issue in the ordinal consistency loss. Can the authors elaborate on how significant this problem is, and whether any mitigation strategies were considered? How does the current framework handle imbalanced sampling?\n2. Although AdaTS is said to have minimal computational overhead during pretraining, how does this scale with increasing dataset size or sequence length?\n3. Validate TFC against wavelet-based coherence metrics on datasets with extreme non-stationarity (e.g., ACIDS during terrain transitions). Ablation study comparing STFT vs. wavelet transforms would clarify the approach’s limits and potential improvements.\n4. In multi-modal datasets (e.g., MOD with acoustic and seismic signals), each modality may exhibit distinct non-stationarity. Does AdaTS currently align modality-specific temporal assignments, or are they treated uniformly? How could cross-modality coherence enhance dynamic weighting?\n5. The method relies on two key hyperparameters, λ_oc and λ_t. How sensitive is the performance to the choice of these values across datasets?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-06-29"}

comments: SUMMARY:
This paper claims that existing contrastive learning methods struggle with defining meaningful similarities and overlook inherent physical correlation and sequence-varying non-stationarity.
To improve the representational quality and real-world adaptability, this paper introduce a soft contrastive strategy, AdaTS, which uses frequency coherence for similarity measurement. Based on this, AdaTS proposes ordinal consistency learning for improving relative instance similarity and adapting to sequence non-stationarity. Empirical experiments demonstrate the effectiveness and flexibility of the proposed method.

STRENGTHS AND WEAKNESSES:
Strengths
- The experimental validation is comprehensive, including multiple datasets and several state-of-the-art baseline frameworks. The results demonstrate consistent performance improvements.
- AdaTS is designed as a pluggable and flexile module. 
- Ablation study is provided for the analysis of the contribution of each component.

Weakness:
- In terms of the problem formulation, this papers claims the imperfection of existing similarity metrics. I think real-world examples should be provided to show the problem of existing metrics. That will strengthen the motivation of this study.

QUESTIONS:
NA

LIMITATIONS:
NA
TITLE: Official Review
CLARITY: 3
IMPACT: 2
ORIGINALITY: 2
DATE: 2025-07-04

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a method for adaptive time series representation learning using contrastive learning. It introduces three components: a physics-guided time-frequency similarity measure, ordinal consistency learning, and dynamic temporal assignment. The goal is to improve representation quality in the presence of non-stationarity and unclear similarity. Experiments show AdaTS enhances performance when integrated with existing contrastive learning methods.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n1. The paper provides a detailed discussion of the limitations of existing contrastive time series methods and motivates the need for adaptivity. This is very good.\n2. The design of the Dynamic Temporal Assignment module is interesting and reasonably novel.\n3. Writing is rich and technically detailed, showing clear effort in execution and presentation.\n\nWeaknesses:\n1. Main concern: The Physics-Guided Time-Frequency Similarity module appears to reuse an existing method and without clear justification for its applicability in the proposed setting.\n2. The problem definition is vague—it is not clearly stated what the actual task is (e.g., classification over full sequences, segments, or time points), which makes it hard to interpret the results.\n3. The experimental baselines are not sufficiently up-to-date, with most comparisons ending at 2023. Several more recent 2024–2025 baselines exist in the time series contrastive learning literature.\n4. The paper would benefit from evaluations on open public time series datasets to better contextualize the results and benchmark generalizability.\n5. Figure 1 is difficult to interpret without sufficient explanation, especially for readers unfamiliar with the overall flow. A step-by-step narrative would help.\n6. Minor formatting issues exist—for example, unclear spacing between the main text and footnotes on the first page.\n\nQUESTIONS:\n1. Can the authors clarify why Time-Frequency Coherence is an appropriate choice for the datasets used? In what scenarios is this metric applicable or not applicable? Are there cases where it might fail or mislead similarity estimation?\n\n2. What is the downstream task being evaluated—full sequence classification, segment-level prediction, or pointwise classification? Please make this clear in the main text.\n\n4. Can the authors include more recent and competitive baselines? This would help position AdaTS relative to the SOTA.\n\n5. Are there results of testing AdaTS on open time series datasets or other benchmark collections? That would enhance reproducibility and comparability.\n\n6. Please revise Figure 1 with more descriptive labeling and stepwise explanation in the caption or main text to guide the reader through the proposed pipeline.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 3, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-12"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper presents a novel time series representation learning method through soft contrastive learning. The three major components of AdaTS are Time-Frequency Coherence (capturing harmonic structures and physical correlations), Ordinal Consistency Learning (avoiding noisy absolute similarity values and preserving semantic relationships across sequences), and Dynamic Temporal Assignment (adapting to varying non-stationarity). The final composite loss is designed as a linear combination of the above components.\n\nSTRENGTHS AND WEAKNESSES:\nThe major strengths of this paper:\n\n1. **Performance Gains**: \n\t- The experiment results demonstrate a prominent accuracy boost compared to sota benchmarks in Table 1, indicating that the proposed method is effective on benchmark datasets.\n2. **Efficiency & Adaptability**:\n\t- TFC reduces memory by **2×** vs. DTW with minimal runtime overhead (Table 3).\n\t- DTA dynamically handles diverse non-stationarity (e.g., abrupt vs. smooth sequences).\n3. **Generalizability**:\n\t- Pluggable into SOTA frameworks (TS2Vec, TS-TCC, FOCAL), improving all (Table 1).\n\t- Consistent gains on UCR/UEA benchmarks (Figure 2b).\n\nThe major weakness of this paper:\n\n1. **Oversampling in Ordinal Learning**:\n\t- Frequent sampling of highly similar instances may shrink embedding space diversity.\n2. **TFC’s Stationarity Assumption**:\n\t- STFT assumes local stationarity, limiting performance on highly non-stationary signals.\n3. **Parameter Sensitivity**:\n\t- Margin *δ* (ordinal loss) and temperature $\\tau_T$ (DTA) require per-dataset tuning (Figure 5).\n\nQUESTIONS:\n1. How can we prevent oversampling-induced shrinkage of embedding spaces in ordinal consistency learning? Could adaptive triplet mining or distance regularization help?\n2. Can wavelet transforms or Wigner-Ville distributions replace STFT in TFC to capture transient dynamics (e.g., mechanical faults or bio-signal spikes) without sacrificing efficiency?\n3. How to integrate cross-modal coherence (e.g., joint TFC for acoustic-seismic pairs) to leverage complementary information while preserving modality-specific features?\n4. Can the temporal unpredictability metric be computed incrementally for streaming data, enabling online adaptation to sudden signal changes?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 4, "IMPACT": 4, "ORIGINALITY": 4, "DATE": "2025-07-16"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes AdaTS, an adaptive soft contrastive learning framework for time series representation learning. AdaTS introduces three core components:\n\n- Time-Frequency Coherence (TFC) – a spectral-domain similarity metric meant to reflect physics-guided relationships\n- Ordinal Consistency Learning (OCL) – a ranking-based loss that enforces relative similarity between samples rather than absolute distances\n- Dynamic Temporal Assignment (DTA) – a mechanism that assigns temporal similarity weights per sequence, based on a dissimilarity statistic.\n\nAdaTS is designed as a plug-and-play module that can be integrated into existing contrastive learning frameworks (e.g., TS2Vec, FOCAL, TS-TCC). The authors demonstrate consistent improvements in performance across a wide range of datasets (IoT, HAR, UCR/UEA), particularly under low-label regimes. Ablation studies and hyperparameter sensitivity analyses are also provided.\n\nSTRENGTHS AND WEAKNESSES:\n## Strengths\n- The paper addresses important challenges in contrastive learning for time series, especially in handling non-stationarity and noisy similarities.\n- The framework is modular and integrates easily with existing self-supervised learning (SSL) methods, offering strong performance gains with minimal overhead.\n- The proposed dynamic temporal assignment (DTA) is a practical and well-motivated solution to adapting similarity decay per sequence, and it is supported by reasonable empirical evidence.\n- The empirical improvements are consistent across different baselines and settings, especially in challenging, low-label regimes.\n\n## Weaknesses\n- The use of Time-Frequency Coherence (TFC) is adapted from classic signal processing literature [6], but lacks novelty. Related works like TF-C [1] and MF-CLR [2] have already explored frequency-domain similarity in contrastive learning. The contribution here appears more engineering-oriented than conceptual.\n- The Ordinal Consistency Learning (OCL) component is based on well-established triplet ranking loss formulations. It is not clearly distinguished from prior works such as SoftCLT [3] and TNC [4], and lacks qualitative justification (e.g., embedding smoothness or class structure).\n- The κₙ statistic used for temporal assignment is heuristically defined based on log similarity of adjacent time steps. This approach is sensitive to noise, and the paper does not investigate learning-based alternatives or validate the statistic's robustness (e.g., as explored in StatioCL [5]).\n- The evaluation is heavily focused on downstream accuracy, with no exploration of embedding space quality (e.g., t-SNE visualizations, clustering metrics, intra-class variance) to support the claim of improved representation.\n- The term “physics-guided” is frequently used but weakly justified. There are no domain-specific case studies or physical grounding (e.g., phase alignment, system resonance) that explain how TFC captures meaningful physical signal characteristics.\n- The gains on less dynamic datasets (UCR/UEA) are modest, raising the question of whether AdaTS consistently benefits static time series, and under what conditions it provides significant value.\n\n[1] Zhang, Xiang, et al. \"Self-supervised contrastive pre-training for time series via time-frequency consistency.\" NIPS, (2022): 3988-4003.\n\n[2] Duan, Jufang, et al. \"MF-CLR: Multi-frequency contrastive learning representation for time series.\" ICML, 2024.\n\n[3] Lee, Seunghan, Taeyoung Park, and Kibok Lee. \"Soft Contrastive Learning for Time Series.\", ICLR\n\n[4] Tonekaboni, Sana, Danny Eytan, and Anna Goldenberg. \"Unsupervised Representation Learning for Time Series with Temporal Neighborhood Coding.\" ICLR, 2021\n\n[5] Wu, Yu, et al. \"StatioCL: Contrastive Learning for Time Series via Non-Stationary and Temporal Contrast.\" ICKM, 2024.\n[6] Zhan, Yang, et al. \"Detecting time-dependent coherence between non-stationary electrophysiological signals—A combined statistical and time–frequency approach.\" Journal of neuroscience methods 156.1-2 (2006): 322-332.\n\nQUESTIONS:\nQ1: How does TFC compare, both conceptually and empirically, with prior frequency-domain contrastive metrics like TF-C [1] and MF-CLR [2]? What are the unique advantages of your TFC integration?\n\nQ2: Have you considered learning the decay parameter κₙ via a small network or attention mechanism? Would this allow more flexible and robust adaptation than handcrafted statistics?\n\nQ3: Does ordinal consistency remain effective under noisy or highly dynamic conditions? Can you provide a robustness study or embedding analysis to support this?\n\nQ4: Can you include visualizations (e.g., t-SNE, PCA) of the learned embedding space with and without AdaTS to clarify how the representation quality improves?\n\nQ5: Are there scenarios where AdaTS fails or provides no improvement—especially on more stationary or synthetic datasets? Have you analyzed such cases?\n\nQ6: What physical phenomena or applications specifically benefit from the use of TFC? Can you show that TFC scores correlate with known physical behaviors (e.g., gait cycles, vibration modes)?\n\nLIMITATIONS:\nnil", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-21"}