Title: Selective Learning for Deep Time Series Forecasting
Venue: NeurIPS 2025
Accepted: yes
Keywords: selective learning; time series forecasting; deep learning
Abstract: Benefiting from high capacity for capturing complex temporal patterns, deep learning (DL) has significantly advanced time series forecasting (TSF). However, deep models tend to suffer from severe overfitting due to the inherent vulnerability of time series to noise and anomalies. The prevailing DL paradigm uniformly optimizes all timesteps through the MSE loss and learns those uncertain and anomalous timesteps without difference, ultimately resulting in overfitting. To address this, we propose a novel selective learning strategy for deep TSF. Specifically, selective learning screens a subset of the whole timesteps to calculate the MSE loss in optimization, guiding the model to focus on generalizable timesteps while disregarding non-generalizable ones. Our framework introduces a dual-mask mechanism to target timesteps: (1) an uncertainty mask leveraging residual entropy to filter uncertain timesteps, and (2) an anomaly mask employing residual lower bound estimation to exclude anomalous timesteps. Extensive experiments across eight real-world datasets demonstrate that selective learning can significantly improve the predictive performance for typical state-of-the-art deep models, including 37.4% MSE reduction for Informer, 8.4% for TimesNet, and 6.5% for iTransformer.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nDeep learning models for time series forecasting suffer from overfitting because all future steps, including noisy and abnormal ones, are taken into consideration by the loss function. This work proposes a dual-mask mechanism to filter out noise and anomalies. Noise is filtered by an uncertainty mask derived from residual entropy. Anomalies are filtered by an anomaly mask derived by comparing the residual with that of a pretrained estimation model. Experiments show that the proposed approach can improve the predictive performance of deep models.\n\nSTRENGTHS AND WEAKNESSES:\n### Strengths\n\n- The idea of using selective learning for TS forecasting is interesting and novel.\n- Extensive experiments were conducted to validate the effectiveness of the proposed approach.\n- The proposed approach is agnoistic to backbone and can be used as a plug-and-play module.\n\n### Weaknesses\n- Some derivations are unclear, for example:\n  1. For the uncertainty mask, the inclusion of entropy is confusing. It's hard for me to understand the role of Equations 6 and 7. Moreover, $\\epsilon^{(i)} = 100, \\forall i$, this timestamp has an entropy of 0 accroading to the defination.\n  2. For anomaly mask, also using theoretical lower bound is reasonable, the rational of using another estimation model to detect anomalies needs more explanations, considering lightweight weak models are used here.\n\n- The proposed approach seems to be sensitive to the hyper-parameter masking ratio, which may limit its application:\n  1. Best performance is achieved only masking ratios of uncertainty and anomaly masks are jointly tuned, tuning one of them can not get the optimal performance (Figure 3).\n  2. Different datasets require different ratios (Table 6).\n\n- Other than runing time estimation in Table 7, detailed theoretical computational efficiency analysis is required.\n\nQUESTIONS:\n- If assuming that the dataset contains a significant amount of noise and anomalies, does the input side also need adjustment, rather than just at the output end?\n- The current approach always filters some timestamps even when the dataset is clean. Is there some way to handle this?\n- See my questions in the #Weaknesses Section.\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-16"}

comments: SUMMARY:
The work proposes a novel selective learning strategy that dynamically ignores spurious timesteps by applying two masks. The uncertainty mask removes any step showing unusually high entropy. Additionally to it the anomaly mask filters out observations that fall outside the lower‑bound forecast generated by a linear model $g$. Experiments show the improved performance and large applicability to several modern TSF approaches.

STRENGTHS AND WEAKNESSES:
Strengths: 

- Well written and structured work.
- Easy integration approach, can be plugged in any TSF regression model.   
- Interesting idea on filtering timesteps based on uncertainty and anomaly detection. 

Weaknesses: 

- Entropy mask depends on a unimodal residual fit Gaussian. How SL will behave under a mixture or non-parametric fit?
- Early uncertainty estimation looks a limitation of the approach.

QUESTIONS:
- Have the authors tried non-gaussian residuals? 
- Both masks rely on noisy residuals before the backbone is warm. The authors mention a warm-up phase, can the authors give more details on that? 
- Have the authors tried a held-out validation set to train model $g$? I asked because looks a bit problematic to use the same training set for that too.

LIMITATIONS:
Yes
TITLE: Official Review
CLARITY: 3
IMPACT: 3
ORIGINALITY: 3
DATE: 2025-06-26

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes Selective Learning, a novel training strategy for deep time series forecasting (TSF). Unlike conventional methods that uniformly optimize all timesteps via MSE loss, this method selectively computes loss only over generalizable timesteps while masking out those identified as uncertain or anomalous. The authors propose a dual-mask mechanism consisting of: an uncertainty mask using residual entropy and an anomaly mask based on residual lower-bound deviation. Extensive experiments on 8 real-world datasets and 6 popular TSF models demonstrate significant improvements, particularly for overfitting-prone architectures.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\n1. The paper addresses a core weakness in current TSF training—overfitting on noisy or anomalous data.\n2. The experiments are comprehensive, covering diverse model backbones (Transformer, CNN, MLP), datasets with various periodicity and non-stationarity levels.\n3. The paper is very well written with clear problem setup, motivating examples and thoughtful appendices with implementation details and limitations.\n\nWeaknesses and Questions:\n\n1. Concept Drift vs. Anomalies: Many works on non-stationary TSF emphasize concept drift. In realistic scenarios, drift and anomalies often co-exist. How does the proposed method distinguish between the two? Please provide precise definitions and clarify whether “non-generalizable” includes drifted patterns.\n2. The proposed approach appears closely related to SIN [1], which also filters out unstable regions. Please clarify the technical differences and include empirical comparison to strengthen the novelty claim.\n[1] SIN: Selective and Interpretable Normalization for Long-Term Time Series Forecasting, ICML 2024.\n3. The selective learning principle overlaps with ideas from curriculum learning, hard example mining, and prioritized sampling. The paper should provide a clearer conceptual and methodological differences from these approaches in §2.2.\n4. The residual entropy and estimated lower-bound both depend on the training dynamics (e.g., optimizer, batch size, learning rate). While Theorem 1 attempts to bound the variance estimation error, empirical sensitivity analysis (e.g., different η, batch sizes) is missing.\n5. The use of a second model to estimate residual lower bounds (e.g., DLinear) could introduce biases if the estimation model underfits or overfits. Clarify the generalizability of the estimation model and whether it affects downstream learning in challenging regimes (e.g., high-noise datasets).\n6. The method relies on masking ratios (ra, ru) as key hyperparameters. These appear to be set as fixed percentages. This could lead to a situation where on a \"clean\" dataset, the model is forced to discard perfectly valid timesteps, or on a very \"noisy\" dataset, it fails to discard enough. How robust is the method for the choice of these ratios, especially across datasets with varying noise levels?\n7. The anomaly mask depends on a separate, lightweight estimation model (g(·; φ)). The paper shows that simpler models work better here, which makes sense to avoid overfitting in the estimator itself. However, the performance of the main model is now implicitly dependent on the quality and convergence of this secondary model. How does the performance of Selective Learning degrade if the estimation model g is poorly chosen or poorly trained? Is there a risk of a negative feedback loop where a bad estimator creates bad masks, which further hurts the main model?\n\nQUESTIONS:\nSee the Strengths And Weaknesses section.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces selective learning for deep time series forecasting. Selective learning only calculate the MSE loss over a subset of timesteps, guiding the model to focus on generalizable timesteps while disregarding non-generalizable ones. Technically, the paper propose a dual-mask mechanism (including an uncertainty mask and an anomaly mask) to target timesteps. Extensive experiments across eight real-world datasets demonstrate that selective learning can significantly improve the performance for state-of-the-art deep models.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\nS1. This paper proposes a novel training strategy for deep time series forecasting and offers a valuable insight: optimizing a generalizable subset of the time series is more effective than optimizing the entire sequence due to the inherent vulnerability of time series to anomalies and noise.\n\nS2. Extensive experiments and rigorous analysis demonstrate the effectiveness of selective learning. The performance gain from selective learning is significant, particularly for those models prone to overfitting (e.g., Informer, Crossformer).\n\nS3. The paper is well-written with clear explainations and is easy to follow.\n\nWeaknesses:\n\nW1. In lines 170-176, the authors state that the static masking approach is suboptimal. However, the experiment does not include a comparative analysis between the current dual-masking mechanism and static masking approach.\n\nW2. While the idea of selective learning is general, the current technical implementation of selective learning is confined to in-domain prediction task and cannot be applied to the training of time series foundation model.\n\nQUESTIONS:\nQ1. Although selective learning demonstrates reduced MSE and MAE metrics, could the masking strategy potentially introduce bias to models?\n\nQ2. In Figure 5, why does it present results of Informer for 30 epochs while other models for only 10 epochs?\n\nQ3.In Assumption 4, what does $\\au$ stand for?\n\nQ4. The paper compares selective learning with other training objectives (e.g., PS loss and FreDF). If I understand correctly, selective learning appears theoretically compatible with these objectives. Could the idea of selective learning be combined with existing objectives to further enhance their performance?\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 4, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-02"}