Abstain Mask Retain Core: Time Series Prediction by Adaptive Masking Loss with Representation Consistency
Authors: Renzhao Liang
Sizhe Xu
Chenggang Xie
Jingru Chen
Feiyang Ren
Shu Yang
Takahiro Yabe
Venue: NeurIPS 2025 spotlight ()
Keywords: Time Series Forecasting
Redundant Feature Suppression
Adaptive Masking Loss
Primary area: deep_learning
TL;DR: The paper designs a training objective to suppress the learning of redundant features in time series models.

Abstract: Time series forecasting plays a pivotal role in critical domains such as energy management and financial markets. Although deep learning-based approaches (e.g., MLP, RNN, Transformer) have achieved remarkable progress, the prevailing "long-sequence information gain hypothesis" exhibits inherent limitations. Through systematic experimentation, this study reveals a counterintuitive phenomenon: appropriately truncating historical data can paradoxically enhance prediction accuracy, indicating that existing models learn substantial redundant features (e.g., noise or irrelevant fluctuations) during training, thereby compromising effective signal extraction. Building upon information bottleneck theory, we propose an innovative solution termed Adaptive Masking Loss with Representation Consistency (AMRC), which features two core components: 1) Dynamic masking loss, which adaptively identified highly discriminative temporal segments to guide gradient descent during model training; 2) Representation consistency constraint, which stabilized the mapping relationships among inputs, labels, and predictions. Experimental results demonstrate that AMRC effectively suppresses redundant feature learning while significantly improving model performance. This work not only challenges conventional assumptions in temporal modeling but also provides novel theoretical insights and methodological breakthroughs for developing efficient and robust forecasting models. We have made our code available at \url{https://github.com/MazelTovy/AMRC}.

Abstain Mask Retain Core: Time Series Prediction by Adaptive Masking Loss with Representation Consistency
Renzhao Liang* Beihang University
Sizhe Xu" New York University sx24900nyu. edu
liangr enw.ao@uzzaaaa edu cn
Chenggang Xie
Jingru Chen Peking University
Feiyang Ren New York University fr2303Enyu edu
Beihang University
xiechenggang@buaa edu.
2401212839epku. edu. cn
Shu Yang New York University sy42548nyu. edu
Takahiro Yabe
New York University
takahirooabee@ny.. edu
Abstract
Time series forecasting plays pivotal role in critical domains such as energy man- agement and financial markets. Although deep learning- based approaches (e.g.. MLP, RNN, Transformer) have achieved remarkable progress, the prevailing "long- sequence information gain hypothesis" exhibits inherent limitations. Through systematic experimentation, this study reveals counterintuitive phenomenon: appropriately truncating historical data can paradoxically enhance prediction ac- curacy, indicating that existing models learn substantial redundant features (e.g., noise or irrelevant fluctuations) during training, thereby compromising effective signal extraction. Building upon information bottleneck theory, propose innovative solution termed Adaptive Masking Loss with Representation Consis- tency (AMRC), which features two core components: 1) Dynamic masking loss, which adaptively identified highly discriminative temporal segments to guide gra- dient descent during model training; 2) Representation consistency constraint, which stabilized the mapping relationships among inputs, labels, and predictions. Experimental results demonstrate that AMRC effectively suppresses redundant feature learning while significantly improving model performance. This work not only challenges conventional assumptions temporal modeling but also pro- vides novel theoretical insights and methodological breakthroughs for developing efficient and robust forecasting models. We have made our code available httpps//github... wtwettuub..oo///aaeelooo////////
Introduction
Time series forecasting, pivotal technology in critical domains such as energy management and financial markets, directly influences decision-making quality and economic efficiency [I]
13
19
20
23 Recent breakthroughs in deep learning have driven revolutionary advancements in
'Equal contribution.
Now at University of Leeds. Corresponding author.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

time series prediction. Contemporary frameworks including Multilayer Perceptron (MLP) based architectures [
18
29
30
33
Recurrent Neural Networks (RNNs) with their variants 9
14
22
and attention mechanism-basee models exemplified by the Transformer 12 6 17
21
36
37
39 have
achieved remarkable breakthroughs in modeling complex temporal patterns through the construction of elaborate hierarchical temporal dependencies.
Current mainstream forecasting models predominantly adhere the "long-sequence information gain hypothesis," which posits that extending historical data length enhances the availability of temporal dependencies [16] 34]. However, through systematic experimental analysis, this study challenges this conventional assumption. As shown in Table [ we observed a counterintuitive phenomenon across multiple benchmark datasets and diverse model architectures: appropriately truncating early segments of input sequences can significantly improve prediction accuracy. This finding reveals critical issue in modern predictive models: during training, models inadvertently capture substantial number of redundant features. These features not only fail to enhance performance but also interfere with the learning process, thereby limiting the models' potential to achieve optimal results.
Through systematic analysis, we have identified two typical manifestations of redundant features and their underlying mechanisms. First, input truncation optimization experiments (as shown in Figure 25 Table [ demonstrate that selectively masking partial historical data can significantly improve model prediction performance. This phenomenon reveals the current model's s inefficient utilization of long historical windows. Second, representation similarity analysis (as illustrated in Figure 2a) shows that both the model's prediction results and intermediate embeddings exhibit an abnormally concentrated distribution, which significantly deviates from the natural dispersion characteristics of the input and label. Collectively, these observations indicate that existing models exhibit low efficiency when processing long historical windows, often encoding substantial noise or irrelevant variables rather than truly predictive signals.
Building upon information bottleneck theory 10 method called Adaptive Masking Loss with Representation Consistency (AMRC). The core method- ology comprises: 1) An adaptive masking mechanism that dynamically identifies key segments with high discriminative power sequential data and leverages these informative segments to guide the gradient optimization process (as illustrated in Figure B 2) A representation consistency constraint that establishes stable mapping relationships among the input feature space, label space, and predicted outputs, thereby effectively enhancing the model's generalization capability. Experimental results (as demonstrate that the AMRC method significantly reduces the complexity of the training solution space by suppressing the model's reliance redundant features, fully exploits the performance potential of the model architecture, and consequently improves prediction accuracy.
24
26 27). this study proposes an innovative
shown in Table 2
The primary contributions of this study include:
Theoretical Insight: Through rigorous experimental validation, We demonstrate that existing time series forecasting models are prone to learning redundant features, which in turn constrain their performance. Building on the theory of information bottlenecks, we construct novel theoretical framework for time series modeling and propose an innovative optimization pathway, offering new theoretical perspective for advancing the field of time series forecasting.
Methodological Innovation: We propose an optimization framework Adaptive Masking Loss with Representation Consistency, By dynamically selecting discriminative temporal segments to guide gradient descent (as illustrated in Figure TT] while enforcing input-label-prediete consistency, our method effectively suppresses redundant feature learning. Extensive experiments demonstrate consistent performance gains across diverse benchmarks and architectures.
Our work advances the understanding of temporal pattern learning mechanisms while offering practical pathway to enhance the efficiency and reliability of time series forecasting systems.
2 Related Work
The Information Bottleneck (IB) method was first introduced by Tishby et al. 26 as an information- theoretic framework that aims to compress input signals while preserving as much relevant information as possible about the target output. In the field of machine learning, IB theory has been widely adopted as a regularization technique. For instance, Alemi al. I proposed the Variational Information Bottleneck (VIB), which leverages variational inference to construct a tractable lower bound on the

---

IB objective. Building upon this, Tishby and Zaslavsky further explored the applicability of ietomatiinnthhoreee objectives deep neural networks. Research on IB has also extended into the domain of clustering. Slonim et al. [24] developed distributional clustering algorithm based on mutual information maximization and demonstrated its effectiveness on the 20 Newsgroups dataset, achieving substantial compression with minimal loss of relevant information. More recently, Hu et al.
[0]] conducted a comprchensive survey the IB literature, reviewing over two decades of theoretical developments, methodological advances, and practical applications.
In the context of deep learning, time series forecasting methods can be broadly categorized into MLP, RNN, and Transformer-baut approaches. Among MLP-based models, DLinear (33) and TSMixer are representative examples, featuring relatively simple architectures while achieving strong performance across multiple datasets, RNN-based methods, such as Segrnn [6] and LSTMlong [2]. focus on structural modifications to address challenges related to parallel prediction and long- long-sequence modeling. Transformer-base Informer introduces a sparse attention mechanism to improve the scalability of traditional attention for time series modeling; Autoformer incorporates frequency-domain -ommai information to enhance attention; and iTransformer further extends attention across channels by embedding multivariate sequences for variable-aware representation.
models include Informer [7 Autoformer [21]. and i'Transfommer [I5]
Another key research area concerns noise robustness and representation learning. Early work, such as Informer [37], used sparse attention for information distillation in long sequences, while TS2Vec [3T] adopted contrastive learning to regularize temporal representations. More recently, dedicated frameworks have been proposed. For instance, TS-CoT (35) employs dual-encoder architecture and a cross-view prototype alignment mechanism to achieve global semantic consistency. Similarly, DECL [38) guides contrastive learning to acquire denoising capabilities by constructing positive samples from denoised data and leveraging an adaptive denoiser.
Analysis of Redundant Feature Learning
Given multivariate time series X E RT'D, where T is the number of timesteps and D is the number of variables, the objective of time series forecasting is to learn a mapping function So that transforms historical observations X,- r.t E RLXD (where L denotes the input length into future values X1+1++H E " (where represents the forecasting horizon).
Conventional time series forecasting models follow the long-sequence information gain hypothesis 35 32 37,, which holds that increasing the input length L improves forecasting accuracy. However, our experiments (Table T on multiple standard benchmarks reveal a counterintuitive result: truncating the input-sun such as masking the first & timesteps often improves forecasting performance, which measured by Mean Squared Error (MSE). We found that models tend to learn redundant features, which degrade model performance even after convergence. This finding is supported by two key observations:
Original Data
Data With Mask
Trapped
Avoid Local Optimal
Local Optimal
AMRC
Figure Illustration of the effect AMRC method. Without regularization, the model tends to overfit redundant input features, leading to suboptimal convergence. By suppressing redundant input features, AMRC restructures the optimization landscape, promoting more efficient representation learning and facilitating better convergence.

---

3.1 Input Truncation Optimization
Based on the baseline model configuration (input length L 48, forecasting horizon H 48), we design an input truncation comparative experiment by applying a masking operator M& to the input sequence. When we have an input sequence of length L at time step t, denoted as x! the masking operator M&(.) is mathematically defined as:
Mx (41
if S k:
x
(I)
otherwise
Here, & E {l....,Li L} denotes the masking step size.
To probe redundant features, we employ an Optimal Masking strategy: Given an input sequence of length L, we generate L masked variants (M (zero-padded to preserve dimensionality). For instance, yields L' = 43 (first positions zeroed). The optimal mask length k* is selected as the configuration minimizing MSE, thereby defining the theoretical upper bound for redundancy climination:
[[(((Ma(x())) - -
arg min k (1.2,....))
Table 1: Performance Gains via Optimal Masking Across Time Series Models. Ratio quantifies the percentage of training samples demonstrating prediction error reduction through Optimal Masking, calculated as number of masked series/number of total series x 1000%
Model
ETThl
ETTh2
Solar- Kolar-Fnergy
Weather
Metric
MSB
MSF*
Ratio
MSF
MSF*
Ratio
MSF
MSF
Ratio
MSF
MSE
Ratio
SOFTS
Train Set
0,278
0.254
56.54%
0.318
0.259
61.65%
0.182
0.155
11.80%
0.421
0.400
45.10%
Test Set
0,408
0.365
64.24%
0.326
0,303
28.73%
0,293
0.184
41.58%
0,205
0.185
54.93%
TTansformmrr
Train Set
0,298
0.270
57,87%
0.315
0.261
64.19%
0.410
0,281
61. 61%
0.436
0.389
62.98%
Test Set
0.413
0,289
60.07%
0.329
0.299
32.16%
0.395
0.271
68.43%
0.209
0.170
80.26%
PatchTST
Train Set
Test Set
0,424
0,343
0.402
0,303
65.57%
65.51%
0.327
0.329
0.269
0.298
42.46%
69.35%
0.374
0.366
0.344
0.277
51.66%
35.89%
0.227
0,215
0,180
0.180
42.43%
45.55%
TSMixer
Train Set
0,372
0.342
55.79%
0.544
0.431
73.96%
0.233
0.195
26.30%
0.363
0.348
37.57%
Test Set
0.402
0.372
59.19%
0.324
0.289
42.13%
0,288
0,250
40.12%
0,322
0.195
70.88%
Train Set
0,200
0.262
57.96%
0.300
0.251
59.36%
0.142
0.112
13.58%
0.403
0.353
63 63.93%
TimeMixer
Test Set
0,393
0.366
58.04%
0.318
0.285
44.52%
0,288
0.253
36.25%
0.197
0.172
63%
As demonstrated in Table [ the experimental results confirm that masked models consistently achieve lower MSE, with more than 50% of samples exhibiting improved predictive performance (Ratio> > 50%). Notably, the phenomenon of redundancy learning shows strong architecture architecture-agnostic characteristics. On the Weather dataset, both iTransformer (a Transformer-aase model) and TSMixer (an MLP-based model) demonstrate similar relative improvements: iTransformer achieves an MSE reduction from 0.209 to 0.170 (- (-18.7%) while TSMixer improves from 0.222 to 0.195 (-..2%). These results indicate that the effectiveness of our masking strategy is not dependent on specific model architectures.
3.2 Representation Similarity Paradox
To further investigate the redundant feature learning phenomenon, we apply t-SNE to project the SOFTS model's high-dimensional representations of the input, embedding, prediction, and label onto a 2D plane (Figure. 2ap, after normalizing all features to the [0, 1] range.
As illustrated in Figure. 2 Normalized input (Znn E TR and output (Zout E R embeddings show clear contrast: inputs remain dispersed, while embeddings and preds cluster tightly despite large differences in their corresponding labels. This suggests that the model encodes redundant, task-irreeevant features that misrepresent semantic relationships and distort the input-output mapping.
3.3 Information Bottleneck Constraints on Redundancy
In time-series forcasting models, the input sequence typically encoded into latent representation Z, from which a decoder then predicts the target sequence Y. The optimization objective is to learn

---

Input
Embedding
Pred
Labels
Data2
1.0
1.0
1.0
Data4
Data5
0.5
0.5
0.5
0.5
Dataf
Data7
0.0
0.0
0.0
0.0
Dataš
0.5
1.0
0.5
1.0
0.5
1.0
0.5
1.0
Data9
Datal0
(a) Normalized t-SNE Projections Input, Embedding, Prediction, and Label
Input
Prediction Masked Prediction
Mask Region
Input
Prediction Masked Prediction
Region
True Values
True Values
Original Prediction
Masked Prediction
Original Prediction
Masked Prediction
0.75
0.75
0.5
0.5
0.50
0.50
MULL
0.0
0.0
-0.5
Steps
Steps
(b) Masked vs. Unmasked Prediction Performance
Figure 2: Embedding Distributions and Masking Effects of Our Method.
an optimal representation Z that maximally preserves information relevant to while discarding irrelevant details from X. According to the Information Bottleneck (IB) Theory [24], this process can be viewed as a bottleneck that compresses input information. The informational relationships among X, Y. and Z, which governed by the model's 's learnable parameter A, can be quantified using mutual information. The objective can be thus formally expressed as maximizing the mutual information between the representation Z and the target Y:
1(,,Y:0) - dx dy p(z, 0) log p(z, y 0)
p(= 0 0)p(v 0)
Due to inherent limitations in the data and model capacity, the amount of information that can be extracted and transmitted during training is bounded. Consequently, the representation capacity is subject upper information constraint Ic. Based on this, the objective of the time series prediction model can be equivalently formulated as the following constrained optimization problem:
max (2, Y; s.t. 1(X, 0) < Le-
This constrained optimization problem can be transformed into an unconstrained form using the method of Lagrange multipliers, leading the maximization of the following objective I
(O) = I(Z; Y ;)) - (I (2; ;)
(5)
There implementation paths under this objective: one maximize the mutual information
T(Z; Y) between Z and Y; the other is to minimize the mutual information [(Z; X) between Z and X. Most current sequential prediction models focus improving I I(Z; through iterative training, but have not explicitly optimized performance by penalizing redundant features via minimizing / ((;X)) Therefore, propose adaptive loss function that aims minimize the mutual information between and Z, offering novel optimization path for improving the performance of sequential prediction models.
4 Proposed Method
4.1 Adaptive Masking Loss (AML)
applying ideal masking to input data reduces the information T(X) while improving prediction accuracy. This indicates that the representation Zk-, generated by encoder pe from masked features X,,,- contains less redundancy and better approximates the minimal sufficient statistics (i.e., with smaller 7(X, Zz-;0), Based on this insight, we propose the Adaptive Masking Loss (AML) to explicitly reduce mutual information T(X, Z; 0) by guiding the encoder's output representation toward ZE+, thereby suppressing redundant feature learning and unleashing model potential. The overall framework of AML is illustrated in Figure
As discussed in Section 3.1
5

---

Original data
Find best masks
Lost-rgghl sighal
B
from distribution
Embedding
Data with mask
60 20-2)
max
Mask
Backbone
Loss;
En
Mask:
Loes;
Prodictor
Min(Laus)
Mask
Model
Loss,
Model
Original
Embedkiig
Backbone
Predictor
MSR
MSER
MSE,
MSE
AML
Model W.
K Masked
Embedkling
Backbone
Predictor
Figure 3: Overview of the Adaptive Masking Loss (AML) framework. The upper half illustrates how the optimal mask length K* is selected by evaluating prediction losses over sampled masks. A weighting coefficient / is computed based the gain over the unmasked loss. The lower half shows the AML loss, calculated as the sum of representation differences between the original input and the K masked input across embedding, backbone, and predictor layers.
4.1.1 Implementation
The exhaustive search for optimal mask k* by enumerating all possible mask lengths k E {1...., L} results prohibitive (L) time complexity for long sequences. We therefore adopt an efficient stochastic approximation strategy:
1. Random Mask Generation:
Independently sample m mask indices (kam from uniform distribution d(k-) = Uniform(1, (1, LJ. each generating masked variant:
X f, Mr.
Mx.(X.() (x(44)
2. Loss Evaluation: Compute prediction losses for both masked and original data:
f, = (()o( x(())
(7)
6= (= y!"!)
(8)
Optimal Representation Selection: If the corresponding representation Ž, po Po
(x(L))
satisfies (xx(",, 2.) < 1(x("), 2). where Z Po (x) is the original representation. signifies that masked input variant can achieve better predictive performance than the original input. This provides clear indication that the removed information was redundant rather than
essential. The optimal mask variant is selected by:
arg max(E - Cs)
4.1.2 Loss Formulation
To promote compact and informative representations, AML minimizes the distance between the original representation 7 and the optimal masked variant 7,-:
LAML = B Di x D2
2 21-
(10)
The adaptive weight 3 = max(0, (l-l,-))/ ensures that this regularization term is only active when a better-performing masked representation found. Such setup dynamically scales the optimization intensity, guaranteeing more substantial influence from mask variants with greater loss reduction.

---

4.2 Embedding Similarity Penalty (ESP)
Time series forecasting models often encounter two issues: semantic inconsistency, where seman- tically similar inputs lead to substantially different predictions, and representation collapse, where dissimilar inputs result nearly identical outputs. While consistency regularization methods like Temporal Ensembling [12 and Mean Teacher [25] address stability for individual samples under augmentation, they do not explicitly consider the relational structure between different samples. We therefore introduce the Embedding Similarity Penalty (ESP), strategy that directly addresses this by comparing the geometry the embedding space with that the output space for pairs of samples within mini-batch.
Pairwise distances. For batch B = {(X,, Y) encoder output and keep the ground-truth ER PxD.
we denote by Z; = fenc E R...D the The (normalised) squared Frobenius distances
1
1
Ze
Zy1li,
A 0
PXD IY -YI isi
(11)
LXD
Consistency penalty. Ideally A and D. should match: semantically similar inputs ( 0) ought to produce similar outputs (A 0). and vice versa. Deviation is quantified element-wise through
Ps = ReRLR A D + RELU A = D A
(12)
where ReLU(x)= max(0, 2) and Penalty then reads
|+ denotes the non-negative part. The
1
n
n
LESP
Ps..
(13)
n
i=1
==
Equation 13) back-propagates smooth, unbiased gradients that jointly reshape the encoder and the predictor so that input and output manifolds remain geometrically aligned. The detailed implementa- tion of the Embedding Similarity Penalty (ESP) is provided as pseudocode in Appendix C Algorithm
4.3 Overall Training Objective
introduced the Adaptive Masking Loss LAML that discourages the learning of redundant temporal prefixes, while Section 4.2 proposed the Seabeedding-Siiilart Penalty LESP to enforce comantic-behavioure consistency. Combined with the standard prediction loss Lpeed (e.g., MSE
Section 4.1
between the forecast Y and the target Y), our final objective is
Liotll
Lpred + AAML LAML. + ÀESP LESP,
(14)
where AAML., λESP > 0 control the strength each auxiliary term. Minimizing (14) jointly (i) identifies the informative prefix for every sequence, (ii) preserves the intrinsic topology of the data, and (iii) improves predictive accuracy and interpretability without adding inference-time overhead.
5 Experiment
5.1 Experiment Setup
Datasets. We evaluate our proposed method using seven widely recognized benchmark datasets for multivariate time series forecasting: ETTh1, ETTh2, ETTm1, ETTm2, Solar-Energy, Electricity, and Weather. These datasets encompass variety of application scenarios with different temporal resolutions, seasonality patterns, and dynamic structures. Detailed descriptions of each dataset, including their specific characteristies and collection periods, are provided in the Appendix
Task formulation. In our experimental setup, the forecasting task is formulated as sequence-to- sequence regression problem, applicable to multivariate time series. Each model is trained to predict future sequence (!#) e HD from fixed-length historical input sequence x! x
(18) 18X

---

where H. denotes the prediction length and D is the number of variables. We adopt multiple prediction horizons H E {48, 72, 96, 120, 144, 168, 192].
Baselines. Our method is compared against five diverse baseline models: SOFTS g iTrrnsformer 15]. PatchTST [17], TSMixer [7], and TimeMixer [28]. These baselines are implemented us-
ing their official codebases and recommended hyperparameters to ensure a fair comparison under consistent experimental conditions.
Implementation details. All models are implemented in PyTorch and trained on a single NVIDIA A100 80GB GPU. To ensure fair comparison and allow both baseline models and those augmented with our proposed modules to fully exploit their capacity, we train each model for up to 100 epochs using the Adam optimizer with an initial learning rate of 1 x 10 a cosine annealing scheduler. and a batch size of 32. Early stopping is applied based on validation loss with a patience of 20 epochs. The pest-performing checkpoint on the validation set is selected for final evaluation on the test set.
Hyperparameter selection. For the AMI,, the input sequence prefix length configured as with the mask sampling cardinality parameterized as 12. We fix both AAML and XESP to 1 for all experiments. These settings follow standard benchmark configurations commonly used time series forecasting.
48,
5.2 Forecasting Results
We present the forecasting performance of our method Adaptive Masking Loss with Representation Consistency (AMRC)-in comparison with five representative baseline models across seven widely used time series benchmark datasets. Table D reports the Mean Squared Error (MSE) and Mean Absolute Error (MAE) for each model, both with and without the incorporation of AMRC.
Table 2: Performance Comparison of Time Series Forecasting Models With and Without AMRC. In the experimental results, we highlighted in bold the parts where the AMRC model improved by more than 0.005 in MSE and MAE metrics compared to the baseline model. The detailed hyperparameter configurations for each model can be found in Appendix [R Full results are listed in Appendix [..] Table 5 Furthermore, a detailed statistical analysis presenting results mcan + standard deviation over 10 runs, along with significance tests, is provided in Appendix [.] Table 6
To further validate
the robustness of AMRC, we conducted additional experiments on the ExchangeRate dataset and the challenging. law- Illness dataset, as detailed in Appendix 7
Model
ETTbl
ETT12
ETTml
ETTm2
Solar-Eacrg
Electricity
Weather
Metric
MSE
MAE
MAE
MSE
MSB
MAE
MAD
MSE
MAE
SOFTS
Original
0.408
0.414
0.326
0.359
0.484
0.434
0,210
0.285
0.293
0,314
0.169
0.255
0.205
0.234
AMRC
0,389
0,393
0.311
0,362
0.475
0.423
0.198
0.265
0,290
0.309
0.162
0,244
0,196
0.220
iransformmrr
Original
AMRC
0.413
0.402
0.399
0.415
0.329
0.324
0.362
0.356
0.517
0.502
0.448
0.447
0.213
0.211
0.290
0.280
0.395
0.392
0.352
0.342
0.176
0.163
0.260
0.239
0,201
0.209
0.221
0.237
TimeMixer
Original
AMKC:
0.388
0.393
0.401
0.316
0.339
0.466
0.447
0.405
0.204
0.285
0.269
0.284
0.317
0.194
0.188
0.277
0.197
0.186
0.237
0.228
PatchTST
Original AMRC
0.424
0.424
0.327
0.358
0.461
0.422
0.211
0.287
0.374
0.382
0.211
0.283
0.215
0,280
0.411
0.415
0.319
0.356
0.456
0.196
0.271
0.361
0.376
0.207
0.210
0.264
TSMixer
Original
0.402
0.412
0.324
0.357
0.440
0.413
0.201
0.279
0.288
0.314
0.172
0.258
0.222
0.288
AMRC
0.386
0.397
0.340
0.432
0.196
0.257
0.313
0.169
0.212
0.281
Consistent Performance Gains, Across all models and datasets, our method consistently yields performance improvements. For example, the MSE of the SOFTS model decreases from 0.408 to 0.389 on the ETThl dataset. Similar trends are observed in iTransformer, where the MSE on Electricity drops from 0.176 to 0.163, The enhancements demonstrate that AMRC effectively mitigates redundant or noisy temporal segments, thereby improving prediction stability and accuracy.
Architecture-Agnsste Effectiveness. AMRC delivers significant performance gains not only on Transformer-basee architectures such as iTransformer and PatchTST, but also on MLPed models including TimeMixer, SOFTS, and TSMixer. For instance, on the ETTm2 dataset, the MSE of PStchTST model decreases from 0.211 to 0.196 (a reduction of approximately 7.11%), while the MSE of SOFTS model drops from 0.210 to 0. 0.198 (approximately 5.71 % reduction). These results demonstrate the strong architettre--gnostic generalization ability of AMRC, highlighting its broad applicability across a wide range of time series forecasting models.
8

---

Generalization on Low-Channel Datasets. On datasets with fewer input channels (ETThI, ETTh2,
ETTml, ETTm2), AMRC effectively enhances model performance. For instance, on ETTml, the MSE of iTrrasffrmer decreases from 0.517 to 0.502, and that of TSMixer drops from 0.440 to 0.432. These results demonstrate AMRC's ability to mitigate overfitting and improve prediction accuracy in low-dimensional time series forecasting tasks.
Robustness on High- Channel Datasets. For high-dimensional datasets such as Weather (21 chan- nels) and Solar-Energy (137 channels) see in Appendix E AMRC consistently improves robustness by reducing the impact of signal noise and inter-channel redundancy. On the Weather dataset, TimeMixer's MSE decreases from 0.197 to 0.186 and MAE from 0.237 to 0.228, while i'Transfrmmer sees an MAE drop from 0.237 to 0.221. On Solar- Golar-Energy: PStchTST's MSE drops from 0.374 to 0.361, and SOFTS sees a slight MAE reduction from 0.314 to 0.309. These enhancements highlight AMRC's effectiveness in managing complexity in multivariate time series with high channel counts.
Generalizable Training Framework. The consistent performance improvements observed across all models validate the strong scalability and integrability of AMRC. As constraint-based optimization strategy, AMRC does not rely on any specific model architecture, making it highly generalizable, It serves versatile training framework for enhancing both the efficiency and accuracy of time series forecasting models.
5.3 Ablation Study
Setup. We evaluate ablation variants four diverse datasets: ETThl and ETTh2, representing hourly electricity load with varying degrees of seasonality; Solar-Energy, which exhibits weather- driven variability and periodicity; and Weather, a multivariate meteorological dataset with complex inter-variable dependencies. We adopt a fixed input horizon following standard benchmarks. We also analyzed the sensitivity the number of sampled masks, m, used in AML. While larger m allows for a more extensive search, it incurs greater computational cost. Our analysis, detailed in Appendix Table reveals diminishing returns mincreases. Consequently, set m 12 for all experiments to effectively balance performance and computational efficiency.
Evaluation protocol. For each dataset, we apply the ablation study to five baseline models SOFTS, iTransformer, TimeMixer, PatchTST, and TSMixer under four configurations: I) baseline + AML, 2) baseline ESP, and 3) baseline + both AML and ESP. This design allows us assess the standalone effectiveness of each module as well as their combined synergy.
Findings. We evaluate the individual and joint effects of the AML and ESP components using five representative forecasting architectures across four datasets. As shown in Table both components contribute measurable performance gains in isolation, while their combination AMRC consistently leads to the best forecasting accuracy in terms of MSE and MAE. AML provides stronger improve- ments across most settings, supporting its role in suppressing redundant prefixes during training. ESP, while often delivering smaller standalone gains, remains beneficial by promoting geometric alignment between embedding and output spaces. Together, these findings demonstrate that each component addresses distinct source of generalization eтror.
Component impact across architectures. The benefits of AML and ESP are consistently ob- served across all backbone models, regardless of architectural differences. For instance, models with strong expressiveness, such as iTransformer and TimeMixer, benefit significantly from AML, achieving notable MSE reductions on datasets like Weather and ETTh2. Even architectures with- out attention mechanisms, such as SOFTS and TSMixer, exhibit consistent gains. highlighting the broad applicability of adaptive prefix masking- In contrast, the improvements from ESP are often more dependetd being particularly effective on high-dimensional multivariate inputs where representation alignment plays critical role. For example, ESP yields non-trivial reductions in MAE on Weather, where multiple variables evolve under shared dynamics. Notably, we observe relatively smaller improvements on the Solar-Energy dataset for transformer-based models such PatchTST and iTransformer, which may be attributed to their reliance on longer input sequences for stable attention computation.
Complementarity and synergy. The AMRC configuration, which jointly applies AMI and ESP, consistently outperforms its ablated variants across all benchmarks. The performance improvement

---

Table Ablation Study Results on Different Model Components
Model
ETThi
ETTh2
Solar-Energa
Weather
Mettic
MSE
MAB
MSE
MAE
MSE
MAE
MSE
MAE
AMI.onh only
0.322
0.192
0.228
SOFTS
ESP only
0.393
0.398
0.318
0.351
0.295
0.318
0.208
0.241
AMRC
0.389
0.362
AML only
0.410
0.413
0.328
0.363
0.398
0.347
0.205
0.230
iTransoomer
ESP only
AMRC
0.402
0.399
0.324
0.356
0,392
0,342
0.201
0.221
only
0.232
TimeMixer
ESP only
0.391
0.406
0.317
0.347
0.293
0.322
0.202
0.248
0.316
0.186
AML only
0.419
0,420
0.325
0.361
0.369
0.579
0.214
0.274
PatchTST
ESP only
0.417
0.418
0.322
0.357
0.384
0.217
AMRC
0.411
0.415
0.319
0.356
0.361
0.376
0.210
0.264
AML only
0.396
0.404
0.324
0.356
0.285
0.317
0.216
0.283
TSMixer
ESP only
0.390
0.799
0.322
0.352
0.291
0.323
0.224
0.292
0.340
0.212
from combining both components generally exceeds the stronger of the two individual effects, indicating synergistic interaction. This complementarity can be attributed their distinct operational scopes: AML operates on the input level by learning to suppress non-in con-informative temporal segments, while ESP regularizes the latent space to align representations across semantically related inputs. As result, AMRC improves both the quality of features learned from the data and the consistency of their usage in prediction. The robust gains observed across datasets and architectures suggest that jointly addressing input redundancy and representation inconsistency is critical for improving generalization in time series forecasting.
Table 4: AMRC Effectiveness with Prefix Masking at a Fixed Input Length (L = 48). Ratio is the percentage of training samples with reduced MSE under prefix masking. Ratio $ is the same metric after training with AMRC. Average results across all lengths are in Appendix D. Table T0.
Model
ETTh2
Solar-Fnergy
Metric
Ratio
Rati
Ratio
Ratio
Ratio
Ratio
Ratio
Ratio"
57.33%
28.72%
20.28%
41.58%
33.49%
47.12%
Trunsformer
60.05%
49.95%
32.16%
23.28%
68.43%
63.21
80.26%
70.29%
TimeMixer
58.04%
46.29%
44.52%
34.17%
36.25%
27.90%
66.13%
52.28%
PatcTTT
65.51%
51.63%
42.46%
26.19%
$1.66%
47.64%
42.43%
30.78%
TSMixer
59.19%
46.62%
42.13%
27.98% %
40.12%
28.36
70.88%
58.23%
Effectiveness of AMRC in Reducing Redundant Features We evaluate the model's robustness to redundant input by computing the proportion of training samples with improved MSE under prefix masking Ratio and compare it to the value after applying AMRC Ratio*. As shown in Table 4 AMRC consistently improves or maintains this ratio, indicating its effectiveness in suppressing the impact of redundant temporal information.
6
Conclusion
This study pioneers the investigation into the negative effects of redundant feature learning time series forecasting and introduces AMRC, plug-and-ple solution that suppresses such learning without requiring architectural modifications. Unlike prior work focused on enhancing predictive features, AMRC improves accuracy by reducing reliance on redundant features while maintaining model flexibility, Its key advantages include: 1) seamless integration with existing models, 2) effective suppression of feature redundancy, and 3) strong generalization performance across benchmark tests. By addressing the cong-overlooked issue of redundant learning, this research provides a novel and practical methodology for optimizing forecasting models.
10

---

References
[1] Alexander Alemi, lan Fischer, Joshua V Dillon, and Kevin Murphy. Deep variational information bottleneck. arXiv preprint arXiv:1612.00410, 2016.
[2] Peng Chen, Yingying Zhang, Yunyao Cheng, Yang Shu, Yihang Wang, Qingsong Wen, Bin Yang, and Chenjuan Guo. Pathformer: Multi-scale transformers with adaptive pathways for time series forecasting. In International Conference on Learning Representations, 2024.
[3] Zihang Dai, Zhilin Yang, Yiming Yang, Jaime Carbonell, Quoc Le, and Ruslan Salakhutdinov. Transformer-xl: Attentive language models beyond fixed-length context. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. Association for Computational Linguistics, 2019.
[4] Abhimanyu Das, Weihao Kong, Andrew Leach, Shaan K Mathur, Rajat Sen, and Rose Yu. Long-term forecasting with tide: Time-series dense encoder. Transactions on Machine Learning Research, 2023.
[5] Jiayu Ding, Shuming Ma, Li Dong, Xingxing Zhang, Shaohan Huang, Wenhui Wang, Nanning Zheng, and Furu Wei. Longnet: Scaling transformers to 1,000,000,000 tokens. arXiv preprint arXiv:230.002486, 2023.
[6] Alexandre Drouin, fitienne Marcotte, and Nicolas Chapados. Tactis: Transformer-attennionna copulas for time series. In International Conference Machine Learning, pages 5447-5493.
PMLR, 2022.
[7] Vijay Ekambaram, Arindam Jati, Nam Nguyen, Phanwadee Sinthong, and Jayant Kalagnanam. Tsmixer: Lightweight mip-mixer model for multivariate time series forecasting, In Proceedings of the 29th ACM SIGKDD conference on knowledge discovery and data mining, pages 459-469, 2023.
[8] Lu Han, Xu- Yang Chen, Han-Jia Ye, and De-Chuan Zhan. Softs: Efficient multivariate time series forecasting with series-core fusion. Advances in Neural Information Processing Systems, 37:64145 64175, 2024.
[9] Hansika Hewamalage, Christoph Bergmeir, and Kasun Bandara. Recurrent neural networks for time series forecasting: Current status and future directions. International Journal of Forecasting, 37(1):388 427, 2021.
[10] Shizhe Hu, Zhengzheng Lou, Xiaoqiang Yan, and Yangdong Ye. A survey on information bottleneck. IEEE Transactions Pattern Analysis and Machine Intelligence, 2024.
[11] Nataliia Kashpruk, Cezary Piskor-Ignaton likkt--gnnttoicz, and Jerzy Baranowski. Time series prediction in industry 4.0: a comprehensive review and prospects for future advancements. Applied Sciences, 13(22):12374, 2023.
[12] Samuli Laine and Timo Aila. Temporal ensembling for semi-supervised learning. In Interna- tional Conference on Learning Representations, 2017.
[13] Bryan Lim and Stefan Zohren. Time-series forecasting with deep learning: survey, Philosoph- ical Transactions of the Royal Society A, 379121944:222202209 2021.
[14] Shengsheng Lin, Weiwei Lin, Wentai Wu, Feiyu Zhao, Ruichao Mo, and Haotong Zhang, Segrnn: Segment recurrent neural network for long-term time series forecasting. arXiv preprint arXiv:2308. 1200, 2023.
[15] Yong Liu, Tengge Hu, Haoran Zhang, Haixu Wu, Shiyu Wang, Lintao Ma, and Mingsheng Long. itransformer: Inverted transformers are effective for time series forecasting. In International Conference on Learning Representations, 2024.
[16] Chao Ma, Yikai Hou, Xiang Li, Yinggang Sun, and Haining Yu. Long input sequence network for long time series forecasting. arXiv preprint 20X.v:2407.15869, 2024.

---

[17] Yuqi Nie, Nam H. Nguyen, Phanwadee Sinthong, and Jayant Kalagnanam. A time series is worth 64 words: Long-term forecasting with transformers. In International Conference on Learning Representations, 2023.
[18] Boris N Oreshkin, Dmitri Carpov, Nicolas Chapados, and Yoshua Bengio. N-beats: Neural basis expansion analysis for interpretable time series forecasting. arXiv preprint arXiv: 1905. 10437, 2019.
[19] Asiye K Ozcamli, Fatma Yaprakdal, and Mustafa Baysal. Deep learning methods and applications for electrical power systems: comprehensive review. International Journal of Energy Research, 44(9):716--7557 202..
[20] Fotios Petropoulos, Daniele Apiletti, Vassilios Assimakopoulos, Mohamed Zied Babai, Devon Barrow, Souhaib Ben Taieb, Christoph Bergmeir, Ricardo J Bessa, Jakub Bijak, John E Boylan, al. Forecasting: theory and practice. International Journal of forecasting, 38(3):705-871, 2022.
[21] Yankun Ren, Longfei Li, Xinxing Yang, and Jun Zhou. Autotransformer: Automatic transformer architecture design for time series classification. In Pacific-Asia Conference on Knowledge Discovery and Data Mining, pages 143-155. Springer, 2022.
[22] Koushik Roy, Abtahi Ishmam, and Kazi Abu Taher. Demand forecasting in smart grid using long short-term memory. 202/ International Conference on Automation, Control and Mechatronics for Industry 4.0 (ACMI), pages 1-5. IEEE, 2021.
[23] Zezhi Shao, Fei Wang, Yongjun Xu, Wei Wei, Chengqing Yu, Zhao Zhang, Di Yao, Tao Sun, Guangyin Jin, Xin Cao, et al. Exploring progress in multivariate time series forecasting: Comprehensive benchmarking and heterogeneity analysis. IEEE Transactions on Knowledge and Data Engineering, 2024.
[24] Noam Slonim and Naftali Tishby. Agglomerative information bottleneck. Advances in neural information processing systems, 12, 1999.
[25] A Tarvainen. Mean teachers are better role models: weight-averaged consistency targets improve semi-supervised deep learning results. Advances in Neural Information Processing Systems, 30: 1 1196, 2017.
[26] N TISHBY. The information bottleneck method. In Proceedings of the 37-thAnnual Allerton Conference on Communication, 2000, 2000.
[27] Naftali Tishby and Noga Zaslavsky. Deep learning and the information bottleneck principle, In 2015 ieee information theory workshop (itw), pages 1-5. leee, 2015.
[28] Shiyu Wang, Haixu Wu, Xiaoming Shi, Tengge Hu, Huakun Luo, Lintao Ma, James Y Zhang, and JUN ZHOU, Timemixer: Decomposable multiscale mixing for time series forecasting. In International Conference on Learning Representutions, 2024.
[29] Zhijian Xu, Ailing Zeng, and Qiang Xu. Fits: Modeling time series with 10k parameters. CoRR, 2023.
[30] Kun Yi, Qi Zhang, Wei Fan, Shoujin Wang, Pengyang Wang, Hui He. Ning An, Defu Lian, Longbing Cao, and Zhendong Niu. Frequency-domain mips are more effective learners in time series forecasting. Advances in Neural Information Processing Systems, 36.76656-76679, 2023.
[31] Zhihan Yue, Yujing Wang, Juanyong Duan, Tianmeng Yang, Congrui Huang, Yunhai Tong, and Bixiong Xu. Ts2vec: Towards universal representation of time series. Proceedings of the AAAI conference artificial intelligence, volume 36, pages 8987, 2022.
[32] Manzil Zaheer, Guru Guruganesh, Kumar Avinava Dubey, Joshua Ainslie, Chris Alberti, Santiago Ontanon, Philip Pham, Anirudh Ravula, Qifan Wang. Li Yang, et al. Big bird: Transformers for longer sequences. Advances in neural information processing systems, 33: 17283-17297, 2020.
12

---

[33] Ailing Zeng, Muxi Chen, Lei Zhang, and Qiang Xu. Are transformers effective for time series forecasting? Proceedings of the AAAI conference on artificial intelligence, volume 37, pages 11121-11128, 2023.
[34] Ailing Zeng, Muxi Chen, Lei Zhang, and Qiang Xu. Are transformers effective for time series forecasting? In Proceedings of the AAAI conference on artificial intelligence, volume 37, pages 11121-11128, 2023.
[35] Weiqi Zhang, Jianfeng Zhang, Jia Li, and Fugee Tsung. A co-training approach for noisy time series learning. In Proceedings of the 32nd ACM International Conference on Information and Knowledge Management, pages 3308-3318, 2023.
[36] Yunhao Zhang and Junchi Yan. Crossformer: Transformer utilizing cross-dimension dependency for multivariate time series forecasting. In The eleventh international conference on learning representations. 2023.
[37] Haoyi Zhou, Shanghang Zhang. Jieqi Peng, Shuai Zhang, Jianxin Li, Hui Xiong, and Wancai Zhang, Informer: Beyond efficient transformer for long sequence time-series forecasting. In Proceedings of the AAAI conference on artificial intelligence, volume 35, pages 11106-11115, 2021.
[38] Shuang Zhou, Daochen Zha, Xiao Shen, Xiao Huang, Rui Zhang, and Korris Chung. Denoising- aware contrastive learning for noisy time series. Proceedings of the Thirty-Third International Joint Conference Artificial Intelligence, pages 5644 5652, 2024.
[39] Tian Zhou, Ziqing Ma, Qingsong Wen, Xue Wang. Liang Sun, and Rong Jin. Fedformer: Frequency enhanced decomposed transformer for long-term series forecasting, In International conference on machine learning, pages 27268-27286. PMLR, 2022.
13

---

A Limitations
Despite the demonstrated effectiveness (Table 2 of our approach, AMRC has several limitations related to its underlying assumptions, interpretability, and practical trade-offs, which highlight important directions for future work.
Limitations of AML AML's efficacy is bound by two key factors: the temporal characteristics of the data and the interpretability of its masking mechanism.
The prefix-makking strategy assumes that redundant information often resides in the initial segments of a time series. In scenarios where the most critical predictive informa- tion lies exclusively the later portions of the input sequence, AML's core mechanism becomes ineffective. Masking the prefix will not improve the prediction loss, causing the adaptive coefficient / to remain zero and deactivating the regularization.
significant limitation is the "black box" nature of the masking process. While AMI. designed to identify and suppress redundancy, it is difficult determine precisely what kinds of patterns are being masked-whether they represent noise, outliers, or simply outdated information. The adaptive-weight mechanism improves efficiency, but the decision process is not transparent. Clarifying this is a crucial direction for future work to enhance the method's interpretability.
2. Dependency on Data Dimensionality and the Role of ESP
We observe that ESP's improvements are more pronounced on datasets with lower feature dimensionality (e.g., the ETTh family). On higher-dimensional datasets like Weather (21 channels) and Solar-Energy (137 channels), its standalone gains are comparatively smaller.
This occurs because ESP aligns the geometric structure between the embedding and output spaces. As feature dimensionality increases, the optimization directions for this alignment grow exponentially, introducing greater uncertainty during training and potentially yielding diminished returns.
This limitation is effectively mitigated within the combined AMRC framework. High- dimensional datasets often contain significant feature redundancy, which is precisely the condition where AML excels. Therefore, the two components are highly comple- mentary: ESP is most effective in lower-dimensional settings, while AML provides the primary benefit in higher-dimensional, redundant settings, ensuring that AMRC remains robust across diverse data types.
3. Inherent Design Trade-offs The search for an optimal mask requires evaluating m variants per batch, increasing the training cost by a factor of approximately m. This makes it less suitable for latency-sensitive applications.
The optimal mask is found via stochastic sampling of m candidates, which is an approxima- tion of an exhaustive search. This practical compromise means that some redundancy may remain, though it strikes balance with computational feasibility.
B Details of the Baseline Model
All models are reproduced based on their official open opun-source impllmennatiiinnnnns
1. SOFTS from fttph//tgggggggtttooooooo/oollar
2. TimeMixer from fttpmh/tggg/ggghhh.....o////u///
3. iTransformer http://pgg///gttooooooco///////r
4. PatchTST from http://twt///ggttt...ooo///////a
5. TSMixer from http:ttppp..ttttcccco///aaaaaatp
The hyperparameters for each model different datasets follow the official configurations provided in their corresponding GitHub repositories. For the PatchTST model on the Solar-Energy dataset, since no official configuration was provided, we adopted the hyperparameter settings from iTransformer.
14

---

C
Model Detail
C.1
ESP
Algorithm I:
Penalty (ESP) for Time Series Forecasting
Input: Mini-batch B {(X,, Y.) encoder feucs predictor fpred
Output: Penalty loss LESP
Forward pass compute encoder outputs
for i 4- to do
3
74 < Senc X;) end
#encoder output E LRD
5 2. Compute pairwise Frobenius distances
Initialize A A E R
for 4- to do
8
for j <- to n do
9
A E
1j f- LxD
Z4 1
#embedding similarity #output similarity #symmetry
AQ
[Y, Y
A E
J + A
11
Ej+
AO JR + AO
12
end
14 end
3. Compute pairwise penalties
16 for i 1-+1 to m do
for j + do
Ps - -
#element-wiee consistency penalty
end
20 end
Compute final regularization loss
22 LESP n² Ey=E,=1 P.j
23 5. Backward pass and update
Update using forecasting loss +AESP CESP
15

---

D Full Results
D.1 Experimental Result Details
Table 5: Multivariate forecasting results with prediction lengths / E {48, 72, 96, 120, 144, 168, }92] and fixed input window length L 48. Red highlights indicate performance improvements > 0.005 using our method, while blue highlights denote improvements > 0 but < 0.005.
Models
SOPTS
TimeMiser
Wianstomee
PacaTST
TSMixe
ociginal
AMBC
original
AMBC
original
AMRC
original
AMBC
ociginal
AMBC'
Metnc
MSSH
MAD
MSI:
MAR
MSS
MAR
MSS
MAB
MSS
MAA
MSSUI
MAB
MSS
MAG
MISI:
MAA
MSS
MAE
MSS::
MAB
0.339
0.364
0.381
0.367
0.377
0.367
0.375
96
0.394
0.377
0.376
0.399
0.344
0.401
0.908
0
0.387
0.411
0.417
0.997
0.405
0.389
0.37%
01,95
120
0.418
0.421
0.400
0.401
0.395
0.410
0.397
0.404
0.19
0.419
0.410
0.403
0.428
0.426
0.415
0.418
0.436
0.415
0.386
0.491
194
0.676
0.424
0.464
01.02
0,415
0.471
114
0.413
0454
0.427
I.420
0.407
0.465
0.634
0.433
0.434
0.419
0.427
0.405
1
168
0.438
0.434
0,416
0.409
0.425
0.427
0.420
0.417
0.433
0.434
0.431
0.421
0.456
0.441
0.441
0,429
0.433
0.431
0.412
0.416
0.450
0.427
0.410
0.435
0.490
0.488
0.443
11449
0.411
0.488
0.448
0.446
0.421
Ang
0.408
0:401
0:413
0.415
11,12
0.10
0.424
0.411
0,417
46
0.236
0.304
0.221
0.303
0.285
0.302
0.230
0.290
0.246
0.312
0.237
0.306
0.341
0.306
0.329
0.301
0.241
0.302
0.229
0.277
72
0
0.333
0.275
0.344
0.273
0.335
0.250
0.309
0.283
0.336
0.250
0.327
0.261
0.312
0.274
0.325
0.276
0.328
0.270
0.299
BTMC
120
0.326
0.319
0.361
0.350
0:315
0,307
0.366
0.304
0.323
0.248
0.359
03385
0331
11:08
0.33
0.338
0,388
0.30
11352
0.266
0.30
1.32
0.389
0.545
0.351
0.300
0.361
0.309
0:336
0.299
0.357
0.39
01300
0.331
0.361
0.345
0.354
03714
0.330
0.314
0.316
0:175
8-153
0.94
0:11
9,62
0.14
11372
06366
0.36
0.353
0.334
6:60
0.112
0.355
0.971
0.181
0.308
0.271
0.354
0-35
0368
0260
0,20
0.279
0.291
0,576
0.392
0.376
0.88
6:367
0.901
0.400
0.98
0.10
0.402
0.307
0.990
6.90
Aug
0.326
0.399
0.311
0.363
0.318
0.355
0316
0.339
0.329
0.362
0.34
0.356
0.327
0.356
0.319
0.33g
0.334
0.357
0.310
0.340
0.697
0-434
0.463
0,422
0.422
0,423
0,443
0,39
0,5c3
0.653
0.529
0.465
0.481
0.02
0.472
0-417
0.452
0.41
0.442
0.404
32
0.062
0/421
0.41
0-118
0/51
0/20)
0,99
0
11358
1.879
0.438
0.653
[111
0153
0-400
0:419
0.999
0.40
0 1.99
BT
0.647
6/188
0:40
0.49
6.37
0,15
0,408
0,302
0,475
0311
0,31
11917
0
0.42
0,50
0422
0.002
0,20
0:30
0.4%
6.995
0.304
0.907
0.525
0.496
0.531
0.457
0.517
0,477
0.43
0.901
0.445
0.488
0.430
0.470
0.45
0.447
0.409
0.521
0.452
IL504
0.451
0,465
0.431
0.489
0,418
0.451
0.451
0.444
Aug
0.484
0.434
0.415
0.423
0.465
0.429
0.447
0.405
0.517
0.448
0.502
0.447
0.461
0.422
[ 0.456
0,413
0.440
0.413
0.432
0.412
0.154
0:441
0.226
6.177
0.251
0.130
0.190
0.255
0.158
0:342
0.160
01251
6:151
0.341
0.147
6.141
0.208
01166
0:346
0173
03778
11208
1178
0.361
0.176
0.203
0:159
ETTm2
0.111
0.180
0.367
0.360
0.300
0.16
03365
1500
6368
6.356
0314
0750
1210
0.280
0.313
0.561
0.191
0.258
0.281
0.118
01261
144
0.236
0,302
6.311
0.300
0.381
0.30
0266
6385
0256
030
1255
6.296
0.353
0,362
0.330
0.266
0.228
0.295
0.219
0.272
168
0.346
0:111
0.355
0.300
0.345
0.11
6285
9251
11515
1211
6.301
0338
0.300
6:311
0.291
01346
0.105
0:335
1..79
192
0.261
0.316
0.245
0.293
0.255
0.313
0.20
0295
0200
0.221
0.257
0.312
0.200
0.317
0.340
0.300
0.257
0.313
0.243
0.290
Ang
0.210
0.285
0.098
0.365
0.309
0.285
0.204
6,609
0213
0.250
0,211
0.200
0.311
0.287
0.196
0.201
0.279
0.096
0.256
0.294
0.350
0,357
0.34
1.354
0.302
0.386
0.248
Solue- Elerpy
0.333
0.313
0:33
0.363
03511
0342
0441
03511
11442
0,373
0.424
0.250
0.415
0.414
96
0.308
0.334
0.36
6:03
0.69
0.345
0304
0345
0331
0.45
0.363
0.400
0.417
0.301
0.400
0.306
0.334
0.301
0.346
120
0.285
0.311
0.388
0330
0111
11515
0,390
0.36
0.356
6365
0.30
01250
0.115
0.3K3
1.309
144
0.256
0.316
0.291
0:30
0:288
0.300
0256
0205
0200
1131
1266
0.322
0364
0.355
0.331
0:351
0.250
0.304
0:275
0.301
0.28%
0.388
0:228
0.100
0.228
0.90
0172
0.356
0.19
0.300
6:536
0.347
(1266
0.11
0.306
0.304
0.316
0.288
0.308
0.29
0.17
0.20
0.119
0,392
0,251
0,29
0.342
0.309
0.356
0.500
0.352
0.297
0.321
0.30
0.295
0.345
0/352
0.543
0,314
|
0.258
45
[.16]
0.388
0.152
0.174
0.153
6.199
0.145
0.183
0.159
11119
1:147
0.177
0.159
0.361
6.185
0.345
1.109
0.237
0.158
0.236
0.176
02212
0.174
0.203
0.179
0220
0.165
0.205
0.180
1211
1.17
0.204
0.208
0.220
6.30
0:262
0.200
0.273
0.996
0.265
Woitte
36
0.201
0:253
0.395
0:221
0.203
02511
0.991
0.241
0200
112
1197
0:335
0.119
0.26
0314
0:276
01215
0.288
0:15
0.289
0.204
0:33
0.155
0.37
0213
0.20
0.205
0.1%6
0.322
0.20
0:10
0.218
0.300
6.114
0.259
11212
0:33
0.343
0.333
0.36
0,245
0.571
0.200
0.214
0.566
Ang
0.205
0.254
0.196
0.220
0.197
0.257
0.186
0.328
0,209
11237
0.201
0.331
0.215
0.250
01100
0.364
01222
0.285
0.312
0.261
0.146
0.233
0.138
0.221
0.172
0.250
0.164
0.5%
0.101
11.3%%
0.136
0:216
0.189
0.264
0.264
0.148
0.230
0.141
0.234
72
0.161
0.163
0.253
0.158
0:325
0.306
0.229
6:303
0.278
0.165
0.351
0.236
56
0.171
0.774
11262
[.16]
0.156
0.28
6:11
0.300
0284
0.322
8.136
0.761
0.168
0.305
11:91
0881
175
11755
0.245
0.315
0.661
0:300
0.150
0.384
67.66
11264
163
0.176
0.262
0.166
0.348
0.199
0.255
1154
0224
6381
0.56
0.165
0.349
0.211
0.394
0306
0.266
0.181
0.365
0.
0.252
190
0.181
0.766
0.170
0.252
0.300
0.761
10
0.086
0.210
1.176
0.250
0.314
0.908
0111
0.701
0.186
0.361
01 11
11.2%
Avg
0.169
0.255
0.362
0.344
0.194
0.275
0.188
0.22
0.120
11260
| 0.163
0.339
0.211
01283
0.30
0.265
0.173
0.258
0.106
0.347
16

---

Table 6: Detailed statistical analysis of AMRC effectiveness. This table presents the mean + standard deviation over 10 runs for original and AMRC-enhanced models. The 'Conf (%)² row indicates the confidence level from significance tests comparing AMRC to the baseline.
Model
Metric
ETTh
ETTa2
ETTml
Ef'm2
MSB
MAE
MSE
MAE
MSE
MAE
MSE
MAE
Original
0 0.008 0,004
0.444 1 0.005
0.528
1.0003
0.350
1.000
0,454 1 0.044
0,434 1 0.000
0.210
0.002
0 0.555
0.001 0.1M6
SOFTS
AMKC
0.011
0.384
0.009
0.311
0.008
0.302
0.000
0,475
0.006
0.423
0.198
0.307
0.265
Conf
99
Original
0.001
0.529
0:00
0.002
TTrnnfoommer
AMRC
0.402
0.004
0,208
0.324
0,336
0.004
0.467
0.211
0.003
Coof
Original
0.093 a 0.005
0.408
1 1,0005
0.518
0.006
0,355
1 0.08
0,896
1. 0.004
0,429 1: 0.006
0.209
0.002
0.285
0.004
TimeMixer
AMRC
0.388 = 0.0006
0.401
= 0.000
0.008 0.316
0.007 0.339
0.447 + 0.008
0.405 + 0.000
0.007 0.203
0.269 = 0.006
Conf (%)
99
99
99
99
99
99
99
99
Original
0.424 a 0.005
0.424
1 0.002
0.527
0.001
0.358
1 0.00
0.061
1 0.003
0.422 1. 0.002
0,211
1. 0.002
0.287
0.003
PatchTST
AMBC
0.411 = 0,00
0.415
= 0.00
0.319
0.004
0.356 a 0.004
0,436 + 0.003
0,413 a 0.000
1.NMS 0.196
0.271
0.1M44
Conf (58)
90
99
95
99
99
99
99
Original
0.402-
0.005
0.122
+ 0. 0.005
0.524
0.004
0.357 +: 0.004
0,440 + 0.063
0.415 0.006
0.201
0.005
0.
0.005
TSMaser
AMRC
0.386
0,010
0.397
0.008 "
0.319
0.007
0.540
0.011
0.432 1 0.010
0,412
0.196
0.007
267 0.013
Conf
99
Solar- Energy
Electricity
Weather
Model
Metric
MSE
MAE
MSE
MAE
MSE
MAE
Original
0.293 = 0.003
0.314 + 0.004
0.169 = 0.003
0.255
0.004
0.205 =
0.002
0,234 = 0.003
SOFTS
AMRC
0.290 + 0.007
0.309 + 0.007
0.162
0.006
0.244
0.007
0.005 0.196
0.186 + 0.004
Conf (%)
95
95
99
99
Original
0.395 + 0.002
0.352 + 0.002
0.176
0.002
0.260
0.003
0.209
0.003
0.237 + 0.002
iTransformer
AMRC
0.392 = 0.006
0.342 + 0.005
0.163 = 0.004
0.239 + 0.007
0.201 = 0.005
0.221 + 0.008
Conf (%)
99
99
Original AMRC
0.288
0.003
0.317
0.000
0.194
0.010
0.279
0.006
0.197
0.010
0.237 +: 0.009
TimeMixer
0.284 + 0.008
0.317 + 0.008
0.188 + 0.012
0.277 + 0.008
0.014 0.186
0.228 + 0.011
Conf (%)
95
90
99
99
99
Original AMRC
0.374 + 0.003
0.383 + 0.004
0.211 + 0.002
0.283
0.002
0.215
0.002
0.280 + 0.003
PatchTST
0.361 + 0.006
0.376 + 0.007
0.207 = 0.004
0.002 0.285
0.003 0.210
0.264 + 0.003
Conf (%)
95
99
99
95
99
99
Original AMRC
0.288 + 0.004
0.314 + 0.004
0.173 = 0.005
0.258
0.006
0.222
0.002
0.288 + 0.007
TSMixer
0.280 = 0.011
0.313 = 0.005
0.169
0.009 99
0.247
0.006 95
0.212 0.010
0.281 + 0.009
Conf (%)
99
95
99
99
Table 7: Additional experiments on the Illness and ExchangeRate datasets using the SOFTS backbone. Results are reported as mean + standard deviation over 10 runs. Due to the small size of the Illness dataset (967 samples), the experimental setup was adjusted (Input L = 48, prediction lengths H E (24, 36, 48, 60)) following the PStchTST protocol [17]
/
AMRC MSE
AMRC MAE
Original MSF
Original MAE
Conf-MSF. (S)
Conf-MAFF (%)
24
16633
1 0.07
0.789 1 0.06
1.776 1. 0.14
0.852
0.03
95
95
18888
0.8541
1,92 1
90
Illness
48
2.035
1 0.07
0.916 1 0.05
2.153 1 0.12
0.954 0.03
95
95
2.054
0.9355
2.113
0.958
90
90
48
0,03913
0.001
0,13016 +: 0.000
0.01208 = 0.001
0,13728 + 0.008
99
95
72
0.06788 0.002
0.16132 + 0.008
0.06093
0.003
0.17116 + 0.009
95
90
96
0.07927 + 0.002
0.18782 + 0.005
0.08329 + 0.004
0.20196 + 0..00
99
99
ExchangeRate
120
1000033- 0.001
0.21698
+ 0.002
0.10095 = 0.001
0.22840 + 0.001
99
99
144
0. 1.124177 + 0.002
0.24484 0.002
0.125 + 0.002
0.25241 + 0.001
99
99
0.14602
0.002
0.25876
0.001
0.16047
0.003
0.28234
0.003
99
99
192
0,17457 0.007
0.29181 + 0.006
08366
0.007
0.30397 + 0.007
95
99
17

---

Table 8: Hyperparameter sensitivity analysis for the mask count (m) on the ETThl dataset with the Trannsformer backbone. We report the average MSE and MAE m varies, The results show diminishing returns, justifying our choice of 12.
Model
MSE (mean)
(mean)
1/8
01.007
0.416
0.409
1/4
0.389
0.393
SOFTS
0.384
0.388
1/2
0
0.385
0.380
0.384
1/8
0.412
0.412
0.417
Transformer
1/4
0.402
0.399
1/3
0.398
0.395
1/2
0.395
0.392
3/4
0.394
0.390
1/8
0,396
0.405
0.391
0,409
TimeMixer
1/4
0.388
0.401
1/3
0.385
0.39%
0.383
0.396
3/4
0,382
0.395
1/8
0.422
0,420
1/6
0.419
0.421
PaccTTT
1/4
0.411
0.415
1/3
0.406
0.411
0.409
3/4
0,401
0.408
1/8
0.401
0.411
1/6
0.395
0.408
0
0.397
TSMixer
1/3
0.381
0.392
3/4
0.376
0.387
Table 9: The robustness of AMRC SOFTS. Results averaged experiments, each tested with different random seeds.
Solar-Fnergy
Prediction
MSP
MAR
MSE
MAE
MSF
MAR
MSE
MAR
0.334
0.003
0.359 0.002
0.221 0.001
0.303
0.002
0.253
0.002
0.289
0.032
0.152
0.301
0.174
0.035
0.377
0.002
0.358
0.002
0.307
0.002
0.364
1: 0.001
0.308
0.002
0.322
0.002
0.195
0,002
0.221
0.002
120
0.400
0.002
0.400 + 0.005
0.315
0.001
0.368 0.002
0.282
0.002
0.299
0.002
0.197
0.001
0.223
0.003
0.404
0.002
0.402
0.002
0.333
0.002
0.371
0.002
0.002
0.309
0.003
0.002
0.233
0.001
0.002
0.408
0.002
0.354
0.002
0.345 0.001
0.002
0.304
0.003
0:238
0.427
0.002
0.410 + 0.002
0.373
0.002
0.399
0.005
0.298
0.001
0.308
0.001
0.232
0.003
0.249
0.002
18

---

Table 10: AMRC Effectiveness with Ideal Masking Averaged Across All Input Lengths. Ra- tio is the percentage of samples with reduced MSE under ideal masking. Ratio is the same metric after training with AMRC. These results are averaged across all input lengths (24, 48, 96, 120, 144, 168, 192)) to show overall robustness.
Models
SOFTS
TimeMixer
iTransformer
PatcTTST
TSMixer
Metric
Ratio
Ratio
Ratio
Ratio
Ratio
Ratio
Ratio
Ratio
Ratio
Ratio
ETThi
57.14%
48.7%
49.69%
37.81%
51.88%
42.93%
57.81%
42.91%
52.92%
39.1%
ETTh2
30.99%
21.09%
47.16%
34.89%
33.28%
24.11
43.54%
27.91%
44.29%
29.63%
Solar- Solar-Energy
44.87%
33.83%
37.52%
29.61%
71.18%
67.72%
53.26%
48.02%
41.66%
30.11%
Weather
54.63%
48.33%
67.39%
52.78%
79.4%
69.36%
41.98%
29.86%
69.32%
56.26%
D.2
Visualized Prediction Comparison Chart
Prediction Comparison
Prediction Comparison
Input
Input
True Values
True Values
Original Prediction
Original Prediction
Optimal Masking
Optimal Masking
Our Method
0.4
Our Method
-0.2
Mask Region
Mask Region
0.3
36
60
60
(a) SOFTS in ETTh1
(b) SOFTS in ETTh2
Prediction Comparison
Prediction Comparison
Input
Input
True Values
0.5
True Values
Original Prediction
Original Prediction
Optimal Masking
Optimal Masking
-0.2
Mask Region
Our Method
0.3
Mask Reglon
Our Method
(c) TixeMixer in ETTml
(d) TimeMixer in ETTm2
Comparison
Prediction Comparison
Input
0.4
0.0
True Values
Original Prediction
eoocimott
True
Original Prediction
Our Nethod
Optimal Masking
0.0
Optimal Masking
0.4
Mask Region
Mask Region
24
48
72
96
Time
Steps
(c) iTransformer in Solar-Energy
(f) iTransformer in weather
Prediction Comparison
Prediction Comparison
Input
Input
True Values
True Values
Tedegc
1.0
Original Prediction
18.6
Optimal Masking
Optimal Masking
Our Mothod
Our Method
Region
Region
12
24
35
60
72
54
36
Stops
(g) PStchTTT in weather
(h) TSMixer in ECL
Figure 4: Qualitative comparison of prediction performance. Each subplot provides visual compari- son of the ground truth, the baseline model, the optimal masking result, and the forecast from AMRC on specific model and dataset. The mask region highlights the prefix portion of the input.
19

---

E
Dataset description
Here we provide detailed descriptions along with download links for each dataset:
1. ETT (Electricity Transformer Temperature) 37
This collection includes two hourly- resolution datasets (ETTh) and two -e-minute-resolutii datasets (ETTm). Each dataset captures seven key operational metrics (including oil and load measurements) from electricity transformers, spanning from July 2016 to July 2018.
2. Electricity Comprising hourly power consumption records from 321 customers, this dataset covers the period from 2012 to 2014.
3. Weather:
Featuring 21 meteorological indicators (such as air temperature and humidity), this dataset provides semmintt-inneeree recordings throughout 2020, sourced from weather stations in Germany.
4. Solar-Energy:
Documents the solar power generation output of 137 photovoltaic plants in 2006, with measurements taken at 10-minute intervals.
Table 11: Detailed Dataset Descriptions. The table summarizes key characteristics of the time series datasets, including the number of channels, prediction lengths, dataset splits, temporal granularity, and application domains.
Dataset
Channels
Prediction Length
Dataset Split (Train,
Granularity
Domain
ETThI, ETTh2
[48. 96, 120, 144, 168, 192]
(8545, 2881, 2881)
Hourly
Electricity
ETTml,, ETTm2
[48, 72, 96. 120, 144, 168, 192
(34465, 11521, 11521)
15min
Electricity
Weather
[48, 72, 96, 120, 144, 168, 192]
(36792, 5271, 10540)
10min
Weather
ECL.
321
[48. 72, 96, 120, 144, 168, 192]
(18317, 2633,, 5261)
Hourly
Electricity
Solar-Energy
137
[48, 72, 96, 120, 144, 168, 192]
(36601, 5161, 10417)
10min
Energy
20

---

NeurIPS Paper Checklist
The checklist is designed encourage best practices for responsible machine learning research, addressing issues of reproducibility, transparency, research ethics, and societal impact. Do not remove the checklist: The papers not including the checklist will be desk rejected. The checklist should follow the references and follow the (optional) supplemental material. The checklist does NOT count towards the page limit.
Please read the checklist guidelines carefully for information on how to answer these questions. For each question in the checklist:
You should answer [Yes], [No], or [NA].
[NA] means either that the question Not Applicable for that particular paper or the relevant information is Not Available.
Please provide short (1-2 sentence) justification right after your answer (even for NA).
The checklist answers are an integral part of your paper submission. They are visible to the reviewers, area chairs, senior area chairs, and ethics reviewers. You will be asked to also include (after eventual revisions) with the final version of your paper, and its final version will published with the paper.
The reviewers of your paper will be asked use the checklist as one of the factors their evaluation. While "[Yes] is generally preferable to "[No]" ", it is perfectly acceptable to answer "[No] providedd proper justification is given (e.g.. "error bars are not reported because it would be too computationally expensive" or "we were unable to find the license for the dataset we used"). In general, answering binary way, we
"[No] or "[NA]" is not grounds for rejection. While the questions are phrased acknowledge that the true answer is often more nuanced, please just use your best judgment and write justification to elaborate. All supporting evidence can appear either in the main paper or the supplemental material, provided in appendix. If you answer [Yes] to a question, in the justification please point to the section(s) where related material for the question can be found.
IMPORTANT, please:
Delete this instruction block, but keep the section heading "NeurIPS Paper Checklist",
Keep the checklist subsection headings, questions/answers and guidelines below.
Do not modify the questions and only use the provided macros for your answers.
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
The main claims are clearly written in the abstract and introduction.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results be expected to generalize other settings.
is fine include aspirational goals as motivation long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
We discussed the limitation of our method in Appendix A.
21

---

Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g.. independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or speech-to-text system might not be used reliably provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faireess
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full of assumptions and a complete (and correct) proof?
Answer: [Yes]
Justification: All the theories and hypotheses we proposed are supported by experimental and mathematical derivations.
Guidelines:
The answer NA means that the paper does not include theoretical results. All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
Justification: We provide detailed descriptions of the hyperparameters in the paper and appendices, along with an anonymous link to the experimental demo the abstract.
Guidelines:
The answer NA means that the paper does not include experiments.
22

---

If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g.. with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors welcome describe the particular way they provide for reproducibilityy In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: We have included link an anonymous demo of our experiments in the abstract.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips. cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (c.g., for a new open-source benchmark).
The instructions should contain the exact command and environment needed to to reproduce the results. See the NeurIPS code and data submission guidelines (https
//nips.cc/public/eee.c wttp://ddd//ooeeuumiieeeeeeeeeee ssi ssionyolicy) for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
23

---

Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification:
We provide the experimental setup details both the main text and appendices.
Guidelines:
The answer NA means that the paper does include experiments.
The experimental setting should be presented in the core the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer:
[Yes]
Justification: The margin of error is reported in the appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
should be clear whether the error bar is the standard deviation or the standard error of the mean.
It is OK to report I-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not show in tables figures symmetric error bars that would yield results that are out of range (c.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: We provide sufficient computational resource details for each experiment in both the main text and appendices.
Guidelines:
The answer NA means that the paper does include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.

---

The paper should provide the amount of compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary failed experiments that didn't make into the paper).
9. Code ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics htttps:/ http://sss.cplllleccccccccceeeee
Answer: [Yes]
Justification: Our methodology and implementation fully adhere to the ethical code standards set forth by NeurlPS.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No. they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g.. there a special consid- eration due to laws regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification: We have discussed the broader impact of time series forecasting in both abstract and introduction.
Guidelines:
The answer NA means that there no societal impact of the work performed.
the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deploymentt of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (c.g., gated release models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility ML).
11. Safeguards
Does the paper deseribe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [No]
Justification: This paper does not have this risk.
25

---

Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but encourage authors to take this into account and make best faith effort.
Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification:
We included it in implementation details and appendix.
Guidelines:
The answer NA means that the paper does not existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset used and, possible, include a URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g.. website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of in the package should be provided. For popular datasets, paperssithcode com datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that re- re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's creators.
13. New assets
Question: Are assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: N/A.
Guidelines:
The answer NA means that the paper does release new assets.
Researchers should communicate the details the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
26

---

Answer: [NA]
Justification: N/A.
Guidelines:
The answer NA means that the paper does involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurIPS Code of Ethies, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals equivalent for research with human subjects
Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Question:
Answer: [NA]
Justification: N/A.
Guidelines:
The answer NA means that the paper does involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing. editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, originality of the research, declaration is not required.
Answer: [No]
Justification:
We did not use any large language models (LLMs) in this work.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non stansadd components.
cc. cc//oonfeeenccss/2222555 Please refer to LLM policy (https://neur))) http:://wwurppppecoeooeeeeeeeeee for what should or should not be described.
27


Reviews:
[{"review_id": "01h3C3KpaS", "reviewer": "Reviewer_EQP6", "summary": "The authors make an interesting observation that recent NN architectures trained on standard benchmarks can improve their performance by masking the context windows, and that latent representation lack desirable properties. They leverage these empirical findings to construct representation penalizations to train time series models with better generalization. They demonstrate their approach in benchmark settings.", "questions": "1. Although the use of IB in representation learning may customary, it would be useful in section 2.3 to clarify the relationship between X, Y, Z and $\\theta$ \n2. In 3.1 this can be said of any transformation applied to X that reduces entropy, to me the important part is the invariance or preservation of predictive power, or in other terms the mutual infomation between the label and the transformed input.\n3. why does the first inequality in line 148 hold?\n4. Is the ESP novel in the time-series literature - I didn't see any references on it.\n5. The combination of AML and ESP is reminiscent of contrastive objectives in (self and semi-supervised) representation learning, have you explored connections?", "limitations": "Yes", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n* The empirical observation about performance of SOTA architectures with context masking is - to the best of my knowledge - novel, and of independent interest\n* Augmenting time series reconstruction error objective with the proposed representation regularizations/losses  is also novel and well motivated\n* Experimental results using standard benchmark settings and models demonstrate performance improvements\n* Authors provide code\n\nWeaknesses:\n* Minor, some of the information theory connections/motivations/rationales are unclear to me in its current presentation.", "was_revised": true, "final_justification": "I mantain my positive score, but increase my confidence in it thanks to the authors rebuttal clarifications.", "rebuttal": "Thank you sincerely for your valuable comments. Your recognition of our empirical findings on context masking and so on is deeply encouraging. At the same time, the issues you identified—such as insufficient clarity in the theoretical exposition—provide important directions for further improving the work. We have carefully examined each of your points and made corresponding revisions (with changes in the manuscript). Below, we respond to your specific comments point by point, and we respectfully welcome any further guidance.\n\n**Q1**: Although the use of IB in representation learning may customary, it would be useful in section 2.3 to clarify the relationship between $X$, $Y$, $Z$ and $\\theta$.\n\n**A1**: Thank you for your valuable suggestion! To more clearly clarify the relationships among $X$, $Y$, and $Z$, we will add the following clarification in Section 2.3: In time-series forecasting models, the input sequence $X$ is encoded into a latent representation $Z$, and the decoder then predicts the target $Y$ based on $Z$. The optimization objective is to learn an optimal representation $Z$ that maximally preserves the information relevant to $Y$. The informational relationships among $X$, $Y$, and $Z$ can be quantified using mutual information.\n\n**Q2**: In 3.1 this can be said of any transformation applied to X that reduces entropy, to me the important part is the invariance or preservation of predictive power, or in other terms the mutual infomation between the label and the transformed input.\n\n**A2**: You have pinpointed the core idea of our approach—eliminating redundant information without compromising predictive performance—which is precisely what motivates AML. Concretely, we introduce an adaptive coefficient $\\beta = \\max\\left(0, \\frac{\\ell - \\ell_s^{*}}{\\ell}\\right)$ and apply the information-bottleneck regularization only when the representation-learning loss $\\ell_s$ is lower than the original prediction loss $\\ell$ (i.e., $\\ell_s < \\ell$). This setup guarantees, by construction, that the transformed representation preserves the same predictive capability as the raw input. Your feedback helps us articulate the theoretical underpinnings more clearly. We will highlight this point more prominently in Section 3.1 of the revised manuscript.\n\n**Q3**: Why does the first inequality in line 148 hold?\n\n**A3**: What follows is a proof of this statement.\n\n**Assumptions.**\n\n- Original input: $X$\n- Masked sample: $\\tilde{X} = M_k(X)$\n- Encoder: $p_{\\theta}(\\cdot)$\n- Original latent representation: $Z = p_{\\theta}(X)$\n- Masked latent representation: $\\tilde{Z} = p_{\\theta}(\\tilde{X})$\n\n**By the definition of mutual information.**\n\n$$\nI(X; Z)=H(Z)-H(Z\\mid X),\\qquad\nI(X; \\tilde{Z})=H(\\tilde{Z})-H(\\tilde{Z}\\mid X).\n$$\n\n**Under a deterministic encoder.**\n\nWhen the encoder parameters $\\theta$ are fixed, $Z=p_{\\theta}(X)$ and $\\tilde{Z}=p_{\\theta}(\\tilde{X})$ are deterministic functions. Hence,\n\n$$\nH(Z\\mid X)=0,\\qquad H(\\tilde{Z}\\mid X)=0.\n$$\n\nTherefore\n\n$$\nI(X; Z)=H(Z),\\qquad I(X; \\tilde{Z})=H(\\tilde{Z}).\n$$\n\nFurthermore, since $\\tilde{X}$ is obtained from $X$ by masking, and\n\n$$\nZ=p_{\\theta}(X),\\qquad \\tilde{Z}=p_{\\theta}(\\tilde{X}),\n$$\n\nassert that\n\n$$\nH(\\tilde{Z}) < H(Z).\n$$\n\nTherefore\n\n$$\nI(X; \\tilde{Z}) < I(X; Z).\n$$\n\n**Q4**: Is the ESP novel in the time-series literature - I didn't see any references on it.\n\n**A4**: Thank you gives us an opportunity to clarify the innovative contribution of the ESP method. The citations to prior work you pointed out are indeed important, and we will add a discussion of them in the Related Work section of the supplementary material.\n\nThe core idea of ESP is to impose an input–output consistency constraint using relative distances: if two inputs x1,x2 are similar, then their corresponding outputs y1,y2 should also be similar. We note that the semi-supervised learning literature already contains consistency-regularization approaches with a related spirit. For example, the π-Model [1] applies two different perturbations (e.g., noise or data augmentation) to the same input and forces the model’s perturbed predictions to be consistent; Temporal Ensembling [1] constructs an unsupervised loss by taking a moving average of predictions across training epochs; and Mean Teacher [2] introduces a teacher model (often with stronger augmentations) as the consistency target for the student model in order to stabilize predictions.\n\nHowever, ESP differs fundamentally from these methods in its problem setting, object of constraint, and optimization objective. Consistency methods in semi-supervised learning primarily focus on the stability of a single sample’s predictions under different perturbations. By contrast, ESP is designed specifically for the unique challenges of time-series forecasting (e.g., semantic inconsistency and representation collapse): it explicitly aligns the geometric structure of the input embedding space with that of the output space to enhance robustness, while not relying on unlabeled data or data augmentation—making it simpler and more efficient to use. The novelty of ESP lies in introducing the idea of manifold alignment into time-series forecasting for the first time, directly constraining pairwise relative distances between samples rather than enforcing per-sample perturbation consistency, which better matches the characteristics of sequential data.\n\n[1] Laine S, Aila T. Temporal ensembling for semi-supervised learning[J]. arXiv preprint arXiv:1610.02242, 2016.\n\n[2] Tarvainen A, Valpola H. Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results[J]. Advances in neural information processing systems, 2017, 30.\n\n**Q5**: The combination of AML and ESP is reminiscent of contrastive objectives in (self and semi-supervised) representation learning, have you explored connections?\n\n**A5**: Thank you for raising this insightful question! We have indeed thought deeply about the connections between AML/ESP and (self-/semi-)supervised representation learning, and we agree this is a very worthwhile direction to explore.\n\nSpecifically, the design of ESP is inspired by the idea of consistency regularization in self-supervised learning—akin to positive-pair alignment in contrastive learning. ESP improves representation robustness by enforcing geometric consistency between the input and output spaces (i.e., $\\Delta E \\approx \\Delta O$). Unlike standard contrastive methods, however, ESP does not rely on data augmentation or explicit negative sampling. Instead, it directly leverages the supervisory signal—the output-space distance $\\Delta O$—to shape the geometry of input representations, which makes it particularly well suited to time-series forecasting.\n\nSimilarly, the masking mechanism in AML does have parallels to semi-supervised learning: a masked input and its original counterpart naturally form a contrasting pair, and the model’s predictions for the masked variant can play a role analogous to pseudo-labels."}, {"review_id": "Oi2YdUk8ri", "reviewer": "Reviewer_nsY4", "summary": "This paper investigates the phenomenon that longer historical data does not always lead to better accuracy in time series forecasting, primarily due to the model learning redundant or noisy features. To address this issue, the authors propose a simple, plug-and-play training framework called Adaptive Masking Loss with Representation Consistency (AMRC), which features two core components: 1) Dynamic masking loss and 2) Representation consistency constraint. AMRC dynamically masks potentially redundant temporal information during training, encouraging the model to focus on more informative segments and improving its generalization ability.", "questions": "Please check the weaknesses I mentioned earlier.", "limitations": "- The paper does not extensively discuss the robustness of the method to hyperparameter choices or provide guidelines for their selection.\n- Limited discussion on potential scenarios where AMRC might be less effective or may introduce drawbacks, such as over-masking or hindering the learning of genuinely informative long-term dependencies.", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 3, "originality": 2, "strengths_and_weaknesses": "Strengths:\n- Apparently, this is an effective method to enhance time series forecasting by mitigating the negative effects of redundant feature learning.\n- The authors evaluate AMRC across multiple datasets with varying characteristics and several baseline models. The consistent performance improvements validate the effectiveness and robustness of the approach.\n\nWeaknesses:\n- The method relies on masking strategies and hyperparameters (e.g., prefix length, mask sampling), but the paper lacks discussion on its robustness to these choices or guidance for tuning them.\n- The writing quality needs improvement, and there are several inconsistencies throughout the manuscript, such as:\n    * Inconsistent use of \"Figure,\" \"Fig.,\" and \"Fig\"\n    * Lack of uniformity in font sizes across text elements within and between figures", "was_revised": true, "final_justification": "Upon revisiting the paper, other reviews and the authors’ responses, I acknowledge that while some concerns regarding robustness remain, in my opinion, the paper presents valuable technical contributions to the field and demonstrates strong potential. I appreciate the authors’ openness to further discussion and their commitment to improving the work. Overall, I now lean towards accepting the paper. I have updated the score from \"Weak accept\" to \"Accept\".", "rebuttal": "Dear Reviewer:\n\nWe are sincerely grateful for your thorough review and valuable comments—especially your careful identification of writing-quality issues, which has been pivotal in improving the paper’s professionalism and readability. We have addressed the problems you noted in the revised manuscript. Your suggestions have not only strengthened the rigor of the paper but also clarified directions for methodological improvement. In accordance with your feedback, we have undertaken a comprehensive revision; our detailed point-by-point responses are as follows:\n\n**Q1:** The paper does not extensively discuss the robustness of the method to hyperparameter choices or provide guidelines for their selection.\n\n**A1:** You are correct that AMRC’s performance can be influenced by hyperparameter choices. We have added the following analysis:\n\n**Mask count m:** In principle, a larger m allows the model to search for the most informative segments over a longer history (expanding the window from K to K+n). However, under our current search procedure this enlarges the optimization space and increases computational cost. Our hyperparameter sensitivity study(The results are shown in the table below) shows that while the loss continues to decrease as \\(m\\) grows, the rate of decrease diminishes (i.e., diminishing returns). We therefore set \\(m=12\\) as a balance between computational cost and efficiency.\n\n Here, *m* denotes the number of sampled masks and *L* is the input length; “MSE (mean)” and “MAE (mean)” are averaged across different output lengths.\n\n| Model        |  m/l | MSE (mean) | MAE (mean) |\n|:-------------|:----:|-----------:|-----------:|\n| SOFTS        |  1/8 |      0.407 |      0.416 |\n| SOFTS        |  1/6 |      0.407 |      0.409 |\n| SOFTS        |  1/4 |      0.389 |      0.393 |\n| SOFTS        |  1/3 |      0.384 |      0.388 |\n| SOFTS        |  1/2 |      0.381 |      0.385 |\n| SOFTS        |  3/4 |      0.380 |      0.384 |\n| iTransformer |  1/8 |      0.412 |      0.412 |\n| iTransformer |  1/6 |      0.411 |      0.417 |\n| iTransformer |  1/4 |      0.402 |      0.399 |\n| iTransformer |  1/3 |      0.398 |      0.395 |\n| iTransformer |  1/2 |      0.395 |      0.392 |\n| iTransformer |  3/4 |      0.394 |      0.390 |\n| TimeMixer    |  1/8 |      0.396 |      0.405 |\n| TimeMixer    |  1/6 |      0.391 |      0.409 |\n| TimeMixer    |  1/4 |      0.388 |      0.401 |\n| TimeMixer    |  1/3 |      0.385 |      0.398 |\n| TimeMixer    |  1/2 |      0.383 |      0.396 |\n| TimeMixer    |  3/4 |      0.382 |      0.395 |\n| PatchTST     |  1/8 |      0.422 |      0.420 |\n| PatchTST     |  1/6 |      0.419 |      0.421 |\n| PatchTST     |  1/4 |      0.411 |      0.415 |\n| PatchTST     |  1/3 |      0.406 |      0.411 |\n| PatchTST     |  1/2 |      0.403 |      0.409 |\n| PatchTST     |  3/4 |      0.401 |      0.408 |\n| TSMixer      |  1/8 |      0.401 |      0.411 |\n| TSMixer      |  1/6 |      0.395 |      0.408 |\n| TSMixer      |  1/4 |      0.386 |      0.397 |\n| TSMixer      |  1/3 |      0.381 |      0.392 |\n| TSMixer      |  1/2 |      0.378 |      0.389 |\n| TSMixer      |  3/4 |      0.376 |      0.387 |\n\n**Q2**:Limited discussion on potential scenarios where AMRC might be less effective or may introduce drawbacks, such as over-masking or hindering the learning of genuinely informative long-term dependencies.\n\n**A2**:Thank you for raising this key question. Our method does have limitations on some datasets. For AML, if the information most relevant to predicting $Y$ lies in the prefix time steps, AML’s prefix-masking policy (starting at 0) becomes suboptimal: multiple rounds of random masking increase computation, and the penalty coefficient $\\beta = \\max\\left(0, \\frac{\\ell - \\ell_s^{*}}{\\ell}\\right)$ remains identically zero, making performance gains unlikely. This highlights AML’s sensitivity to key information that does not lie in the prefix. For ESP, the improvements are relatively limited on high-dimensional datasets (e.g., Weather and Solar-Energy), because ESP enhances robustness by explicitly aligning the geometry of the input embedding space with that of the output space and directly constraining pairwise relative distances; as the number of channels (feature dimensionality) increases, the optimization directions grow exponentially with dimensionality, introducing greater uncertainty during optimization and potentially yielding smaller gains than on lower-dimensional datasets."}, {"review_id": "d8hqBNl2fU", "reviewer": "Reviewer_mCJD", "summary": "The authors propose Adaptive Masking Loss with Representation Consistency (AMRC), a training scheme for time-series models that draws on information-bottleneck theory. It introduces a dynamic masking loss that automatically highlights the most informative temporal segments during training, steering gradient updates away from redundant patterns, and it adds a representation-consistency constraint to keep the relationship among inputs, labels, and predictions stable. By jointly reducing noise and preserving meaningful structure in the learned features, AMRC improves forecast accuracy and demonstrates a more efficient, principled way to build robust temporal models.", "questions": "Q1. It seems like once masks are learned for a given task there could be some additional optimization to increase inference efficiency. Even if there was no performance gain, it would be helpful for storage and memory reasons to reduce the length of the sequence of any given time series problem. Is there any work the authors have explored to analyze what these efficiency wins may be under different task settings?\n\nQ2. In some previous works related to time series interpretability there has been analysis of applying a dynamic mask [https://arxiv.org/pdf/2106.05303] or sliding window [https://arxiv.org/pdf/2107.14317] to analyze how predictions change. Would further gains be possible if the masking operator was defined for some sequence from k to k+n rather than just from time 0:k.", "limitations": "Yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 4, "significance": 3, "originality": 3, "strengths_and_weaknesses": "S1. The core idea of the paper is interesting and novel. While there has been work on improving interpretability of time series predictions through the application of IB-inspired masking this hasn’t been applied directly to improving the underlying learned representation to the best of my knowledge. Other methods such as TimeSieve [https://arxiv.org/pdf/2406.05036] have only recently started exploring how to improve the underlying representations in an efficient way. \n\nS2. Framework flexibility: the authors do show some performance gains on a range of model types and tasks. The design of the framework is such that it should be easy to extend as needed in many places. \n\nS3. Ablation studies were useful for understanding the impact of the ESP step combined with AML as well as how the method scales across tasks and model types when considering what % of training samples show improved performance after masking. These studies suggest the method is effective in its aims of reducing redundant feature learning.\n\nW1. It is clear that certain tasks and data types will benefit more from this method than others. It would be helpful to provide guidance on how to identify these cases where the potential performance gain may outway the efficiency costs of introducing a more heavyweight method like this one.", "was_revised": true, "final_justification": "The authors have made a great effort to further examine the reviews of all reviewers and provide additional experimentation where helpful to validate their results. I am convinced by the sensitivity analysis and comparisons with alternative methods that were further added to the current results and will keep my recommendation as is.", "rebuttal": "Dear Reviewer:\n\nWe sincerely appreciate your insightful and constructive comments on our manuscript, and we are especially grateful for your careful identification of typographical errors, which we have corrected in the revised manuscript. Your professional review has greatly helped us improve the quality of the paper. In response to your questions and suggestions, we have made thorough revisions and added clarifications; our detailed replies are as follows:\n\n**Q1:**  It seems like once masks are learned for a given task there could be some additional optimization to increase inference efficiency. Even if there was no performance gain, it would be helpful for storage and memory reasons to reduce the length of the sequence of any given time series problem. Is there any work the authors have explored to analyze what these efficiency wins may be under different task settings?\n\n**A1:** Thank you for raising this important question. We have indeed thought carefully about efficiency optimizations at inference time. Building on AML’s core idea—improving representation quality by masking out redundant information in the input sequence—we find that while the current random-masking strategy improves upon exhaustive search (with complexity $O(L)$), there is still room for further optimization. Given that redundancy in real-world scenarios often exhibits continuity (e.g., prolonged repeated patterns or stable fluctuations), the following strategies may help improve efficiency:\n\n1\\) Dynamic binary search strategy: This method assumes redundancy is contiguous and models the mask-length space $[0, L]$ as an ordered search space. Through iterative binary search, it can quickly locate the contiguous interval containing redundant information: in each round, evaluating the performance of masks at two midpoints halves the search space. This method applies only when redundant features are contiguous.\n\n2\\) Gradient-based adaptive sampling: This method leverages AML’s loss-feedback mechanism. By analyzing the trend of the loss across neighboring mask lengths, it dynamically adjusts the sampling direction. When a significant drop in loss is detected, it indicates that the current mask length is closer to the optimal de-redundancy region; the procedure automatically shifts in that direction, achieving a hill-climbing search. This method adapts to different data distributions and is also effective for non-contiguous redundancy.\n\nFuture work can further explore combining masking with retrieval to achieve more precise redundancy removal and more efficient representation learning. Regarding efficiency gains across different scenarios, our method performs well in most settings. However, when predictive signals are concentrated very early in the timeline, AML’s prefix-masking policy (starting at $0$) may, after multiple masking evaluations, yield a penalty coefficient $\\beta = \\max\\left(0, \\frac{\\ell - \\ell_s^{*}}{\\ell}\\right)$ that remains identically zero, adding extra spatiotemporal cost without improving performance. In addition, ESP shows relatively limited gains on high-dimensional datasets (e.g., Weather and Solar-Energy) because ESP is based on relative distances and explicitly aligns the geometry of the input embedding space with that of the output space. For datasets with many channels, a fixed set of relative distances leaves more degrees of freedom, making distance-metric optimization more challenging. For example, in 2D, knowing a point’s relative distances to two other points can determine its position, whereas in 3D one needs three points.\n\n**Q2：** In some previous works related to time series interpretability there has been analysis of applying a dynamic mask [https://arxiv.org/pdf/2106.05303] or sliding window [https://arxiv.org/pdf/2107.14317] to analyze how predictions change. Would further gains be possible if the masking operator was defined for some sequence from k to k+n rather than just from time 0:k.\n\n**A2:** Thank you very much for the suggestion. Your proposal to mask the interval $[k,k+n]$ is indeed more advantageous, and we agree it is preferable. However, due to computational constraints, our current choice of a prefix mask $[0,k]$ (i.e., $0{:}k$) is based on two considerations: (1) exhaustive search over $[k,k+n]$ leads to exponentially growing computational complexity, which is prohibitive with our current unoptimized retrieval procedure; (2) many time-series settings adopt a Markov assumption, under which later segments implicitly encode earlier information, so we temporarily mask the earlier segment. Exploring a joint masking–retrieval approach is a promising direction for future work—thank you for highlighting this important direction."}, {"review_id": "QR5Oz9Zsrx", "reviewer": "Reviewer_9riX", "summary": "The paper challenges the common assumption that longer input sequences always benefit time series forecasting. To this end, it introduces two techniques aimed at improving both accuracy and efficiency by filtering out irrelevant information: Adaptive Masking Loss (AML), which suppresses temporal prefixes that contribute little to predictions, and Embedding Similarity Penalty (ESP), which promotes semantic alignment between input embeddings and outputs to enhance generalization. The proposed methods are tested across several datasets (ETTh1, ETTh2, Solar-Energy, Weather) and architectures, including Transformers and MLP-based models. The authors provide ablation studies that highlight the contributions of AML and ESP individually and in combination. The results show consistent performance improvements and point to new directions for optimization in time series forecasting.", "questions": "Please address the weaknesses.", "limitations": "yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths\n* Both AML and ESP are conceptually simple and easy to grasp.\n* The methods consistently improve performance across various models and datasets, with solid experimental backing.\n\nWeaknesses\n* The core motivation questioning the value of longer input sequences may seem at odds with prior work, where longer sequences often help by providing additional useful context. While long inputs can certainly introduce noise, this is a well-studied challenge in time series forecasting, and recent studies (e.g., TimeMoE) have demonstrated clear performance gains with longer sequences. The authors could clarify their framing to better align with or differentiate from these established findings.\n\n* The connection to existing work on noise robustness and time series representation learning feels underdeveloped. The paper focuses mainly on applying the proposed methods to baseline models, but it would benefit from a clearer discussion of how it builds on or differs from related approaches.\n\n* The claim that AML’s stochastic sampling reduces computational cost isn’t substantiated with quantitative evidence. There’s no analysis of how varying the sample size affects training time, resource usage, or accuracy.\n\n* While the ablation results show that ESP complements AML, the paper doesn’t provide insight into why ESP is particularly helpful for certain datasets or tasks.\n\n* There’s little analysis of the kinds of patterns AML tends to mask, whether it targets noise, outliers, or outdated information is left unclear.\n\n* The argument that optimal mask positions mostly fall in prefix regions isn’t supported by quantitative or visual analyses (e.g., distributions of k, per dataset and model).\n\n* Some recent baseline models and datasets are missing or feel outdated (e.g., Traffic, ExchangeRate, Illness), limiting the scope of comparison.", "was_revised": true, "final_justification": "The authors have thoroughly addressed the concerns raised during the review and discussion process. In light of their clarifications and additional analyses, I have adjusted my score upward to reflect a more positive evaluation.", "rebuttal": "Dear Reviewer:\n\nThank you for your careful and in-depth review. From the perspectives of theoretical rigor, completeness of experimental design, and the connection between our methodological innovations and existing research, your suggestions are highly constructive and crucial to improving this paper. The core tensions you identified have all clarified the key directions for revision. We will address these issues one by one below.\n\n**A1：** We are deeply grateful for your insightful comments. Existing research—including strong work such as **TimeMoE**—indeed shows that longer sequences can improve model performance by providing richer contextual information. It may be that we did not convey this clearly; we fully agree with this view and have built upon it with further exploration.\n\nOur study focuses on the **selective utilization** of information within long sequences. While longer sequences do increase the total amount of information, our experiments indicate that some portions may be unhelpful for the prediction target or even introduce interference (which aligns with classic findings on noise in time-series forecasting). To address this, our masking mechanism aims to more effectively filter and leverage key subsequences, dynamically adjusting via the parameter $\\beta$ (defined as $\\beta = \\max\\left(0, \\frac{\\ell - \\ell_s^{*}}{\\ell}\\right)$). This allows us to increase the information density of the learned representation without degrading predictive performance.\n\nFollowing the evaluation protocol in Section 2, we applied an optimal masking analysis to a trained TimeMoE model (official GitHub code and hyperparameters) to diagnose redundancy. Due to time constraints, we evaluated the single-expert configuration on ETTh1 with input length L=1024 and forecast horizons of 96 and 192. As shown in Table following, while longer inputs provide richer context, they also introduce substantial redundant features（ratio denotes the proportion of samples in the dataset whose prediction MSE decreases after applying the  optimal mask）.\n\n| Prediction length | Dataset | Original loss | Masked loss |   Ratio  |\n|:-----------------:|:-------:|--------------:|------------:|---------:|\n|        96         | train   |         0.106 |       0.096 |  99.13%  |\n|        96         | test    |         0.522 |       0.463 |  98.24%  |\n|       192         | train   |         0.157 |       0.149 |  98.67%  |\n|       192         | test    |         0.722 |       0.674 | 100.00%  |\n\n**A2：** Thank you very much for the suggestion. Prior to our work, on **noise robustness**, Informer [1] observed that as input sequences grow longer, the computational complexity of traditional models increases, which can degrade forecasting performance; it addresses this via sparse attention for information distillation. For **representation learning**, TS2Vec [2] adopts contrastive learning to regularize temporal representations. However,most long-sequence forecasting studies emphasize architecture design, whereas our paper approaches the problem from the perspective of objective optimization.\n\nThe distinction between AML and prior noise-robustness research is that AML uses information-theoretic guidance to eliminate redundant information. The contribution of ESPto representation learning is to constrain input–output consistency. Due to space limitations in the main text, we will add these comparisons to the Related Work section of the camera-ready supplementary material.\n\n\n【1】Zhou H, Zhang S, Peng J, et al. Informer: Beyond efficient transformer for long sequence time-series forecasting[C]//Proceedings of the AAAI conference on artificial intelligence. 2021, 35(12): 11106-11115.\n\n【2】Yue Z, Wang Y, Duan J, et al. Ts2vec: Towards universal representation of time series[C]//Proceedings of the AAAI conference on artificial intelligence. 2022, 36(8): 8980-8987.\n\n**A3：** Thank you for pointing this out; we did not state it clearly in the original manuscript. The AML method uses a random-sampling strategy that effectively reduces computational cost, striking a balance between efficiency and performance. Compared with the high complexity of exhaustively evaluating all possible masks (time complexity $O(L,n)$), fixing the sample to $m$ masks reduces the complexity to $O(m,n)$. We analyzed how the sampling size affects training time, resource usage, and accuracy, and found that although the loss decreases as $m$ increases, the rate of decrease diminishes. We therefore set $m=12$ as a trade-off between computational cost and efficiency.\n\n Here, $m$ is the number of masks and $L$ is the input length; “MSE (mean)” and “MAE (mean)” are averaged over different output lengths.\n\n\n| Model        |  m/l | MSE (mean) | MAE (mean) |\n|:-------------|:----:|-----------:|-----------:|\n| SOFTS        |  1/8 |      0.407 |      0.416 |\n| SOFTS        |  1/6 |      0.407 |      0.409 |\n| SOFTS        |  1/4 |      0.389 |      0.393 |\n| SOFTS        |  1/3 |      0.384 |      0.388 |\n| SOFTS        |  1/2 |      0.381 |      0.385 |\n| SOFTS        |  3/4 |      0.380 |      0.384 |\n| iTransformer |  1/8 |      0.412 |      0.412 |\n| iTransformer |  1/6 |      0.411 |      0.417 |\n| iTransformer |  1/4 |      0.402 |      0.399 |\n| iTransformer |  1/3 |      0.398 |      0.395 |\n| iTransformer |  1/2 |      0.395 |      0.392 |\n| iTransformer |  3/4 |      0.394 |      0.390 |\n\n**A4：** Thank you for the insightful comments. We observe heterogeneous performance of ESP across datasets of different dimensionalities: ESP performs better on ETTh1, ETTh2, ETTm1, and ETTm2, while the gains are smaller on higher-dimensional datasets such as Weather (21 channels) and Solar-Energy (137 channels). The main reason is that ESP enhances robustness by explicitly aligning the geometric structure of the input embedding space with that of the output space, directly constraining pairwise relative distances between samples; as the number of channels (feature dimensionality) increases, the optimization directions grow exponentially with dimensionality, introducing greater uncertainty during training and potentially yielding smaller improvements than on lower-dimensional data. However, high-dimensional datasets typically contain more redundant information, on which AML performs well; combining ESP with AML therefore yields consistent and effective improvements across diverse datasets.\n\n\n**A5:** Thank you very much for the valuable comments. The issue you raised regarding the types of mask patterns in AML is indeed important and helps improve the interpretability of our method. AML dynamically adjusts the optimization strength via an adaptive weight $\\beta$, so mask variants that yield larger reductions in loss exert a stronger influence on the training objective, guiding the model to learn features with less redundancy. We fully understand the reviewer’s interest in clearly distinguishing the types of feature patterns being masked. Although the current masking strategy improves representation efficiency, it does have limitations in interpretability: while the adaptive-weight mechanism is effective, its decision process remains something of a black box, making it difficult to clearly identify the specific types of masked feature patterns (e.g., noise, outliers, or stale information). We take this concern seriously and consider it an important direction for future work.\n\n**A6:** Thank you for the valuable comments. We acknowledge that searching for the optimal mask over the interval $[K, K+n]$ may outperform restricting the mask position to the prefix region. However, under our current search procedure, retrieval over $[K, K+n]$ is prohibitively expensive. Our focus on the prefix region is motivated by the Markov chain assumption: many real-world time series (e.g., traffic flow, energy consumption) exhibit Markov properties, where future states depend primarily on recent history and the contribution of distant history decays with temporal distance. Accordingly, we posit that later segments already contain the information present in the prefix, making redundancy more prevalent in the prefix. Once we have improved the retrieval method, we plan to expand the search range to $[K, K+n]$.\n\n**A7：** We sincerely appreciate the reviewer’s valuable suggestions. The benchmark datasets you mentioned—Traffic, ExchangeRate, and Illness—are indeed important standards in time-series forecasting. In designing our experiments, we primarily considered two factors: (1) selecting public datasets consistent with those used in the original baseline papers to ensure comparability; and (2) adopting the officially released hyperparameter settings to ensure fairness in reproduction (especially given the sensitivity of time-series models to hyperparameters). Accordingly, we used, wherever possible, the intersection of datasets for which the baseline models provide official hyperparameters.\n\nRegarding the benchmarking issue you raised, we encountered several practical difficulties. For the Traffic dataset, the large data scale and the already high computational complexity of the baseline models made testing challenging under limited compute, especially since our method introduces additional spatiotemporal complexity. In addition, for the ExchangeRate and Illness datasets, some baselines do not publicly release hyperparameter settings, which complicates fair comparison. If you deem it necessary, we are willing to conduct supplemental experiments on these datasets; however, we note that, due to hyperparameter uncertainty, the results may differ from the baselines’ performance reported on other datasets.\n\nMeanwhile, we have been actively tracking the latest developments in the field; for example, SOFTS is a NeurIPS 2024 paper. If there are recent open-source models of interest, we would be happy to include them in our comparisons. Your professional advice will greatly help us improve this work."}]

Author rebuttal:

Meta-review: This paper presents a novel approach to time series forecasting, Adaptive Masking Loss with Representation Consistency (AMRC), which aims to mitigate redundant feature learning by selectively masking temporal segments and enforcing representation consistency.  Across several datasets and backbones, AMRC yields consistent accuracy gains; ablations indicate complementarity between AML and ESP. 

Strengths: This paper is based on an interesting empirical observation: masking/truncation can outperform full-context training. The proposed 
AMRC enjoys simplicity and flexibility: AML and ESP are conceptually simple, architecture-agnostic, and plug-and-play. Empirical results have indicated improvements across multiple datasets and model families. Code is provided.

Weaknesses:  There are several limitations that may reduce the overall impact. The conceptual connections to information theory are not always clear, and the framing around the limits of long input sequences seems at odds with prior work, requiring more careful positioning. The paper lacks discussion of efficiency trade-offs and robustness to hyperparameter and masking choices, as well as clarity on what types of features AML tends to suppress. Some baseline models and datasets are missing, and the empirical claims about efficiency are not substantiated. Reviewers also note issues with writing consistency and insufficiently developed comparisons to related work.

Rebuttals: The authors have provided thorough responses to the concerns. Reviewers are confident with the potential values of this technical contribution.

Overall, this work proposes a promising and novel idea with solid empirical results, but it requires stronger theoretical grounding, clearer positioning against existing literature, and deeper analysis of efficiency and masking behavior to reach the bar of a top-tier venue.
Decision: Accept (spotlight)