Feature-aware Modulation for Learning from Temporal Tabular Data
Authors: Haorun Cai
Han-Jia Ye
Venue: NeurIPS 2025 poster ()
Keywords: tabular deep learning
distribution shift
temporal shift
machine learning
Primary area: deep_learning

Abstract: While tabular machine learning has achieved remarkable success, temporal distribution shifts pose significant challenges in real-world deployment, as the relationships between features and labels continuously evolve. Static models assume fixed mappings to ensure generalization, whereas adaptive models may overfit to transient patterns, creating a dilemma between robustness and adaptability.
In this paper, we analyze key factors essential for constructing an effective dynamic mapping for temporal tabular data. We discover that evolving feature semantics—particularly objective and subjective meanings—introduce concept drift over time. Crucially, we identify that feature transformation strategies are able to mitigate discrepancies in feature representations across temporal stages.
Motivated by these insights, we propose a feature-aware temporal modulation mechanism that conditions feature representations on temporal context, modulating statistical properties such as scale and skewness. By aligning feature semantics across time, our approach achieves a lightweight yet powerful adaptation, effectively balancing generalizability and adaptability.
Benchmark evaluations validate the effectiveness of our method in handling temporal shifts in tabular data.

Feature-aware Modulation for Learning from Temporal Tabular Data
Hao-Run Cai
Han-Jia Ye
School of Artificial Intelligence, Nanjing University, China National Key Laboratory for Novel Software Technology, Nanjing University, China {caihr, yehj @lamda. edu. cn
Abstract
While tabular machine learning has achieved remarkable success, temporal distribu- tion shifts pose significant challenges in real-world deployment, as the relationships between features and labels continuously evolve. Static models assume fixed map- pings to ensure generalization, whereas adaptive models may overfit to transient patterns, creating a dilemma between robustness and adaptability. this paper, analyze key factors essential for constructing effective dynamic mapping for temporal tabular data. We discover that evolving feature semantiss-particulant particularly objective and subjective meanings-inttroduce introduce concept drift over time. Crucially, we identify that feature transformation strategies are able to mitigate discrepancies in feature representations across temporal stages. Motivated by these insights, we propose feature-aware temporal modulation mechanism that conditions feature representations temporal context, modulating statistical properties such scale and skewness. By aligning feature semantics across time, our approach achieves a lightweight yet powerful adaptation, effectively balancing generalizability and adaptability. Benchmark evaluations validate the effectiveness of our method in handling temporal shifts in tabular data.
Introduction
Tabular data, structured in rows and columns, forms the foundation of critical decision-making across diverse domains such as healthcare (39], finance 442 and e-commerce 33
Traditional machine and modern deep architectures 12 [3 16 52) commonly assume independent and identically distributed However, in real-world scenarios [54], temporal distribution shifts 16 41), characterized by evolving relationships between features and labels, present significant challenges. For instance, latent factors such economie fluctuations, policy changes, evolving user behaviors can render static models ineffective, as previously learned mappings quickly become obsolete.
learning models for tabular data data-including tree-based methods [5]
7
22
36
(i.i.d.) data 3 30
Most existing approaches assume i.i.d. data and primarily focus on improving generalization based on a fixed mapping from the feature space to the label space. Statie models, although easy to train and capable of reasonable generalization, fundamentally lack the capacity to adapt to evolving data patterns without explicit temporal awareness, as depicted in Figure) T For example, ensemble meth- ods [13] and classical gradient boosting decision trees (GBDTs) 7 22 36) exhibit robustness across diverse distributions [6 41], I but they lack mechanisms to capture evolving temporal dependencies.
To better understand learning under temporal shift and the limitations of existing approaches, we begin temporal tabular datasets. Beyond the classical notions of covariate shift, label shift, and concept shift refeeriefeeng respectively to changes in p(x), p(y), and ((y]z)) over time SI temporal shifts involve additional complexities, such as dynamic transformations within the feature space X' and the label space ) themselves [ These
by examining the nature of evolving distribution in
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Deratty
Hight High (EEN))
Besaaty
location
Statio Magpion
Sbiling
S44
Salory (rm))
02:22 Coundaatic (raw)
Denadte
2000
Dynaniic Mappirts
C Prine teveitaed in Ailggee
Figure 1: Left: Static models assume a inverinvariant mapping f for all temporal subset Di, while adaptive model fe dynamically adjust to each temporal stage. Right: Raw salary and location values (top) encode shifting subjective concepts like "high income" or "prime location," which vary across time. Our method aligns these semantics (bottom) by modulating feature distributions using temporal statistics (e.g., mean, std, skewness) to preserve concept consistency over time.
challenges necessitate learning an adaptive model with distributional extrapolation capability, enabling effective generalization to previously unseen distributions.
However, learning such adaptive models is inherently difficult. Limited samples specific time points hinder the accurate estimation of instantancous distributions. Moreover, naïve adaptive implementations often fail to leverage temporal context effectively, limiting their ability transfer knowledge from historical data to future deployment stages [14]. These limitations impede the development of models with temporal extrapolation capability, increasing the risk of overfitting to short-term patterns. This situation creates a dilemma: static models achieve strong generalization but ignore temporal dynamics, whereas adaptive models focus on immediate adjustments the cost of long-term stability. Therefore, establishing a principled balance between generalizability and adaptability is essential for effectively learning from temporally evolving tabular data.
In this paper, we aim to develop an adaptive model to address temporal shifts in tabular data. We first identify key factor in temporal generalization: the evolving semantics of features. Specifically, a feature's 's meaning can be interpreted either based on its absolute value (objective semanties) or in relation to the data distribution (subjective semantics). For example, a person's salary or house's ' coordinates carry objective semantics tied to fixed numerical values. In contrast, identifying someone as part of a "high-income group" or a location as a "prime area" depends on subjective semantics, which are context-dependee remain unchanged, their implied meaning may evolve. Conversely, values may change while the concept remains stable. For instance, average salaries may rise over time, but the threshold for being considered "high income" is defined relative to the distribution. Similarly, coordinates remain fixed, but the concept of "prime location" may evolve due to urban development, as illustrated in Figure ! (right top). These evolving semantics challenge static models, which fail to interpret such changes.
and can shift over time. Thus, even if raw feature values
Based on these insights, we propose a feature-aware temporal modulation mechanism that adjusts feature representations based on temporal context. We observe that the distributional statisties of features-specificttee specifically, their mean, std, and skewness play a critical role in shaping semantics, corresponding bias, scale, and distributional shape, as shown in Figure We introduce a learnable transformation that modulates features according to these statistics, allowing the model to align semantics across temporal stages (Figure [ right). This alignment helps maintain stable relational structure among features, which is essential for learning generalizable representations under temporal shifts. As a result, the model gains distributional extrapolation capability to mitigate concept drift, and temporal extrapolation capability by leveraging similarity across time to generalize to previously unseen or sparsely observed periods. Experiments demonstrate the effectiveness of our approach in handling temporal shifts across real-world datasets. The contributions are summarized as follows:
We analyze the challenge of temporal distribution shifts in tabular data and highlight the role of evolving feature semantics.
We propose a novel feature-aware temporal modulation mechanism that leverages distributional statisties (mean, std, skewness) to conditionally align feature representations over time. Our method achieves a balance between generalization and adaptability, enabling both distributional and temporal extrapolation with low cost.
2

---

2 Related Work
2.1 Tabular Machine Learning
Tabular data is a widely-used format across various real-world applications [1921]] including healthcare [39], finance [42]. and e-commerce [33] Classical tree-based methods, such Random Forest [5) XGBoost (7). LightGBM [22], and CatBoost [3), remain competitive due to their robust- ness, interpretability. and high performance in practice. In recent years, deep learning approackes for tabular data have gained significant attention. Notably, FT-Transformer (FT-T) [10] leverages the Transformer architecture to model feature interactions, while retrieval-based methods like TabR [12] and MCdernNCA (52) predict labels by retrieving neighbors in the learned representation space. TabM 13] introduces an ensemble strategy that integrates multiple MLPs, while other methods, such as SNN (24). DCNv2 [50], MLP-PLR [II], and RealMLP [18), enhance the MLP architecture itself. Additionally,
approaches like TabPFN and its variants [16] [26] (37 have shown remarkable versatility and generalization across various tabular tasks. Despite significant progress and strong results on established benchmarks
22 51 the challenge of deploying these
models effectively in dynamic, non-stationary environments is increasingly critical 6 19
41 54
2.2 Distribution Shift in Tabular Data
Existing methods for addressing distribution shift in tabular data can be broadly categorized into two groups: static methods, which aim to learn robust and generalizable representations across distributions, and adaptive methods, which adjust dynamically to distributional changes. In static methods, Ruhachev et al. [41] observe that TabM 13 performs well under temporal shift, while Cai & Ye [6] show that GBDTTs 7 22 36) remain competitive when using refined training protocols. Adaptive methods, on the other hand, focus primarily on domain shifts from a source domain to a
target domain [8. 23 25 43 45/ However, these methods typically require access to target domain data in advance and are often designed to handle static
shifts, which are ill- suited for capturing the intra-domain dynamics common in temporal distribution shifts. Cai & Ye (6) propose incorporating temporal embedding to capture trends and periodicities, introducing temporal adaptivity into the model to some extent. contrast, we identify key factors that contribute model adaptability and condition feature representations on temporal embeddings through a feature- aware modulation mechanism. Our approach facilitates semantic alignment across time, overcoming the limitations of previous methods.
2.3 Hypernetwork
Hypernetworks, which generate the weights for another neural network 14], have become an effective strategy for making model behavior adaptive. This concept has been further developed in recent years, with specialized architectures achieving impressive results tabular data 44 32). However, when applied to tabular data with temporal shifts, the conventional hypernetwork approach considered computationally prohibitive and data-inefficient [49]. Feature-wise Linear Modulation (FiLM) approach (35) proposes a lightweight alternative, using hypernetworks for feature modulation rather than generating complete weights. FiLM has shown promising results in visual reasoning. Inspired by this, we explore effective modulation mechanisms for temporal tabular data, introducing feature-aware temporal modulation approach that preserves the adaptive benefits of hypernetworks while ensuring computational efficiency for handling time-varying distributional shifts.
Preliminaries
3.1 Learning from Tabular Data with Temporal Shift
tabular dataset with n examples is generally represented as ((x,, y:) y: where X and y denote the feature space (e.g., Rd)) and the label space (e.g., classes or real values for classification or regression tasks), respectively. The goal is to learn mapping f X + y from the examples, where (xx;) = fi Rs y/s Typically, we assume that the pairs (x) are sampled i.i.d. from the joint distribution of (X', J²). and expect / to predict the label of an unseen instance xx sampled from the
same distribution. The prediction model f can be implemented using tree-based models 5 7 22 36
or deep neural networks [10 12
13
7 18 52).
3

---

In real-world applications, tabular datasets are often collected sequentially over time, meaning the underlying data distribution evolves over time. We define a temporal tabular dataset as = , D,- which represents a union of subsets 'D.- = {a Mi, [)] where denotes the number of instances collected time Each subset Dr is attached with timestamp t. The goal remains to learn a mapping/ f from U, Dr to predict the label for future instance x² collected at time t". However, since the i..d. assumption no longer holds in this case, temporal shift arises both within training datasets and between test instances [6].
Moreover, for two different timestamps + t', there may be changes in covariant distributions (i.e.. p(æ;) * p(ze) )) label distributions (i.e., p(B( * p(ye)) )), posterior distributions (i.e., p(y (z;) of p(ye- [xy]) (3) and even in the feature space (i.e., X' 7 Xy.) and label space (i.e., V' + Vv) ) [6].
These complexities in temporal distribution shifts lead to prediction deviations in the learned mapping f. making difficult to predict labels for future instances accurately.
3.2 Basic Solutions to Deal with Temporal Shift
We categorize methods addressing temporal shift in tabular data into two categories: static and adaptive methods. The static method learns a mapping ff that is agnostic to the time index t from D, often using empirical risk minimization 48)
min
Σ (D.D
me (yi, ds f(x:))
Here, ((.-)) measures the discrepancy between the target and the predicted labels, such cross- entropy for classification or mean squared error for regression, and wl; represents the instance-specific weight to emphasize important training examples. The static mapping f is trained to generalize across temporal shifts among training examples, with the expectation of being robust to future instances with varying shifts. Rubachev et al. [41] investigate the generalization ability of representative tabular models trained using this static approach.
An alternative solution is to learn sample reweighting scheme to align the training distribution with the distributions of future instances. Various reweighting strategies have been proposed in [25] 43
45 46). where training sample weights are derived from test instances available during training to minimize the distribution gap between source and target domains. However, these methods assume access to target domain data during training, and primarily designed to adapt static domain shifts, failing to account for the dynamic and continuous nature of temporal shifts.
In contrast to fixed models, the adaptive method conditions the model f on the timestamp, allowing the model to hehave differently based on the given time. For example, [40] 44] learn adaptive weights W; in Equation (1) solely based on the training set DD while still maintaining uniform behavior across test instances. Recently, Cai & Ye [6] proposed a temporal embedding approach, which concatenates temporal embedding (tt) to the model input, enabling the model to learn temporal patterns in an end-to-end manner and incorporating temporal adaptability. Specifically, (+t) maps the timestamp / to vector, making the mapping fæ,, become (()). thereby incorporating temporal information. While this approach has been validated, the limited encoding dimension of the temporal embedding restricts the model's 's ability extract fine mee-grained temporal features, and it is often affected by covariate and label shifts, resulting in fragmented feature concept alignment.
3.3 Challenges of the Adaptive Model for Temporal Shift
To address temporal shifts in tabular data, we aim learn an adaptive model f. from D. A straight- forward approach is to make the parameters of f dependent on the timestamp. Define an auxiliary mapping h that maps the subset D, to the parameters of the model f. and then use the generated weights for making predictions for a given input. We can express this process as ft h(D)). However, generating all parameters of f makes the output space of h large, making h computationally prohibitive, data-inefficient, and prone to overfitting 4 14 lightweight sxpernetwork-basee solution.
15 32
To address this, we focus on

---

Later:
Laket
Density
Original 0.1
Osiginal
Luboat
Dometty
Origital
Tizse
Eatret
A 15
Value
Value
Value
Figure 2: Top: Empirical feature distributions over time, with colors ranging from dark (early periods) bright (recent periods), exhibit clear non-stationarity in bias (left), scale (middle), and skewness (right). Bottom: Schematic illustration of learnable transformations applied to feature distributions: shifting the mean (9) aligns bias (left), adjusting standard deviation (7) alters scale (middle), and modulating asymmetry (A) reshapes skewness (right). These transformations enable semantic alignment across temporal stages, thereby strengthening both generalization and adaptability.
4 Aligning Feature Concept under Temporal Shift
Given the evolving nature of feature semantics under temporal shifts, a central question arises: how we align feature representations across time maintain semantic consistency? Traditional approaches treat features as fixed entities, assuming their concept remain stable throughout the dataset. However,
have observed, features with subjective semantics, those defined relative the current data distribution, can shift in meaning even when their raw values remain unchanged, as discussed in Section and Figure [ This phenomenon invalidates the i.i.d. assumption that underpins most standard learning algorithms and motivates reature-aware semantic modeling approach.
From theoretical standpoint, the challenge can be viewed through the lens of representation learning under distribution shift. Suppose we denote the representation function as o(æ; f), parameterized by 9. When the semantics of æ shift over time, e.g.. the notion of "high income" changes due inflation, the optimal mapping & should adapt such that semantically equivalent inputs under different D, are mapped to similar representations. In other words, seek semantic invariance:
δ(æ; 0e) p(x'; (vr), if ~ D., x' ~ Dr, and x, x' are semantically equivalent.
This highlights the need for temporal modeling at the feature level, where each feature æ; E æ may experience unique semantic shifts, and the model must dynamically adjusts its interpretation accordingly over temporal context.
Furthermore, the subjective nature of many real-world features suggests that distributional statistics, such as the mean, standard deviation, and skewnexs, can act as proxies for semantic context. Figure illustrates the temporal distributions of selected features from real-world datasets within the TabReD benchmark [41] Our analysis demonstrates that adjustments to the three aforementioned distributional statistics effectively characterize the majority of temporal distribution shifts observed. Inspired by this, we hypothesize that semantic alignment can be achieved by recalibrating features informed by their temporal distributional profile. Such approach enables the model to restore temporal invariances while embedding inductive biases conducive to extrapolation beyond observed periods.
In the following section, we operationalize this intuition by introducing feature-aware temporal modulation mechanism. This mechanism conditions feature representations on temporal information through the modulation of distributional statistics computed at each timestamp. By explicitly adapting these statistical summaries, it offers lightweight and interpretable way to align feature semantics over time, thereby improving the model's generalization under non-stationary conditions.
5

---

Temps Eanhs
In. Modiulator
Rep. Modialattr
Out, Misdilator
x
Linear
Norm
ReLU
Drop
Linete
8
V
Temporal Enibedding S
Temperal Madulator
Fourier
Lineax
>
y
Lincar
KelU
(t((t
B
Fourier
Norm
Linear
Figure 3: Overview of our feature-aware temporal modulation framework. Temporal modulation can be applied on raw feature input, intermediate representation, and output logits. The modulator conditions temporal context (t) to predict parameter y, 3, À for modulation based on Equation (
5 Feature-aware Temporal Modulation
To achieve semantic alignment across temporal stages, we introduce feature-aware temporal modulation mechanism that conditions feature transformations on learned representations of time. The core idea is to explicitly reparameterize each input feature according the evolving distributional context, thereby enabling the model to interpret feature values in a temporally consistent manner.
Let t R denote the timestamp associated with given instance, and (+) E R represent the temporal embedding [6]. which captures both short-term and long-term temporal dynamics. This embedding serves contextual signal, and is used input to lightweight modulator that outputs a set of modulation parameters for each feature dimension.
Concretely, for each feature R" and E R, the modulator produces three parameter vectors r(b(t)), (((()) e R", which correspond to scale, bias, and nonlinear transformation coefficient, respectively. We define the temporal modulation function as follows:
c = (+b(()) YJ(m;; (x((())) + (+)()),
where YJ(;;A); X;) denotes the Yeo-Johnson transformation
53 defined as:
YJ(z;AA) = ( ((2: + 1- (1= (1- 11)
if x; 2 0,
(2 A.) if x < 0.
This transformation allows the model to dynamically adjust the distribution of features by applying a smooth, nonlinear transformation that adapts both skewed and heavy-tailed distributions. The inclusion of the Yeo-Johnson transformation enables nonlinear reshaping of feature distributions, which is critical for adapting to the temporal shifts in feature semantics. By leveraging the temporal context (t),, this design provides a feature-wise, time-dependent translormation, where the same raw input æ; can be interpreted differently depending on the temporal context (t).. thus capturing the evolving semantics of each feature. Unlike linear modulation schemes (35]. our method explicitly accommodates the nonlinear evolution of feature semantics, facilitating semantic extrapolation over time. Our temporal modulation mechanism applied to an MLP is illustrated in Figure 3 The modulation can be flexibly applied to the raw feature, within the intermediate representations, or to the predicted logits, offering enhanced adaptability across different stages of the network.
As demonstrated in the pilot study in Section 6.3 and Figure the temporal adaptivity enabled by our modulation approach is grounded in unified feature semantics. After a single modulation applied to the raw input features, the model able learn stable and consistent decision boundary over the modulated representations. This explains why our method achieves superior performance without explicitly injecting temporal information into the model architecture. Moreover, while our modulation servers as an approach to handle concept drift, the covariant and label shifts inherently embedded in the temporal dimension-iie, the changes in p(æ) and p(y) over time-are naturally mitigated through the alignment of feature semantics. Once the model learns a temporally coherent representation space, it can extract generalizable knowledge across di differnnt temporal phases, rather than overfitting to the local variations at specific timestamps.

---

Methods
Hrt
E01
HD
SHY
CT4
DEL
MR.
WFJ
Rank
Classical Baselines
Linear
0.9388
0.5944
0.8231
0.2435
0.4864
0,5596
0.1680
1.7464
18,250
0:0000
+:00105
1:0,006
+0000007
1:00008
1:0,0008
:10.00014
XGBoost
0,9625
0.6199
0.8644
0.2262
0,4792
0,5520
0.1610
1.4700
6.375
000000
+0.0005
+0,0002
-0,000
0.0001
$:000000
+:0.9001
2000001
CatBoost
0.9639
0.6242
0.8620
0.2292
0,4792
0.5495
0.1610
1.4654
4.375
20000000
1000018
10:000
2000000
00000
1:0.000
250,0000
2:0.0011
LightGBM
0.9631
0.6164
0.8599
0.22260
0.4877
0,5500
0,1612
[14654
7.375
Statie Methods
-0
20.008
1.10000
20,0000
20.0002
1:00001
2:00000
20.0012
RandomForest
0,9580
0.6068
0.8171
0.2400
0,4841
0,5588
0.1647
1,5845
17.375
0.000
100000
1:0.000
0:000001
100000
1:00001
11000004
Deep Methods
MLP
0.9360
0.6220
0.5508
0.2641
0,4821
0,5515
0.1619
1.5362
16.000
20.0053
:000000
1:000015
=0.0093
20.000
40.0012
1:0,000
2000009
MLP-PLR
0.9596
0.6185
0.8166
0.2361
0.4799
0.5481
0,1616
1,5235
10.875
20,0004
1:00072
4:0.0044
2.000
2000003
10000T
1:0,000
2:0.0033
FT-T
0.9591
0,6159
0.5746
0.2438
0,4807
0.5503
0,1623
1,5146
15.125
20.0048
100022
1:000119
20.01
20.0008
1:00011
1:0,000
2:0.005)
TabR
0,9605
0.6148
0.8342
0.2370
0,4883
0,5550
0.1623
1.4732
13.750
+0000009
+-00095
+-00044
+00004
+000017
4:0.0040
+-0.0004
+.0000776
ModemNCA
0,9610
0.6341
0.8378
0.2325
0,4804
0,5520
0,1619
1,4857
9.125
-0.00009
+0,0000
1:00009
-0.00
200000
2:00007
+00000
2:000054
TabM
0.9640
0.6325
0,8290
0.2306
0,4813
0,5500
0,1612
1.4887
7.250
20.002
+500000
800000
2000000
4:00014
1:0,0044
200004
Deep Methods with Temporal Embedding 6
MLP
0,9471
0.6252
0,5519
0.2431
0,4801
0,5518
0.1621
1,5319
14.375
+0.00
+0.003
+-00013
=000152
+00000
+00.008
#:0,0002
=-0.0440
MLP-PLRR
0,9607
0.6110
0.8158
0.2338
0,4803
0.5494
0.1617
1,5133
11.250
20.0015
100053
1:0000
20.005
100004
1:00002
2:00066
Adaptive Methods
FT-T
0,9608
0.6211
0,5563
0.2363
0,4778
0,5503
0,1622
1,5118
11.125
20.0046
100111
+000259
20.0266
20.006
10.0018
1:0.0002
2000062
TabR
0.9606
0.6233
0.8426
0,2392
0.4827
0.5497
0.1627
1.4620
10.125
200.0038
1000057
1:0.1:00
20,011
2.00043
1.00004
1:0,000
2:0,0068
ModemNCA
0.9620
0.6356
0.8316
0.2255
0,4791
0,5535
0.1617
1.4903
7.625
0000000
10.0166
+500088
000004
200000
4-00019
2:0:0002
2-0.0055
TabM
0,9629
0.6271
0,8363
0.2321
0.4791
0,5488
0.1609
1,4812
5.125
+0000
#0,0092
+000126
#-0.0006
+000054
4000012
+-0,000
+000070
Deep Methods with Temporal Modulation (Ours)
MLP
0.9593
0,6230
0,5532
0.2345
0.4782
0.5502
0.1616
1,5179
11.000
20:0007
100004
1:00001
20.002
1000011
1:0000
2:000040
MLP-PLR
0.9591
0.6133
0.8190
0.2340
0.4780
0.5474
0.1616
1.5134
10.000
200000
10.0054
1.0.012
0.0027
200000
1000008
1.000002
220.0045
TabM
0.9641
0.6318
0.8457|
0.2225
0.4773 00000005
0,5491
0.1610 2503000
1.4794
3.500
200000
2500019
1:00015
20000115
2-00014
3-0.0040
Table 1: Main results on the TabReD benchmark [41]. The best performance is shown in bold, with the 2nd to 4th best results [boxed. Metrics reported are averaged AUCt and RMSE, over 15 runs, along with the corresponding standard deviation and the average rank across all datasets. The results demonstrate that our temporal modulation approach consistently improves performance, surpassing both static models and temporal embedding baselines (6. Our approach also enhances stability, it naturally decouples the temporal modality from the input modality.
The ablation study in Section 6.4jand Table 2 further confirms the effectiveness of applying modulation multiple levels of the MLP. Specifically, modulating the input features, intermediate representations, and predicted logits all contribute positively to performance. This highlights the complementary role of each modulation layer and suggests that multiple levels of temporal modulation are beneficial for capturing complex temporal dynamics.
Additionally, by sharing the temporal embedding across all instances and modulators, our method ensures both parameter efficiency and temporal coherence, enabling the model to generalize effec- tively across time while avoiding overfitting to specific timestamps. As we demonstrate later in our experiments, this modulation mechanism improves both predictive performance and robustness in the presence of temporal distribution shifts.
current limitation of full-stage modulation lies its incompatibility with PLR embedding [.] which is commonly integrated into state-of-the-an tabular models [IK 52 PLR embedding transform numerical features into combinations of sine and cosine, resulting in an expected arc- sine distribution rather than the semantically interpretable distributions discussed in Section D and
7

---

Figure 4: Pilot study on aligning feature semanties. The left plot illustrates the decision boundary learned by static MLP. highlighting that such methods struggle to capture separability under temporal shifts. The top panel visualizes the evolving decision boundaries learned by an MLP with our temporal modulation, where modulation is applied once at the input layer. Each of the five subplots corresponds to a different temporal segment, revealing how the model adapts its decision boundary in response to temporal dynamics. The middle panel displays feature distributions after modulation, which are better aligned across time. This alignment enables the model to form consistent decision boundary, as shown in the bottom panel. These results demonstrates that our lightweight modulation mechanism effectively aligns feature semantics, allowing the backbone network to operate within a unified conceptual space over time.
This mismatch can undermine the effectiveness our modulation strategy. Nevertheless, inspired by the ablation study, we find that applying the modulation once at the raw feature level still effective for models using PLR embedding, partially mitigating this limitation.
Section 4
Experiments
6.1
Setup
We conduct experiments on the TabReD benchmark proposed by Rubachev et al. [41]. using the refined training protocol introduced by Cai & Ye (6). Our preprocessing, training, evaluation, and hyperparameter tuning setup follows the practices established in Cai & Ye 66. Detailed experimental setup is provided in Section B
6.2 Results
Our main results on the TabReD benchmark [4] are summarized in Table 1 Among static models, we compare classical baselines including Linear, XGBoost (7). CatBoost (36]. LightGBM (22,, and Random Forest [5), as well as deep tabular models such as MLP, MLP-PLR M.] FT-Transformee
10. TabR 12, ModernNCA [52]. and TabM [3] While these methods lack temporal adaptation capabilities, missebllebbased models still demonstrate strong performance due to their robustness and generalization ability. For adaptive models, we follow the approach from Cai Ye (6) by concatenating temporal embedding to the input features of the six aforementioned deep tabular models. This serves as simple yet effective baseline. Most of these models benefit from this temporal input, underscoring the importance of dynamic modeling in the presence of temporal shifts.
Building on this, we apply our temporal modulation approach to MLP, MLP-PLR, and TabM. The MLP uses full modulation across all stages, while MLP-PLR and TabM employ single-step mod- ulation on raw features, as mentioned Section S Our method yields consistent performance improvements, significantly outperforming static baselines and surpassing the temporal embedding baselines across the board, as shown Figure Notably, although our method does not explicitly feed temporal information into the model, it still achieves robust gains, highlighting the effectiveness of our design. More specifically, we observe that even the simplest MLP achieves superior perfor- mance over most deep learning methods when equipped with full temporal modulation. TabM with our modulation achieves the highest average rank (3.50) among methods. To the best of our knowledge, this is the first instance where deep tabular method consistently outperforms GBDT-

---

Temporal Embedding
Temporal Modulation (Ours)
In.
Rep.
Out.
Imp. (Relative)
Rank
2. 7,0035
Percastagn Emp.
+2.0%%
2.0035
X
X
X
0.00%
7,000
1.50%
+1.33%
X
V
1.02%
74)
5.125
1.00%
V
0.26%
(12.6%)
5.250
+0.38%
0.00%5
+0.07%
J
V
1.19%
(56.8%)
5.500
X
X
1.83% (87.4%)
3.250
MLP
MLP-FLR
TabM
MIP
MLP-PLR
TabM
X
V
1.54%
(73.6%)
3.625
Figure 5: Improvement in performance with temporal modulation. The bar chart compares the percentage im- provement in performance across different models: MLP, MLP-PLR, and TabM. The left side shows results with temporal embeddings. while the right side demonstrates the improvement using our proposed temporal modulation method. Temporal modulation yields significant improve- ments, particularly for the MLP model, achieving 2.09% increase, while other models exhibit more moderate gains.
V
1.62% (77.2%)
3.750
2.09%
2.500
Table 2: Ablation study on the effect of modulation placement. While all modulators have positive impact on performance improvement, input-level modulation contributes the most to per- formance improvement.
based models under temporal distribution shifts. In summary, these results collectively demonstrate the strong effectiveness and generalizability of our proposed temporal modulation approach.
6.3
Pilot Study
The results of our pilot study, presented in Figure 4 are designed to evaluate whether our proposed modulation effectively aligns model representations. We use an MLP as the backbone and apply single modulation on raw features to clearly separate and visualize the model's learned representations. To this end, construct synthetic dataset with temporal distribution shift. Under the i.i.d. assumption, the dataset is non-separable, causing static models to fail in learning a valid decision boundary, shown in Figure (left). After applying our temporal modulation, the model is able to adaptively adjust its decision boundaries at each temporal stage, as illustrated in Figure (right top).
We further visualize the feature distributions before and after modulation (Figure 4 right middle), and observe that the temporal shift in the input space is effectively corrected. While the post- modulation feature distributions are still non-i.i.d., they become aligned enough to enable consistent and separable decision boundaries, as shown in Figure (right bottom). These results demonstrate that the modulated model learns in unified representation space and capture temporally generalizable knowledge, thus validating our motivation of aligning feature semantics through temporal modulation.
6.4 Ablation Study
The results of our ablation study, presented in Table 2 investigate the relationship between model performance and the location of temporal modulation. We conduct this analysis using an MLP back- bone, evaluating all combinations of the three modulation positions: raw feature input, intermediate representation, and output logits. As expected, applying modulation at all three stages yields the best performance on the TabReD benchmark, with an average improvement over the baseline MLP of 2.09%. Notably, all combinations outperform the baseline MLP without modulation, indicating that each modulation layer contributes positively temporal generalization. These findings validate the design choice of multi-level modulation and demonstrate the complementary benefits of applying temporal adaptation throughout the network.
Beyond overall performance, we also observe several interesting findings. Applying a single mod- ulation layer only the raw feature level achieves 87.4% of the performance gain obtained by the full modulation setup. In contrast, removing the modulation at the raw feature layer reduces the performance gain to just 56.8% of the full modulation. This highlights the critical importance of early-stage modulation. At the raw input level, the model has access to the most complete and unaltered information, making it the most effective stage for semantic alignment. If modulation is omitted at this stage, subsequent modulations are less effective, as the model may have already inter- nalized misaligned representations. Although any modulation applied at deeper layers of a sufficiently expressive network can approximate the optimal transformation based on the universal approximation property of neural networks theory [34], our method achieves competitive performance with only a single modulation at the input level. This efficiency is particularly encouraging, it enables seamless

---

integration of our modulation mechanism into existing models by simply adding a modulation layer the imput, without requiring changes to the internal structure of the backbone.
7 Conclusion
In this paper, we address temporal distribution shifts in tabular data by identifying evolving feature semantics as a core challenge. We show that conventional static and adaptive models face a trade-off between generalization and adaptability. To bridge this gap, propose feature-aware temporal modulation mechanism that conditions feature representations on temporal context through learnable transformations of distributional statistics. This enables the model to align semantics across time, facilitating both distributional and temporal extrapolation. This lightweight yet effective strategy enables stable learning under temporal shift and improves generalization future data. Extensive experiments validate the robustness and adaptability of our method in temporal tabuar learning.
Acknowledgments and Disclosure of Funding
This work is partially supported by Key Program of Jiangsu Science Foundation (BK20243012) CCF-ALIMAMA TECH Kangaroo Fund (NO.2024005), Collaborative Innovation Center of Novel Software Technology and Industrialization. We thank Si- Si-Yang Liu, Yu-Sheng Li, Huai-Hong Yin, Jun-Peng Jiang, and Jin-Hui Wu for their insightful discussions. We also appreciate the reviewers for their thoughtful comments.
References
[1] Akiba, T., Sano, S., Yanase, T., Ohta, T., and Koyama, M. Optuna: next-generation hyperparameter optimization framework. In KDD, pp. 2623-2631. ACM, 2019.
[2] Arik, S. Ô. and Pfister, T. Tabnet: Attentive interpretable tabular learning. In AAAI, pp. 6679-6687, 2021.
[3] Bishop, C. M. Pattern recognition and machine learning, 5th Edition. Springer, 2007.
[4] Bonet, D., Montserrat, D. M., Giró-i-Nieto, X., and loannidis, A. G. Hyperfast: Instant classification for tabular data. In AAAI, pp. 11114-11123, 2024.
[5] Breiman, Random forests. Mach. Learn., 45:5-32, 2001.
[6] Cai, H.-R. and Ye, H.-J. Understanding the limits of deep tabular methods with temporal shift. In ICML, pp. 6366-6386, 2025.
[7] Chen, T. and Guestrin, C. Xghoost: A scalable tree boosting system. In KDD, pp. 785-794, 2016.
[8] Ganin, Y., Ustinova, E., Ajakan, H., Germain, P., Larochelle, H., Laviolette, F., Marchand, M.. and Lempitsky, V. S. Domain-adversarial training of neural networks. J. Mach. Learn, Res., 17: 59:1-59:35, 2016.
[9] Gardner, J., Popovic, 7.,, and Schmidt, L. Benchmarking distribution shift in tabular data with tableshift. In NeurlPS, pp. 53385-53432, 2023.
[10] Gorishniy, Y., Rubachev, I., Khrulkov, V., and Babenko, A. Revisiting deep learning models for tabular data. In NeurIPS, pp. 18932- 18943, 2021.
[11] Gorishniy, Y., Rubachev, I., and Babenko, A. On embeddings for numerical features in tabular deep learning. In NewrlPS, pp. 24991-25004, 2022.
[12] Gorishniy, Y., Rubachev, I., Kartashev, N., Shlenskii. D.,, Kotelnikov, A., and Babenko, A. Tabr: Tabular deep learning meets nearest neighbors. In ICLR, 2024.
[13] Gorishniy, Y., Kotelnikov, A., and Babenko, A. Tabm: Advancing tabular deep learning with parameter-efffcient ensembling. In ICLR, 2025.
10

---

[14] Ha, D., Dai, A. M., and Le, Q. V. Hypemetworks. In ICLR. 2017.
[15] Helli, K., Schnurr, D., Hollmann, N., Müller, S., and Hutter, F. Drift-resilient tabpfn: In-context learning temporal distribution shifts on tabular data. In NeurIPS, pp. 98742-98781, 2024.
[16] Hollmann, N., Müller, S., Eggensperger. K., and Hutter, F. Tabpín: transformer that solves small tabular classification problems in second. In ICLR, 2023.
[17] Hollmann, N., Müller, S., Purucker, L.,, Krishnakumar, A., Körfer, M., Hoo, S. B., Schirrmeister, R. T., and Hutter, F. Accurate predictions on small data with tabular foundation model. Nature, 637:319-3266 2025.
[18] Holzmüller, D., Grinsztajn, L.,, and Steinwart, I. Better by default: Strong pre-tuned mips and boosted trees on tabular data. In NeurlPS, pp. 26577-26658, 2024.
[19] Jiang, J.-P., Liu, S.-Y., Cai, H.-R., Zhou, Q. and Ye, H.-J. Representation learning for tabular data: A comprehensive survey. CoRR, abs/2504.16109, 2025.
[20] Jiang, J.-P. Xia, Y., Sun, H.-L., Lu, S., Chen, Q.-.,, Luo, W., Zhang, K., Zhan, D.-C., and Ye, H.-
J. Multimodal tabular reasoning with privileged structured information. CoRR, abs/2502.05564, 2025.
[21] Jiang. J.-P., Zhou, T., Zhan, D.-C., and Ye, H.-J. Compositional condition question answering in tabular understanding. In ICML, pp. 27831-27850, 2025.
[22] Ke, G., Meng, Q., Finley, T., Wang, T., Chen, W., Ma, W., Ye, Q., and Liu, T.-Y. Lightghm: A highly efficient gradient boosting decision tree. In NIPS, pp. 3146-3154, 2017.
[23] Kim, C., Kim, T., Woo, S., Yang, J. Y., and Yang, E. Adaptable: Test-time adaptation for tabular data via shift-aware uncertainty calibrator and label distribution handler. CoRR, abs/2407.10784, 2024.
[24] Klambauer, G., Unterthiner, T., Mayr, A., and Hochreiter, S. Self Self-normalizing neural networks. In NIPS, pp. 971-980, 2017.
[25] Lipton, Z. C., Wang, Y., and Smola, A. Detecting and correcting for label shift with black box predictors. In ICML, pp. 3128-3136 2018.
[26] Liu, S.-Y. and Ye, H.-J. Tabpfn unleashed: A scalable and effective solution to tabular classifi- cation problems. In ICMI,, pp. 40043-40068, 2025.
[27] Liu, S.-Y., Cai, H.-R., Zhou, Q.-L., and Ye, H.-J. TALENT: A tabular analytics and learning toolbox. CoRR, abs/2407.04055 2024.
[28] Loshchilov, I. and Hutter, F. Decoupled weight decay regularization. In /CLR, 2019.
[29] McElfresh, C., Khandagale, S., Valverde, J., C., V. P., Ramakrishnan, G., Goldblum, M., and White, C. When do neural nets outperform boosted trees on tabular data? In NeurlPS, pp- 76336-76369, 2023.
[30] Mohri, M., Rostamizadeh, A., and Talwalkar, A. Foundations of Machine Learning. MIT Press, 2012.
[31] Moreno- Moreno-Torres. J. G., Raeder, T., Alaíz Rodríguez, R., Chawla, N. V., and Herrera, F. A unifying view on dataset shift in classification. Pattern Recognit, 45:521-530, 2012.
[32] Mueller, A. C., Curino, C. A., and Ramakrishnan, R. Mothernet: Fast training and inference via hyper-network transformers. In ICLR, 2025.
[33] Nederstigt, L. J., Aanen, S., Vandic, D., and Frasincar, F. FLOPPIES: A framework for large-scale ontology population of product information from tabular data in e-commerce stores. Decis. Support Syst., 59:2969911111 311, 2014.
[34] Park, S., Yun, C., Lee, J., and Shin, J. Minimum width for universal approximation. In ICLR, 2021.

---

[35] Perez, E., Strub, F.. de Vries, H., Dumoulin, V., and Courville, A. C. Film: Visual reasoning with general conditioning layer. In AAAL, pp. 3942-3951, 2018.
[36] Prokhorenkova, L. O., Gusev, G., Vorobev, A., Dorogush, A. V., and Gulin, A. Catboost: unbiased boosting with categorical features. In Neur/PS, pp. 6639-6644 2018.
[37] Qu, J., Holzmüller, D., Varoquaux, G., and Morvan, M. L. Tabicl: A tabular foundation model for in-context learning on large data. In ICML, pp. 50817-50847, 2025.
[38] Quioneno-Candela, J., Sugiyama, M., Schwaighofer, A., and Lawrence, N. D. Dataset Shift in Machine Learning. MIT Press, 2009.
[39] Rajkomar, Oren, E., Chen. K., Dai, M., Hajaj. N.. Hardt, M., Liu, P. J., Liu. X., Marcus,
J., Sun, M.. Sundberg, P.. Yee, H., Zhang, K., Zhang, Y., Flores, G., Duggan, G. E., Irvine, J.. Le, Q.. Litsch, K., Mossin, A., Tansuwan, J., Wang, D., Wexler, J., Wilson, J., Ludwig, D.,
Volchenboum, S. L.,, Chou, K., Pearson, M., Madabushi, S., Shah, N. H., Butte, A. J., Howell, M. D., Cui, C., Corrado, G. S., and Dean, Scalable and accurate deep learning with electronic health records. npj Digit. Medicine, 2018.
[40] Ren, M., Zeng, W., Yang. B., and Urtasun, R. Learning reweight examples for robust deep learning. In ICML, pp. 4331-4340, 2018.
[41] Rubachev, I., Kartashev, N., Gorishniy, Y., and Babenko, A. Tabred: A benchmark of tabular machine learning in-the-wild. In ICLR, 2025.
[42] Sattarov, Schreyer, M., and Borth, D. Findiff: Diffusion models for financial tabular data generation. In ICAIF, pp. 64-72 ACM, 2023.
[43] Shimodaira, H. Improving predictive inference under covariate shift by weighting the log- likelihood function. Journal of Statistical Planning and Inference, 90:227-244, 2000.
[44] Shu, J., Xie, Q.. Yi, L., Zhao, Q., Zhou, S., Xu, Z., and Meng. D. Meta-weight- Learning an explicit mapping for sample weighting. In NeurlPS, pp. 1917-1928, 2019.
[45] Storkey, A. J. and Sugiyama, M. Mixture regression for covariate shift. In NIPS, pp. 1337-1344, 2006.
[46] Sugiyama, M., Krauledat, M., and Müller, K. Covariate shift adaptation by importance weighted cross validation. Mach. Learn. Res., 1005, 2007.
[47] Sun, and Saenko, K. Deep CORAL: correlation alignment for deep domain adaptation. In ECCV Workshops (3), pp. 443-450, 2016.
[48] Vapnik, V. Principles of risk minimization for learning theory. In NIPS. pp. 831-838, 1991.
[49] von Oswald, J., Henning, C., Sacramento, J., and Grewe, B. Continual learning with hypernetworks. In ICLR, 2020.
[50] Wang, R., Shivanna, R., Cheng, D. Z., Jain, S., Lin, D., Hong, L., and Chi, E. H. DCN V2: improved deep & cross network and practical lessons for web-scale learning rank systems. In WWW. pp. 2021.
[51] Ye, H.-J,, Liu, S.-Y., Cai, H.-R., Zhou, Q.-I,, and Zhan, D.-C. A closer look at deep learning methods on tabular datasets. CoRR, abs/2407.00l 00956, 2024.
[52] Ye, H.-J., Yin, H.-H., Zhan, D.-C., and Chao, W.-L. Revisiting nearest neighbor for tabular data: A deep tabular baseline two decades later. In ICLR, 2025.
[53] Yeo, I.-K. and Johnson, R. A. A new family of power transformations improve normality symmetry. Biometrika, 87:954-959, 2000.
[54] Zhou, Z.HH. Open environment machine learning, National Science Review, 9:nwac122 123, 2022.
12

---

A Code and Data Availability
We release the complete implementation of our method at the following repository: https:/
Modulation
We the TabReD 41 benchmark to evaluate the performance of our models. Furthermore, adopt the refined training protocol and the data preprocessing procedures proposed by Cai & Ye [6].
B Detailed Experimental Setup
Our experiments are run under Linux using Python 3.10 and PyTorch 2.0.1. We adopt the training,
evaluation, and hyperparameter tuning setups from Cai & Ye [6, Ye et al. [31], and Liu et al. [27]. Hyperparameter optimization is performed using Optuna [I], with 100 trials for most methods.
Due to computational constraints, FT- FT-Transformer and TabR are tuned with 25 trials. The search
space strictly follows the configurations used Cai & Ye (6 and Rubachev al. 41 and is also documented in our source code (available in the config/ folder).
identified, each method is trained using 15 random seeds, and we report the average performance across these runs, For all deep learning methods, we use a batch size of 1024 and the AdamW optimizer [28] Classification tasks are evaluated using AUC (higher is better), while regression tasks are evaluated using RMSE (lower is better). Model selection is based on the best performance on the validation set. Following Rubachev et al. [4]] and Cai & Ye (6), we adopt an early stopping strategy with patience of 16 epochs based on validation performance.
Once optimal hyperparameters
Regarding temporal embedding, Table 3 compares the hyperparameter search spaces used in Cai & Ye [6] and in our work. While Cai & Ye [6] conducted separate hyperparameter searches for each periodic component and the trend, adopt unified design with fixed embedding dimension of 128 across all periodic priors while retaining the trend component, resulting in a single hyperparameter Lembedding to be tuned. Unlike prior work that relies on delicate hyperparameter balancing, our formulation explicitly disentangles the input modality from the temporal modality. This separation mitigates mutual interference and alleviates the scaling issue commonly observed in temporal embeddings- -weere increasing dimensionality often leads to degraded performance. By contrast, our design maintains stable improvements as d_embedding grows, demonstrating more robust and effective modeling of temporal patterns. We further provide an ablation study in Table 4 to validate the scalability of d_ embedding.
Parameter
Distribution
Parameter
Distribution
year_ _oder
{0, PowerInt
year_ _oder
Fixed 128
nonth_order
{0, Powerlnt
month_ order
Fixed 128
day_order
(0. PowerInt
day_
Fixed 128
hour_ordes
00, PowerInt [1
hour_order
Fixed 128
trend d_embedding
True, Falsc}
trend
Fixed Truc
{0, PowerInt(11,55 [1,
d_ d_embedding
{0, PowerInt 11]}
Table Left: Temporal embedding hyper- pyper-parammttr search space in Cai & Ye (
Right: Hyper-
parameter search space in this work. Here, PowerInt(a, (a, b) denotes the set of integer powers of two in the range [2 24] - e.g., PowerInt(1, = {2, 8, 16,32).
C Additional Experimental Results
In this section, we include additional analyses and validations to complement the results presented in the main paper. Specifically, we provide: (1) an ablation study on the temporal embedding dimension, (2) statistical significance tests, and (3) additional pilot visualizations. Furthermore, we report the complete numerical results with standard deviations, and verify the robustness our conclusions under alternative training protocols.
We conduct an ablation study on the temporal embedding dimension (Table which shows that con- ventional temporal embeddings deteriorate with increasing dimensionality, whereas our modulation
13

---

Methods
dam's
HIf
E01
HDf
SH
CTV
DEI
MRJ
WEL
Imp.
MLP (Statie)
0.9360
0.6220
0.5508
0.2641
0.4821
0.5515
0.1619
1.5362
8
0.9394
0.6245
0.5515
0.2558
0.4800
0.5605
0.1621
1.5529
+0.21%
32
0,9400
0.6194
0.5514
0.2497
0.4805
0.5607
0.1620
1.5448
+0.45%
MLP (+Embedding)
128
0,9442
0.6200
0.5529
0.24599
0.4799
0.5605
0.1622
1.5422
+0.76%
512
0,9491
0.6260
0.5536
0.2584
0.4806
0.5591
0.1618
1.5439
+0.40%
0,9439
0.6249
0,5423
0.2471
0.4786
0.5622
0.1618
1.5342
+0.65%
32
0.9439
0.6255
0.5400
0.2507
0.4785
0.5579
0.1615
1.5317
+0.58%
MLP (+Modulatim))
128
0.9468
0.6242
0.5547
0.2399
0.4785
0.5624
0.1617
1.5314
+1.33%
512
0.9563
0.6213
0.5558
0.2415
0.4779
0.5589
0.1616
1.5276
+1.48%
Table 4: Ablation on the embedding dimension demb- The results may be suboptimal due to the restricted hyperparameter search space. Performance of conventional temporal embeddings (6] tends to deteriorate at higher dimensions, whereas our modulation approach maintains consistent gains, demonstrating scalability and a stronger capacity capture fineggrained temporal dependencies.
approach exhibits stable and scalable performance. The statistical significance analysis in Table further confirms that our method significantly outperforms static approaches, while prior temporal embedding methods fail to achieve comparable improvements.
Additional pilot visualizations in Figure analyze the behavior of our method under different types of distribution shifts. While temporal shilts are primarily concept shifts and our approach is explicitly designed handle such cases, we observe that, the presence of sample noise, the proposed modulation further aligns feature distributions across various shift types, enabling the backbone network to learn more stable and robust decision boundaries.
ML.P
(Static)
(4Emhedding)
(+ (-Medulatiaa)
TabM
(Strii))
(+Fmholdinn)
(+Medolatioo)
(Stnii)
1,0000
0.4237
0.0033
(Sttic)
1,0000
0.7336
0.0631
(+Eimbedding)
0.4237
1,0000
0.1122
(+Immbeddingg)
0.7336
1.0000
0.2909
(+Modulation)
0.0033
0.1122
1.0000
(-Modulation)
0.0631
0.2909
1.0000
Table 5: Statistical significance analysis using Nemenyi post-hoc tests for MLP and TabM variants. The proposed modulation mechanism significantly outperforms the static baselines, whereas simply concatenating temporal embeddings [6] does not yield significant gains, demonstrating the effective- ness of our approach.
Given the absence of a standardized training protocol for temporal tabular data, we further conduct a comparative analysis of model performance under different training setups. Table reports the results of each method when trained and evaluated randomly split training and validation sets, instead of the temporal training protocol proposed by Cai & Ye (6]. In addition, Table presents the results obtained under the original protocol proposed by Rubachev et al. [41]. These results show that our temporal modulation approach achieves generally competitive performance across different training protocols, highlighting its robustness to various data splitting strategies. Furthermore, we argue that the effects of temporal adaptivity and validation splitting strategies are largely orthogonal: while temporal adaptation techniques enable distributional and temporal extrapolation, the choice of validation splitting mainly determines the foundation of effective model learning.
Table reports the numerical results from the ablation study discussed in the main text. We observe that the best performance on each task is consistently achieved when input-level modulation is applied, highlighting its importance for capturing temporal dynamics effectively.
14

---

(a) No shift
Tea d
Taae Ske
Now Siee
Tre Sloee
Covariant shift
(c) Label shift
Figure 6: Additional pilot study under different types of shifts. While temporal shifts in tabular data predominantly manifest as concept shifts (i.e., changes in p(y[x)), we further analyze our method under other types of shifts, including covariate shift (i.e., changes in p(x)) and label shift (i.e., changes in p(y)), as well as no-shift scenario. Each subfigure follows the same visualization scheme as Figure[4 illustrating the evolving decision boundaries and feature alignments over time in our method compared to static baseline. The results show that our temporal modulation, though primarily designed to mitigate concept shift, also promotes temporal alignment of feature distributions under covariate and label shifts, leading to more stable decision boundaries. Moreover, under the no-shift setting, it introduces no adverse effect, confirming that the modulation mechanism remains neutral when temporal shift is present.
15

---

In.
Rep.
Out.
Hr
EO1
HDt
SHY
CT4
DR.
MR.
WFJ.
Imp.
Rank
X
X
0,9360
0.6220
0.5508
0.2641
0.4821
0.5515
0.1619
1.5362
0.00%
7.000
X
V
0,9468
0.6126
0.5524
0.2460
0.4820
0.5512
0.1619
1.5176
1.02%
5.125
0,9400
0.6221
0.5533
0.2640
0.4814
0,5514
0.1620
1.5206
0.26%
5,250
V
0,9463
0.6185
0,5525
0.2439
0.4819
0,5508
0.1621
1,5211
1.19%
5,500
X
X
0,9565
0.6241
0,5519
0.2388
0.4779
0.5495
0.1623
1,5162
1.83%
3.250
X
V
0,9574
0.6242
0.5559
0.2392
0.4781
0.5600
0.1620
1.5355
1.54%
3.625
V
X 0.9567
0.6272
0.5515
0.2418
0.4782
0.5502
0.1615
1.5365
1.62%
3.750
V
0.9593
0.6230
0.5532
0.2345
0.4782
0.5502
0.1616
1.5179
2.09% %
2.500
Table 6: Detailed results for ablation study on the TabReD benchmark 41
as an extension to
Table 2
The best performance shown in bold, with the 2nd best results underlined.
Methods
HIt
E01
HD
SH4
CT
DEL
MR.
WE
Rank
Classical Baselines
Linear
0,9397
0.5895
0.8235
0.2458
0,4867
0,5591
0.1685
1.7425
15.125
20.00009
2000000
2000018
20.0072
20.000
1:000004
1:0.0073
2:0.0020
XGBoost
0.9625
0.6200
0.8452
0.2298
0,4806
0.5468
0.1611
1.4566
5..250]
200.000
1000004
1:00000
310.00b
CatBoost
0.9639
0.6213
0,8580
0.2340
0,4805
0.5471
0.1613
I.4556
5.375
220,0008
10.0025
1000022
200011
200000
120000
11:000001
200.0022
LightGBM
0,9616
0.6136
0.8334
0.2322
0,4807
0.5469
0.1616
1.4471
7,500
Statie Methods
+00000
+00005
4-0.001
+0000015
200000
4-0,0004
RandomForest
0.9580
0.6254
0.8142
0.2427
0.4846
0.5588
0.1649
1.5694
13.250
0,0000
1000029
1:0.0177
0.00002
+00,0001
4:0.0003
150,0000
:30000044
Deep Methods
MLP
0,9383
0.6225
0.5532
0.2509
0.4814
0.5521
0.1619
1.5252
13.750
21000042
100.031
1:0.0111
1:001199
:10.0004
1:0.006
1:0,0001
110.0059
MLP-PLR
0,9599
0.6225
0.8208
0.2406
0,4800
0.5507
0.1616
1,5097
9,875
20.0014
100.055
1.0.002
0.0013
2.0000
1:0.0010
1:0.0002
2:0.0163
FT-T
0,9616
0.6268
0,5846
0.2369
0,4804
0,5503
0.1622
1,5001
9.375
20.0000
500095
+00099
0.002
0000007
4:0.014
#50.0002
=60.0064
TabR
0,9543
0.6206
0.8147
0.2384
0.4880
0.5548
0,1622
1.46299
12.250
20.0052
10.005
1:00166
0068
20.0037
100003
1:00000
2:00006
ModemNCAA
0,9617
0.6246
0.8399
0.2299
0,4806
0,5510
0.1621
1.4773
7.750
200000
100010
201007
-0.000
20,000
1000015
150.0004
20.0101
TabM
0.9629
0.6332
0.8282
0.2305
0.4794
0,5495
0.1607
1,4681
4.000
1000030
1:0.0128
0,000
2000007
1000013
1000000
2:0.0047
Deep Methods with Temporal Embedding 6
MLP
0.9451
0.6267
0.5530
0.2610
0.4796
0.5569
0.1618
1.5333
12.375
Adaptive Methods
20000022
::0.0025
1:0.0009
20.01
220.0002
1000008
1:0.0001
2:0.0076
MLP-PLR
0,9600
0.6252
0.8210
0.2368
0.4798
0.5528
0.1612
1.5048
8.6255
200000
0007
2.00009
=: 20.004
20,0000
1000017
1:0.000
2000043
TabM
0.9636
0.6299
0.8427
0.2317
0.4784
0.5492
0.1609
1,4891
3.750
10000
200.014
1:00009
0.0.00
Deep Methods with Temporal Modulation (Ours)
MLP
0.9567
0.6267
0,5562
0.2499
0.4786
0.5594
0.1618
1.5122
11.500
20.00044
2000022
1000011
100000
2:
1:000012
150.0001
200.0052
MLPP-PLR
0.9607
0.6237
0.8104
0.2365
0.4792
0.5564
0.1613
1.5314
9.750
2010
1:0.002
0,007
1:00000
1:00000
2:0.007
TabM
0.9633
0.6319 2011110
0.8380
0.2306
0.4785
0.5491
0.1608
1.4717
3.500
1:0.0000
20.004
200000
2200012
220.0002
20000774
Table 7: Main results on the TabReD benchmark 41 using random split, results are averaged over three distinct partitions, with 15 seeds per partition. The best performance is shown in bold, with the 2nd to 4th best results boxed]. Metrics reported are averaged AUC and RMSE,, along with the corresponding standard deviation and the average rank across all datasets. We follow the random splits used in Cai & Ye (6)
16

---

Methods
HIT
E01
HD1
SH4
CT
DEL
MR.
WEJ
Rank
Classical Baselines
Linear
0.9388 0,000
0.5731
0.8174
0.2560
0,4879
0,5587
0.1744
1.7465
14.750
1000058
1:0.008
+:0011
+00.0004
4:0.0008
1:0,0107
-1000003
XGBoost
0.9609
0,5764
0.8627
0,2475
0,4823
0,5459
0,1616
1.4699
5.625
20.00
50.000
1:00005
20,0004
2000001
2000001
+:0,0000
2:0,000
CatBoost
0,9612
0.5671
0.8588
0.2469
0,4824
0,5464
0.1619
1.4715
7.000
000000
1000100
1:0000
000001
00000
2002
+:000000
200.0TTG
LightGBM
0,9600
0,5633
0,8580
0,2452
0,4826
0,5474
0,1618
1.4723
7.375
Static Methods
20,0002
0000
1:00004
2000001
0000
1:00000
200.001
RandomForest
0.9537
0,5755
0.7971
0.2623
0,4870
0.5565
0.1653
1.5839
14.000
20:0001
20,000
1:00008
20,0006
20.0001
100000
20,0000
20000004
Deep Methods
MLP
0,9404
0.5866
0,4730
0.2802
0,4820
0,5526
0,1624
1.5331
12.125
20.002
20.003
2:00006
-000281
0.000
2000012
1:0,000
2000005
MLP-PLR
0,9592
0,5816
0.8448
0.2412
0,4811
0,5533
0.1616
1,5185
7.250
=0.0000
$0.0085
+.00008
00.004
0.0000
2:00004
100.000
=:0.0046
FT-T
0.9562
0,5791
0.5301
0.2600
0,4814
0,5534
0.1627
1.5155
11.125
20.0092
1000061
1:00078
000144
200000
1:000028
1:0,0008
2:0.0058
TabR
0,9527
0,5727
0.8442
0,2676
0.4818
0.5557
0,1625
1.4782
11.625
2.00002
4.0.0665
1.00011
0000100
000000
1:00006
1:0.0005
ModemNNA
0,9571
0,5712
0.8487
0.2526
0,4817
0.5523
0.1631
1.4977
10.000
000000
1:0.0037
1:0.018
1:00018
2.0,0052
TabM
0,9590
0,5952
0.8549
0.2465
0.4799
0,5522
0.1610
1,4852
4.625
20.0018
2500057
+000024
-000002
0000077
2000014
#000001
3:000046
Deep Methods with Temporal Embedding 6
MLP
0.9399
0.5877
0.4740
0.2795
0,4815
0,5589
0.1625
1,5363
13.000
Adaptive Methods
0:0.01121
+-0.0000
+0000201
+00,0002
4-0.0112
4:0,0001
:50.0001
MLP-PLR
0,9593
0,5807
0.8472
0.2356
0.4796
0,5548
0.1617
1,5137
6.750
0,0005
+-0.0035
+-0.0025
0.0000
+-0.0027
4-0,0002
::00.0442
TabM
0.9616
0.5951 2500045
0.8548
0.2469
0.4795
0.5528
0.1615
1.4788
4.250
2000016
150.004
20.012
2000005
1:0.003
200,000
2:00003
Deep Methods with Temporal Modulation (Ours)
MLP
0.9538
0,5832
0.4753
0.2658
0.4811
0.5587
0.1623
1.51557
11.250
220,0005
1:00030
1:0.0010
0.015
210,0003
1:00014
1:0.0001
:200.0022
MLP-PLR
0,9576
0,5824
0,8434
0.2353
0,4805
0,5605
0,1616
1,5188
8.625
+0.0010
#0.0102
4-00029
4-0.00099
0.0000
4-0.0016
+-0.0001
=-0.0054
TabM
0.9634
0,5978
0.8557
0,24155
0.4796
0.5533
0.1615
1.4813
3.625
-00000004
4-0.001
+00.0006
+-0.0125
Table 8: Main results on the TabReD benchmark [1] using original temporal splits. The best performance is shown in bold, with the 2nd to 4th best results boxed]. Metries reported are averaged AUC1 and RMSFJ over 15 runs, along with the corresponding standard deviation and the average rank across all datasets.
D Computational Resources
All deep learning methods were trained on 20 NVIDIA RTX 4090 (24 GB) GPUs, Classical machine learning methods were executed on 4 Intel Xeon Platinum 8352S CPUs. All experiments were run in parallel, and the total wall-clock time for all reported experiments was approximately 14 days.
E Societal Impact Discussion
This work studies temporal distribution shifts in tabular data and proposes methods for improving model robustness and adaptability over time. As tabular data is widely used in high-stakes domains such as healthcare, finance, and public policy, understanding and addressing temporal shifts is critical for ensuring long-term model reliability and fairness.
Our contributions are primarily methodological and empirical. We do not use sensitive personal data, and the benchmark datasets employed in our experiments are publicly available and anonymized. While improved temporal adaptation techniques may enhance performance in real-world systems, we believe that our work supports the development of more trustworthy machine learning systems by providing tools and insights for coping with real-world temporal dynamics. We encourage future work to further investigate responsible deployment practices.
17

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: Please refer to Section and Section 6
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results be expected to generalize other settings.
is fine include aspirational goals as motivation as long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: Please refer to Section 5
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification. asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low images are taken in low lighting. Or speech-to-text system might be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach address problems of privacy and faimess.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
18

---

Justification:
This paper does not include theoretical results.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
Justification: Please refer Section 6.1 and supplementary material.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make clear how to reproduce that algorithm. (b) the contribution is primarily a new model architecture, the paper should describe the architccture clearly and fully.
(c) If the contribution is new model (e.g., a large language model), then there should either way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reprodueibility. the case of closed-source models, may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
19

---

Answer: [Yes]
Justification:
Please refer to supplementary material.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.cm
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https
/ ccpppp.ccppulli//uuuii wwwiiicgguie///eeeeeeeeic CoodeSubeiiieeee ssi wswonnolli onnolicy)))) for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: Please refer to supplementary material. Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification: Please refer to supplementary material.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, at least for the experiments that support main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
should be clear whether the error bar is the standard deviation the standard error of the mean.
20

---

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of enrors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: Please refer supplementary material.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g.. preliminary or failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https: //te:::////uis..eeeee http://s...c/uuictctttcccusuieea
Answer: [Yes]
Justification:
The research conducted in the paper conform.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., if there is a special consid- eration due to laws or regulations their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: Please refer supplementary material.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that improvement in the quality of generative models could be used to
21

---

generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Question:
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Answer: [NA]
Justification:
The research has such risk.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: The creators original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g., website). the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in package should be provided. For popular datasets, wwpprswi www.swtththc....c com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that re- re-packaged, both the original license and the license of the derived asset (if has changed) should be provided. If this information is not available online, the authors are encouraged to reach out to the asset's 's creators.
13. New assets
22

---

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification:
No new assets have been introduced in the paper.
Guidelines:
The answer NA means that the paper does release new assets.
Researchers should communicate the details the dataset/code/mooel as part their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember anonymize your assets (if applicable). You either create an anonymized URL or include an anonymized zip file.
Crowdsourcing and research with human subjects
For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Question:
Answer: [NA]
Justification:
No crowdsourcing and research in this paper.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: The research has such risk.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, formatting purposes and does not impact core methodology, scientific rigorousness, originality of the research, declaration is not required.
23

---

Answer:
[NA]
Justification:
The core method development in this research does not involve LLMs.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, or non non-standard components.
Please refer to our LLM policy fattps://neurre cc CCCCon/errencee/2225//I for what should or should not be described.
24


Reviews:
[{"review_id": "MOtvHkJYsT", "reviewer": "Reviewer_dcxr", "summary": "The paper aims to tackle the problem of temporal distribution shift in tabular data. The authors assert that distribution shifts can arise because the semantics of individual features (e.g., high income) can change as the global feature distribution drifts. To mitigate this problem, the authors propose Feature-Aware Temporal Modulation: a lightweight hypernetwork that, given a temporal embedding, generates the parameters of a lightweight linear and nonlinear transform that we can insert into multiple parts of a neural network to mitigate temporal drift. Experiments on the TabReD benchmark show that the proposed method achieves the best average rank among the experimented baselines.", "questions": "Please refer to the Weaknesses section.", "limitations": "yes", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "**Strengths**\n\n- The work tackles temporal distribution shift in tabular data, which is an important but under-recognized problem.\n- The modulator is very lightweight - it has a small parameter count.\n\n**Weaknesses**\n\n- The reported improvements on TabReD are small (1–2 % points). To further assess the method’s distinctive strengths, I believe that the authors could construct and experiment on more controlled synthetic datasets that can show the strengths of the proposed method. For example (these are examples, not definitive nor exhaustive):\n    - Experiment on synthetic covariate-shift, label-shift, and concept-shift scenarios.\n    - Observe how latent representations drift over time under each shift scenario.\n    - Compare the latent space of the proposed method compared to supplying vanilla temporal embeddings to the baseline model.\n- The paper does not clarify the boundary between settings where the proposed method helps and where just supplying a temporal embedding to the model and “letting the network figure it out” could be better:\n    - Distribution shape change (e.g., what if a feature that is initially unimodal becomes bimodal)\n    - High dimensionality of input feature space\n    - Categoricals or ordinal features", "was_revised": true, "final_justification": "During the review and discussion period, my raised issues and current thoughts are:\n\n**[Point 1]** The empirical gains are very small (1~2%). Against the competing baseline(temporal embedding[1]), the performance is very neck-and-neck, with the competing baseline outperforming the proposed method on multiple datasets. This is even more so when considering the variance of the experiments.\n\nSince the gains on real-world datasets are marginal, I have requested measurements against a synthetic dataset, which resulted in relatively more gains than real datasets. However, the gains are attributed to other types of shifts, while the main focus of writing in the manuscript was concept shift. To this end, I think that integrating this result would be challenging without a major revision of the manuscript.\n\n\n**[Point 2]** To the best of my understanding, the paper tries to implicitly claim the contribution credit for raising the importance of the temporal distribution shift problem in tabular data. However, this contribution is attributed to a prior work[1] and not this manuscript. Although the authors do cite [1], the writing and visual analyses of the manuscript attenuate or avoid the presence of the prior work. For instance, the paper does not clarify the boundary between settings where the proposed method helps and where just supplying a temporal embedding[1] to the model and “letting the network figure it out” could be better. \n\nTo this end, I have requested active comparisons between the proposed method and [1] - e.g., to compare the evolution of decision boundaries over time (Figure 4 of the manuscript) so that we can observe whether [1] fails to capture the evolution while the proposed method succeeds. However, the authors say that *\"Similar to our method, temporal embedding allows the model to change its decision boundary over time.\"* which entails that [1] can also capture the evolution of the decision boundary. Moreover, incorporating this result into the manuscript would require a major revision of the narrative and visualizations.\n\n\nConsidering the points above, I wish to keep my current rating while lowering the originality score.\n\n[1] Cai, H.-R. and Ye, H.-J. Understanding the limits of deep tabular methods with temporal shift. CoRR, abs/2502.20260, 2025.", "rebuttal": "Thank you for your detailed and constructive review! We will address your concerns in the following responses.\n\n> The reported improvements on TabReD are small (1–2%). To further assess the method’s distinctive strengths, I believe that the authors could construct and experiment on more controlled synthetic datasets that can show the strengths of the proposed method.\n\nWe are very grateful for your valuable suggestions and have supplemented some further analysis:\n\n> > Experiment on synthetic covariate-shift, label-shift, and concept-shift scenarios.\n\nThe **Fig. 4** in paper has already demonstrated the concept shift. We further constructed the above three different shifts based on a same dataset and performed numerical comparisons.\n\n| Shift | MLP        | MLP-Temporal | MLP-Modulated |\n| -------------- | ---------- | ------------ | ------------- |\n| None           | **0.9940** | 0.9758       | 0.9928        |\n| Concept        | 0.8850     | 0.9833       | **0.9878**    |\n| Label          | 0.9216     | 0.9176       | **0.9455**    |\n| Covariant      | 0.9045     | 0.9039       | **0.9146**    |\n\nThe above results show that both temporal embedding and temporal modulation **focus on addressing concept shift**, which is consistent with our expectations and the focus of the paper. However, for covariate shift and label shift (which we obtained by sampling different time slices of the dataset), we found that **simply adding temporal embeddings resulted in performance degradation, while our temporal modulation improved performance**. We believe this may be because the model's response to covariate shift and label shift primarily depends on its inductive bias, while temporal modulation directly acts on the decision boundary and always attempts to **correct any potential inductive bias**.\n\nThis also explains why our method is better than simply adding temporal embedding: the temporal shift in real data is often a complex superposition of the three types of shift, and our method achieves better performance on all three shifts.\n\nThese results are encouraging and we will discuss them further in the revision. In addition, we will also include **visualization results** in the revision.\n\n> > Observe how latent representations drift over time under each shift scenario.\n\nIf we analyze the overall distribution of the latent representation, we believe this may be similar to the conclusion of the MMD heatmap in [1], that is, the model representation $\\phi(x)$ is nearly **stable when no temporal information is used**, but the target mapping $f$ is constantly changing. Therefore, we should pay more attention to the changes in the mapping and how the modulation is corrected, rather than how the model representation itself changes. If we try to analyze it on feature level, since each dimension of the model representation is often a fusion of multiple features, it is actually difficult to know whether the type of shift in the concept of each feature is as shown in **Fig. 1**, or whether there is no shift at all. If you have better ideas or suggestions, please feel free to raise them!\n\n> > Compare the latent space of the proposed method compared to supplying vanilla temporal embeddings to the baseline model.\n\nThis is also an interesting research direction, and we will include visualizations in the revision if possible. In our understanding, the fundamental difference between temporal modulation and simply adding temporal embedding is that **temporal modulation isolates temporal information from the modality of the conditional input, rather than conflating it with the original input modality**. This prevents the model input from being **affected** by overly complex temporal embedding and enables more **refined** temporal feature extraction. (For example, when simply adding temporal embedding, the embedding dimension tuned by Optuna often does not exceed 100, as the model's input feature dimension is also only ~100. However, in our temporal modulation, although we set an upper limit of 1024 for the embedding dimension, many experiments still reach this limit.) In contrast to latent representation, we believe that simply adding temporal embedding can lead to a serious **curse of dimensionality**, while temporal modulation **fundamentally avoids this problem**.\n\nBesides, we believe that a 1-2% performance improvement is not small. Without incorporating any temporal information, the performance gap between the most basic MLP model and the strongest tested TabM model is approximately 2.7%. All deep methods except TabM and MNCA have a performance gap of less than 2% compared to MLP. This makes the dynamic MLP with our temporal modulation outperform all static deep methods except TabM and MNCA under temporal shifts. This can also be seen in **Tab. 1**.\n\n> The paper does not clarify the boundary between settings where the proposed method helps and where just supplying a temporal embedding could be better.\n\nWe acknowledge that this is an important question, one that the paper does not explore further. This may require more detailed experiments. Here we only list a few possible observations:\n\n1. For **ordered but discrete numerical features** (e.g., int-type numerical features), temporal modulation does not necessarily yield better results than temporal embedding. This may be because the knowledge learned by the model is already discrete, and directly adding modulation to it makes it difficult to optimize, resulting in worse results than adding temporal embedding to the extra dimension.\n2. Compared to **categorical features**, we did not observe the results in 1. This may be because we use one-hot encoding for categorical features, allowing our modulation to apply independently to each category. This provides greater flexibility.\n3. For **features with strong non-normality** (e.g., long-tailed distributions), our temporal modulation yields greater improvements. This may be because our modulation includes a correction for skewness, which allows the model to learn better.\n4. We rarely observed distribution changes such as \"unimodal to bimodal\" (in other words, we generally found corresponding relationships with different peaks, although the density differences may be large), but we believe this may be a difficult issue. Our time modulation may only be able to achieve this correction by transforming all distributions into more normal distributions.\n5. **Input dimensionality is indeed a key issue**. As mentioned in the previous question, our time modulation fundamentally addresses the curse of dimensionality that can result from input temporal embedding. **You may be interested in the response to reviewer kbVu**, where we analyze the performance of temporal embedding and our temporal modulation **when different time dimensions are fixed**. The results show that our method achieves both performance and scalability. This means that temporal modulation will perform better in scenarios with more complex temporal features.\n\nWe hope this response addresses your concern. Thank you again for your constructive feedback, which will help us improve our work! Please feel free to raise any further questions or suggestions!\n\n---\n\n[1] Cai, H.-R. and Ye, H.-J. Understanding the limits of deep tabular methods with temporal shift. CoRR, abs/2502.20260, 2025."}, {"review_id": "1kFzylgHif", "reviewer": "Reviewer_Gi1z", "summary": "The paper tackles a problem of temporal shift in tabular data. It first identifies that evolving feature semantics (the difference between relative and absolute feature values) is tied to temporal shift in tabular data (via visualization of feature distributions across time, hypothesis and examples). Authors then propose a dynamic adaptation method which is a feature/neuron/logit modulation based on timestamp -- which helps normalize features and model internals across timestamps. The authors show significant gains on a real world tabular benchmark TabReD.", "questions": "-   Could you provide results for the default TabReD splits, and discuss those together with results on random splits? It is interesting how the proposed technique interacts with different validation and splitting strategies (for example, why are there gains in random splits too, where there is presumably less shift).\n-   What encourages the modulator behaviour illustrated in Figure 4? A deeper investigation in mechanisms behind this effect would be very insightful.", "limitations": "Limitations are adequately discussed", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 4, "significance": 3, "originality": 4, "strengths_and_weaknesses": "**Strengths**:\n\n-   The authors find a compelling explanation on how temporal shift in tabular data happens and use it to address it in a clear and simple way via feature modulation.\n-   The empirical results seem strong (modulo notes discussed in the weaknesses section below).\n-   The writing is clear, engaging and easy to understand.\n\n**Weaknesses**:\n\n-   The authors use a validation setup from the follow-up paper, which may be less standard than the one proposed in the original benchmark. In addition to including results on random splits, it would be great if authors provide results on the original splits from the TabReD benchmark.\n-   Writing is not precise enough on how the modulator is trained &#x2013; I assume that it's just trained as ususal, this leads to the next question (*most important in this review for me* - addressing it decently would lead to score increase with high probability)\n-   What precisely encourages such \"normalizing\" behaviour (well illustrated on synthetic data in Figure 4). Is it the training procedure plus the hold-out validation on a shifted subset? Is it some kind-of optimization dynamic specific to the modulator architecture? A deeper investigation beyond simple high level ablation would be very meaningful and make paper much stronger.", "was_revised": true, "final_justification": "I've read all the other reviews and author responses. I belive the submission clearly meets the bar for acceptance. \n\nI also want to address the concern of reviewer `dcxr` regarding small 1-2% gains on real-world datasets – those gains are actually substantial when it comes to this particular benchmark, furthermore seing *more or less* consistent gains over strong baselines such as MLP-PLR or TabM is also an encouraging signal regarding the methods applicability.", "rebuttal": "We appreciate your thoughtful comments! We will address your concerns in the following responses.\n\n> In addition to including results on random splits, it would be great if authors provide results on the original splits from the TabReD benchmark.\n\nWe further show the model performance before and after adding our temporal modulation to the **original temporal split in TabReD** [1].\n\n| Method            | HI         | EO         | HD         | SH         | CT         | DE         | MR         | WE         |\n| ----------------- | ---------- | ---------- | ---------- | ---------- | ---------- | ---------- | ---------- | ---------- |\n| Linear            | 0.9388     | 0.5731     | 0.8174     | 0.2560     | 0.4879     | 0.5587     | 0.1744     | 1.7465     |\n| XGBoost           | 0.9609     | 0.5764     | **0.8627** | 0.2475     | 0.4823     | **0.5459** | 0.1616     | **1.4699** |\n| CatBoost          | 0.9612     | 0.5671     | 0.8588     | 0.2469     | 0.4824     | 0.5464     | 0.1619     | 1.4715     |\n| LightGBM          | 0.9600     | 0.5633     | 0.8580     | 0.2452     | 0.4826     | 0.5474     | 0.1618     | 1.4723     |\n| RandomForest      | 0.9537     | 0.5755     | 0.7971     | 0.2623     | 0.4870     | 0.5565     | 0.1653     | 1.5839     |\n| MLP               | 0.9404     | 0.5866     | 0.4730     | 0.2802     | 0.4820     | 0.5526     | 0.1624     | 1.5331     |\n| MLP-PLR           | 0.9592     | 0.5816     | 0.8448     | 0.2412     | 0.4811     | 0.5533     | 0.1616     | 1.5185     |\n| FT-T              | 0.9562     | 0.5791     | 0.5301     | 0.2600     | 0.4814     | 0.5534     | 0.1627     | 1.5155     |\n| TabR              | 0.9527     | 0.5727     | 0.8442     | 0.2676     | 0.4818     | 0.5557     | 0.1625     | 1.4782     |\n| ModernNCA         | 0.9571     | 0.5712     | 0.8487     | 0.2526     | 0.4817     | 0.5523     | 0.1631     | 1.4977     |\n| TabM              | 0.9590     | 0.5952     | 0.8549     | 0.2465     | 0.4799     | 0.5522     | **0.1610** | 1.4852     |\n| MLP-Modulated     | 0.9538     | 0.5832     | 0.4753     | 0.2966     | 0.4811     | 0.5587     | 0.1623     | 1.5157     |\n| MLP-PLR-Modulated | 0.9576     | 0.5824     | 0.8434     | **0.2353** | 0.4805     | 0.5605     | 0.1616     | 1.5188     |\n| TabM-Modulated    | **0.9634** | **0.5978** | $--$       | 0.2415     | **0.4796** | 0.5533     | 0.1614     | 1.4813     |\n\nThe results are **consistent** with those in paper, and our method still achieves strong performance on the original temporal split of TabReD. This shows the **robustness** of our method in different validation strategies.\n\n> It is interesting how the proposed technique interacts with different validation and splitting strategies.\n\nIn our understanding, the key to resolving temporal shift lies in achieving **distributional and temporal extrapolation capabilities** (**L37-L47**). The difference is that the training protocol in [2] modifies the validation split to allow the model's performance to be better extrapolated to test time (so does the random split), which is the foundation for model learning. Our temporal modulation (or directly concatenate temporal embedding) finds a solution to the extrapolation capability within a given split. We believe that **the effects of these two approaches are orthogonal**, affecting the final model performance in different ways. This also explains why our method has similar effects on all validation splits.\n\n> Writing is not precise enough on how the modulator is trained – I assume that it's just trained as usual.\n\nYes, the modulator is trained **end-to-end along with the model backbone**.\n\n> What precisely encourages such \"normalizing\" behaviour?\n\nComparing this phenomenon with normalization is quite instructive. Although we haven't discussed this before, we believe that our modulation method can **indeed be considered a special form of normalization**, but a learnable and more general form because it supports not only scale changes but also skew changes. From this perspective, we believe that this is not the result of hold-out validation (as just mentioned, these two approaches are orthogonal in our understanding), but is more similar to the **RevIN method [3] in time series prediction**: in time series prediction, by learning a scaling and offset coefficient for each subsequence, the distribution shift of the time series can be effectively addressed. **Corresponding to our method**, we first use the temporal embedding to guide the model \"which samples belong to the same subsequence and should be scaled together\" in temporal tabular data, and then learn the coefficients of each \"subsequence\" in an end-to-end form, thereby learning a **transformation for each specific temporal stage**.\n\nWe hope this response addresses your concern. We're open to further discussion and would appreciate any additional thoughts you may have!\n\n---\n\n[1] Rubachev, I., Kartashev, N., Gorishniy, Y., and Babenko, A. Tabred: A benchmark of tabular machine learning in-the-wild. In ICLR, 2025.\n\n[2] Cai, H.-R. and Ye, H.-J. Understanding the limits of deep tabular methods with temporal shift. CoRR, abs/2502.20260, 2025.\n\n[3] Kim, T., Kim, J., Tae, Y., Park, C., Choi, J.-H., and Choo, J. Reversible Instance Normalization for Accurate Time-Series Forecasting against Distribution Shift. In ICLR, 2022."}, {"review_id": "ZG3HnmikGE", "reviewer": "Reviewer_kbVu", "summary": "The paper addresses the problem of temporal distribution shifts in tabular data by proposing a \"feature-aware temporal modulation\" method. This approach leverages a temporal embedding to dynamically adjust statistical properties (mean, standard deviation, skewness) of features, aiming to maintain semantic consistency over time. Empirical evaluations on the TabReD benchmark demonstrate improvements over both static and basic adaptive approaches.", "questions": "* How sensitive is the model’s performance to changes in the temporal embedding dimension? Could you elaborate on how you selected the embedding dimension used in your experiments?\n\n* The focus seems to be limited on one type of distribution shifts. How does the approach perform under sudden, irregular, or non-smooth temporal shifts compared to gradual shifts? \n\n* What is the standard deviation over different repetitions of the model? It would be very interesting to see", "limitations": "* Evaluation on 8 datasets is very limited, especially as the rank was used as an aggregation statistic. Without per-dataset significance tests, no statistical claims about methods being superior to others are possible.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strengths:\nThe paper effectively identifies and illustrates the challenge of evolving semantics in temporal tabular data, which is a unique and novel perspective. Particularly, the objective vs. subjective semantics perspective is inspiring.\n\n* Data-driven and problem-oriented analysis with insightful examples clearly illustrating the nature of distribution shifts and their impact.\n\n* The introduction of feature-aware modulation seems like a conceptually innovative and intuitive idea for tabular data. (Although similar approaches already exist for time-series)\n\n* Convincing evaluation on a well-designed benchmark (after the improvements of Cai et al.) with realistic prediction tasks\n\n\nWeaknesses: \n* The hyperparameter for the temporal embedding dimension is briefly mentioned but not thoroughly analyzed. An additional ablation study would strengthen the claims.\n\n* Computational Efficiency: Although presented as a lightweight method, there is insufficient quantitative analysis regarding computational overhead. \n\n* Limited discussion of scales of shift: Doesn’t yet characterize how severe temporal shifts must be before static models fail, leaving some uncertainty about practical thresholds.\n\n* Some referenced adaptive methods (e.g., domain–adversarial, reweighting) are only briefly mentioned without analysis of their failure modes on temporal tasks. \n\n* Limited analysis of the statistical significance of the approach. The authors use the mean rank over datasets to establish the superior performance of the proposed method. However,  on the datasets where the proposed approach ranks first, the difference to the next method always seems very small (Table 1). Given the missing analysis of significance per dataset, I consider it very likely that the proposed method is not truly better than the state-of-the-art. However, since the authors never claim that, I consider that a minor weakness. Even if it doesn’t improve performance, the approach is conceptually interesting and a good baseline for future work.", "was_revised": true, "final_justification": "The paper presents a novel and well-motivated approach to handling temporal distribution shifts in tabular data via feature-aware temporal modulation. My initial concerns included the sensitivity to temporal embedding dimensionality, lack of significance analysis, and limited discussion of failure modes. The rebuttal addressed most of these points with new ablation studies, significance tests (Nemenyi), and clarifications on design choices and computational overhead. While the evaluation remains somewhat limited in dataset diversity and shift types, the conceptual contribution is valuable and well-supported by the provided evidence. I therefore maintain my borderline accept recommendation, as the strengths outweigh the remaining limitations.", "rebuttal": "We are grateful for your constructive suggestions! We will address your concerns in the following responses.\n\n> The hyperparameter for the temporal embedding dimension is briefly mentioned but not thoroughly analyzed. An additional ablation study would strengthen the claims.\n\nWe further demonstrate the model performance achieved by **tuning parameters after fixing different temporal embedding dimensions**. Due to the limitation of hyperparameter space, all results are lower than those in Tab. 1.\n\n| Dimension | MLP-Temporal | MLP-Modulated |\n| --------- | ------------ | ------------- |\n| 8         | +0.20%       | **+0.65%**        |\n| 32        | +0.60%       | **+0.68%**        |\n| 128       | +0.92%       | **+1.47%**        |\n| 512       | +0.40%       | **+1.47%**        |\n\nWe find that our **temporal modulation consistently outperforms temporal embedding**. Furthermore, there is an **inflection point** between the dimension and performance of temporal embedding, but not our temporal modulation. This demonstrates that our approach combines strong performance with scalability. By separating the conditional input (temporal) modality from the original feature modality, we reduce the impact of temporal input on the model backbone while enabling more refined temporal feature extraction, as also declared in response to reviewer dcxr.\n\n> How sensitive is the model’s performance to changes in the temporal embedding dimension? Could you elaborate on how you selected the embedding dimension used in your experiments?\n\nThe temporal embedding dimension is indeed a very sensitive hyperparameter, as demonstrated in the experiments above. In our experiments, all hyperparameters were tuned using Optuna. The hyperparameter space for the temporal encoding dimension is int: [0, 8, 16, 32, 64, 128, 256, 512, 1024].\n\n> Doesn’t yet characterize how severe temporal shifts must be before static models fail, leaving some uncertainty about practical thresholds.\n\nThis is a critical issue and difficult to quantify. However, in the design of temporal embedding, the hyperparameter search space already includes the option of not adding temporal embedding (i.e., dimension = 0), at which point the method degenerates to conventional methods. In other words, in our experiments, **whether to add temporal embedding is determined entirely through hyperparameter search**.\n\nYou may also be interested in the response to the reviewer dcxr, which analyzes when to use our temporal modulation and when to use temporal embedding directly.\n\n> Some referenced adaptive methods (e.g., domain–adversarial, reweighting) are only briefly mentioned without analysis of their failure modes on temporal tasks.\n\nDomain-adversarial methods can only address domain-to-domain shifts and require partial target domain information, making them inappropriate for the temporal shift setting used in this paper. Reweighting methods can only address covariate and label shift, not concept shift, which is often severe in temporal shift datasets. Our temporal feature modulation aims to address concept shifts primarily caused by semantic evolution.\n\n> Limited analysis of the statistical significance of the approach.\n\nDue to the large dataset size in TabReD, there aren't enough experiments to support significant comparisons between all methods. Below are the Nemenyi tests for MLP and TabM:\n\n|               | MLP    | MLP-Temporal | MLP-Modulated |\n| ------------- | ------ | ------------ | ------------- |\n| MLP           | 1.0000 | 0.4237       | 0.0033        |\n| MLP-Temporal  | 0.4237 | 1.0000       | 0.1122        |\n| MLP-Modulated | 0.0033 | 0.1122       | 1.0000        |\n\nIt can be seen that adding our modulation method to MLP significantly improves performance, while adding temporal embedding alone does not.\n\n|                | TabM   | TabM-Temporal | TabM-Modulated |\n| -------------- | ------ | ------------- | -------------- |\n| TabM           | 1.0000 | 0.7336        | 0.0631         |\n| TabM-Temporal  | 0.7336 | 1.0000        | 0.2909         |\n| TabM-Modulated | 0.0631 | 0.2909        | 1.0000         |\n\nSimilar results are seen for TabM.\n\nIn addition, our paper presents multiple evaluation metrics, including average rank (Tab. 1) and average percentage change (Fig. 5), providing a multi-perspective evaluation.\n\n> The focus seems to be limited to one type of distribution shifts. How does the approach perform under sudden, irregular, or non-smooth temporal shifts compared to gradual shifts?\n\nWe acknowledge this limitation, but in the analysis in [1], the MMD heatmap shows that most real datasets have shifts in the form of trends and cycles. The data may indeed have unknown shifts in the test set, but for models deployed before this, there is no effective way to predict this.\n\n> Although presented as a lightweight method, there is insufficient quantitative analysis regarding computational overhead.\n\nWe will introduce more detailed quantitative analysis in the revision. Our MLP with temporal modulation still significantly outperforms other models (such as the transformer) in terms of training efficiency and achieves better performance, which in part demonstrates the lightweight nature of our approach.\n\n> What is the standard deviation over different repetitions of the model? It would be very interesting to see.\n\nThe following table provides the standard deviations of the random split results in the Appendix.\n\n\n| Method            | HI     | EO     | HD     | SH     | CT     | DE     | MR     | WE     |\n| ----------------- | ------ | ------ | ------ | ------ | ------ | ------ | ------ | ------ |\n| Linear            | 0.0009 | 0.0040 | 0.0018 | 0.0072 | 0.0003 | 0.0004 | 0.0073 | 0.0029 |\n| XGBoost           | 0.0003 | 0.0024 | 0.0091 | 0.0016 | 0.0002 | 0.0002 | 0.0001 | 0.0064 |\n| CatBoost          | 0.0008 | 0.0025 | 0.0022 | 0.0017 | 0.0003 | 0.0002 | 0.0001 | 0.0021 |\n| LightGBM          | 0.0005 | 0.0045 | 0.0081 | 0.0015 | 0.0003 | 0.0003 | 0.0004 | 0.0042 |\n| RandomForest      | 0.0002 | 0.0029 | 0.0017 | 0.0024 | 0.0001 | 0.0003 | 0.0000 | 0.0004 |\n| MLP               | 0.0042 | 0.0031 | 0.0011 | 0.0119 | 0.0004 | 0.0006 | 0.0001 | 0.0059 |\n| MLP-PLR           | 0.0014 | 0.0055 | 0.0102 | 0.0131 | 0.0003 | 0.0010 | 0.0002 | 0.0163 |\n| FT-T              | 0.0036 | 0.0095 | 0.0391 | 0.0082 | 0.0007 | 0.0014 | 0.0002 | 0.0084 |\n| TabR              | 0.0052 | 0.0055 | 0.0165 | 0.0068 | 0.0037 | 0.0033 | 0.0003 | 0.0067 |\n| ModernNCA         | 0.0007 | 0.0110 | 0.0074 | 0.0037 | 0.0005 | 0.0015 | 0.0004 | 0.0101 |\n| TabM              | 0.0014 | 0.0030 | 0.0128 | 0.0033 | 0.0007 | 0.0013 | 0.0002 | 0.0047 |\n| MLP-Modulated     | 0.0004 | 0.0028 | 0.0011 | 0.0068 | 0.0003 | 0.0012 | 0.0001 | 0.0052 |\n| MLP-PLR-Modulated | 0.0010 | 0.0041 | 0.0082 | 0.0027 | 0.0003 | 0.0010 | 0.0001 | 0.0078 |\n| TabM-Modulated    | 0.0008 | 0.0020 | 0.0090 | 0.0034 | 0.0006 | 0.0012 | 0.0002 | 0.0073 |\n\nWe hope this response addresses your concern. Please feel free to raise any further questions or suggestions!\n\n---\n\n[1] Cai, H.-R. and Ye, H.-J. Understanding the limits of deep tabular methods with temporal shift. CoRR, abs/2502.20260, 2025."}, {"review_id": "TSJJLfXiCx", "reviewer": "Reviewer_UGg7", "summary": "This work is motivated by the challenges of modeling tabular data that shifts over time. They propose an efficient feature-aware temporal modulation mechanism, inspired by feature-wise affine transformations such as FiLM, that adjusts feature representations based on temporal context in a learnable way, thereby allowing for the modulation of features over time. The proposed method is evaluated on a series of distributional and temporal extrapolation tasks from the TabRed benchmark.\n\nThe method is intuitively motivated — semantics of features vary over time, and can be modulated based on “subjective semantics” (i.e. with reference to the rest of the data distribution at a similar point in time), and the results are fairly clearly in favor of the proposed method, at least as applied to the TabRed benchmark. I raise a few questions below, but feel there is a strong case for acceptance for this paper.", "questions": "See above.", "limitations": "See above.", "rating": 5, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "# Major comments\n\n* The paper is well-written and does a good job setting up some of the main issues in this field, for example, the tension between generalizability and adaptability for temporal models.\n* The TabRed benchmark is certainly one that aligns with the authors’ intended domain application, but it is also not very widely used. At the same time, many other tabular benchmarks contain at least a subset of tables with temporal columns. It would be nice to see these results applied to more “standard” tabular benchmarks (but where there is temporal information that is often not fully utilized) — this would provide further evidence of the advantages of the proposed approach, and would particularly emphasize how utilizing temporal information “as temporal information” would improve the model performance.\n* In principle, it seems that there is nothing limiting this approach to temporal shifts — indeed, if this improves performance by modulating feature values over time, it would also be interesting to see whether it improves performance by similarly modulating based on the value of another continuous (or categorical) feature, effectively swapping the “time” modulator for some other signal. (This is a direction for future work, but an interesting one that would require effectively no changes to the authors’ methods.)\n\n# Minor comments\n* L258: “Our preprocessing, training, evaluation, and hyperparameter tuning setup follows the practices established in Cai & Ye [5]” - please provide some further details here.\n* I find the colors of the points in Figure 4 quite difficult to see, especially due to the small size. Perhaps this can be improved with (1) a higher-contrast palette and (2) changing the marker shape for different classes. I also feel that the figure would be easier to interpret with some indication of the values of the “time axis” for the smaller panels in the plot (perhaps by labeling the title of each subfigure with the time range it represents).\n\n# Typos etc\n* L249 “These embedding transform” —> these embeddings\n* L261 “Unlike prior work [5] requires” —> work [5], which requires", "was_revised": true, "final_justification": "I have read the author response, and will keep my score of 5. As I said in my original review, I feel there is a clear case for acceptance for this work.", "rebuttal": "Thank you for your thorough review and insightful feedback! We will address your concerns in the following responses.\n\n> It would be nice to see these results applied to more “standard” tabular benchmarks.\n\nWe acknowledge that TabReD has not received the attention it deserves. However, even if the datasets in existing benchmarks contain temporal columns, they are often not suitable for direct evaluation. These problems include data leakage caused by non-temporal splitting, extensive feature engineering of timestamps, and sample grouping that is not suitable for temporal splitting. **This has limited our options.** The Appendix of the TabReD paper also provides a detailed analysis [1]. Here we provide results on **two additional Kaggle datasets**:\n\n| Method            | Avocado Prices | tabular-playground-series-jul-2021 |\n| ----------------- | -------------- | ---------------------------------- |\n| MLP               | 4.5141         | 2.0932                             |\n| MLP-Temporal      | 4.3536         | 2.1141                             |\n| MLP-Modulated     | **4.0379**     | **2.0294**                         |\n| MLP-PLR           | 5.0073         | 1.9825                             |\n| MLP-PLR-Temporal  | 5.1699         | 2.0638                             |\n| MLP-PLR-Modulated | **4.7394**     | **1.9684**                         |\n| TabM              | 4.3938         | 1.9521                             |\n| TabM-Temporal     | **3.6955**     | 1.9773                             |\n| TabM-Modulated    | 3.8976         | **1.8204**                         |\n\nUnlike the dataset in TabReD, **the temporal shift of these two datasets is not obvious**, so directly concatenating temporal embedding may not be effective. **Our method shows robustness**.\n\n> It would also be interesting to see whether it improves performance by similarly modulating based on the value of another continuous (or categorical) feature.\n\nThis is a very good idea, and we've previously explored it, including combining it with ensemble methods. However, **current temporal embedding already incorporates many assumptions** (periodicity and trends). While features that satisfy a linear relationship or have a temporal period can be successfully used for modulation, other features with unknown periods are not. Furthermore, **selecting suitable features from among all available features is a key challenge**, as using unsuitable features for modulation can lead to missing input information and degraded performance. This is definitely a direction for future work, and it holds great potential in scenarios where the shift patterns can be formalized.\n\n> L258:  please provide some further details here.\n\nThanks for pointing this out. Due to space constraints, we moved the details of this section to the appendix when we submitted the main paper. We will include more details here in the revision.\n\n> I find the colors of the points in Figure 4 quite difficult to see, especially due to the small size. \n\nIndeed, we will enlarge the sample points, highlight the different categories, and include a time axis in the revision to improve readability.\n\nWe hope this response addresses your concern, and we will carefully address all the suggestions and correct any typos in the revision. Please feel free to raise any further questions or suggestions!\n\n---\n\n[1] Rubachev, I., Kartashev, N., Gorishniy, Y., and Babenko, A. Tabred: A benchmark of tabular machine learning in-the-wild. In ICLR, 2025."}, {"review_id": "vlv768Sc8q", "reviewer": "Reviewer_kWjp", "summary": "This paper presents a method for dealing with temporal drift in tabular ML settings that uses a modulator that adjusts features and/or model outputs over time. In experiments on the TabReD benchmark they compare their approach with both static and adaptive tabular ML methods. The also include experiments on synthetic datasets, and an ablation study.", "questions": "1. it is not clear where and how modulation is applied in your model. I understand from Section 5 and eq. 2 that you learn mappings \\psi, \\gamma, \\beta, and \\lambda. From eq. 2 it appears that these mappings are used to modify features x before or during processing by the model. Sub-questions below.\n\n2. How do you apply modulation? It seems the easiest way would be to modulate your input features according to eq. 2. But Figure 5 suggests other ways - e.g. during processing and after processing. Does this mean that your model must use hidden states with the same dimensionality as your input features?\n\n3. If there are multiple ways to apply modulation, then how do you apply modulation in your experiments? I cannot determine this from your experiment description.\n\n4. If you can apply modulation to input features prior to processing by the ML model, then you could use any ML model with this approch, right? (E.g. xgboost or SVM). In this case, why do your experiments only use neural network-based methods with modulation? It is remarkable that catboost, xgboost, and lightgbm perform so well even _without_ modulation. So why not test them _with_ modulation as well?\n\n5. How do you learn mappings \\psi, \\gamma, \\beta, and \\lambda? This is not clear at all. Please include a description around eq. 2 and also in your experiments section for how you learned these mappings in your experiments.", "limitations": "yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 3, "significance": 4, "originality": 3, "strengths_and_weaknesses": "This paper is very clear and well motivated. Their new method appears both novel and technically sound (see below for some questions and concerns). Their experiments appear to be very robust, and they clearly demonstrate the utility of their modulation approach. I have several questions and concerns about their method described in the questions section.", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your valuable feedback! We will address your concerns in the following responses.\n\n> How do you apply modulation?\n\nOur feature-aware temporal modulation can be **flexibly applied at three stages**: input features, intermediate representations, or output logits (**L226**). The ablation study in **Table 2** explores the relationship between the application position of modulation and performance improvement, and it is found that modulation at all stages has a positive impact on performance (**Sec. 6.4**).\n\n> Does this mean that your model must use hidden states with the same dimensionality as your input features?\n\nThis design does not require hidden states to match input dimensionality. Modulation parameters are dynamically generated per layer, allowing dimension-agnostic adaptation.\n\n> How do you apply modulation in your experiments?\n\nAs specified in Sec. 6.2 (**L274**), the MLP uses full modulation across all stages, while MLP-PLR and TabM only employ single-step modulation on input features.\n\n> How do you learn $\\psi$, $\\gamma$, $\\beta$, and $\\lambda$?\n\nAll parameters of the temporal embedding and modulator are **jointly optimized with the backbone model** using the AdamW optimizer and standard losses (e.g., cross-entropy for classification and RMSE for regression).\n\n> You could use any ML model with this approch, right? Why do your experiments only use neural network-based methods with modulation?\n\n**The modulation mechanism is not decoupled from model training**. Instead, it is learned through end-to-end optimization and relies on gradient-based methods. \n\nConsequently, our modulation strategy is compatible with any deep learning model, but non-differentiable models (e.g., tree-based ensembles) are unable to optimize the parameters of $\\psi$, $\\gamma$, $\\beta$, and $\\lambda$ due to the absence of gradient flow.\n\n**In summary**, our modulation is a deep learning-specific, gradient-based adaptation mechanism applied flexibly across network stages. Its strength lies in jointly learning temporal representations and feature transformations, which is infeasible for non-differentiable models. We appreciate your comment and will acknowledge this limitation in our revision. \n\nWe hope this response addresses your concern. Please feel free to raise any further questions or suggestions!"}]

Author rebuttal:

Meta-review: The paper introduces a feature-aware modulation mechanism to handle temporal distribution shifts in tabular data by aligning feature semantics over time through lightweight modulation of distributional statistics such as scale, skewness, and bias. This approach enables models to better balance generalization and adaptability compared to static models or simple temporal embeddings, and empirical evaluation on the TabReD benchmark demonstrates consistent, if modest, improvements. To further strengthen the work, the authors are encouraged to candidly discuss its limitations, particularly around the magnitude of empirical improvements, the boundaries of when modulation is superior to temporal embedding, and potential weaknesses in high-dimensional or categorical feature settings. A more open reflection on these limitations would enhance the credibility of the contribution and help guide future research on temporal shift in tabular learning.
Decision: Accept (poster)