ICML 2025

Understanding the Limits of Deep Tabular Methods with Temporal Shift
Hao-Run Cai 1 2 Han-Jia Ye 1 2
Abstract
Deep tabular models have demonstrated remark-
able success on i.i.d. data, excelling in a variety
of structured data tasks. However, their perfor-
mance often deteriorates under temporal distri-
bution shifts, where trends and periodic patterns
are present in the evolving data distribution over
time. In this paper, we explore the underlying rea-
sons for this failure in capturing temporal depen-
dencies. We begin by investigating the training
protocol, revealing a key issue in how model se-
lection performs. While existing approaches use
temporal ordering for splitting validation set, we
show that even a random split can signiﬁcantly
improve model performance. By minimizing the
time lag between training data and test time, while
reducing the bias in validation, our proposed train-
ing protocol signiﬁcantly improves generalization
across various methods. Furthermore, we analyze
how temporal data affects deep tabular represen-
tations, uncovering that these models often fail
to capture crucial periodic and trend information.
To address this gap, we introduce a plug-and-play
temporal embedding method based on Fourier se-
ries expansion to learn and incorporate temporal
patterns, offering an adaptive approach to han-
dle temporal shifts. Our experiments demonstrate
that this temporal embedding, combined with the
improved training protocol, provides a more ef-
fective and robust framework for learning from
temporal tabular data.
1. Introduction
Tabular data is one of the most prevalent data formats in a
wide range of real-world applications, such as e-commerce
(Nederstigt et al., 2014) and healthcare (Hassan et al., 2020).
1School of Artiﬁcial Intelligence, Nanjing University, China
2National Key Laboratory for Novel Software Technology,
Nanjing University, China.
Correspondence to: Han-Jia Ye
<yehj@lamda.nju.edu.cn>.
Proceedings of the 42 nd International Confere...