Functional Complexity-adaptive Temporal Tensor Decomposition
Authors: Panqi Chen
Lei Cheng
Jianlong Li
Weichang Li
Weiqing Liu
Jiang Bian
Shikai Fang
Venue: NeurIPS 2025 poster ()
Keywords: functional tensor decomposition，Bayesian Method
Primary area: probabilistic_methods

Abstract: Tensor decomposition is a fundamental tool for analyzing multi-dimensional data by learning low-rank factors to represent high-order interactions. While recent works on temporal tensor decomposition have made significant progress by incorporating continuous timestamps in latent factors, they still struggle with general tensor data with continuous indexes not only in the temporal mode but also in other modes, such as spatial coordinates in climate data.  Moreover, the challenge of self-adapting model complexity is largely unexplored in functional temporal tensor models, with existing methods being inapplicable in this setting.  To address these limitations, we propose functional Complexity-Adaptive Temporal Tensor dEcomposition (Catte). 
 Our approach encodes continuous spatial indexes as learnable Fourier features and employs neural ODEs in latent space to learn the temporal trajectories of factors. To enable automatic adaptation of model complexity, we introduce a sparsity-inducing prior over the factor trajectories. 
 We develop an efficient variational inference scheme with an analytical evidence lower bound, enabling sampling-free optimization. Through extensive experiments on both synthetic and real-world datasets, we demonstrate that Catte not only reveals the underlying ranks of functional temporal tensors but also significantly outperforms existing methods in prediction performance and robustness against noise.

widely applied Functional Complexity-adaptive Temporal Tensor Decomposition
Panqi Chen'
Lei Chengl.2 Jianlong Li Weichang Li"
Weiqing Liu Jiang Bian Shikai Fang
College of Information Science and Electronic Engineering, Zhejiang University 2Zhejiang Provincial Key Laboratory of Multi-Modal Communication Networks and Intelligent Information Processing Microsoft Research Asia
Abstract
Tensor decomposition is a fundamental tool for analyzing multi-dimensional data by learning low-rank factors to represent high-order interactions. While recent works on temporal tensor decomposition have made significant progress by incor- porating continuous timestamps in latent factors, they still struggle with general tensor data with continuous indexes not only in the temporal mode but also in other modes, such spatial coordinates in climate data. Moreover, the challenge of self-adapting model complexity is largely unexplored in functional temporal tensor models, with existing methods being inapplicable this setting. To address these limitations, we propose functional Commplexit-Adaaptie dEcomposition (CATTE). Our approach encodes continuous spatial indexes as learnable Fourier features and employs neural ODEs in latent space to learn the temporal trajectories factors. To enable automatic adaptation of model complex- ity, introduce a sparsity-inducinn prior over the factor trajectories. We develop an efficient variational inference scheme with an analytical evidence lower bound, enabling sampling-frr free optimization. Through extensive experiments on both syn- thetic and real-world datasets, we demonstrate that CATTE not only reveals the underlying ranks of functional temporal tensors but also significantly outperforms existing methods in prediction performance and robustness against noise. The code is available httht:::////tt //ti:/b//tttttt. cww.//ceaaTTAALa///CTT...
Temporal Tensor
Introduction
Tensor is ubiquitous data structure for organizing nulti-dimenssonal data. For example, four- mode tensor (longitude, latitude, depth, time) can serve as a unified representation of spatiotemporal signals in the ocean, such temperature flow speed. Tensor decomposition is prevailing frame- work for multiway data analysis that estimates latent factors to reconstruct the unobserved entries. Methods like CANDECOMPPPARAEAN (CP) and Tucker decomposition[[] across fields, including climate science, oceanography, and social science.
An emerging trend in tensor community is to leverage the continuous timestamp of observed entries and build temporal tensor models, as the real- world tensor data is often irregularly collected in time, e.g., physical signals, accompanied with rich and complex time-varying patterns. The temporal tensor methods expand the classical tensor framework by using polynomial splines [3], Gaussian processes [ 5], ODE [6]] and energy-based models [Ø]] estimate the continuous temporal dynam- ics in latent space, instead of discretizing the time mode and setting a fixed number factors.
Correspond to lei lei_cheng @ zjj.edu..n, fsk @sk@zju.ddu.cn
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Despite the successes of current temporal tensor methods, they inherit fundamental limitation from traditional tensor models: they assume tensor data at each timestep must conform a Cartesian grid structure with discrete indexes and finie-dimensional modes. This assumption poorly aligns with many real-world scenarios where modes are naturally continuous, such as spatial coordinates like (longitude, latätuee, depth). To fit current models, we still need to discretize continuous indexes, which inevitably leads to a loss of fine-grained information encoded in these indexes. From high- level perspective, while current temporal tensor methods have taken crucial step forward by mod- eling continuous characteristics in the temporal mode compared to classical approaches, they still fail to fully utilize the rich complex patterns inherent in other certinoous-indexee modes.
Another crucial challenge lies in automatically adapting the complexity functional temporal ten- sor decomposition model to the data, which is determined by the tensor rank. As a key hyperpa- rameter in tensor modeling, the rank directly influences interpretability, sparsity, and model expres- siveness. While classical tensor literature offers extensive theoretical analysis and parning-based solutions [8 H. ICI this topic has been largely overlooked in emerging functional temporal tensor methods. The introduction of dynamical patterns significantly complicates the latent landscape, and the lack of investigation into rank selection makes temporal tensor models more susceptible to hy- perparameter choice. Exisiting approaches [8] 9. I[] focus on modeling discrete factors, which can not be straightforwardly extended to the functional regimes.
To fill these gaps, we propose CATTE, complexity complexity-ddaptive method for modeling temporal tensor data with continuous indexes across all modes. Our method models general temporal tensor data with continuous indexes not only in the time mode but also in other modes. Specifically, CATTE organizes the continuous indexes from non-temporal modes into a Fourier-feature format, encodes them as the initial state of latent dynamics, and utilizes the neural ODE [[II] to model the factor tra- jectories. To enable self-adaptive complexity in functional temporal tensor decomposition, CATTE extends the classical Bayesian rank selection framework [8] by placing sparsity-inducing priors over factor trajectories. For efficient inference, we propose novel variational inference algorithm with an analytical evidence lower bound, enabling sampling-free inference of the model parameters and latent dynamics. For evaluation, we conducted experiments on both simulated and real-world tasks, demonstrating that CATTE not only reveals the underlying ranks of functional temporal tensors but also significantly outperforms existing methods in prediction performance and robustness against noise.
2
Preliminary
2.1 Tensor Decompusition
Tensor decomposition represents multi-dimensionn arrays by decomposing them into lower- dimensional components, thus revealing underlying patterns in fiigh-dimensional data. We denote a K-mode tensor as y E - x x..xX1 Ixx, where the k-th mode consists of L7 dimensions. Each entry of Y, termed y/, is indexed by K-tuple = (i1, ... ik. ...,)), where is denotes the index of the node along the mode & (1 S k & K). For tensor decomposition, set of factor trices [U*} ===1 are introduced to represent the nodes in each mode. Specifically, the k-th factor matrix U is composed of Iş latent factors, i.e., U = [u] ....... u
uf, IT E R Rd and u Tu I. u Th u Rs 1T E RR, where R2 denotes the rank of mode-k. The clas- (CP) decomposition [I]] aims to decompose tensor into a sum of
sic CANDECOMPPPARAA
rank-one tensors. It sets R
Rs Rx
R and represents each entry using
R
K
ΣI
r=1
where 1 E R is the all-one vector and @ is Hadamard product of a set of vectors defined as Lut = (u) @ u k
Here, @ is Hadamard product.
2.2 Automatic Tensor Rank Determination
The tensor rank R determines the complexity of the tensor model. An improper choice of the rank can lead to overfitting underfitting to the signal sources, potentially compromising the model

---

interpretability. However, the optimal determination of the tensor rank is known to be NP-hard 19 I12 T3. Bayesian methods have been introduced to facilitate Tucker/CP decomposition with automatic tensor rank learning [14, 8 9 [Ta]. These methods impose sparsity-rromotin priors (e.g..
the Gamma prior and Laplacian prior) on the latent factors.
For example, Bayesian CP decomposition with Gaussian-Gamma priors models the mean and precision of all latent factors with zero elements and set of latent variables À [A,,"* Ar. .. AR] E E² respectively:
R p i, AA) (uf,, 10,A A "), "),K, p(A) II Gamma (Arla,, b.)).
where A = diag A) is the inverse covariance matrix shared by all latent factors over K modes. Note that components of u are assumed to be statistically independent and the distribution of the r-th component is controlled by A,. For example, if X, is large, then the density function peaks at mean zero, so the r-th component concentrated at zero. Otherwise, if X, is small (which leads to heavy tails), it allows the component to spread out to wider range of values. The conjugated Gamma priors are assigned to A. Here Gamma(z|a, b) -ー for x 2 0, which represents the Gamma distribution for In this context, a and denote the shape and rate parameters respectively, and I()) denotes the Gamma function. {af, 1 pre-determined hyperparameters. The tensor rank will be automatically determined by the inferred posteriors of X.
[(a]
2.3 Generalized Tensor with Continuous Modes
Real-world tensor data often contains continuous modes, prompting increased studies general- ized tensor data with continuous indexes. Existing approaches can be broadly classified into two categories:
1. Temppral tensor model with continuous timestamps: Recent studies encode the representations of continuous timestamps into the latent factor of CP model [S. [[S] the tensor core of Tucker model [+]] Taking CP as an example, the temporal tensor model with continuous timestamps can be written as:
v1(t) ~= 1 1 (t)),
where is the tensor index, is the continuous timestamp. u (1) is the factor trajectory of the ig-th node in the k-th mode. Although this modeling approach effectively captures complex temporal dynamics, is inadequate for generalizing to data with continuous indexes over the entire domain, such as spatiotemporal data which has continuous coordinates on both spatial and temporal modes [U6]].
Functional tensor model: Another popular model to handle continuous-indexed modes is the func- tional tensor [IZ7, I8, [99), which assumes that the cecttnuous-nddexeee tensor be factorized set of mode-wise functions and the continuous timestamp is simply modeled as an extra mode. Still taking CP an example, the functional tensor model can be written as:
(t) Re 1 T [ (ix) Tu
Temporal (t)],
where (ix) R4 - R is the latent vector-valued function of the k-th mode, which takes the continuous index input and outputs the latent factor. u Temporal (L) I2, RR is the latent function of the temporal mode. The fully-factorized form of (H) models each mode equally and independently. It often overlooks the complex dynamics of the temporal mode, which requires special treatment (UGG).
3 Methodology
Despite recent advances in modeling temporal tensors, most of these methods are still unsuitable for generalized tensor data with continuous indexes across all domains. While functional tensor meth- ods offer greater flexibility, they simply treat temporal dynamics as an independent mode, which tends underfit the inherent complexity of the temporal dynamics. Furthermore, rank determina- tion remains less explored issue in temporal tensor models. To address these issues, propose
3

---

Decompasition
Continuous indexes entry
Fourier features
Neural ODEs
solver
Revcoling
trajectories
Eo
Observations of general temporal tensor data
Reconstruction
Latent space
Figure 1: Graphical illustration of the proposed CATTE (the case of K = 3).
CATTE, a novel temporal tensor model that integrates the continuous-indexel features into latent ODE model with rank-revealing prior.
Without loss of generality, we consider a K-mode generalized temporal tensor with continuous indexes over all domains, and actually corresponds to a function F(i,, ikk t) RK + R to map the continuous indexes and timestamp to the tensor entry, denoted as yı(t) Fii,.... ix t). We assume the function F(11," ".. ,ik,) CP format with rank R, i.e.
, t) can be factorized into K factor trajectories following the
vi(t) F(i1. ... ,kk,t) t) Ro 1" (ix.
(5)
where u (ix, () R R is the trajectory of latent factor, which is R-size vector-valued func- tion mapping the continuous index ig mode-k and timestamp to a Rt-dieensional latent factor. We claim that the proposed model (55) is generalization of existing temporal tensor methods (L) via modeling continuous -indxxe patterns not only in the temporal mode but in all modes. If we restrict i finite and discrete, (5S) will degrade to (B). Compared to the fully- fully-factorized functional ten- sor (H), the proposed method (5)) explicitly models the time-varying factor trajectories of all modes, known dynamic factor learning [D) [T]. Given the fact that temporal mode always dominates and interacts with other modes, the proposed method is expected to improve the model's 's capability by learning time-narying representations in dynamical data.
3.1 Continuous-indexed Latent-ODE
To allow flexible modeling of the factor trajectory and continuous-inde indexxd information, we propose a temporal function g" (ix, t) TR. + R based on encoder-decoder structure and neural ODE [I]]
to approximate the factor trajectory u (ix, t) of mode-k. Specifically, we have:
x (ix, 0) = (inccoder(cos(22rb/:: si2
(6)
x" (ix, () =z (in, 0) f( ha, (z" (ixs s)ds,
g" (ix, 1) =Decoder (2 (ic;
(8)
Eq. (6) shows that how we obtain z(i, 0) R the initial state of the latent dynamics by encod- ing the continuous index is. In particular, the input coordinate in is firstly expanded into a set of Fourier features [cos(2.rbú4a sinn E R.2.M. where bą E R. M is a learnable vector that scales is by M different frequencies. This effectively expands the input space with high-frequency components [20],, helping to capture fine- ane-grained index information. The Fourier features are then fed into an encoder Encoder(.) R22M
_ R to get x (ix, 0). Give the initial state, we then ap- ply a neural network he, (z s), - + IR'' to model the state transition of the dynamics each timestamp, which parameterized by Os, and the state value can be calculated through integrations as shown in (Z)). Finally, we will pass the output of the latent dynamics through a de- coder Decoder(-) RR R obtain g g t) the approximation of the factor trajectory, as described in (E). We simply use the multilayer perceptrons (MLPs) to parameterize the encoder and the decoder.
Note that (α)) actually represents the neural ODE model [21, 11, and we handle the integration to obtain z (ix, t) in (7Z) at arbitrary by using numerical ODE solvers:
z Lix, t) ODESolve(2* (ix. 0), her).
(9)

---

Algorithm 1: Training process of CATTE
Input:
Training data = {yn, in in Jn=1 Collect all possible Ig. indexes for K modes and T possible timestamps. Initialize {wi 1: 3r 3 0,
while convergence do
Construct set of initial ODE state tables Zo using Fourier features and Encoder, encompassing all possible indexes.
for 1.2. T do
[(t)) ODESolve(Z(tt-1), (he (ooo)tttttttttt) Compute necessary (ix, 6x) from (t) using (8).
end
Take gradient step on (I
end
For computing efficiency, we concatenate the initial states of multiple indexes together, and construct matrix-valued trajectory, where each row corresponds the initial state of unique index. Then, we only need to call the ODE solver once to obtain the factor trajectory of observed indexes. For simplicity, we denote the all learnable parameters in (G)-() as We for mode-k, which includes the frequency-scale vectors by as well as the parameters of neural ODE O2 and the encoder-decoder.
3.2 Functional Rank-revealing Prior over Factor Trajectories
To automatically determine the underlying rank in the functional dynamical scenario, apply the Bayesian sparsity-promotte priors. Different from the classical framework [8], stated (2), we assign a dimension-wise
prior to the factor trajectory.
p(u" (ix, t) = N( (is, t) Ak,
(10)
where dAag(A) and = p...... ,y.....,p..... ARI E RR We assign Gamma priors to A: p(A) I1 Gamma (ommma(Aylofa) [a, b9), identical to (2)). Then, the rank-revealing prior over all factor
trajectories is:
plu, P(A)) II p pu f)A),
(11)
where U denotes the set of factor trajectories (1= We are to emphasize that the sparsity- inducing prior is assigned over family of latent functions. rather than tors /88, MJ. Consequently, classical inference techniques developed are not applicable in the func- tional tensor context. In the next section, introduce novel gradientdescenthased inference method that efficiently prunes redundant components from these latent functions.
set of static fac-
With finite observed entries D {yn, ins J. where Yn denotes the n-th entry observed at continuous index tuple i, = ... ...)) and timestamp t,- Our goal is to learn factorized function described in (5S) construct direct mapping from (in fn) ya. Therefore, for each observed entry (Un, in, tn ), we model the Gaussian likelihood as:
plyn 7) (V, T
[ou" (12 (,)))
(12)
where is the inverse of the observation noise. We further assign Gamma prior, p(T) Gamma(r|c.) dd), and the joint probability can be written as:
p(u, p(u, ()p(T) II (()
(13)
We illustrate CATTE with the of K in Figure II

---

Model Inference
4.1 Factorized Functional Posterior and Analytical Evidence Lower Bound
It is intractable to compute the full posterior of latent variables in (T3) due to the high- dithnssinnsii integral and complex form of likelihood. We take a workaround to construct a variational distribu- tion q(4,, T) to approximate the exact posterior p(l,, X, T|D). Similar to the widely-used mean-
field assumption, we design the approximate posterior in fully factorized form: qu,, (()))))))((T)
Specifically. the conditional conjugate property of Gaussian Raassian-Gamma distribution motivates to formulate the corresponding variational posteriors as follows:
N
K
q(u)) = II
II == R=1
NN (1f (a)ttllg"" (12,)))
(14)
where g.( (..-) is the mode-wise latent temporal representations parameterized by Wk, as we men- tioned in Section B3ll, and a is the variational variance shared by all u². Similarly, we formulate
g()).((T) as:
R II Gamma(Arla,,,))))
(A)
= Gamma(rlp;
(15)
where 3,),,,), p, are the variational parameters to characterize the approximated pos- teriors. Our goal is estimate the latent ODE parameters w/ and variational parameters Har, 3, 3- p, in (44 (LL) to make the approximated posterior g(4,, x, as close as possible the true posterior p(u, (D)) To do so, follow the variational inference framework [22] and
construct the following objective function by minimizing the Kullback-Leibler (KL) divergence be- tween the approximated posterior and the true posterior (L(g(4, X) (p) (DD), which leads to the maximization of the evidence lower bound (ELBO):
ELBO- Euru,A,;)lp(((DL X, r)] + BY(UAA) In p(u[>) KL(O(A)II(()) - kL(u(T))lp(T)).
9(44)
(16) The ELBO is consist of four terms. The first term is posterior expectation of log-likelihood while the last three are KL terms. Usually, the first term is intractable if the likelihood model is complicated and requires the costly sampling-basee [23, [Σ]. Fortunately, by leveraging the well-designed conjugate priors and factorized structure of the posterior, we make an endeavor to derive its analytical expression:
approximation to handle the integrals in the expectation
N
N (1b(p) lm
E ( [In (D/U,, x)
In ln(2
N Σ 2 ==1
2yn (1g,4))) + vee((" (1
( (w) (TT, + oDr)]] (17)
where (1): fn) the r-th element of the k-tth mode's latent temporal representation g g t.). We provide the detailed derivation in Appendix A. The second term of (IT) computes the KL divergence between prior and posterior of factor trajectories, which is also with a closed from:
p(([A)
E Equ.s (In
(L((()))) E,,()))
(())
N
K
In
Br
ar
ar 102
lg (ig tx ( 1}.
(18)
a,,
B,
where (A) EE,()), (AR)) |/1/33,.... an/3n]" This term encourages rank re- duction, as drives the posterior mean of >r be large, thereby forcing the corresponding r-th component of K factor trajectories (-)) k) to be zero. The combination of the above two terms enables an automatic rank determination mechanism by striking a balance between capacity of representation and model complexity. As the prior and posterior of X and are both Gamma distribution, the last two KI. terms the ELBO are analytically computable, as shown in ((7) in Appendix Aw

---

We highlight that each term in (LG)) admits closedform expression, eliminating the need for sam- plingbased approximations. Consequently, during training we can compute the ELBOs gradient with respect to the variational parameters exactly, allowing us to employ standard gradientbased optimization methods. We present the differences from previous methods in Appendix Q Our pro- posed method scales efficiently to jointly optimize both the variational and latent ODE parameters:
argmax (or.)) ELBO.
(19)
We summarize the inference algorithm in Algorithm When N is large, we can use the mini- batch gradient descent method to accelerate the optimization process. After obtaining the variational posteriors of the latent variables, we can further derive the predictive distribution for new data at arbitrary indices. More details can be found in Appendix A. To distinguish our method from previous approaches, we refer as a functional automatic rank determination mechanism (FARD).
Related Work
Many existing approaches augment the tensor with a time mode to integrate temporal informa- tion [24L. 25, 26, 27]]. To leverage continuous timestamps, [ modeled the tensor core while [S]] modeled the latent factors as time functions with Gaussian processes within the Tucker decomposi- tion. Similarly, [LS] captured the temporal evolution of latent factors the frequency domain by employing Gaussian process prior, with the factor trajectories subsequently generated via inverse Fourier transform. contrast, [6]] directly applied a neural ODE model to represent tensor entry values function of the corresponding latent factors and time. However, it learns iiveiinvarinnt latent factors to control the derivatives of the entry dynamics, limiting its expressiveness. The most recent work [28] constructed a multipartite graph to encode interactions across different modes into the evolution of latent factors. However, the above methods are unable to model the temporal tensor data with all modes being continuous- continoous-indexed.
Within burgeoning literature on functional tensors, existing methods often rely on deterministic ten- sor models, such as the Tucker model [17]] or tensor train models [20, R0 IR, (1 (2] to approximate multivariate functions with low-rank structures. However, these methods are sensitive to data noise and lack the capability to provide uncertainty quantification. Alternatively, tensor-based Bayesian models with Gaussian process priors have been proposed to represent continuous-indee latent fac- tors [6% [T] Despite their advantages, these approaches do not explicitly model temporal dynamics, limiting their effectiveness in capturing complex patterns.
Experiment
6.1
Synthetic Data
We first evaluated CATTE on a synthetic task. We generated a two-mode temporal tensor, and each entry is defined as:
Y(i, 12, 1 ju (11, t) @ u2(,,,)) f)], where u (i1, cos (2Tt + 2.5mii);;
(20)
u (12, t) sin(3nt 3.5miz))
We randomly sampled 25 x 25 50 off off-grid indexes entries from interval [0. 1] x [0. 1] x 1]. We added Gaussian noise 6 ~ (0, 0.05) the generated data. We randomly selected 20% of the data (6250 points in total) as the training data. Detailed model set- tings can be found in Appendix [H_ In Figure 2 we showed the predictive trajectories of entry value in- dexed in different coordinates. The dotted line repre- sents the ground truth and the full line represents the the predictive mean learned by model. The cross symbols represent the training points. The shaded region represents the predictive uncertainty region.
1.0
M
0.5
-0.5
0.0
Uncortaiity
-1.0
Time
-0.3
Time
(a) (0.112,,,.823, ()
(b) (0.992, 0.982, t)
Figure 2: Prediction results on different coor- dinates.
One can see that although the training points are sparse and noisy, CATTE accurately recovered the ground truth, demonstrating that it has effectively captured the temporal dynamics. Figure C depicts
7

---

R components of the learned factor trajectories at timestamp = 0.235. One can see that CATTE identifies the underlying rank (i.e.. 1) through uniquely recovering the real mode functions and other four components are learned to be zero. More detailed interpretations on the rank revealing process were provided in Appendix [B
6.2 Real-world Data
Datasets: We examined CATTE on three real-world benchmark datasets. (I) CA traf- fic, lane-blocked records in California. We extracted three-mode temporal tensor be- tween 5 severity levels. 20 latitudes and 16 longitudes. We collected 10K entry values and their perature logs of Poznan Supercomputing and Networking Center. We extracted three- mode temporal tensor between 3 air conditioning modes (24", 27 and 30)), power usage levels (50%, 75%, 100%) and 34 locations. We collected 10K entry values and their timestamps, trmnestamss.btttss///eeeee (bttps://zenodo http///eoooooorrrrrr///////////e org/reeord/36100/rrtt
(2) Server Room, tem-
Y8SYt3bMJGi); (3) SSF, sound speed field measurements in the pacific ocean covering the region between lati- tudes 17 ~ N0,, longitude 114.7%E 117.7%E and depth Om ~ 200m. We ex- tracted three-mode continuous-nndxeed temporal tensor data contains 10K observations across 10 latitudes, 20 longitudes, 10 depths and 34 timestamps over 4 days. (nttps: 7/ncss Jncss.hycom. oog/ http:/theddd//ncssss/ http://nssss/rri//iiiee BeerrdddGBBy..0///
93 33.00//3//datsset html).
Baselines and Settings: We compared CATTE with state-of-the-arr temporal and functional tensor meth- ods: (1) THIS-ODE [[]]. a continuous-tima decom- position using neural ODE to estimate tensor en- tries from static factors and time; (2) NONFAT (I5],, bi-level latent GP model that estimates dynamic factors with Fourier bases; (3) DEMOTE [28], a neural diffusion-reaction process model for learning dynamic factors in tensor decomposition; (4) Fun- BaT [T a Bayesian method using GPs func- tional priors for continuous-indexxe tensor data; (5) LRTER [ a low-rank functional Tucker model that uses factorized neural representations for de- composition. We followed [28, [9]] to randomly
2 0.235)
0.2353)
0.9
-0.2
-1.3
-0.4
-2.4
-1.2
Figure 3: Visualizations of learned factor tra-
jectories at timestamp & = 0.235. Only the 3rd component is revealed to be informative and others are pruned to be zero.
draw 80% of observed entries for training and the rest for testing. The performance metrics in- clude the root-mean-square error (RMSE) and the mean absolute error (MAE). Each experiment was conducted five times (5fold crossvalidation) and we reported the average test errors with their standard deviations. For CATTE, we set the ODE state dimension = 10 and the initial number of components of the factor trajectories R 10. We provided more detailed baseline settings in Appendix H
Prediction Performance: Table I shows that CATTE consistently outperforms all baselines by substantial margin, without requiring manual rank tuning. Moreover, the integration of FARD leads to significant performance gains, highlighting its effectiveness. The learned ranks of the CA traffic, Server Room and SSF datasets are 5, 7, 7 respectively. We illustrated their rank-learning curves of three datasets in Figure B in Appendix B. We observed that methods which do not consider the continuously indexed mode (e.g., NONFAT, DEMOTE, THIS-ODE) perform poorly on the SSF dataset. In contrast, approaches that leverage this continuity achieve significantly better results. This is because the SSF dataset exhibits strong continuity across three modes, and methods that fail to incorporate this information struggle to deliver satisfactory reconstructions, Additional visualization results of the predictions from various methods and datasets are presented in Fig. D. Fig. E and Fig. M and in Appendix BTll
Revealed Rank Analysis and Interpretability: We analyzed the revealed rank and the learned fac- tor trajectories of the SSF dataset. Figure A a) shows the posterior mean of the variance of the learned factor trajectories (i.e., E., (),, which governs the fluctuations of their corresponding R = 10 com- ponents of factor trajectories. One can see that E. Xg these components concentrate around zero and can be pruned without affecting the final predictions. Thus, our method revealed the rank of the SSF dataset to be 7. Additionally, ( and Eg 2 dominate, indicating that the corresponding Lst and 10th components of the factor trajectories form
1 Eng( } and (2(2)) are small, indicating that
8

---

RMSE
MAE
Datasets
CA Truffic
Server Room
SSF
CA Truffic
Server Room
SSF
THIS-ODE
0.587 0.002
0,083 + 0,002
+ 0.003
NONFAT
0,504 + 0.010
0.129 = 0.002
9.79
0.010
0.167 + 0.009
0,078 + 0.001
8,771 + 0.043
DEMOTE
0.447 0.001
0.131 1: 0.001
9.789 1: 0.001
0.118 0.002
0.090 1: 0.0015
8.757 1 0.001
FunBaT-CP
0.047
0.001
FunkaT-Tucker
0.584
0.009
0.498
0.058
0,730
0.201
0.189
0.014
0.381 1. 0.053
0.614 1 0.128
LKIFR
0,379 + 0.042
0.151 = 0.004
0,595
+ 0.018
0.187 + 0.022
0,110 + 0.002
0.464 = 0.0165
R
THIS-ODE
0.632 0.002
0.132 = 0.003
1,039
0.015
0.552 = 0.001
0.083 + 0.002
1,032 + 0.002
NONEAT
0.501
0.002
0.117 0.006
9.801
0.014
0.152 0.001
0,071 1: 0.004
8.744 1: 0.035
DEMOTE
0.421 +: 0.002
0.105 = 0.003
9.788 +: 0.001
0.103 + 0.001
0,068 1 0.003
8,757 1 0.001
FunBaT-CP
0.547
0.025
0,422 1 0.001
0.675
0.061
0.204 1: 0.052
0.307 1 0.002
0.531 1 0.051
FunBaT-Tucker
0,521
+ 0.097
LRTFR
0,376 0.016
0.167
0.006
0.036 0,532
0.182 + 0.012
0,121 + 0.005
0,418 + 0.003
R=
THIS-ODE
0.628 + 0.007
0.154 = 0.016
1,685
0.009
0.548 + 0.006
0.089 1 0.002
1.674 1 0.008
NONEAT
1: 0.002
DEMOTE
0,389 + 0.005
0,094 = 0.006
9,790 + 0.002
0.091 + 0.001
0.062 + 0.006
8,753 + 0.006
FunBaT-CP
0.545 + 0.009
0.26 = 0.001
0.685 + 0.049
0.204 + 0.037
0.307 + 0.001
0.541 + 0.039
PunBaT-Tucker
0.587 0.011
0.450
0.041
0.642
0.037
0.195 0.022
0,330 + 0.026
0.507 + 0.029
I.RTFR
0.365 0.042
0.156
0.012
0.502
0.033
161
0.014
0,118 1 0.009
0.392 1 0.028
Functional Automatic Rank Determination
CATTE (Ours)
0,284 1 0.016
0.078
0.001
0.373
0.003
0.0K5
0.004
0.047
1 0.003
0.288
1 0.003
CATTE W.D. FARD
0.301
1 0.020
0.091
0.008
0.402
0.013
0.094 1 0.010
0.0657
1 0.005
0.310 1 0.010
10
0.7
2.110)
124
-0.1
1
by componcnt
carne stams UCFA
1-25
10y componett
200
100
1000
3a 2ng 4. 5p 6o 76 9e 9p30m
Time
Time
Length of time series
(a) Posterior mean of } (b) Factor trajectories
(c) Entry predictions
(d) Scalability
Figure 4: Illustrations on (a) the posterior mean of the variance from the SSF dataset (R 10), (b) the dominant components of learned depth-mode factor trajectories from the SSF dataset, (c) the entry value predictions indexed in (17%N, 114.7%E, 30m) of the SSF dataset, (d) the scalability over the length time series on synthetic dataset.
the primary structure of the data. illustrate this, we plotted these two components of the depth- mode factor trajectories at depth 30m in Figure Bk.. As we can see, the trajectories show periodic patterns, which are influenced by the the day-night cycle of ocean temperature. We also compared the predicted curves of CATTE and LRTFR for an entry located at 17 1 and depth of 30m. CATTE outperforms LRTFR, providing more accurate predictions with uncertainty quantifi- cation, even outside the training region (right to the dashed vertical line). This suggests that our model holds promise for extrapolation tasks. The results collectively highlighted the advantage of CATTE in capturing continuous-inden multidimensional dynamics, which is crucial for analyzing real-world temporal data and performing predictive tasks.
CATTE
CATTE w.o. FARD
LRTER (R=5)
DEMOTE (R=5)
Noise Variance
RMSE
0.5
0.305 0.005
0.356 0.010
0.444 = 0.047
0.430 + 0.010
0.516
0.034
1. 0.009
Table 2: Experiments on the robustness of functional automatic rank determination mechanism against the noise on the CA traffic dataset. The results were averaged over five runs.
The incorporated FARD mechanism can reveal the underlying rank of the functional temporal data and prune the unnecessary components of the factor trajectories, enhancing noise robustness. To evaluate its performance, we added Gaussian noise with varying variance levels to the training set of CA traffic dataset and compared the results of different methods, as summarized in Table We disabled FARD by using a simple RMSE criterion as the objective function to constitute ablation study. More detailed results of the basclines with varying ranks
Robustness against Noise:

---

CATTE achieves lower prediction errors than CATTE w.o. FARD,, confirming FARDs benefit. We also demonstrate the robustness of CATTE aganist Laplacian and Poisson noise in Table B (Appendix B.4..
in Table g and Table DI in Appendix H
Teasor size
20 x 20 x 20 25 (An-oder)
20 x 20 x 20 x 20 x 25 -oder)
20 x 20 x 20 x 20 X 20 x 25 (6p
Timets)
0.433
1,101
3.062
Table 3: Per-epoch/iteration running time on different orders of tensors(in seconds).
Sensitivity and Scalability: We examined the sensitivity of CATTE with respect to the variational hyperparamter a.,, 09 and the dimensionality of ODE state J on the CA traffic dataset. The results were given in Table E and Table H respectively. Empirically, CATTE performs consistently well across different af. be and J. We further evaluated the scalability of CATTE with respect to the length of the time series 7 and J. For T, we randomly sampled 25 x 25 x 'I' off-grid index entries from the interval [0, 1] x [0, 1] x [0, 1] using Eq. (2Q) for training. We varied across {50, 200, 500, 1000} and J across {5, 25, 30}. To better quantify the scalability, we employed the simple Euler scheme with fixed step size [64]] for network training. The results shown in Figure Hid) indicate that the running time of CATTE grows linearly with T and is insensitive to J, demonstrating its suitability for large-scale applications.
In addition, we conducted experiments to assess scalability with respect tensor order. We gen- erated synthetic datasets with varying orders while fixing the temporal dimension to 25 and the non-temporal dimensions to 20. From each dataset, randomly extracted 1% of the total data points for training. The average training time per epoch is reported in Table B. The results show that CATTE scales well with tensor order. By decoupling each mode, CATTE translates the addition of modes into an increase in the number of ODE states under our proposed efficiency- driven approach (as discussed in Section I). For example, for 4th-order tensor of size 20 x 20 x 20 25, we only need to solve 20 + 20 + 20 = 60 ODE states, which can be updated simultaneously with a single ODE solver. Additional results the scalability of CATTE provided in Appendix IB.S.
Computational Efficiency: We compared per-iteration runtimes of CATTE and baselines on system with an NVIDIA RTX 4070 GPU, Intel 19-13900H CPU, 32 GB RAM, and 1 TB SSD (Table M Appendix B. CATTE is slightly faster than NONFAT and DEMOTE, and much faster than THIS-ODE.
7 Conclusion
We have presented CATTE, a complexity- model for functional temporal tensor decomposi- tion. CATTE automatically infers the underlying rank of temporal tensors with continuously indexed modes, while achieving state-of-the-art predictive performance and capturing interpretable temporal patterns efficiently. A current limitation of our work is the lack of explicit incorporation of physical laws into the modeling process. In future work, we plan to integrate CATTE with domain-specific physical knowledge, enabling more accurate long-range and wide-area physical field reconstruc- tions.
8 Acknowledgment
The work was supported part by the National Natural Science Foundation of China under Grant 62371418, in part by the Fundamental Research Funds for the Central Universities (226-2025- 00168) and in part by Zhejiang University Education Foundation Qizhen Scholar Foundation.
10

---

References
[1] Richard Harshman et al. Foundations of the parafac procedure: Models and conditions for an explanatory multi-modal factor analysis. UCLA working papers in phonetics, 16(1):84, 1970.
[2] Nicholas D Sidiropoulos, Lieven De Lathauwer, Xiao Fu, Kejun Huang, Evangelos E Papalex- akis, and Christos Faloutsos. Tensor decomposition for signal processing and machine learning. IEEE Transactions on signal processing, 65(13))3551-355) 2017.
[3] Yanqing Zhang, Xuan Bi, Niansheng Tang, and Annie Qu Dynamic tensor recommender systems. Journal of machine learning research, 22(65): 2021.
[4] Shikai Fang, Akil Narayan, Robert Kirby, and Shandian Zhe. Bayesian continuous tucker decomposition. In Kamalika Chaudhuri, Stefanie Jegelka, Le Song, Csaba Szepesvari, Gang Niu, and Sivan Sabato, editors, Proceedings of the 39th International Conference Ma- chine Learning. volume 162 of Proceedings of Machine Learning Reseurch, pages 6235-6245. 6245. PMLR, 17-23 Jul 2022.
[5] Shikai Fang, Xin Yu, Shibo Li, Zheng Wang, Robert M. Kirby, and Shandian Zhe. Streaming factor trajectory learning for temporal tensor decomposition. In Proceedings of the 37th In- ternational Conference on Neural Information Processing Systems, NIPS '23, Red Hook, NY,
USA, 2024. Curran Associates Inc.
[6] Shibo Li, Robert Kirby, and Shandian Zhe. Decomposing temporal high-order interactions via latent odes. In International Conference on Machine Learning, pages 12797-12812. PMLR, 2022.
[7] Zerui Tao, Toshihisa Tanaka, and Qibin Zhao. Undirected probabilistic model for tensor de- composition. Thirty-sevnnth Conference on Neural Information Processing Systems, 2023.
[8] Qibin Zhao, Liqing Zhang, and Andrzej Cichocki. Bayesian ep factorization of incomplete tensors with automatic rank determination. JEEE transactions pattern analysis and machine intelligence, 37(9):1751-177 2015.
[9] Lei Cheng, Zhongtao Chen, Qingjiang Shi, Yik-Chung Wu, and Sergios Theodoridis. Towards flexible sparsity-awand modeling: Automatic tensor rank learning using the generalized hyper- bolic prior. IEEE Transactions on signal processing, 70:1834-1888 1849, 2022.
[10] Piyush Rai, Yingjian Wang, Shengbo Guo, Gary Chen, David Dunson, and Lawrence Carin. Scalable bayesian low-rank decomposition of incomplete multiway tensors. In Eric P. Xing and Tony Jebara, editors, Proceedings of the 31st International Conference on Machine Learning. number 2 in Proceedings of Machine Learning Research, pages 1800-1808, Bejing, China, 22-24 Jun 2014. PMLR.
[11] Ricky TQ Chen, Yulia Rubanova, Jesse Bettencourt. and David K Duvenaud. Neural ordinary differential equations. Advances in neural information processing systems, 31, 2018.
[12] Tamara G Kolda and Brett Bader. Tensor decompositions and applications. SIAM review,
5 51(3):455-500 2009.
[13] Johan Hästad. Tensor rank is np-complete. In Automata, Languages and Programming: 16th International Colloquium Stresa, Italy, July 1 11-15, 1989 Proceedings 16, pages 451-460. Springer, 1989.
[14] Morten Mørup and Lars Kai Hansen. Automatic relevance determination for multi-way models. Journal of Chemometrics: A Journal of the Chemometries Society, 23(7-8):352-363, 2009,
[15] Zheng Wang and Shandian Zhe. Nonparametric factor trajectory learning for dynamic ten- sor decomposition. International Conference on Machine Learning, pages 23459-23469. PMLR, 2022.
[16] Ali Hamdi, Khaled Shaban, Abdelkarim Erradi, Amr Mohamed, Shakila Khan Rumi, and Flora D Salim. Spatiotemporal data mining: a survey on challenges and open problems. Artifi- cial Intelligence Review, pages 1-48, 2022.

---

[17] Yisi Luo, Xile Zhao, Zhemin Li, Michael K Ng, and Deyu Meng. Low-rank tensor function representation for multi multi-dimensional data recovery. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023.
[18] Rafael Ballester-Ripoll, Enrique G. Paredes, and Renato Pajarola. Sobol tensor trains for global sensitivity analysis. Reliability Engineering &amp; System Safety, 083:3113223 Mar 2019.
[19] Shikai Fang, Xin Yu, Zheng Wang, Shibo Li, Mike Kirby, and Shandian Zhe. Functional bayesian tucker decomposition for continuous-indexen tensor data. In The Twelfth Interna- tional Conference on Learning Representations, 2024.
[20] Matthew Tancik, Pratul Srinivasan, Ben Mildenhall, Sara Fridovich-Keil. Nithin Raghavan, Utkarsh Singhal, Ravi Ramamoorthi, Jonathan Barron, and Ren Ng. Fourier features let net- works learn high frequency functions in low dimensional domains. Advances in neural infor- mation processing systems, 33:7537-7547, 2020.
[21] Morris Tenenbaum and Harry Pollard.
Ordinary differential equations: an elementary textbook for students of mathematics, engineering, and the sciences. Dover Publications eBooks, Dover Publications eBooks.
[22] Cheng Zhang, Judith Bütepage, Hedvig Kjellström, and Stephan Mandt. Advances in varia- tional inference. IEEE Transaction on Pattern Analysis and Machine Intelligence, (8):2008 2026, 2019.
[23] Carl Doersch. Tutorial on variational autoencoders. arXiv preprint arXiv: 2016.
[24] Liang Xiong, Xi Chen, Tzu-Kuo Huang, Jeff Schneider, and Jaime G. Carbonell. Temporal collaborative filtering with bayesian probabilistic tensor factorization. In Proceedings of the 2010 SIAM International Conference Data Mining, Apr 2010.
[25] MarkC. Rogers, Lei Li, and Stuart Russell. Multilinear dynamical systems for tensor time series. Neural Information Processing Systems, Information Processing Systems, Dec 2013.
[26] Shandian Zhe, Kai Zhang, Pengyuan Wang. Kuang-Chih Lee, Zenglin Xu, Yuan Qi, and Zoubin Ghahramani. Distributed flexible nonlinear tensor factorization. Neural Information Processing Systems,Neural Information Processing Systems, Apr 2016.
[27] Yishuai Du Yimin Zheng, Kuang-chih Lee, and Shandian Zhe. Probabilistic streaming tensor decomposition. In 2018 IEEE International Conference on Data Mining (ICDM), pages 99- 108, 2018.
[28] Zheng Wang, Shikai Fang, Shibo Li, and Shandian Zhe. Dynamic tensor decomposition via neural diffusion-reaction processes. Advances in Neural Information Processing Systems, 36, 2023.
[29] Alex Gorodetsky, Sertac Karaman, and YoussefM. Marzouk. Function-train: A continuous analogue of the tensor-train decomposition. arXiv: Numerical arrii: Numerical Analysis, Oct 2015.
[30] Daniele Bigoni, AllanPeter Engsig-Kanun and YoussefM. Marzouk. Spectral tensor-train de- composition. Siam Journal on Control and Optimization, Journal on Control and Opti- mization, Aug 2016.
[31] Andrei Chertkov, Gleb Ryzhakov, Georgi Novikov, and Ivan Oseledets. Optimization of func- tions given in the tensor train format. arXiv preprint arXiv:2209. 14808, 2022.
[32] Andrei Chertkov, Gleb Ryzhakov, and Ivan Oseledets. Black box approximation in the ten- sor train format initialized by anova decomposition. SIAM Journal on Scientific Computing, A211--2210- 2023.
[33] Mikkel N. Schmidt. Function factorization using warped gaussian processes. In Proceedings of the 26th Annual International Conference on Machine Learning, page 921928, Jun 2009.
12

---

[34] Eckhard Platen and Nicola Bruti-Liberati. Numerical solution of stochastic differential equa- tions with jumps finance, volume 64. Springer Science & Business Media, 2010.
[35] Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. Pytorch: An imperative style. high-performance deep learning library. Advances in neural information processing systems, 32, 2019.
[36] Diederik P Kingma. Adam: A method for stochastic optimization. 1rXiv;1412.6980, 2014.
arXiv preprint
13

---

NeurIPS Paper Checklist
1. Claims
Question:
Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: We clearly state contribution in the abstract and introduction part of the main paper.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
We discuss the limitations in the conclusion part of the main paper.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are discussed in the paper.
The authors are encouraged to create separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (c.g., independence assumptions, noiseless settings, model well-specification. asymptotic approximations only holding locally). The au- thors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, c.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the ap- proach. For example, facial recognition algorithm may perform poorly when image resolution is low images taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to ad- dress problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers grounds for rejection, worse outcome might be that reviewers discover limitations that aren't it acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
14

---

Justification:
We include the theoretical results with full assumptions and complete proofs in Section of the main paper and Appendix (A.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theo- rems.
The proofs can either appear the main paper or supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be comple- mented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclu- sions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
Justification:
We include all implementation details for reproducibility in the Appendix E.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is a dataset and/or model, the authors should describe the steps taken make their results reproducible verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or the contribution a specific model and empirical evaluation, it may be necessary either make it possible for others replicate the model with the same dataset, or provide access to the model. In general. releasing code and data also be provided via
often one good way accomplish this, but reproducibility detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of model checkpoint, other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all sub- missions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution primarily new algorithm, the paper should make it clear how to reproduce that algorithm.
(b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to re- produce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case au- thors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
15

---

Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Question:
Answer: [Yes)
Justification:
We provide the data and code supplementary material.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.cr
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so No an acceptable answer. Papers cannot be rejected simply for including code, unless this is central to the contribution (e.g., for a new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (nttps:
/
for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing much information possible in supplemental material (appended to the paper) recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: We include implementation details of data and sampling in the experiment section of main paper and Appendix B.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment statistical significance
Question:
Does the paper report error bars suitably and correctly defined or other appropri- ate information about the statistical significance of the experiments?
Answer: [Yes]
Justification:
We report the standard deviation of the results in experiment part main
paper.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
16

---

The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should prefer- ably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not show in tables figures symmetric error bars that would yield results that are out of range (c.g. negative emror rates).
If error bars reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification: We talk about the compute resources in experiment part of the main paper.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g.. preliminary or failed experiments that didn't t make it into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics
Answer:
[Yes]
Justification:
We follow the NeurlPS Code of Ethics in every respect.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., if there is a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification:
We have discussed the broader impacts of our paper in Appendix D
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact spe- cific groups), privacy considerations, and security considerations.
17

---

The conference expects that many papers will be foundational research and not tied particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitiga- tion strategies (e.g., gated release of models, providing defenses addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility MI..
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g.. pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
The paper poses no such risks.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by re- quiring that users adhere usage guidelines or restrictions access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do require this, but we encourage authors to take this into account and make best faith effort.
12. Lieenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [NA]
Justification: The paper does not use existing assets.
Guidelines:
The answer NA means that the paper does existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
assets are released, the license, copyright information, and terms use in the pack- age should be provided. For popular datasets, paperswi thcode com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the cense of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
18

---

If this information is not available online, the authors are encouraged to reach out to the asset's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documenta- tion provided alongside the assets?
Answer: [NA]
Justification: The paper does not release new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/mmodl part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the pa- per include the full text of instructions given to participants and screenshots, if applicable, well as details about compensation (if any)?
Answer: [NA]
Justification: The paper does not involve crowdsoureing or research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contri- bution of the paper involves human subjects, then as much detail as possible should be included the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, cura- tion, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper deseribe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification:
The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equiva- lent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
19

---

Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology. scientific rigorousness, or originality the research, declaration is not required.
Answer: [NA]
Justification: The core method development in this research does not involve LLMs any important, original, or non-standard components.
Guidelines:
The answer NA means that the method development in this research does involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy ottpas///eaarrrsscccc//oorrreere for what should or should be described.
20

---

APPEXNDIX
A Details of derivations
A.1 Log-marginal likelihood
(D) = q/4, A, In) / x(,, p(u, x,,,),TD)) p(u, X, T
aldddd
= ((u.A.T)) in puu, 2) OldAdT
p(u, X, r(D)q( A,
=faUAA p(u,,,TT71 dudddr / q(U,,,T) >((1) alddddr (21) q(U,),T))
q(,
= [In plu,,, ( EQCLAAF) p(U, T)ID T C(g) KL(g(U, X,
q(u, A,
A.2 Lower bound of log-marginal likelihood
pl, (h T, D)] - E g(1, A, r)]
((g) = E [In
[n + E Eq(UA (lu X)] Eu
(n + E E p((11))
+ In p())
IIn p((T)
(22)
((1) ((2)
((T))
p((12); (L(((()) - (L(((())l)(()))
(n A, + EqULA
q(U)
The first term of evidence lower bound (posterior expectation of log- likelihood) can be written
N In((T) +
N E3
NV
E (I =
[n
E
1T c u (1p (.))
2
2
m=1
N
N (b()) - In
1
(23)
In (T) +
Σ
(1V
1 u (TH
2
2
n=1
where (-) is the digamma function. The posterior expectation of model error is:
[y T
u (()) = u 2y T @6
g" (2)
+ T @ vee(g (i (n)g (1h LT) TT +021).
(24)
If a 0, then posterior expectation of model error becomes (yn 1 @ u ti),
The second term of evidence lower bound can be written as
Eq( [In p(1 )( / qu4, A) p(([)) auda / 9(U, ( / q(()))
((1)
g((1)
/ ((4) In p(u Ey(A) (A))dlt - -1 g((4) In ((1)
(25)
-KL((())))(() Ey())) - N
Br 0,,
Or (0² + (.))) 1],
)
Br
where g. (f%) is the r-th element of g((2
"The KL divergence between two Gaussian distributions p(æ) N (xxlu, of) and q(x) (x
can be computed using KL(pllg) = |n[ o2 (7t64-m22) 1). Detailed derivation can be found in Appendix A
21

---

The third term of evidence lower bound can be written as
R 3r KL(g(A)llp(A)) Σ
I(ao) bo T(a,)
+ (ax, ao)bbaa)) - (bo - bi) ar
ln
In
(26)
3,
The fourth term of evidence lower bound can be written
I((o))
(L(((T)IlP(T)) = In in
do T(p) + (p co)o(() - (do 2)
(27)
A.3
KL divergence of two Gaussian distribution
The Kullback-Leibler (KL) Divergence between two probability distributions p and q is defined as:
(L(pl1)) = - p(x)
p(x)
dr.
g(x))
Let p ~ ((u,, and ~ N N(2,, o.)), where the probability density functions (PDFs) are given by:
1
(r- 11)
p(x)
exp
v 2
20
(x: - 122 )2
g(x)
exp
V
20
Substitute the Gaussian PDFs into the definition of KL divergence:
(- (
(L(pll4) - {
(x 1-112
exp
207
exp
In
dx.
2
20²
(x (w- 22
exp
2
20
Simplify the logarithmic term:
p(z)
2na
(xr 22)
222
+
((x)
1
20
20
2
02
(x
111)
(xr - 222
In
+
207
20
Thus, the integral for KI. divergence becomes:
KL(pllg) = [
(- (x
(x-
mn)
02) -
02
(-(x--
11)
(x - 122 ))
exp
In
+
dr.
V 2
20
20
20
Simplifying the Integral: We can now break the integral into two parts: 1. The constant term:
exp
(x
In
02
dr.
2mO
20
/2n07 evaluates to:
Since
exp
(x- |
2n
is the PDF of Gaussian distribution, its integral is 1, this term
02
In
2. The difference of squared terms:
m1)2
(
(x p1)²
(x U22
exp
dx.
2TT
20
20
20
'The KL divergence between two Gamma distributions p(x) Gamma(z jaz, by) can be computed using KL(p q) ag In by In [(a2)
Gamma(z|aa, b1) and g(z)
(as a2) (b2 b1) 21
F(a))
Detailed derivation can be found in A
22

---

This term can be split into two parts: The first part involves f1. and after calculation, it simplifies to:
of
2
2
The second part involves the difference between la1 and p2, and after calculation, it simplifies to:
(0i
u22
20
Combining all parts, the KL divergence between two Gaussian distributions is:
of
1
(n
22)²
KL(pllg) - In
20
2
20
A.4 KL divergence of two Gamma distribution
The Kullback-Leible (KL) Divergence between two probability distributions p and q is defined as:
111))) KL ll g) = - p(x:) In
p(x)
dx.
((x))
Let p ~ Gamma(aj, b1) and q ~ Gamma(a2, b2), where the probability density functions (PDFs) with rate parameters are given by;
p(x) = 1 -1 exp (-b1x)
x 2 0,
I(a1)
bg x 42 exp (bbz)
((x)
x 2 0.
((a2)
Substitute the PDFs of the Gamma distributions into the definition of KL divergence:
expí bja) b x
b a - cxp (-b1z)
Maa
KL(p ll
In
dr.
I(a1)
by 22--2- expí - b2x)
[(02]
Simplify the logarithmic term:
p(æ)
= 10'001 - exp (- b1xx)
[(a2)
In
(xx)
6 202-0=11 exp (-byx)
I((a1)
T((2)
= = a2) a2) In(x) + (- x + byx) + In
+ (a1 In(bi) - lnbb))).
[((a))
Thus, the integral becomes:
1 1 exp a-) (a1-az) az) In(x)) + (b2 61)x + In
T(a2)
KL(p 1)
+ (a, In(b1) a₂ In(bb)) dx.
I(a1)
[(a,)
now break this into four separate integrals.
20 b f exp (-b)a))
I
(a1 a2) In(z)dz.
T(a1)
This integral can be solved using the properties of the Gamma distribution and the digamma function (a)):
11 = (a2 - a1) (In(b)) - (aaa)) where (a) is the digamma function, the derivative of the logarithm of the Gamma function.
12 8 I exp exp (-b1z)
(b2 by )x.
r(a:)
After performing the integration, obtain:
17 = (b2 - b1) T(a + +1)
1
(b2 by b1)01 'n
I(a1) b
23

---

b -l exp (-b1z)
T((22)
13
In
[(a1) dr.
r(a1)
Since this is constant term, we can immediately evaluate it:
T((2)
13 = (((a))
b x a
exp (-b1z)
(a1 In(bi) a2 ln(2))d)
T(a1)
This integral simplifies to:
44 4 a2 In(61) d2 In(b)).
Combining all the parts, the KL divergence between two Gamma distributions with rate parameters is:
T(22) T(ai)
KL(p q) = (a2 a1) (In(bi) vaa)) (b2 bijaa by + In
+ In(bi) a2 In(b2).
A.5 Predictive distribution
Through minimizing the negative log-marginal likelihood with observed training data, infer the distributions of the latent variables q, with which a predictive distribution can be derived. Given index set (i.... 2 (p), are to predict the corresponding value, have:
PD) ~ / pu (n g. D ( 1K
Ro (a(T)d((((())) ((
1K 1 (
1k=))dr
= =/ N( 1u @
K
o
)d(
22., (q(T)
k=1
K
/ / N( (u
II k=1
N N Ig (P2,)))) o M)d(( (aamma(rlo..)dr
/ N( 1u
Ww , Ig tp) o I)d(
II Nu
(Ch, tp). n)d(u (ammaarlpp.))
k#1
= N(
& u Tul T W . 1g (g,, o2)),
Rxll
II
N(u 1g (p),tt)) o l) Gamma (Tlp. 1)d
(28)
k#1
=/ʃ N
(i, 1p),, - +
)
c up
k#1
II (u 1g (Ph 4) o Mi (Gamma(Tll.l) k#1
ff (V IT (g((1))))) + o²
K
c
(2,44)
gz (2)))))) Gamma (rlp, ()T kgj
1T (g o
Σ
g
We found the prediction distribution follows the student's s-t distribution.

---

A.6 Closed Form of Predictive Distribution
After obtaining the variational posteriors of the latent variables, we can further derive the predic- tive distribution of the new data with arbitrary indexes. Given the index set f & tp} for prediction, we can obtain the variational predictive posterior distribution, which follows a Student's I-distribution (See Appendix A.3 for more details):
pu D) ~ TUpll Sp. Up);
Mp = 1 (cg
Vy 2p,
(29)
K
1 + 0² Σ
s g (ih
1 @ (2,4)) kz)
where Up. Sps My is the mean, scale parameter and degree of freedom of the Student's t-distribution, respectively. The closed-form predictive distribution is great advantage for the prediction process, as it allows to do the probabilistic reconstruction and prediction with uncertainty quantification over the arbitrary continuous indexes.
25

---

B Additional experiment results
B.1 Experiment settings:
synthetic data
The CATTE was implemented with PyTorch [35] and torchdiffeq library lbttps://github hoa//rrgicchen//oochhiiie.
We employed a single hidden-layer neural network (NN) to pa- rameterize the encoder. Additionally, we used two NNs, each with two hidden layers, for derivative learning and for parameterizing the decoder, respectively. Each layer in all networks contains 100 neurons. We set the dimension of Fourier feature M 32, the ODE state J and the initial number of components of the latent factor trajectories R = 5. The CATTE was trained using Adam [36]] optimizer with the learning rate values of learnable parameters {ar, 3r },) 11 o², 1 are set to 1e (so that all the initial posterior means of A equal 1). We ran 2000 epochs, which sufficient for convergence.
5e The hyperparamters {ah, b --1==1 ea, d and initial
B.2 Experiment settings: real-world data
For THIS-ODE, we used two-layer network with the layer width chosen from {50, 100). For DEMOTE, we used two hidden layers for both the reaction process and entry value prediction, with the layer width chosen from (50, 100). For LRTFR, we used two hidden layers with 100 neurons to parameterize the latent function of each mode. We varied R from (3. 5, 7} for all baselines. For deep-learning based methods, all used tanh activations. For FunBa'T, we varied Matémn Kernel {1/2, 3/2} along the kernel parameters for optimal performance for different datasets. We use ADAM optimizer with the learning rate tuned from (5e-4, 1e- 5e le 12-
B.3 Rank learning curves of different datasets
1000
800
4,000
600
comporent
Au componere
2.667
comppeent
curpprrant
200
1.333
500
1000
1500
2000
0.000
0
500
1000 Epoch
1500
2000
Epoch
(a) Power of learned factor trajectories.
(b) Posterior means of A.
Figure 5: Rank learning curves of the synthetic data.
Fig. plots the rank-learning curves during the gradient descent iterations of the synthetic data. That is, the evolutions of (a) the power of R components of the estimated posterior mean of the factor trajectories B and (b) the values of estimated posterior mean of {A, R Note that the power of r-th component of factor trajectories is conditioned by X (as shown (LE) and Remark I), and we plot the pair with the same color. One the factor trajectories are forced to 0, which aligns with the increments of 4 corresponding X,s. And manually exclude the four components, which will not affect the final prediction results Only the third component (r = 3) is activated after convergence and Ag settles at a small value correspondingly. This indicates that our method successfully identifies the true underlying rank (i.e.. 1) of the synthetic data while effecttiely pruning the other four components. Fig. (6) plots the rank- learning curves of the CA traffic, Server and SSF datasets respectively. In the same sense, we can infer that the revealed ranks of these three datasets are 5,7.7 respectively.
that as the epoch increases, the power of 4 components of
WWe define the power of r-th component of factor trajectories as: Σk Sa. [.(x (ix, digd,, which represents the contribution of the r-th component to the final output. The power can be approximated using
Ex (g. (
Our criterion for excluding the r-th rank is when E(A,) is large and power EN,, Σk I (32, t.)) relatively small.
26

---

104
5,000
104
Power
104
componentt
3.333
101
compunor.
comppnon::
compurere
componet::
comppnenn
31. cont03n111
109
1.667
10-1
0
250
500
750 1000
1250 1500 1750
0.000
Epoch
500
1000 Epoch
1500
2000
(a) Power of learned factor trajectories (CA traffic).
(b) Posterior means of À (CA traffic).
10a
4,000
104
Rower
10a
10a
1.333
104
1000
1750
0.000
500
1000 Epoch
1500
2000
(c) Power of learned factor trajectories
(d) Posterior means (Server).
(Server).
10a
4.000
10a
Power
103
Ze componert
comparentt
compareey compuree comppeee campppee 38g companat
2.667
component
compenent
109
comperert
1.333
10-1
10
500
1000
1500
2000
0.000
Epoch
500
1000
1500
2000
Epoch
(e) Power of learned factor trajectories
(f) Posterior means of X (SSF).
(SSF).
Figure Rank learning curves three datasets.
B.4
Noise robustness
Here, we show the robustness of our proposed FARD against the noise on the CA traffic dataset by comparing with the baselines with varying ranks Tab. ann Tab. EL Then, show the robustness of CATTE against varing levels and types of noises on Tab. H The results shows that CATTE do not show significant performance degrade with varing levels of noises and even non-Gaussian noises, demonstating its robustness.
CATTE
CATTE W.o. FARD
LRTFR (R=5)
DEMOTE (R=5)
Noise Variance
MAE
0.5
0.134 1 0.001
0.169 1 0.022
0.213 1 0.007
0.148 1 0.003
0.182 1 0.006
0.219 0.006
0.305 0.009
0.219 1 0.034
Table 4: Extra experimental results the robustness of functional automatic rank determination mechanism against the noise on the CA traffic dataset. The results were averaged over five runs.
27

---

LRTFR (R=7)
DEMOTE (R=7)
LRTFR (R=10) DEMOTE (R=10) LRTFR (R=15) DEMOTE (R=15)
Noise Variance
RMSE
0.484 0.023
0,423 0.019
0.469 0.053
0.4375 + 0.005
0.475 0.018
0.455 + 0.005
0.5475 + 0.011
0.517 = 0.020
0.621 + 0.038
0.547 0.024
0.708 + 0.013
0.552 + 0.003
Noise Variance
MAE
0.224 0.019
0.157 = 0.006
0.232 0.012
0.148 0.001
0.245 0.011
0.140 + 0.002
03335 0.018
0.209 = 0.020
0.402 + 0.032
0.312 + 0.013
0.469 + 0.0135 0.319 + 0.0025
Table 5: Extra experimental results on the robustness of functional automatic rank determination mechanism against the noise on the CA traffic dataset. The results were averaged over five runs.
Gaussion noise
RMSE
MAE
=0000
0.056 0.004
0.045 0.003
=0=0.10
0.090 0.005
0.068 = 0.005
0==0.15
0.112 0.006
0.086 = 0.005
Laplacsian noise
RMSE
MAE
o0=0.05
0.065 0.006
0.052 = 0.006
0'=0.10
0.094 0.008
0.069 0.006
0==0.15
0.117 + 0.07
0.089
0.007
Poisson noise
RMSE
MAE
=0.05
0.064 0.004
0.049
0.005
=010.
0.096 0.010
0.073
0.009
00.15
0.115 + 0.012
0.081
0.01
Table 6: Extra experimental results on the robustness of functional automatic rank determination
mechanism against various noise on the synthetic dataset. The results were averaged over five runs.
B.5
Running time
Here we provide the comparisions of running time different methods Tab
CA traffic
Server Room
SSF
THIS-ODE
283.9
144.8
158.9
NONEAT
0.331
0.81
0,67
DEMOTE
0.84
7.25
1.08
FunBaT-CP
0.059
0.027
0.075
FunBaT-Tucker
2.13
3.20
2.72
LRTFR 0.098
0.178
0.120
CATTE
0.227
0.83
0.247
Table Per-epoch/iteration running time of different methods (in seconds).
B.6 Hyperparameter Analysis
We first test the influnence of the dimension of ODE latent state in Tab. & We observe that the final results are robust to the selection of J.
RMSE
0.279
0.294
0.284
0.290
MAB
0.090
0.088
0.085
0.088
Table 8: Performance of CATTE under different J on the CA traffic dataset. The results were averaged over five runs.
28

---

Then, we test influnence of the hyper- parameters af and by in Tab. N. results are robust the selection of a. and 6..
We observe that the final
= = le"
Or
8 = le-
Gr le" 2e
RMSE
0.056 + 0.004
0.056 = 0.003
0.054 + 0.005
MAE
0.045 + 0.003
0.044 + 0.003
0.044 + 0.004
Table 9: Performance of CATTE under different af. and BΩ. on the synthetic data experiment. The results were averaged over five runs.
B.7 Additional visualization results with other baselines
Here, we show more visualization results predictions on different methods and datasets on Fig. Fig. & and Fig. B. Our method yields qualitatively better visual results compared to the baselines.
Ground
45
Ground tMch
CAFTE
CATTE toust
LFTFR
LFTFR (R
NONEST
NOWAT (5a
Demore (R-T]
Demote (8-T)
Thaining
Theiring points
-05-
(a) (1,4,13)
(b) (5,3,12)
0.6
0.00
truth
0.
He x
-0.4
(c) (1,3,1)
(d) (0,16,6)
Figure 7: Additional visualization results the CA Traffic dataset different coordinates.
B.8
Additional results on scalability
To further demonstrate the scalability of CATTE, we conducted synthetic experiments across varying configurations of timestamps x spatial points. All experiments employed the same network architec- ture. with 5% of the total data points randomly selected for training. Therefore, the size of the training dataset ranges from 10$ to 10?, We hereby report the average training time (in seconds) per epoch Table IIO.
One can see that our method also seales well on the spatial mode since our method decouple each mode and convert the expanding spatial resulotion to the increase of the ODE states. For example, for the spatial resolution 50 x 50, we only need to solve 50 + 50 100 ODE states. Likelywise, for the spatial resolution 200 200, only need to solve 200 200 400 ODE states. We will supplement the results in the latest version.
29

---

-02
toual
Ite/1
(R-7)
NONPAT
Demate
"haingg
-06-
(a) (6,2,5)
(b) (1.1.1)
Ground buc
Ground buth
CATTE toust
CATle (ouns)
LFIF It
LFTPA (he
NOWFT (Sx1)
NONPAT (R-2]
Demate (Su:))
Demate (R-21
X
Thaining points
Teringg portes
Time
-09
ine
(c) (5,6,2)
(d) (2,6,3)
Figure 8: Additional visualization results on the Server Room dataset at different coordinates.
1538
Ground buth
foora)
CATTE (ours)
LFTFR (h-1)
(tw?]
Rurea1 CP (R=T)
PurBaTTuucee (8a7)
X
Training polnts
1535.2
1534
(a) (1.1.1)
(b) (5,1,10)
15.00.
Ground truth
CATEE louns) LETRR (R
dhw?)
FundaNTIPP (R
Aunfia-C (RaT)
Funtenthiicer (R:71
X
Training po 1915
Thatning
1535.
13344
Time
15345
Time
(c) (3,2,7)
(d) (7,10,15)
Figure 9: Additional visualization results on the SSF dataset at different coordinates.
C More Comments on the functional automatic rank determination mechanism
Our proposed functional automatic rank determination mechanism is different from previous method|[4, g. MJ which can not be directly applied for functional tensors. The differences are:
30

---

timestamps spatial resolution
x 50
100 x
200 x 50
100 x 100
200 x 200
7 100
0.398
0.673
1.011
1.016
2.392
T 200
0.814
1.088
1.670
1.714
5.081
T 500
1.688
2.162
3.755
3.721
10.79
Table 10: Per-epoch/iteration running time on different size of tensors(in seconds).
a) Derivation of ELBO: Previous methods predefine separate latent factors and other latent vari- ables (e.g.. X) as variational parameters and then derive the ELBO. However, maintaining these parameters quickly consume memory as the dataset grows. Instead, we characterize the variational posterior mean of the latent factors using newly proposed continuous-indexed latent ODEs. This not only distinguishes our ELBO derivation from earlier methods but also overcomes scalability issues when handling large datasets.
b) Optimization of ELBO: In previous methods, seperate variational parameters are updated it- erativelyadjusting one variable time while keeping the others fixed which limits the ability leverage distributed computing resources. In contrast, we use functional parameterization of the pos- terior distribution and derive closed-form ELBO that can be efficiently optimized using gradient descent. This allows all variational parameters to be updated simultancously, making our method highly parallelizable and well-suited for GPUs.
Furthermore, previous methods struggle to infer out-of-bound data points while our model can han- dle them effectively.
Overall, CATTE integrates the strengths of deep learningscalability. powerful representations, and flexible architectureswith the benefits of Bayesian learning, including robustness noise, uncer- tainty quantification, and adaptive model complexity, to offers an elegant solution for generalized temporal tensor decomposition.
D Impact Statement
This paper focuses on advancing temporal tensor decomposition techniques to push the boundaries of tensor decomposition. We mindful of the broader ethical implications associated with techno- logical progress in this field, Although immediate societal impacts may not be evident, we recognize
the importance of maintaining ongoing vigilance regarding the ethical use of these advancements. is crucial to continuously evaluate and address potential implications to ensure responsible develop- ment and application in diverse scenarios.
31


Reviews:
[{"review_id": "I9oFpf98ay", "reviewer": "Reviewer_a7FC", "summary": "The authors propose a new temporal tensor decomposition method for tensors with continuous modes. Instead of treating time as a separate mode, their method allows latent factor functions, parametrized by neural ODEs, to depend jointly on both the mode index and continuous time. The mean-field variational posterior proposed admits analytically tractable components, allowing for efficient inference. The authors compare their method empirically with other temporal tensor decomposition methods.", "questions": "- L78: Brief mention of rank not having to be scalar (as in Tucker decomposition) would be helpful\n- L94: I think \"automatically determined\" is not a good description of the authors' methodology. See my comments about this below.\n- L105: \"... it is inadequate for...\" Why not? Please explain more concretely.\n- L115: \"... which requires special treatment [16].\" Why? Please explain.\n- L119: What is the best way to handle discrete modes with the proposed method? \n- L156: Please express any potential downsides, if any, of this efficiency-motivated approach.\n- L164: Please clearly state and discuss why the current modeling approach is preferred over a GP for the factor functions, making explicit the trade-offs involved.\n- L199: Please direct the interested reader to the full derivations for the ELBO terms and the factor updates.\n- L186: The mean-field assumption is commonly utilized in variational inference. However, given the potentially intricate interactions between factors, index variables, and time, please discuss the potential problems with this assumption. For example. would this pose a problem in modeling temporal tensors with hierarchical seasonality (months, weeks, days)?\n- L248, Figure 2: Why would the uncertainty be very low in Fig 2, at the top of the second hill?\n- L311: Please discuss why ELBO-based model selection is not utilized, and compare the existing method for model selection with it (in a simpler setting, if needed). You can defer the second part to camera-ready (if accepted) if there are time/compute limitations.", "limitations": "The authors are recommended to express the limitations of their approach taking into consideration the feedback provided above.", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "## Strengths\n- The paper puts forth a clear problem, coherently expresses shortcomings of existing methods, develops a method that addresses these shortcomings, and demonstrates its favorable performance against reasonable baselines. Overall, the paper is written well.\n- The solution offered addresses the original problems adequately. The neural ODE factor functions allow for expressive and smooth temporal dynamics. The probabilistic model and the proposed inference scheme are valid and appropriate, allowing for expressive yet efficient modeling.\n- The empirical investigation is detailed and compares the performance of the method to important baselines. Additional experiments investigate the interpretability of the method, as well as details of rank determination\n\n## Weakness\n- Although overall the proposed method is well-motivated, arguably important modeling choices, trade-offs involved therein, and explicit justification for the choices made are not always presented in sufficient detail. The next section provides more details for this.", "was_revised": true, "final_justification": "The authors provided answers for my remaining concerns, I maintain my positive score.", "rebuttal": "Thank you for your strong support! Here are our responses. C: comments; R: response.\n\n\n\n>L78: Brief mention of rank not having to be scalar (as in Tucker decomposition) would be helpful\n\nR1: Thank you for your suggestion! We will mention it in the latest version.\n\n\n>C2:L105: \"... it is inadequate for...\" Why not? Please explain more concretely.\n\nR2: Good question! Model (3) incorporates temporal info by modeling continuous timestamps into latent factors (i.e., $\\mathbf{u}(t)$), but it cannot handle continuity in other modes (e.g., spatial coordinates). As a result, (3) does not support learning continuous variations (like smooth changes along latitude/longitude or physical parameters) in non-temporal modes.\n\n\n>C3:L115: \"... which requires special treatment [16].\" Why? Please explain.\n\nR3: Great question! Model (4) is a fully factorized form, and treats all modes (including temporal) independently on an equal footing. \n\n While this approach allows continuous indexing across all modes, **it can be overly simplistic by overlooking the fact** that the temporal mode often exhibits much sharper and more complex fluctuations than other modes (e.g, longitude, latitude). This phenomenon  is evident in the dataset we used and is also supported by the literature [16].\n\n\n\n>C4:L119: What is the best way to handle discrete modes with the proposed method?\n\nR4:  In this work, we focus on modeling physical dynamics where the mode indices are inherently continuous. For data with discrete modes, one possible extension is to incorporate ideas from tensor decomposition techniques commonly used in other domains, such as recommender systems, to adapt our model accordingly.\n\n>C5:L156: Please express any potential downsides, if any, of this efficiency-motivated approach.\n\nR5: Good point! The efficiency-driven approach is designed to enable the GPU to batch-process the states of multiple indices. This reformulation is equivalent to the original formulation, which treats each state independently. Therefore, it does not entail any loss in expressiveness or performance.\n\n\n\n>C6:L164: Please clearly state and discuss why the current modeling approach is preferred over a GP for the factor functions, making explicit the trade-offs involved.\n\nR6:  Good question! Neural ODEs have several advangtages over GPs in terms of scalability, flexibility, expressiveness.\n\n* GPs have a computational complexity of $O(N^3)$. To make it practical, various techniques, such as sparsification and conversion to state-space models (as done in FunBaT), are used, further complicating implementation. In contrast, Neural ODEs scale linearly with the number of data, making them a more straightforward choice for large-scale problems.\n\n* Neural ODEs and GPs are both strong approaches for modeling dynamics. However, the expressiveness of GPs heavily depends on the choice of kernels and hyperparameters, which often requires domain expertise and makes hyperparameter optimization challenging. In contrast, Neural ODEs can achieve high expressiveness more easily by simply using MLPs.\n\n\nOne key advantage of Gaussian Processes is their ability to provide uncertainty quantification—a feature that is absent in standard neural ODEs. However, this gap is bridged in our proposed framework, which incorporates uncertainty modeling within a principled probabilistic setting.\n\n>C7:L199: Please direct the interested reader to the full derivations for the ELBO terms and the factor updates.\n\nR7: Thank you for the suggestion. In the revised manuscript, we will direct interested readers to Appendix A, which contains the full derivations of the ELBO terms and the corresponding factor updates.\n\n\n>C8:L186: The mean-field assumption is commonly utilized in variational inference. However, given the potentially intricate interactions between factors, index variables, and time, please discuss the potential problems with this assumption. For example. would this pose a problem in modeling temporal tensors with hierarchical seasonality (months, weeks, days)?\n\nR8:  Good question! We agree that the mean-field assumption simplifies variational inference by assuming independence among latent variables, which can lead to suboptimal posterior estimates. However, in our model, we leverage neural networks to model the posterior means of the latent functions, thereby enhancing their representatitive capacity. The incorporated neural ODEs are effective at capturing complex temporal patterns, such as hierarchical seasonality. Extensive experiments confirm that our approach handles complex temporal tensors well. For empirical evidence, please refer to Appendix B.7, which includes results on datasets with rich seasonal structures, where our method performs robustly.\n\n\n\n>C9:L248, Figure 2: Why would the uncertainty be very low in Fig 2, at the top of the second hill?\n\nR9: Fig.2(b) shows the temporal trajectory indexed at $(0.992, 0.982)$. The peak of the second hill does not contain any training points. However, in a very near trajectory (i.e., at $(0.986, 0.982)$), some training points are observed around the second hill. Our model successfully captures the continuity of the underlying function, resulting in a correspondingly low uncertainty in this region.\n\n>C10:L311: Please discuss why ELBO-based model selection is not utilized, and compare the existing method for model selection with it (in a simpler setting, if needed). You can defer the second part to camera-ready (if accepted) if there are time/compute limitations.\n\n\nR10: Thank you for the suggestion. ELBO-based model selection involves multiple trials, which can be time-consuming. We will compare the existing method for model selection in the latest version."}, {"review_id": "9rn0vdKvt6", "reviewer": "Reviewer_MW7k", "summary": "In this work, the authors try to solve the problem of temporal tensor decomposition with continuous indices in all modes. The CATTE algorithm is developed by employing the encoder-decoder structure and neural ODE to model the factor trajectories. An automatic tensor rank determination is utilized. A sampling-free variational inference algorithm with a closed-form ELBO is proposed. Experimental results on both synthetic and real-world datasets demonstrate the desired performance of the proposed method.", "questions": "1. The main concern of this paper is the limited novelty of the proposed method. As the encoder-decoder structure and neural ODE are common methods for modeling dynamical systems, the current algorithm seems to be an application to the temporal tensor decomposition domain (Specifically, the ODE has already been applied to temporal tensor decomposition [6][28]). Also, the automatic tensor rank determination method has already been developed. The author should give a clear statement on the contribution of the proposed method, such as a more comprehensive related work on encoder-decoder structure and neural ODE, and a detailed list of contributions for the proposed method.\n\n2. How does CATTE scale with the size or dimension of tensors (e.g., 5-order or higher-order tensors and $100\\times100\\times500$ tensors)? Are there computational bottlenecks, especially with the ODE solvers or variational inference?\n\n3. As the author shows the robustness to different noise distributions, it is interesting how about the robustness when the data contains missing entries.\n\n4. Are there any smoothness (stationary) assumptions on the temporal dynamics of the tensor? How is the performance of the CATTE if there is a sudden change?\n\n5. It seems the ref. [28] and ref. [34] are the same. Please check if there are some mistakes.", "limitations": "Yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strength:\n1. A new method that generalizes the modeling the temporal tensor data with continuous indexes in all modes.\n2. Experiments show good performance and robustness across diverse datasets.\n\nWeakness:\n1. The novelty is limited as the encoder-decoder structure, the neural ODE, and the automatic tensor rank determination method have already been used in modeling dynamic systems or tensor decomposition.\n2. The scales and dimensions of the tensor data in experiments are relatively small.", "was_revised": true, "final_justification": "Based on the authors‘ responses, I would keep my score as borderline accept.\nThe main concerns, questions 1 and 2, are properly addressed. The author gives an additional table to highlight the advantages of the proposed method compared with existing methods. Further, the experiments on large tensors are carried out, demonstrating the scalability of the proposed method.", "rebuttal": "Thanks for the careful review! We address the comments below (C: comment, R: response)\n\n\n>C1:About the novelty and contribution\n\nR1: We appreciate the reviewer’s concern. **Our main contribution is to develop an efficient and scalable solution for reconstructing temporal tensors from sparse, noisy, and continuously indexed observations**. This setting  is both *practically significant and technically underexplored*.\n\nWe select three complementary components, i.e., Fourier features, Neural-ODEs, and ARD sparsity, as they are each well-suited to different aspects of this challenging task. However, we emphasize that simply combining these components  does not staightforwardly lead to a functioning model. \n\nTo address this, we propose several methodological innovations to ensure their **compatibility, integration, and effectiveness** within a unified Bayesian  framework. These design choices are key to enabling scale and robust performance under sparse, irregular and noisy data conditions.\n\n\n\nWe are to highlight our novelty and contribution:\n\n**a) Continuous-indexed latent ODE:** We encodes continuous spatial indexes as learnable Fourier features and employs neural ODEs in latent space to learn the temporal trajectories of factors, which enables a new continuous-indexed dynamics modeling.   We also propose an efficiency-driven approach that concatenates the states of multiple indices across different modes and updates them simultaneously using a single ODE solver, thereby accelerating the learning of latent ODE states.\n\nOur framework fundamentally differs from related ODE-based methods [A1, 6, 8], all of which lack support for continuous-indexed temporal tensor modeling. Specifically, [A1] combines a variational autoencoder with an ODE model to handle multivariate time series.  [6] uses time-invariant latent factors as conditional inputs to an ODE. [28] employs a multipartite graph to model cross-mode interactions. Although both target temporal tensors, they also face scalability limitations.\n\n\n\n\n**b) Functional tensor rank automatic determination:** To the best of our knowledge, CATTE is the first work to achieve automatic rank determination in functional tensors, while eliminating the scalibitly and representation issues of previous methods [8,9]. The differences are:\n* Derivation of ELBO: Previous  methods [8,9] predefine separate latent factors and other latent variables (i.e., $\\boldsymbol{\\lambda},\\tau$) as variational parameters and then derive the ELBO. However, maintaining these parameters quickly consume memory as the size of  tensor grows. \nIn CATTE, we replace separate latent factors with latent factor functions and characterize their variational posterior mean using the newly proposed continuous-indexed latent ODEs. This effectively resolves the scalability issues encountered when handling large datasets.  Consequently, we contribute to derivate a new closed-form ELBO.\n\n* Optimization of ELBO: In previous methods [8,9], seperate variational parameters are updated iteratively—adjusting one variable at a time while keeping the others fixed— which limits the ability to leverage distributed computing resources.\nIn contrast, CATTE use functional parameterization of the posterior distribution. It improves the representive power of posterior estimation. Also, through it, a closed-form ELBO can be derived and can be efficiently optimized using gradient descent. This allows all variational parameters to be updated simultaneously, making our method highly parallelizable and well-suited for GPUs.\n\n\nOverall, CATTE  integrates deep learning’s expressive power and architectural flexibility with Bayesian learning’s robustness, uncertainty quantification, and adaptive complexity, providing an effective and principled solution for the target task. **We list the comparison of CATTE\nand other main  methods in the following table:**\n|   Property / Methods   |  **CATTE** | ThisODE|Functional tensor methods | Bayesian neural ODEs | Factor-evolving tensor methods |\n|:-----:|:-----:|:-----:|:-----:|:-----:|:-----:|\n|Continuous  dynamics modeling | **Yes** | Yes | Partial (tend to underfit the complex dynamics) | Yes|Yes|\n|Continuous-indexed  modes modeling | **Yes** | No |Yes| No|No|\n|Scability | **High** | Low |High| Low|Low|\n|Complexity self-adpation| **Yes** | No |No| No|No|\n|Uncertainty-aware| **Yes** | No |Partial| Partial|No|\n\n We will supplement the above dusscussions  in our paper.\n\n[A1] Rubanova, Yulia, Ricky TQ Chen, and David K. Duvenaud. \"Latent ordinary differential equations for irregularly-sampled time series.\" Advances in neural information processing systems 32 (2019).\n\n\n\n>C2:How does CATTE scale with the size or dimension of tensors (e.g., 5-order or higher-order tensors and  $100 \\times 100 \\times 500$ tensors)? Are there computational bottlenecks, especially with the ODE solvers or variational inference?\n\nR2: Good question! We report the experiment with respect to the scale with the oders of tensors.  We generate synthetic dataset with different orders while fix the dimension of temporal mode to 25 and non-temporal mode to 20. We  randomly extract 1% of the total data points for training.\nConsistent with Fig. 4, we report the average training time (in seconds) per epoch.\n\n\n\n|   tensor size   |  $20 \\times 20 \\times 20 \\times 25$ (4-order) |$20 \\times 20  \\times 20 \\times 20 \\times 25$ (5-order)| $20 \\times 20  \\times 20 \\times 20 \\times 20 \\times 25$ (6-order) |\n|:-----:|:-----:|:-----:|:-----:|\n|Time per Epoch | 0.433s |1.101s|3.062s|\n\n\nCATTE scales well with the tensor order. CATTE decouples each mode and translates the addition of modes into an increase in the number of ODE states with our proposed efficent-motivated approach.\nFor instance, for a 4th-order tensor of size $20 \\times 20 \\times 20 \\times 25$, we only need to solve $20 + 20 + 20 = 60$ ODE states using three derivative networks: $h_{\\theta_1}$, $h_{\\theta_2}$, $h_{\\theta_3}$. All the $60$ ODE states can be update simutanously with  ODE solver. Similarly, for a 6th-order tensor of size $20 \\times 20 \\times 20 \\times 20 \\times 20 \\times 25$, only $20 \\times 5 = 100$ ODE states need to be solved, with two additional derivative networks: $h_{\\theta_4}, h_{\\theta_5}$.\n\n\n\n\nRegarding scalability with respect to the spatial and temporal modes, pls refer to R8 of Reviewer q4xE.\n\nAs noted in R1, our deep learning-based variational inference does not suffer from the scalability issues presented in previous methods. **Therefore, with our specific designs, the ODE solvers and variational inference do not pose  computational bottlenecks for our method.**\n\n\nWe will include this clarification in the latest version\n\n\n>C3:As the author shows the robustness to different noise distributions, it is interesting how about the robustness when the data contains missing entries.\n\nR3:We would like to clarify that our method is specifically designed to complete the entire tensor from sparse observations. All experiments are conducted on datasets with a high proportion of missing entries.\n\n\n>C4:Are there any smoothness (stationary) assumptions on the temporal dynamics of the tensor? How is the performance of the CATTE if there is a sudden change?\n\nR4: We do not make explicit smoothness (or stationarity) assumptions on the temporal dynamics of the tensor. However, in the latent space,  implicit assumption on the  smoothness of latent trajectories is made  and we model them using Neural ODEs.  Visualization results of CATTE are provided in Appendix B.7. Figures 8 and 9 show that CATTE remains effective even in the presence of sudden changes.\n\n\n\n\n>C5:It seems the ref. [28] and ref. [34] are the same. Please check if there are some mistakes.\n\nR5:Thanks for the comment! We will fix them in the latest version."}, {"review_id": "D6BkpizEUc", "reviewer": "Reviewer_q3bj", "summary": "This paper proposes CATTE, a method for temporal tensor decomposition that handles tensors with continuous indices across all modes. The key innovation is combining neural ODEs with automatic rank determination for functional temporal tensors.", "questions": "1. Please discuss and clarify the difference and improvement over existing methods like DEMOTE and FunBaT\n2. It seems to the ELBO derivation problems (Eq. 17) has dimensional inconsistencies. It needs clarification - what does vec() of a matrix plus identity $\\mathbf{I}$ mean here? Similar issues raise in other places (e.g., Eq. 24)", "limitations": "Yes. In supplementary", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 2, "originality": 2, "strengths_and_weaknesses": "**Strengthens**\n\n1. Comprehensive baselines: Compares against 6 state-of-the-art methods spanning temporal (NONFAT, DEMOTE), functional (FunBaT, LRTFR), and other approaches\n2. Consistent improvements: Shows great RMSE reduction across datasets, with particularly strong performance on spatialtemporal ones\n3. Computational efficiency: Linear scaling with time series length\n4. Existing methods either handle rank determination or continuous indices, but not both simultaneously. The solution is mathematically sound and showed improvement. \n\n**Weaknesses**\n1. Individual components (Neural ODEs, Bayesian rank selection, functional tensors) all exist. Thus, contribution appears largely incremental - combining existing techniques\n2. Fourier feature dependency: Choice may not be optimal for all data type; What if sparse data. In practice, tensor data is very sparse. Sparse basis functions might be helpful. \n3. Hyperparameter analysis is not enough. For example, having three sets of $a$ and $b$ for Gamma on CA traffic can't conclude the method auto rank selection is robust to initialization. \n4. Missing analysis of related work in Bayesian Neural ODEs literature", "was_revised": true, "final_justification": "Authors addressed some concerns of mine. However the kitchen sink approach (in my view) still affects my evaluation. I will maintain the score", "rebuttal": "We thank the reviewer for the careful review! We address the comments below:( C: comment; R: response)\n\n\n\n>C1: Individual components (Neural ODEs, Bayesian rank selection, functional tensors) all exist. Thus, contribution appears largely incremental - combining existing techniques\n\nR1: We appreciate the reviewer’s concern. However, many works, including the many references in Related Work, also use existent techniques, via their novel application and combination, to address the problem of interest. we believe such works are still valuable contributions to the relevant fields.\n\nOur problem setting, which reconstructing temporal tensors from sparse, noisy, and continuously indexed observations is new and is practically important in environmental and scientific data, but remains largely unaddressed in prior literature. \nTo tackle this challenge, we combine three componants and also propose  **methodological innovations** to ensure the compatibility and effectiveness  of the combinations:\n(1) A  continuous-indexed neural ODEs-based tensor decomposition framework  for handling continuously indexed temporal tensor data with efficiency-driven  approach that scales our model to large datasets. (2) A deep characterization of the variational posterior and new derivation of close-form ELBO to ensure both  effectiveness and scalibility.  **To the best of our knowledge, we are the first to  achieve continuous-indexed temporal tensor modeling,  scale ODE-based tensor decomposition to large datasets, and  enable automatic rank determination in functional tensors.** We list the comparison of CATTE\nand other main  methods in the following table:\n|   Property / Methods   |  **CATTE** | ThisODE|Functional tensor methods | Bayesian neural ODEs | Factor-evolving tensor methods |\n|:-----:|:-----:|:-----:|:-----:|:-----:|:-----:|\n|Continuous  dynamics modeling | **Yes** | Yes | Partial (tend to underfit the complex dynamics) | Yes|Yes|\n|Continuous-indexed  modes modeling | **Yes** | No |Yes| No|No|\n|Scability | **High** | Low |High| Low|Low|\n|Complexity self-adpation| **Yes** | No |No| No|No|\n|Uncertainty-aware| **Yes** | No |Partial| Partial|No|\n\nFor much more details, pls refer to   R1 of Reviewer MW7k.\nWe have validated our method across a wide range of  datasets, demonstrating both strong empirical performance and broad applicability.\n\n\n\n>C2:Fourier feature dependency: Choice may not be optimal for all data type; What if sparse data. In practice, tensor data is very sparse. Sparse basis functions might be helpful.\n\nR2: Thank you for your comment. We would like to clarify that the Fourier features in our work are used  for positional embedding, not for representing the data itself. Besides, our model is specifically designed to handle sparse tensor data by effectively capturing spatial and temporal continuity.  For example, the observation rate is 7.8% for the CA Traffic dataset, 2.6% for ServerRoom, and 11% for SSF.\n\n\n>C3: Robustness of auto rank selection to initialization.\n\nR3: In the objective function (16), $a_{r}^0$ and $b_{r}^0$ only affect the KL term, KL$(q(\\lambda) \\| p(\\lambda))$. Following common practice in previous Bayesian CP works, we set them to small values to define a non-informative prior. This choice has minimal impact on our proposed method.\n\n\n>C4:Missing analysis of related work in Bayesian Neural ODEs literature\n\nR4: Thank you for the suggestion! We will include the following review of the literature on Bayesian Neural ODEs and clarify how our work differs from existing approaches.\n\n\"[B1] combines a variational autoencoder with neural ODE dynamics, replacing the encoder with an ODE-RNN to handle irregularly-sampled time series. [B2] combines a continuous-time GRU with a Bayesian update mechanism to model evolving latent dynamics in  irregularly‑sampled time series. However, these Bayesian Neural ODEs are primarily designed for multivatiate time series, which is not applicable in our setting.\"\n\n[B1] Rubanova, Yulia, Ricky TQ Chen, and David K. Duvenaud. \"Latent ordinary differential equations for irregularly-sampled time series.\" Advances in neural information processing systems 32 (2019).\n\n[B2] De Brouwer, Edward, et al. \"GRU-ODE-Bayes: Continuous modeling of sporadically-observed time series.\" Advances in neural information processing systems 32 (2019).\n\n\n\n\n\n>C5: the difference and improvement over existing methods like DEMOTE and FunBaT\n\nR5: Thank you for your comments. The differences are:\n| | FunBaT | DEMOTE |CATTE|\n|:------:|:------:|:------:|:------:|\n| Formulation  |fully factorized and decoupled: $\\boldsymbol{1}^{\\text{T}}[\\underset{k}{\\circledast}\\mathbf{u}^k(i_k) \\circledast \\mathbf{u}^{\\text{Temporal}}(t) ]$|temporally coupled:$g([\\mathbf{u}^k_{i_1}(t), \\cdots, \\mathbf{u}^k_{i_K}(t)])$, where $g$ is MLP. |Functional factor traj:$\\boldsymbol{1}^{\\text{T}}[\\underset{k}{\\circledast}\\mathbf{u}^k(i_k,t) ]$.|\n| Model & Inference | Gaussian process(GPs) + Bayesian message passing|    Multi-Partite Graphs with diffusion process and  reaction process + Gradient descent on maximum a posteriori (MAP)|Continuous-indexed latent neural ODEs + Gradient descent on analytical ELBO|\n| Hyperparas. | Requires specifying tensor rank and kernel parameters.\t|Requires specifying tensor rank.  |Automatic rank tuning.|\n\nThe improvements are:\n\n  \n\n* The fully factorized form of FunBaT can underfit cases where time mode is complex and naturally coupled with others. DEMOTE requires the discrete factors to be pre-specified and does not support continuous-indexed modeling. Moreover, the evolution of its factors depends on aggregating information from all modes, causing the number of parameters to grow **exponentially** with the number of spatial points. This significantly limits its scalability. **In contrast, CATTE effectively captures time-varying patterns by modeling factor trajectories jointly, and demonstrates excellent scalability, as shown in R4&R8 for Reviewer q4xE.**\n\n* FunBaT reduces computational cost by converting GPs into state-space priors via SDEs, but this limits kernel choices to stationary ones (e.g., Matérn), reducing expressiveness. Its Bayesian message-passing inference is length. DEMOTE uses MAP estimation, which lacks uncertainty quantification and is more sensitive to noise. **In contrast, CATTE  offers better architectural flexibility, enabling richer representations than FunBaT and adopts a Bayesian framework with a closed-form ELBO, enabling efficient gradient-based optimization, uncertainty quantification, and improved robustness to noise.**\n\n\n* FunBaT and DEMOTE requires to pre-set more hyperparameters (tensor rank, kernel paras) than CATTE, making fine-tuning more challenging.\n\nn summary, CATTE integrates the strengths of both FunBaT and DEMOTE while overcoming their respective limitations. Moreover, unlike these two methods, CATTE supports complexity adaptation. We will clarify this in the latest version\n\n\n>C6:It seems to the ELBO derivation problems (Eq. 17) has dimensional inconsistencies. It needs clarification - what does vec() of a matrix plus identity $\\mathbf{I}$ mean here? Similar issues raise in other places (e.g., Eq. 24)\n\n\nR6: Thanks for the question. We would like to clarify that $\\text{vec}\\big( \\mathbf{g} \\mathbf{g}^{T} + \\sigma^2 \\mathbf{I} \\big)$ of  Eq. 17 first performs the addition of two matrices of the same shape, and then vectorizes the result. We will improve the clarity in the latest version."}, {"review_id": "zwVrh2OExV", "reviewer": "Reviewer_q4xE", "summary": "The paper proposes CATTE, a Bayesian CP-style model for generalized temporal tensors whose modes may all be continuous-indexed. CATTE (i) represents spatial coordinates through learnable Fourier features and evolves factors with a latent Neural ODE, (ii) places a dimension-wise Gaussian-Gamma prior over the entire family of factor trajectories, and (iii) derives a fully closed-form evidence-lower-bound that admits sampling-free variational inference. Experiments on one synthetic and three real-world data sets (CA-Traffic, Server-Room and Pacific Sound-Speed Field) show that CATTE outperforms five recent baselines in RMSE/MAE while automatically discovering the ground-truth rank, and exhibiting robustness to several noise types.", "questions": "Could you articulate a specific theoretical insight or property that cannot be achieved by any pair-wise combination of Fourier features, neural-ODE dynamics, and ARD sparsity alone? \nIn other words, what new capability or guarantee emerges uniquely from the full CATTE synthesis?\n\nFor each dataset, what is the total number of trainable parameters in CATTE versus the baselines? \nIf you downscale CATTE so that its parameter budget matches that baseline, how do RMSE/MAE and wall-clock convergence change?\n\nBeyond time horizon, how does CATTE's runtime scale with the number of spatial points? \nA synthetic experiment sweeping both dimensions (timestamps × spatial points) would help practitioners judge feasibility.", "limitations": "Yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 2, "strengths_and_weaknesses": "Strengths\n\nS1. The paper is technically solid: it frames the decomposition in a fully Bayesian manner and derives an evidence-lower-bound whose every term is analytic.\nS2. The motivation and related-work sections clearly position the contribution within the temporal-tensor literature.\nS3. Extending automatic-rank ideas to functional factor trajectories and producing a closed-form ELBO for this setting is, to the best of my knowledge, novel and likely to stimulate further research in probabilistic tensor analysis.\n\nWeaknesses\nW1. (Limited methodological novelty) \nAlthough the paper delivers a well-engineered system, each of its core building blocks (Fourier positional encodings, neural-ODE dynamics, and an automatic-rank-determination sparsity prior) are already well established in the literature. The contribution therefore lies chiefly in how these standard components are combined, rather than in a fundamentally new modelling principle or theoretical insight. \n\nW2. (Parameter efficiency and fairness of comparison) \nEquation (5) defines every latent factor as a trajectory u^k(i_k, t) that is jointly conditioned on the continuous spatial index i_k and on the timestamp t. This modelling choice implies that CATTE's learnable parameter set grows proportionally with #spatial points x #time-stamps x R. The paper does not report the resulting parameter counts, whereas the baselines rely on time-invariant factors or low-order spline/GP coefficients. Only per-iteration runtimes are given (Table 6), so the accuracy gains in Table 1 may partly reflect CATTE's much larger capacity rather than genuine efficiency. A like-for-like parameter budget (or test-error curves versus model size) would be necessary for a fair comparison.\n\nW3. (Limited empirical scale)\nEvaluation is limited to three modest-scale datasets. The benchmarks contain roughly 10,000 observed entries and at most 34 timestamps. No experiment tests very large (>10^6) data, so the linear-time claim remains speculative for industrial-scale workloads.\n\nW4. (Presentation issues)\nSeveral typographical errors (e.g., “Exisiting” -> “Existing”: line 48, “Give” -> “Given”: line 148, “contains” -> “containing”: line 281, “demonstating” -> “demonstrating”: line 559, etc.) distract from the narrative.", "was_revised": true, "final_justification": "The reviewers addressed my concerns, and I raised my score to accept.", "rebuttal": "We sincerely appreciate your thoughtful feedback and  recognition of our work. Below, we address the comments (C: comment, R: response):\n\n\n>C1: Concerns about the methodological novelty.\n\n\nR1:  We list the comparison of CATTE and other main  methods in the following table:\n|   Property / Methods   |  **CATTE** | ThisODE|Functional tensor methods | Bayesian neural ODEs | Factor-evolving tensor methods |\n|:-----:|:-----:|:-----:|:-----:|:-----:|:-----:|\n|Continuous  dynamics modeling | **Yes** | Yes | Partial (tend to underfit the complex dynamics) | Yes|Yes|\n|Continuous-indexed  modes modeling | **Yes** | No |Yes| No|No|\n|Scability | **High** | Low |High| Low|Low|\n|Complexity self-adpation| **Yes** | No |No| No|No|\n|Uncertainty-aware| **Yes** | No |Partial| Partial|No|\n\nFor  more details about our contributions, pls refer to   R1 of Reviewer MW7k.\n\n\n>C2: This modelling choice implies that CATTE's learnable parameter set grows proportionally with #spatial points x #time-stamps x R. \n\nR2: We would like to clarify that the number of learnable parameters in CATTE **does not** grow proportionally with the number of spatial points $\\times$ time-stamps. For each non-temporal mode, we employ a fixed network for continuous index representation learning. Similarly, for the temporal mode, we use a fixed network to learn the derivatives. Our model only grows  with $R$. As a result, our model can handle data with arbitrarily  resolution while maintaining parameter efficiency.  \n\n>C3:The paper does not report the resulting parameter counts, whereas the baselines rely on time-invariant factors or low-order spline/GP coefficients.\n\nR3: Thanks for the comment, we report the parameters in R7.\n\n\n>C4:Limited empirical scale. Evaluation is limited to three modest-scale datasets. \n\nR4: Thank you for the comment. The three real-world datasets we use are widely adopted in this literature [6,15,28], ensuring fair comparison with prior work.  \n\nTo address concerns about scalability, we additionally conducted experiments on a much larger real-world dataset, SSF-large, which spans four dimensions: 38 latitudes × 76 longitudes × 50 depths × 100 timestamps — yielding over $10^7$ data points in total. We randomly selected $10^6$ coordinate–value pairs for training and the rest for testing. This experiment is approximately **100× larger** in scale compared to those in the main section. So the amount training and test data exceed $10^6$. To handle the large data, we increase the size of all model and make their parameters comparable.\n\nWe report the RMSE/MAE, model parameters and training time per epoch in the following:\n|     | CATTE （R=10） | ThisODE (R=10)| DEMOTE (R=10)| LRTFR (R=10)|FunBaT-CP (R=10)|FunBaT-Tucker (R=10)|NONFAT (R=10)|\n|:-----:|:-----:|:-----:|:-----:|:-----:|:-----:|:-----:|:-----:|\n| RMSE | **0.406 $\\pm$ 0.003**| N/A |N/A | 0.713 $\\pm$ 0.0781|0.819 $\\pm$ 0.134|N/A|9.812±0.001|\n| MAE | **0.319 $\\pm$ 0.004** | N/A |N/A |0.547 $\\pm$ 0.0435|0.622 $\\pm$ 0.100|N/A| 8.761±0.001|\n| Parameters | 420K | 428K |650K|462K|N/A|N/A|N/A|\n| Time per Epoch  | 9.38s | >30min |>30min |9.96s|3.70s|>30min|67.2s|\n\nOur method is superior to other baselines in accuracy.\nWe find that among baselines only LRTFR and FunBaT-CP  scales well to the large dataset. NONFAT does not produce meaningful trajectories. For those methods whose per‑epoch runtime is prohibitively long, we do not report the final results.\n\nTo further demonstrate the scalability of our method, we conducted large-scale experiments on synthetic data, as detailed in R8.\n\n\n\n\nWe will include this additional large-scale experiment and its results in the latest version\n\n\n\n\n\n\n\n>C5:Presentation issues\n\nR5: Thank you for the corrections. We will thoroughly review the manuscript and address all typographical errors we found in the revised version.\n\n\n>C6: Could you articulate a specific theoretical insight or property that cannot be achieved by any pair-wise combination of Fourier features, neural-ODE dynamics, and ARD sparsity alone? In other words, what new capability or guarantee emerges uniquely from the full CATTE synthesis?\n\nR6: Thank you for your insightful question. We provide the \n\n\n| Combination Type           | Captures                                  | Missing Capability                                                    |\n|----------------------------|-------------------------------------------|------------------------------------------------------------------------|\n| **Fourier + Neural-ODE**   | Continuous-indexed temporal  dynamics modeling  | Lacks adaptive sparsity; may overfit noisy or irrelevant components; Lack uncertainty quantification.    |\n| **Fourier + FARD**   | Continuous-indexed tensor modeling with adaptive complexity control | No temporal modeling; fails to capture complex dynamic dependencies           |\n| **Neural-ODE + FARD**| Sparse latent temporal dynamics            | Cannot effectively represent  continuous-indexed temporal signals or learn spatial correlations; Scalibility issue for large dataset may arise. |\n\n\nIn contrast, **CATTE** synergistically combines:\n- The expressivity of **Fourier** features for positional representation,\n- The continuity and strong representitive power of **Neural-ODE** latent dynamics,\n- The parsimony induced by **FARD** sparsity,\n\nresulting an efficient and scalable model for reconstructing temporal tensors from sparse, noisy, and continuously indexed observations.\n\n>C7:For each dataset, what is the total number of trainable parameters in CATTE versus the baselines? If you downscale CATTE so that its parameter budget matches that baseline, how do RMSE/MAE and wall-clock convergence change?\n\n\nR7: The total number of trainable parameters in CATTE  versus the baselines on  CA Traffic, ServerRoom and  SSF datasets are:\n|   Parameters   | CATTE （R=10） | ThisODE (R=10)| DEMOTE (R=10)| LRTFR (R=10)|\n|:-----:|:-----:|:-----:|:-----:|:-----:|\n| - | 90K | 89K |105K|91K|\n\nThey are comparable since the number of layers (2) and  layer width (100) are set to same as stated in Appendix B.2.  FunBaT and NONFAT are non-parametric GP-based model and do not have trainable parameters.\n\nWe also varying the rank of the baslines, and when $R=3$, the deep-learning based baselines are:\n\n|   Parameters    | ThisODE (R=3)| DEMOTE (R=3)| LRTFR (R=3)|\n|:-----:|:-----:|:-----:|:-----:|\n| -  | 64K |83K|67K|\n\n\nWe downscale CATTE by setting $R=3$, the number of parameters is 69K. We reconduct the experiment, the results are:\n|    CATTE(R=3)    | CA traffic | Server Room | SSF|\n|:-----:|:-----:|:-----:|:-----:|\n| RMSE | 0.308 $\\pm$ 0.024 | 0.107 $\\pm$ 0.009| 0.435 $\\pm$ 0.020 |\n|  MAE   | 0.113 $\\pm$ 0.011 | 0.084 $\\pm$ 0.005 | 0.033 $\\pm$ 0.017 |\n\nWe can observe  the increase in RMSE/MAE, which is primarily due to underfitting of the signal. Nevertheless, CATTE still outperforms the baseline methods. The convergence is slightly faster than the non-downscaled version of CATTE. \n\nIn practice, CATTE can be manually configured with sufficient model parameters, and it will adaptively adjust its parameters to fit the data effectively. We will clarify these in the latest version \n\n\n\n\n>C8: Beyond time horizon, how does CATTE's runtime scale with the number of spatial points? A synthetic experiment sweeping both dimensions (timestamps × spatial points) would help practitioners judge feasibility.\n\nR8: Thank you for your suggestion. To demonstrate the scalability of our method, we conducted synthetic experiments using the same synthetic function as in Eq. (20), across varying configurations of timestamps $\\times$ spatial points. All experiments employed the same network architecture, with $5\\%$ of the total data points randomly selected for training. As a result, the size of the training dataset ranges from $10^5$ to $10^7$. Consistent with Fig. 4, we report the average training time (in seconds) per epoch.\n\n\n\n|   timestamps \\ spatial resolution   | $50 \\times 50$  | $100 \\times 50$ | $200 \\times 50$| $100 \\times 100$|$200 \\times 200$|\n|:-----:|:-----:|:-----:|:-----:|:-----:|:-----:|\n| T=100 | 0.398 | 0.673 |1.011|1.016|2.392|\n| T=200  | 0.814| 1.088|1.670|1.714|5.081|\n| T=500 | 1.688 | 2.162 |3.755|3.721|10.79|\n\n\n\nOne can see that our method also scales well on the spatial mode since our method decouple each mode and convert the expanding spatial resulotion to the increase of the ODE states. For example, for the spatial resolution $50\\times 50$, we only need to solve $50+50=100$ ODE states. Likelywise, for the spatial resolution $200\\times 200$, we only need to solve $200+200=400$ ODE states. We will supplement the results in the latest version."}]

Author rebuttal:

Meta-review: This paper introduces Catte, a functional complexity-adaptive temporal tensor decomposition method. Unlike prior temporal tensor models that only handle continuous timestamps, Catte generalizes to tensors with continuous indexes across multiple modes, such as spatial coordinates in climate data. The method uses learnable Fourier features to encode continuous spatial inputs and neural ODEs to model temporal factor trajectories. To automatically adapt model complexity, it applies a sparsity-inducing prior over trajectories, coupled with an efficient variational inference scheme that provides a closed-form evidence lower bound without sampling.

The paper is technically solid, though the methodology is a bit complex for practical applications. The design choices are overall appropriate. The reviewers appreciate the linear scaling. While all individual components (Neural ODEs, Bayesian rank selection, functional tensors) exist and the novelty might look a bit incremental, as long as the proposed methodology is technically solid, I am fine with that.
Decision: Accept (poster)