Functional Bayesian Tucker Decomposition for Continuous-indexed Tensor Data
Authors: Shikai Fang
Xin Yu
Zheng Wang
Shibo Li
Mike Kirby
Shandian Zhe
Venue: ICLR 2024 poster (2024)
Keywords: Bayesian model
Tensor decomposition
Gaussian Processes
Primary area: probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)

Abstract: Tucker decomposition is a powerful tensor model to handle multi-aspect data. It demonstrates the low-rank property by decomposing the grid-structured data as interactions between a core tensor and a set of object representations (factors).  A fundamental assumption of such decomposition is that there are finite objects in each aspect or mode, corresponding to discrete indexes of data entries. However,  real-world data is often not naturally posed in this setting. For example, geographic data is represented as continuous indexes of latitude and longitude coordinates, and cannot fit tensor models directly. To generalize Tucker decomposition to such scenarios, we propose Functional Bayesian Tucker Decomposition (FunBaT). We treat the continuous-indexed data as the interaction between the Tucker core and a group of latent functions. We use Gaussian processes (GP)  as functional priors to model the latent functions. Then, we convert each GP into a state-space prior by constructing an equivalent stochastic differential equation (SDE)  to reduce computational cost. An efficient inference algorithm is developed for scalable posterior approximation based on advanced message-passing techniques. The advantage of our method is shown in both synthetic data and several real-world applications. We release the code of FunBaT at {https://github.com/xuangu-fang/Functional-Bayesian-Tucker-Decomposition}.

Published as a conference paper at ICLR 2024
FUNCTIONAL BAYESIAN TUCKER DECOMPOSITION FOR CONTINUOUS-INDENN
TENSOR DATA
Shikai Fang, Xin Yu, Zheng Wang, Shibo Li, Robert M. Kirby, Shandian Zhe University of Utah, Salt Lake City, UT 84112, USA (shikai, xiny, wzuht, shibo, kirby, zhe] êcs wtw..ttahee............ edu
ABSTRACT
Tucker decomposition is a powerful tensor model to handle multi-aspect data. It demonstrates the low-rank property by decomposing the grid-structured data interactions between a core tensor and a set of object representations (factors). A fundamental assumption of such decomposition is that there are finite objects in each aspect or mode, corresponding to discrete indexes of data entries. However, real-world data is often not naturally posed in this setting. For example, geographic data is represented as continuous indexes of latitude and longitude coordinates, and cannot fit tensor models directly. To generalize Tucker decomposition such scenarios. we propose Functional Bayesian Tucker Decomposition (FunBaT). data as the interaction between the Tucker core
We treat the continuous-indexx and group of latent functions. We use Gaussian processes (GP) as functional priors to model the latent functions. Then, we convert each GP into state- space prior by constructing equivalent stochastic differential equation (SDE) to reduce computational cost. An efficient inference algorithm is developed for scalable posterior approximation based on advanced
pessssge-psssingg techniques. The advantage of our method is shown both synthetic data and several real- world applications. We release the code of FunBaT at httttss://gtttiii com/
xuangu- Shaangu-faggFFurne fang/Funct lonal Bayes an- inn-siin---
Tucker puccker-ecooppsstiion
INTRODUCTION
Tensor decomposition is widely used to analyze and predict with multi-way or multi- data in real- world applications. For example, medical service records can be summarized by a four-mode tensor (patients, doctor, clinics, time) and the entry values can be the visit count drug usage. Weather data can be abstracted by tensors like (latitude, longitude, time) and the entry values can be the temperature. Tensor decomposition introduces a set of low-rank representations of objects in each mode, known as factors, and estimates these factors to reconstruct the observed entry values. Among numerous proposed tensor decomposition models, such as CANDECOMP/PARAFAN (CP) (Harshman 1970) decomposition and tensor train (TT) (Oseledets) (011). Tucker decomposition (Tucker 1966) is widely-used for its fair capacity, flexibility and interpretability.
Despite its success, the standard tensor decomposition framework has fundamental limitation. That is, restricts the data format as structured grids, and demands each mode of the tensor must be discrete and finite-dimensional. That means, each mode includes a finite set of objects, indexed by integers, such as user 1, user 2, and each entry value is located by the tuple of discrete indexes. However, in many real-world applications like climate and geographic modeling, the data is represented as continuous coordinates of modes, e.g., latitude, longitude and time. In these cases, each data entry is located by tuple of continuous indexes, and tensor decomposition cannot be applied directly.
To enable tensor models on such data, one widely-used trick is to discretize the modes, for example, binning the timestamps into time steps (say, by weeks or months), or binning the latitude values into set of ranges, and number the ranges by 1, 2, 3... While doing this is feasible, the fine-grained information is lost, and it is hard to decide an appropriate discretization strategy in many cases
"Corresponding author

---

Published as a conference paper at ICLR 2024
(Pasricha et al. 2022 A more natural solution is extend tensor models from grid-structure to continuous field. To the best of our knowledge, most existing methods in that direction are limited to tensor-train (Gorodetsky et al. 2015 Bigoni al. 2016 Mallester-Ripol et al. 2019; Chertkov et al.) 2023) or the simpler CP format (Schmidt 2009), and cannot be applied to Tucker decompasition, an important compact and flexible low-rank representation. What's more, most of the proposed methods are deterministic with polynomial approximations, cannot provide probabilistic inference, and are hard well handle noisy, incomplete data.
To bridge the gap, we propose FunBaT:
Functional Bayesian Tucker decomposition, which gener- alizes the standard Tucker decomposition the functional field under the probabilistic framework. We decompose the continuous- indexed tensor data as the interaction between the Tucker core and group of latent functions, which map the continuous indexes to mode-wise factors. We approximate such functions by Gaussian processes (GPs), and follow (Hartikainen and Särkkä (2100) to convert them into state-space priors by constructing equivalent stochastic differential equations (SDE) to reduce computational cost. We then develop efficient inference algorithm based on the conditional expectation propagation (CEP) framework (Wang and Zhe 2019) and Bayesian filter to approximate the posterior distribution of the factors. We show this framework decomposition to continuous-indexx experiments on both synthetic data and real-world applications of climate modeling. The results show that FunBaT not only outperforms the state-of-the-ar methods by large margins in terms of reconstruction error, but also identifies interpretable patterns that align with domain knowledge.
also be used to extend CP data with a simplified setting. For evaluation, we conducted
2 PRELIMINARY
2.1 TENSOR DECOMPOSITION AND FUNCTION FACTORIZATION
Standard tensor decomposition operates under the following setting: given a K-mode tensor Y E TR Xxx where each mode contains de objects, and each tensor entry is indexed by a K-size tuple i (ii,..., ik), denoted as yi. To decompose the tensor into compact and low- rank form, introduce K groups of latent factors, denoted as U UU' where each U [ut ..... u Each factor u represents the latent embedding of the j-th object in the k-th mode. The classic CP (Harshman
.....,
1970) assumes each factor u is r-dimensional vector, and each tensor entry is decomposed as the product-sum of the factors: // Ri (u o... u 1, where is the glement-wise product. Tucker decomposition (Tucker) 1966) takes one more step towards more expressive and flexible low-rank structure, It allows us to set different factor ranks {ri,... .r.....k
for each mode, by introducing core tensor W E R X=* known as the Tucker core. It models the interaction as a weighted over all possible cross-rank factor multiplication:
yi ve (u,, ... u
where @ is the Kronecker product and vec(.) is the vectorization operator. Tucker decomposition will degenerate CP if we set all modes' ranks equal and W diagonal. tensor train (TT) decomposition (Oseledets) 2011) is another classical tensor method. TT sets the TT-rank (ro, ri,.... -rk) firstly, where To тk 1.Then each factor uz is defined as a r% rk+1 matrix. The tensor entry yi is then decomposed as series of the matrix product of the factors.
Function factorization refers to decomposing a complex multivariate function into a set of low- dimensional functions (Rail 2014 Nouy| 2015). The most important and widely used tensor method for function factorization is TT (Gorodetsky et al., 2015 Bigoni et al. 2016 Ballester- Ripoll et al. 2019). It converts the function approximation into tensor decomposition problem. A canonical form of applying TT to factorize a multivariate function f(x) with K variables x (II.... IK) E R is:
f(x) = (1(z)) x G2(r2)
x Gk(ok),
where each (-) is univariate valuue- function, takes the scaler 2i the input and output matrix (z E RT 187,, It is straightforward to see that G+ is a continuous generalization of the factor uz in standard TT decomposition.
2

---

Published as a conference paper at ICLR 2024
2.2 GAUSSIAN PROCESS AS STATE SPACE MODEL
Gaussian process (GP) (Rasmussen and Williams 2006) is a powerful non-parametric Bayesian model to approximate functions. Formally, a GP is a prior distribution over function f(), such that f(-) evaluated at any finite set of points x = (#1..... ,,NN follows a multivariate Gaussian distribution, denoted f GP (0, K. (z,,")), where (z, x') is the covariance function, also known as the kernel, measuring the similarity between two points. One of the most widely used kernels is the Matérn kernel:
x x²113) ((2)20-1 Ku
12vv
Matté = o²
x' r'11²)
where {o?,
[.v.p p} are hyperparameters determining the variance, length-scalee smoothness, and periodicity of the function, Ky is the modified Bessel function, and [(-) is the Gamma function.
Despite the great capacity of GP, it suffers from cubic scaling complexity ((N3) for inference, To overcome this limitation, recent work (Hartikainen and Särkkä 2010) used spectral analysis to show an equivalence between GPs with stationary kernels and linear time-invariant stochastic differential equations (LTI-SDEs). Specifically, we can formulate a vector-valued latent state z(x² comprising fæ) and its derivatives up to m-th order. The GP f(x) ~ GP (0, x) is equivalent to the solution of
an LTI-SDE defined as:
z( = ((x) df(x)
dfm(x) dx ) dz(x) = F2(x) Lw(())
(4)
dr
dz
where F and L are dime-invariant coefficients, and w(x) is the white noise process with density gs- At any finite collection of points In <... < IN., the SDE in (4) can be further discretized as a Gaussian-Markov chain, also known the state-space model, defined as:
p(z(z)) p(z)) II. == p(zn+1 (2n) (((((z))),
II
N(a(EnH))AAn Qn)
(5)
where An = exp(FA,)). Q - 1 LL AT dtt,, An In+1 Tn. and Poo is the steady- state covariance matrix which can be obtained by solving the Lyapunov equation (Lancaster and Rodman 1995). All the above parameters in (4) and (5) are fully determined by the kernel K and the time interval A.. For the Matérn kernel 3 with smoothness being integer plus half, F, L and Poo possess closed forms (aäkkkä 2013). Specifically, when v = 1/2. we have {m O.F -
1, (0; 1).46 40 (0 where À v3/f. f. With the state
1, 20 22 = o²]; for V 3/2, we have
(0, 1: 10, space prior, efficient (n) inference can be achieved by using classical Bayesian sequential inference techniques, like Kalman filtering and RTS smoothing (Hartikainen and Särkkä 2010). Then the original function f (Ix) is simply the first element of the inferred latent state (.r)
3 MODEL
3.1 FUNCTIONAL TUCKER DECOMPOSITION WITH GAUSSIAN PROCESS
Despite the successes of tensor models, the standard tensor models are unsuitable for continuous- indexed data, such as the climate data with modes latitude, longitude and time. The continuity property encoded in the real-value indexes will be dropped while applying discretization. Additionally, can be challenging to determine the optimal discretization strategy, and the trained model cannot handle new objects with never-seen indexes. The function factorization idea with tensor structure therefore more natural solution to this problem. However, the early work (Schmidi) 2009) with the simplest CP model, uses sampling-basse inference, which is limited capacity and not scalable to large data. The TT-based function factorization methods (Gorodetsky et al. 2015 Bigoni et al. 2016 Ballester-Ripo Ripoll et al.] [2019] Chertkov et al., (023) take deterministic approximation like Chebyshev polynomials splines, which is hard to handle data noises provide uncertainty quantification.
Compared to CP and TT, Tucker decomposition possesses compact and flexible low-rank representa- tion. The Tucker core can capture more global and interpretable patterns of the data. Thus, we aim to
3

---

Published as a conference paper at ICLR 2024
extend Tucker decomposition to continuous-indexed indexed data to fully utilize its advantages Bayesian method, and propose FunBaT: Functional Bayesian Tucker decomposition.
Aligned with the setting of the function factorization (2. we factorize K-variate function f(i) in Tucker format
( with preset latent rank: (ri.... }
f(i) f(i1......... ix) = vec(W)) (U)
(ix)))
(6)
where - (f1,....xx ix) E RKK and W R1xx-xr is the Tucker core, which is the same as the standard Tucker decomposition. However, (*).) is ri-size vector-valued function, mapping the continuous index i mode a "x-dimensional latent factor. We assign independent GP priors over each output dimension of U* and model (-) as the stack of group of univariate scalar functions. Specifically, we have:
(+e [u (1x) (..)....., u (ix)) (ix) ~
~ GP (0, AiRR (.) = l...rk
where ka(ie, (1) R x R
the covariance (kemel) function of the k-th mode. In this work, we the popular Matérn kernel (3) for GPs over all modes.
3.2 STATE-SPACE-PRIOS AND JOINT PROBABILITIES
Given N observed entries of K-mode continuous-indexee tensor V denoted as D {(f", mn)).m where 1" = (7 (?kk..))) is the tuple of continuous indexes, and yn is en- try values, we can assume all the observations are sampled from the target Tucker-format function f(i) and Gaussian noise -1 Specifically, we define the likelihood L, as:
Ln - pVynl ( JE W,+) N (yn vec(MW) (U' @... & UK ;T=))
(8)
We then handle the functional priors over the {U*) As introduced in Section 2.2 the dimension- wise GP with Matérn kernel over uz (ix. is equivalent
M-order LTI-SDE and then
can discrete as a state space model (5). Thus, for each mode's function U we concatenate the state space representation over all Tx dimensions of U*(tx) (ix) to joint state variable Z²
concat (z (fx)...., z (4x) ]] for U*, where (1x) the state variable of (ix). We can build an
ordered index set Is {it ... & which includes the N& unique indexes of mode-k's among all observed entries in D. Then, the state space prior over U on Ia is:
p where p(Z)) (Z Ph::: (Z (2 = NZ (1: ())) ②
p(U*)= = = = p(z (1t)
zt (i p(z))
(9)
(1).
(10)
The parameters in 10p are bock-diagonal concatenates of the corresponding univariate case. BlockDiag(AA ..) and Q
Namely, P
Diagk pk ...P). A
BlockDiag Q ... Q.) where oc. A and Q are the corresponding parameters in (5)
With z (ix). we can fetch the value of U by multiplying with projection matrix H BlockDiag([1, ((, O.. 1.......J) U HZ*
With state space priors the latent functions, we further assign a Gamma prior over the noise precision and Gaussian prior over the Tucker core. For compact notation, denote the set of all random variables as = (W. r. ( K [=]) Finally, the joint probability of the proposed model is:
K
N--1
N
p(D, () (D, Iz k W, -())))(()p00))))))
H
II #=1
pizz
II
Ln.
(11)
k=1
==1
where p(W) (vvee(W) 0, I). p(T) Gam(7 og bu), au and by are the hyperparameters of the Gamma prior, and l, is the data likelihood defined in ().
4 ALGORITHM
The inference of the exact posterior p(O D) with II is intractable, as there are multiple latent functions K == and the Tucker core interleave together in complex manner in the likelihood
4

---

Published as a conference paper at ICLR 2024
term L,, To address this issue, propose efficient approximation algorithm, which first decouples the likelihood term by a factorized approximation, and then applies the sequential inference of Kalman filter and RTS smoother to infer the latent variables { T==1 at each observed index. Finally, we employ the conditional moment matching technique to update the message factors in parallel. We will introduce the details in the following subsections.
4.1 FACTORIZED APPROXIMATION WITH GAUSSIAN AND GAMMA DISTRIBUTION
To estimate the intractable posterior p O D) with a tractable q( ( we first apply the meaa--field assumption and design the approximated posterior as a fully factorized format. Specifically, we approximate the posterior as:
p () R g( ((F))(W)) II K
(Z*)
(12)
where (T) = Gam b). q( (() = (WV | S) are the approximated posterior of r and W, respectively. For q (Z*) we further decompose it over the observed indexes set Iş as
((*)) -I==1 (ZZ). where g (Zz) (i5() = WZ m V=*). Our goal is estimate the variational parameters {a, b, nt S, {m ]= and make (() close to p(OID).
To do so, we use Expectation Propagation (EP) (Minka 2001), to update (()). However, the standard EP cannot work because the complex Tucker form of likelihood term in makes intractable to compute the expectation of the likelihood term L, under q( ()). Thus,
the advanced moment-
matching technique, Conditional Expectation Propagation (CEP) (Wang and Zhe 2019) to address this issue. With CEP, we employ factorized approximation decouple the likelihood term I, into group of message factors {fn):
K fn (Z (i2)) (13)
Nyn vec(W) (U'(i)) (tft) @
Znfa ,")
,
where Zn is the normalized constant, fa(() Gam (un, b,), (n() (Veeee(W) Un, S.).
fa (a (?h) = -N (7%) m V*) are the message factors obtained from the conditional moment-
matching of E.]. Then we update the posterior (T)) and g(VV) by simply merging the message
factors from likelihood (13) and priors:
((T) p())
(T) Gam(rlao, bu) II
n=1 Gam(Tlan; bn);
(14)
((W) = p(W) N
= (Vveo((W) 0, II. N N(vee((W) pns Sn).
=
(15)
The message approximation in 13) and message merging (14 15 are based on the conditional
moment-matching technique and the property of exponential family presented in (Wang and Zhe 2019). All the involved computation is glosed-form and can be conducted in parallel. We provide the details along with the introduction of EP and CEP in the appendix.
4.2 SEQUENTIAL STATE INFERENCE WITH BAYESIAN FILTER AND SMOOTHER
Handling (Z is bit more challenging. It is because the prior of Z has chain structure 9 and we need to handle complicated integration over the whole chain to obtain marginal posterior ((b)) in the standard inference. However, with the classical Bayesian filter and smoother method, we can infer the posterior efficiently. Specifically, the state space structure over Z is:
q( (Zz)) q(Z
(Z())
(16)
DED
where p (ZZ is the transitions of the state given in IT)) and DE is the subset of D. including the observation entries whose k-mode index is ii. namely. D (n: v i, D). If we treat Ihep: fn (Z²), a group of Gaussian message factors, as the observation of the state space model, (66 is the standard Kalman Filter(KF) (Kalman 1960). Thus,
run the KF algorithm and compute gZz)) from s = to N₂ sequentially. After the forward pass over the states, we can run RTS smoothing (Rauch et al. 1965) backward pass to compute the global posterior of q( (Z.). This sequential inference is widely used to infer state space GP models (Hartikainen and Särkkä 2010 Särkkä 2013).
5

---

Published as a conference paper at ICLR 2024
Algorithm 1 FunBaT
Input: Observations D of K-mode continuous indexed tensor, kernel hyperparameters, sorted unique indexes set {Iz} of each mode.
Initialize approx. posterior q(T), s(W). (a(Z*)) and message factors for each likelihood.
repeat
for k = 1 to K do
Approximate messages factors (.) in parallel with CEP Update the approximated posterior ((7), g(W) by merging the message (14) 15
13
Update (Z") sequentially by KF and RTS smoother based on (16)
end for until Convergence
Return: g(T).
The whole inference algorithm organized as follows: with the observation D, we initialize the approximated posteriors and message factors for each likelihood. Then for each mode, we firstly approximate the message factors by CEP in parallel, and then merge them to update ((T), ((W). We run the KF and RTS smoother to infer q( (Z*) at each observed index by treating the message factors as observations. We repeat the inference until convergence. We summarize the algorithm in Table T
Algorithm Complexity. The overall time complexity is O(NKR), where K is the number of modes, N is the number of observations and R is pre-set mode rank. The space complexity is O(NK(R+ + R²)), as we need to store all the message factors. The linear time and space complexity w.r.t both data size and tensor mode show the promising scalability of our algorithm.
Probabilistic Interpolation at Arbitrary Index. Although the state-space functional prior of U or 2 is defined over finite observed index set In. we highlight that we can handle the probabilistic interpolation of Z at arbitrary index it & Ia after model inference. Specifically, with 2 1% 1/ we can infer the (it) by integrating the messages from the transitions of its neighbors and will obtain a closed-form solution:
(12) 1 (
/ ((Z)))) (z, (1)) dz; -N(m", V*) (17)
We leave the detailed derivation in the appendix. This enables us to build a continuous trajectory for each mode, and predict the tensor value at any indexes, for which standard discrete-mode tensor decomposition cannot do.
Lightweight Alternative: FunBaT-CP. As the CP decomposition a special and simplified case of Tucker decomposition, it is straightforward to build functional CP decomposition with the proposed model and algorithm. Specifically, we only need to set the Tucker core W all-zero constant tensor except diagonal elements as one, skip the inference step of W, and perform the remaining steps. We then achieve FunBaTT-CP P a simple and efficient functional Bayesian CP model. In some experiments, we found TunBaTT-CP is more robust than FunBaTT, the dense Tucker core takes more parameters and is easier to get overlitting. We will show the experiment section.
RELATED WORK
The early work to apply tensor format to function factorization is (Schmidt 2009). takes the wrapped-GP to factorize the function with CP format and infers with Monte Carlo sampling, which is not scalable to large data. Applying tensor-traing tensor-train(TT) to approximate the multivariate function with low-rank structure is a popular topic (Gorodetsky et al. 2015 Bigoni et al. 2016 [allester-RRpoll jet al., 2019, Chertkov et al.) 2022 2023), with typical applications simultaneous localization and mapping (SLAM) (Aulinas et al.) 2008). These methods mainly use Chebyshev polynomials and splines as function basis, and rely on complex optimization methods like alternating updates, Bigoni et al. (016) and ANOVA (analysis of variance) representation (Ballester-Riol) et al. 2019 Chertkov et al. 2003). Despite the compact form, they
cross-interpolation (Gorodetsky al. 2015
are purely deterministic, sensitive to initial values and data noises, and cannot provide probabilistic inference. Fang et al.) (2022) 2024) used similar techniques like state-space GP and CEP to our work, but they are designed to capture temporal information in tensor data. More discussions the existing literature can be found in the appendix.

---

Published as a conference paper at ICLR 2024
6 EXPERIMENT
6.1 SYNTHETICC DATA
We first evaluated FunBaT on synthetic task by simulating rank-1 two-mode tensor with each mode designed as a continuous function:
(h) exp(-22j) sin( 2 02 sin (2mi2) cos(22r22.
(18)
The ground truth of the continuous- mode tensor, represented as a surface in Figure Ia was obtained
by: y/ (4 UU We randomly sampled 650 indexes entries from [0, 1] [0. 1] and added Gaussian noise ~ (0, 0.02) as the observations. We used Py'Torch to implement FunBaT, which used the Matérn kernel with v 3/2, and set / 0.1 and o² 1. We trained the model with R = and compared the learned continuous-mode functions with their ground truth. Our learned trajectories, shown in Figure [c and Figure Tdlcearrl revealed the real mode functions. The shaded area the estimated standard deviation. Besides, we used the learned factors and the interpolation (17) to reconstruct the whole tensor surface, shown in Figure [b] The numerical results of the reconstruction with different observed ratios can be found in the appendix.
training data
25
2.
Ground Truth
-3
21
06
28
02 64
08 07
0.
0
0.
0.5
0.0
0.5
1.0
0.0
0.5
1.0
(a) Real tensor surface
(b) Estimated tensor surface
(c) ( (i))
(d) U²
Figure I: Results Synthetic Data
6.2 REAL-WORLD APPLICATIONS
Datasets We evaluated FunBaT four real-world datasets: BeijingAirrPM2..5, BeijingAir-PM10
BeijingAirS022 and US-TEMP. The first three are extracted from BeijingAi which contain hourly measurements of several air pollutants with weather features in Beijing from 2014 to 2017. We selected three cectinoou--nnddxee modes: (atmospheric-pressur,, temperature, time) and processed into three tensors by using different pollutants (PM2.5, PM10, SO2). Each dataset contains 17K observations across unique indexes of 428 atmospheric pressure measurements, 501 temperature measurements, and 1461 timestamps. We obtain US- TEMP from the ClimateChange The dataset contains temperatures of cities worldwide and geospatial features. We selected temperature data from 248 cities in the United States from 1750 to 2016 with three continuous-indexed modes: (latitude, longitude, time). The tensor contains 56K observations across 15 unique latitudes, 95 longitudes, and 267 timestamps.
Baselines and Settings We set three groups of methods as baselines. The first group includes the state- of-art standard Tucker models, including P-Tucker (Oh et al. 2018): scalable Tucker algorithm that performs parallel row-wise updates, Tucker-ALS: Efficient Tucker decomposition algorithm using alternating least squares(ALS) update (Bader and Kolda 2008), and Tucker-SV) (Hoffman al. )013): Bayesian version of Tucker updating with stochastic variational inference The second group includes functional (ensor-train(FTT) based methods with different functional basis and optimization strategy: FTT-ALS [Bigoni al. 2016), FTT-ANOVA
gallester-Ripolle et al. 2019). and FTT-cross (Gorodetsky et al.) 2015). As we can view the prediction task as regression problem and use the continuous indexe features, we add RBF-SVM: Support Vector Machine (Hearst et al. 1998) with RBF kernels, and BLR: Bayesian Linear Regression (Minka 2000) as competing methods as the third group
https: htttps://archiveee wes.icssuuue http://hhhheei....u..eeeee////aa Ming-uuu ti ci-Sitte Site Air r Quality-Data
https://berker http://bkrkkeyyyaaar.....//////e

---

Published as a conference paper at ICLR 2024
RMSE
MAE
Datasets
PM2.5
PMI0
SO2
PM2.5
PMI0
SO2
Resolution: 50 x 50 x 150
P-Tucker
0.805 0.017
0.787 + 0.006
0.686 + 0.02
0.586
0.003
0.595 1 0.005
0.436 0.011
Tucker-ALS 1.032 1 0.049
1.005 + 0.029
0.969 1 0.027
0.729 1 0.016
0.741 1 0.007
0.654 0.034
Tucker-SVI 0.792 + 0.01
0.8 + 0.026
0.701 = 0.08
0.593 + 0.01
0.605 + 0.019
0.423 + 0.031
Resolution: 100 x 100 x 300
P-Tucker
0.8 1: 0.101
0.73 1 0.021
0.644 + 0.023
0.522 1 0.011
0.529 1 0.013
0.402 0.008
Tucker-ALS 1.009 + 0.027
1.009 + 0.026
0.965 = 0.023
0.738 + 0.01
0.754 = 0.007
0.68 + 0.011
Tucker-SVI
0.706 + 0.011
0.783 = 0.067
0.69 0.086
0.509 + 0.008
0.556 + 0.031
0.423 + 0.031
Resolution: 300 300 1000
P-Tukeer
0.914 + 0,126
1.155 + 0.001
0.859 = 0.096
0.401 + 0.023
0.453 + 0.002
0.366 + 0.015
Tucker-AL3
1.025 + 0.044
1.023 = 0.038
1.003 = 0.019
0.742 + 0.011
0.757 : 0.011
0.698 + 0.007
Tucker-SVI
1.735 0.25
1.448 + 0.176
1.376 0.107
0.76 + 0.033
0.747 = 0.028
0.718 0.023
Resolution: 428
501 1461
(original)
P-Tucker
1.256 + 0.084
1.397 + 0.001
0.963 = 069
0.451 + 0.017
0.493 0.001
0.377 + 0.019
Tucker-ALS 1.018 0.034
1.012 + 0.021
0.997 0.024
0.738 0.005
0.756 0.007
0.698 0.011
Tucker-SVI
1.891 + 0.231
1.527 + 0.107
1.613 = 0.091
0.834 + 0.032
0.787 + 0.018
0.756 0.014
Methods using continuous indexes
FTT-ALS
1.020 0.013
1.001 0.013
+
0.705
0.696
0.011
FTT-ANOVA
2.150 +: 0.083
2.007
1.788 0.086
1.499 1.613
0.018
FTT-cross
0.942 + 0.025
0.933
0.026
0.018
0.561
0.467 0.033
RBF-SVM
0.995 + 0.015
0.955
0.006
0.668 0.008
0.486 0.674
BLR
0.998 + 0.013
0
0.837
0.001
0.736 0.007
0.739
0.573 +.000
FunBaT-CP
0.296 + 0.018
0.386
0.009
+
0.233
0.242 0.003
FunBaT
0.288 + 0.008 0.328 = 0.004
0.38
0.183 0.006 0. 0.226 0.002 0.241 + 0.004
Table 1: Prediction error over BejjingAio-MM..5, BeijüngAir-PM100 and BeijingAirS022 with R = 2, which were averaged over five runs. The results for I = 3, 5, are in the supplementary
RMSE
MAE
Modc-Rank
R=3
R=5
R=7
R=3
R=5
R=7
T-Tkkeer
1.306 1 0.02
1.223 + 0.022
1.172 1 0.042
0.7822 1 0.011
0.675 1 0.014
0.611 + 0.007
Tucker-ALS
10
> 10
10
10
> 10
10
Tucker-SVI
1.438 0.025
+
1.39
0.907
0.908
0.875
FTT-ALS
1.613 = 0.0478
+.10==00052
0.055
0.967
0.953
0.942
FTT-ANOVA
5.486 0.031
+
0.059
4.768 0.002
3.123 4.026
FTT-cross
1,415 + 0.0287
+.02200002
0.886 1.205
0.822
0.773
RBF-SVM
2.374 0.047
+
0.017
0.015
0.01
0.01
BLR
2.959 0.041
2.959
+ 0.041
2.959
2.029
0.011
2.029
2.029 0.011
FunBaT-CP
0.805 0.06 0.548
+ 0.03
0.551
0.048
0.448
- 0.06
0.314
FunBaT
1.255 0.108
1
+ 0.117 1.16=
0.736 + 0.069
0.647 + 0.05
0.572 1572:000 0.089
Table 2: Prediction error of US-TEMP, which were averaged over five runs.
We used the official open-source implementations of most baselines. We use the TENEVA library (Chertkov et 2022 2023) test the FTT-based methods. We re-scaled all continuous-moot mode indexes to [0, 1 to ensure numerical robustness. For FunBaT,, we varied Matérn kernels (1/2, 3/2} along the kernel parameters for optimal performance for different datasets. We examined all the methods with rank R {2, 3, 5, 7). We set all modes' ranks to R. Following (Tillinghast ct al., 2020), we randomly sampled 80% observed entry values for training and then tested on the remaining. We repeated the experiments five times, and examined the average root mean-square--rror (RMSE), average meea-abboolute-mme (MAE), and their standard deviations.
To demonstrate the advantages of using continuous indexes rather than index discretization, set four different discretization granularities by binding the raw continuous indexes into several discrete-indexed bins on BeijingAir-PM2.5, BeijingAir- PMI.,, and BeijingAir-S02. We then derived
8

---

Published as a conference paper at ICLR 2024
Obzevered
Latent Trajectory Value
8
Diego..
Dallas,
Learned Trajectory
Chicago, Nashwille,
Saint Louis.
3.25
Latent Trajecto
Weshington, Denives Philadeiphis, Cinclnnatl..
Portland, Minneapolls..
3.00
Retsburgh, Arington.
Anchorage
Seattle Portiand.
2.75
Buffalo..
2.50
Lake Derteer Abuquequee.
Anchorage
2.25
25
30
35
40
45
50
55
60
70
80
100 110 120 130 140 150 Western Longitude(degree)
90
North Latitude(deggre)
(a) U' (f)): Mode of latitude
(b) Ufa): Mode of longitude
Figure 2: Learned mode functions US- (S-TEMP.
the tensors with four different resolutions: 428 x 501 x 1461 (original resolution), 300 300 x 1000, 100 x 100 x 300, and 50 x 50 x 150. We tested the performance of the first group (standard tensor decomposition) baselines on different resolutions. We tested FunBaT, FunBaT-CH and other baselines with continuous indexes at the original resolution.
Prediction Results. Due to space limit, we only present the results with
for BeijingAir-PM PM2.5,
BeijingAir-PM10. and ArrjingAir-O22 datasets in Table while the other results are in the appendix. The prediction results for USMEEMP are listed in Table 2 Our approach FunBaT and FunBaT-CP outperform the competing methods by significant margin in all cases. We found the performance of standard Tucker methods varies lot with different discrete resolutions, showing the hardness to decide the optimal discretization in real-works applications. We also found that the performance of FunBaT much better than FunBaT on US-TEMP. This might be because the dense Tucker core of TunBaT result in overfitting and is worse for the sparse US-TEMP dataset.
Investigation of Learned Functions We explored whether the learned mode functions reveal interpretable patterns that are consistent with domain knowledge. We run FunBaT on US- dataset with A 1, and plotted the learned latent functions of the three modes (latitude, longitude, time), shown Figure 2 and As the first two modes correspond to latitude and longitude, respectively,
Latent Trajectory Value
2.0
2.6
2.4
2.2
representing real geo-locations, we marked out the city groups (the shaded circles) and some city names in Figure 2a and 2b The U of the lantitude mode in Figure [2 shows clear decreasing trend from southern citics like Miami (in Florida) to northern cities like Anchorage (in Alaska), which is consistent with the fact that temperatures are generally higher in the south and lower in the north. The learned longitude-mode U in Figure 2bjexhibits steady trend from east west, but with region (the blue circle) with lower values near the Western longitude 110", That is the Rocky Mountain Area including cities like Denver and Salt Lake City with higher altitudes, and results in lower temperatures. The time-min mode U in Figure provides meaningful insights of the climate change in history. It reveals a gradual increase over past 260 years, with a marked acceleration after 1950. This pattern aligns with the observed trend of rising global warming following the industrialization of the mid-20th century. The function around 1750-177 1770, characterized by lower and oscillating values, corresponds to the Little Ice Age, well-documented period of global cooling in history.
2.0
1750
1880
1850
1900
1950
2000
Timelyear)
Figure 3: U Mode of time
CONCLUSION
We proposed FunBaT, a Bayesian method to generalize Tucker decomposition to the functional field to model iattinoous-indexe tensor data. We adopt the state-space GPs as functional prior and develop an efficient inference algorithm based on CEP. The results on both synthetic and rorl-world tasks demonstrate the effectiveness of our method.

---

Published as a conference paper at ICLR 2024
ACKNOWLEDGMENTS
This work has been supported by MURI AFOSR grant FA9550-20-1-0338 NSF CAREER Award IIS-2046295, and NSF OAC-2311685.
REFERENCES
Josep Aulinas, Yvan Petillot, Joaquim Salvi, and Xavier Lladó, The slam problem: a survey. Artificial Intelligence Research and Development, pages 363-371, 2008.
Brett W Bader and Tamara Kolda. Efficient matlab computations with sparse and factored tensors. SIAM Journal on Scientific Computing, 30(1):205-22) 231, 2008.
Rafael Ballester-Ripoll, Enrique G Paredes, and Renato Pajarola. Sobol tensor trains for global sensitivity analysis. Reliability Engineering & System Safety, 183:311-322, 2019.
Peter J Bickel and Kjell A Doksum. Mathematical statistics: basic ideas and selected topics, volume 1. volume 11 17. CRC Press, 2015.
Daniele Bigoni, Allan P Sinsig-Karuup and Youssef M Marzouk. Spectral tensor-train decomposition. SIAM Journal on Scientific Computing, 3245),42405 2016.
Andrei Chertkov, Gleb Ryzhakov, Georgii Novikov, and Ivan Oseledets. Optimization of functions given in the tensor train format. arXiv preprint arXiv:2209 2022. doi: 10.48550/ARXIV. 14808. URL /
arxia.iiiv....... http///xww.....///////////////// 14808 pdf
Andrei Chertkov, Gleb Ryzhakov, and Ivan Oseledets. Black box approximation in the tensor train format initialized by anova decomposition. SIAM Journal on Scientific Computing, 45(4): A2101-A2118, 2023.
Shikai Fang, Zheng Wang, Zhimeng Pan, Ji Liu, and Shandian Zhe. Streaming bayesian deep tensor factorization. In International Conference Machine Learning, pages 3133-3142 PMLR, 2021.
Shikai Fang, Akil Narayan, Robert Kirby, and Shandian Zhe. Bayesian continuous-time tucker decomposition. In International Conference on Machine Learning, pages 6235-6245 PMLR, 2022.
Shikai Fang, Xin Yu, Shibo Li, Zheng Wang, Mike Kirby, and Shandian Zhe. Streaming factor trajectory learning for temporal tensor decomposition. Advances in Neural Information Processing Systems, 36, 2024.
Alex A Gorodetsky, Sertac Karaman, and Youssef M Marzouk. Function-train: a continuous analogue of the tensor-train decomposition. arXiv preprint
2015.
R. A. Harshman. Foundations of the PARAFAC procedure: Model and conditions for an "eeeeelaaator""nm mwmlimmumooe factor analysis. UCLA Working Papers in Phonetics, 16:1-84, 1970.
Jouni Hartikainen and Simo Särkkä. Kalman filtering and smoothing solutions to temporal gaussian process regression models. 2010 IEEE international workshop on machine learning for signal processing, pages 379-384. IEÉE, 2010.
Marti A. Hearst, Susan T Dumais, Edgar Osuna, John Platt, and Bernhard Scholkopf. Support vector machines. IEEE Intelligent Systems and their applications, 13(4):18 10:1188221 28, 1998.
Matthew D Hoffman, David M Blei, Chong Wang, and John Paisley. Stochastic variational inference. Journal of Machine Learning Research, 2013.
Masaaki Imaizumi and Kohei Hayashi. Tensor decomposition with smoothness. In International Conference on Machine Learning, pages 1597-1606. PMLR, 2017.
Rudolph Emil Kalman. A new approach to linear filtering and prediction problems. 1960.
Peter Lancaster and Leiba Rodman. Algebraic riccati equations. Clarendon press, 1995.
10

---

Published as a conference paper at ICLR 2024
Yisi Luo, Xile Zhao, Zhemin Li, Michael K Ng. and Deyu Meng. Low-rank tensor function representation for multi-dimensional data recovery. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023.
Ben Mildenhall, Pratul P Srinivasan, Matthew Tancik, Jonathan T Barron, Ravi Ramamoorthi, and Ren Ng. Nerf: Representing scenes as neural radiance fields for view synthesis. Communications of the ACM, 65(1):99-106 106, 2021.
Thomas Minka. Bayesian linear regression. Technical report, Citeseer, 2000.
Thomas Minka. Expectation propagation for approximate bayesian inference. In Proceedings of the Seventeenth conference Uncertainty in artificial intelligence, pages 362-369, 2001.
Anthony Nouy. Low-rank tensor methods for model order reduction. arXiv preprint
arXiv:1511.0155. 2015.
Sejoon Oh, Namyong Park, Sael Lee, and Uksong Kang. Scalable Tucker factorization for sparse aensors-algorithms and discoveries. 2018 IEEE 34th International Conference on Data Engineering (ICDE), pages 11 120-1131. IEEE, 2018.
Ivan V Oseledets. Tensor-train decomposition. SIAM Journal on Scientific Computing, 33(5):
2295-23135 2317, 2011.
Ravdeep S Pasricha, Ekta Gujral, and Evangelos E Papalexakis, Adaptive granularity in tensors: A quest for interpretable structure. Frontiers in Big Data, 5:929511, 2022.
Prashant Rai. Sparse low rank approximation of multivariate functions Appiicat-ppliiiaiinnns in uncertainty
guantification. PhD thesis, Ecole Centrale de Nantes (ECN), 2014.
Carl Edward Rasmussen and Christopher K. I. Williams. Gaussian Processes for Machine Learning. MIT Press, 2006.
Herbert E Rauch, F'Tung, and Charlotte T Striebel. Maximum likelihood estimates of linear dynamic systems. AIAA journal, 348):1445-1450, 1965.
Simo Särkkä. Bayesian filtering and smoothing. Number 3. Cambridge University Press, 2013.
Mikkel N Schmidt. Function factorization using warped gaussian processes. In Proceedings of the 26th Annual International Conference on Machine Learning, pages 921-925 2009.
Matthew Tancik, Pratul Srinivasan, Ben Mildenhall, Sara Fridovich-Kell Nithin Raghavan, Utkarsh Singhal, Ravi Ramamoorthi, Jonathan Barron, and Ren Ng. Fourier features let networks learn high frequency functions in low dimensional domains. Advances in Neural Information Processing Systems, 33:7537-7547, 2020.
Conor Tillinghast, Shikai Fang, Kai Zhang, and Shandian Zhe. Probabilistic neural-kerne tensor decomposition. 2020 IEEE International Conference on Data Mining (ICDM), pages 531-540. IEEE, 2020.
Ledyard Tucker. Some mathematical notes three-mode factor analysis. Psychometrika, 31: 279-311, 1966.
Zheng Wang and Shandian Zhe. Conditional expectation propagation. In UAI, page 6, 2019.
11

---

Published as a conference paper at ICLR 2024
A CONDITIONAL EXPECTATION PROPROGATION
A. BREIF INTRODUCTION OF EXPECTATION PROPROGATION(EP)
As general framework of Bayesian learning, expectation proprogation (EP) (Minka 2001 is variational inference method for computing the posterior distribution of latent variable 0 given the observed data D. The exact posterior p(0 (D can be formed following form:
7 II (D 0), 10), p D) - 7
(19)
where P., (D,, 0) the likelihood of n-th data given which may also link to the priors. the normalizer to ensure (19) is valid distribution. The exact computation of (19) is often infeasible
due to the complex form of pm (D,, 0). Therefore, EP naturally adopts a factorized distribution
approximation of real posterior:
p D) = {0 (D) x II (O),
(20)
where (0) is an approximated factor for the actual data likelihood factor pr. (D,, 0). If we can assign proper form of Sn. (0). and make (O) RS pr, (D, 0) as close possible, then we can obtain a good approximation of the posterior distribution. The EP algorithm sets the basic form of factors using the exponential-famity distribution, which includes many distributions such as Gaussian, Gamma, and Beta, and has the canonical formula fn (0) ~ exp(A, A ø (0)). where X, and ó (0) are regarded as natural parameter and sufficient statistics of fn (0), respectively. EP iteratively updates the parameters each fn (0) until convergence. The procedure to update each fa (0) is as follows:
(1) Building a calibrating distribution (a.k.s context factors) by removing current approx. factor:
(0) < (0)/fn fn (0)
(21)
(2) Building a tilted distribution by multiplying back the real data likelihood with context factors:
Pr. (0) ~ p.. (D, 0))\n(()
(22)
(3) Building new approximated posterior q" estimating its parameters by matching moments with the tilted distribution:
Eg (o(0) E.p., (o(0)
(23)
(4) Updating the approximated posterior fn by removing context factors from the new approximated posterior:
(0) x 9" (0)//'²)
(24)
A.2 CONDITIONAL MOMENT MATCH
With complex data likelihood where multiple variables interleave, such as the Tucker-based likelihood in the main paper, computation of Eg(óf ()) in moment match step of classic EP is intractable. To solve this problem, (Wang and Zhe 2019) proposed a conditional moment matching method to
update the parameters of the approximated posterior fo (Z* (12)). Take zt (Zg) for example, the
required moments are o(Z) (Z (1z) Z (iz) z
T ). the key idea of CEP is to decompose
the expectation into nested structure:
Ep(((()) =E E
E 2z) (z))
()()) + (+)p
(25)
( (z))))))
where (z)) A 0z| ((Z)) Therefore, we can compute the conditional moment first, i.e., the inner expectation, with an analytical form. However, the computation of exter-expectation are under the marginal tilted distribution which is still intractable. To solve this problem, we follow the BCTT
12

---

Published as a conference paper at ICLR 2024
(Fang et al. 2022 and apply multivariate delta method 2021 to get.
(Bickel and Doksum
2013
Fang et al.
E o( (
[pn) = Pn (Eg + (x))
(26)
where Pu. denote the conditional moments of tilde distribution.
A.3 CEP UPDATE FOR FUNBAT AND FUNBAT-CP
Following the above paradigm. we can work out the updating formulas for all parameters of the approximated message factors {frr = {ff (())))== (T() for FunBaT and FunBaT-CP.
For fn (Z (it) = (HZ (i) m ) st = (E,, F,, D- m =
(27)
where
a = Wie) (U* (
((h-1) x U (i)),
(28)
and W(x) is the folded tucker core W at mode k.
For fa Gam (T) 3n), the updating formulas are:
on = 2 Bn Uh -yn + trace EE, aa
(29)
where:
a, = vec(W)) (U' 8... (..)))..UUUKC (ix))
(30)
The updating formulas of fa()) NV(voc(W)) [4n Sn) is:
S. = (B,[T Eb, pm Sn (T
(31)
Where
b, = U 8 .... (ix)
(32)
We can apply the above update formulas for FunBaT T-CP by setting the W as constant diagonal tensor. However, We can also re-derive a more convenient and elegant form based on the Hadmard product form of CP. It will result in the similar formats on the update formulas of fn(T) and fa (Z*(2))) )). but
V/ with the different definitions on a and a. They are:
a/. U'
U o U UKK
(33)
and
a, U (1) o......... U (ik)
(34)
A.4 DERIVATION THE PROBABILISTIC IMPUTATION AT ANY INDEX
To derive the probabilistic imputation equation 17) in the main paper, we consider general state space model, which includes sequence of states Xi...., XM and the observed data The states are at time 4, .,..., !M respectively. The key of the state space model is that the prior of the states is Markov chain. The joint probability has the following form,
p XM, D) p(x)) II p(x}+1(x5) (
(35)
Note that here we do not assume the data likelihood is factorized over each state. like those typically used Kalman filtering. In our point process model, the likelihood often couples multiple states together.
13

---

Published as a conference paper at ICLR 2024
Suppose we have some posterior inference to obtain the posterior of these states (X1,, ......... XM ) and we can easily pick up the marginal posterior of each state and each pair of the states. Now we want to calculate the posterior distribution of the state time t such that tmm. t* < tm+1 Denote the corresponding state by x" our goal to compute p(x" (D). To do so, we consider incorporating x" in the joint probability (35)
p(x X x", Xm+ XM, D)
p(xi) II p p(x,+1|x;) p(x" Xm (m))pm)xxmm 1p(xx+1(x'))
p(xj+1(x;) (())x ,,)). (36) j=m+1
II
Now, we marginalize out XI:M)(M (m.++1) {xI..... Xm-1- xm+2............................ XM3. Note that since does not appear in the likelihood, we can take it out from the integral.
p(Xms S, D)
= =f p(x1) j=1
mn-1
M
II p(x)+1l)))
II p(x)+1lx;) (ND)x)....,,)))e XM)dx (.)m j=++1
p((x" (e(x*Iexx 1xmm ) (0
p(xm> Xm+1, D)p(x" ( Xxm (xm)p(xxe (xxm++1x*x))
(37)
p(xm+1| Xmm)
Therefore, we have
p(xm. Xm+1, x" (D) x p(xm; Sm+1lD))(s (xm ) (xxx+x//x))
(38)
Suppose we are able to obtain p(xm: Xm+ ID) As ((xm; Xm++). 1. We now need to obtain the posterior of x". In the LTI SDE model, we know that the state transition is Gaussian jump. Let us denote
p(x1Ix,) = Xm Q1), ((Xm+1/x.) = N Nxxm+||2xx Q2)..
We simply merge the natural parameters of the two Gaussian and obtain
p(xms Xm+1, x (D) = p(Xm xm+11 [D) m2, V*),
(39)
where
(V') = Q + A. A (V*)-² m² Q Aixm + A Q Xm+
(40)
B MORE DISCUSSION ON THE RELATED WORK
Functional Tensor Models. Modeling the inner smoothness and continuity in tensor data in functional manner has been a long-standing challenge and gets increasing attention recent years. Early work like (Schmidi 2009) uses GP with CP form model the functional tensor, but lacks efficient inference to handle large-scale data. The community of low-rank approximation of black- box approximation has raised series of work based on functional tensor-trainte trannTr, such as (Gorodetsky et al. 20155 Bigoni et al. 2016 [Balester-Ripol et al. 2019 Chertkov et al. (2023). However, the series work of FTT depends on tensor-train format and polynomials-based approximation, which is not flexible enough and sensitive to hyperparameters. The similar idea of functional basis has also been used to model the smoothness tensor decomposition (Imaizumi and Hayashi 2017). Most recent work (Luo et al. 2023) also employs the Tucker format and uses the MIP to model the tensor mode functions and shows promising results. However, most of the existing models are purely deterministic and lack the probabilistic inference handle data noise and uncertainty. In contrast, FunBaT is the first work to use the Tucker format to model the functional tensor probabilistic manner, and it enjoys the advantage of the linear-cos: inference handle large-scale data with uncertainty due to the usage of state-space GP.
Differce between FunBaT, BCTT and SFTL. From the technical perspective, BCTT (Fang et al. 2022) and SFTL (Fang et al. 2024) are the most similar work to FunBaT. Those methods utilize the state-space Gaussian Processes (GP) model the latent dynamics in CP/Tucker decomposition
14

---

Published as a conference paper at ICLR 2024
Number of training samples
Observed Ratio
RMSE
130
0.1
0.128
260
0.2
0.102
390
0.3
0.068
420
0.4
0.041
650
0.5
0.027
0.6
0.026
810
0.7
0.025
Table 3: Reconstruct loss of the synthetic data over different observed ratios.
and infer with pressage-passing techniques. This similarity has been noted in short in the related works section of the main paper, and we plan to highlight this more prominently subsequent versions on their differences. The first difference is that BCTT and SFTL focus time-series tensor data, whereas, FunBaT is centered on functional tensor data. This difference in application leads to distinct challenges and modeling: BCTT involves Tucker-core dynamic with static factors, SFTL models time-varying trajectories factors and FunBAT employs a static core with groups of mode-wise dynamics. This fundamental difference in formulation leads varied inferences. Due to the divergent formulations, the inference algorithms between the two methods show significant differences. For each observation, BCTT and SFTL need to infer only one state of the temporal dynamics, as they share the same timestamp, and the inference of all dynamics is synchronous. In contrast, FunBaT requires inferring multiple states of multiple dynamics, depending on each mode's index of the observation, which is more challenging. We will run a loop over tensor mode to do mode-wise conditional moment matching, and then get the message factors fed different functions. The inference of multiple dynamics is asynchronous.
Connection Broader Coordinate-basee based Representation Model. The idea of building parame- terized models (MLP) to map the low-dimensional continuous coordinates to high- dith-dmmeniionll data voxel has boosted attention in recent years, especially in the scenarios computer vision and graphics. The prior work CPNN TTncik et al. 2020) tracks the challenges of classical Gcoordinate-basee MLP" models and proposes the Fourier feature to improve the performance, Nerf (Mildenhall et al.) 2021 one of the most crucial works in graphics recently, uses large positional encoding MLP to reconstruct continuous 3D scenes from a series of 2D) images, which utilizes the Fourier features of the spatial coordinates better capture the high frequency. FunBaT could be seen generalization of these coordinate-based models to the tensor format, The main difference is that FunBaT has dimensional-wiin functional representation, which means the mode-wise functions are independent and can be learned separately. This dimensional-wi tensor data, and the learned mode-wise functions can be easily interpreted and visualized. We believe that the idea of FunBaT can be extended to the broader coordinate- representation model and applications in CV and graphics, and we plan to explore this in future work.
representation fits the low-rank structure of the
C MORE EXPERIMENTS RESULTS
The reconstruction loss of the synthetic data over different observed ratios shown in Table
The prediction results on R [3,5,7 7} SesjingAAr-PM22..0 BeijingAir-PMo PM10, and BeijingAir-SO2 are list Tables Tables Tables
15

---

Published as a conference paper at ICLR 2024
RMSE
MAE
Datasets
PM2.5
PMI0
502
PM2.5
PMI0
S02
Discrete resolution:50
50 150
P-Tucker
0.812 0.054
0.779 0.015
0.668 0.015
0.566 + 0.018
0.56 0.013
0.423 0.005
Tucker ALS
1.063
0.049
1.036 0.062
0.965 + 0.029
0.727 0.021
0.738 0.022
0.628 0.01
Tucker-SVI
0.758 0.017
0.8 4 0.051
0.652 + 0.014
0.554 + 0.014
0.583 0.017
0.421 0.038
Discrete resolution:100 x 100 300
P-Tucker
0.794 0.099
0.767
0.005
0.683 + 0.043
0.486 + 0.009
0.512
0.014
0.402 0.015
Tucker-AI. AL.S
1.02 + 0.032
1.011
0.023
0.97 + 0.027
0.738 + 0.011
0.751 0.004
0.681 0.016
Tucker-SVT
0.681 0.027
0.741 0.071
0.714 + 0.128
0.468 + 0.014
0.526 0.043
0.421 0.038
Discrete Resolution: 300 x 300 x 1000
P-Tucker
1.493 0.125
1.439 0.001
1.264 0.2
0.532 + 0.084
0.575 0.001
0.46 + 0.46 0.004 0.699 = 0.007
Tucker- ALS
1.027 + 0.033
1.027 + 0.038
1.007 + 0.017 1.451 1 0.062
0.743 + 0.01
0.758 + 0.012
Tucker-SVI
1.637 + 135
1.408 1 0.052
0.79 1: 0.016
0.768 1 0.016
0.732 0.016
Discrete Resolution:428 x 501 x 1461 (original)
P-Tucker
2.091 + 0.122
2.316 + 0.001
1.48 + 0.1 1 0.023
0.756 + 0.002
0.79 + 0.001
0.556 - 0.017
Tucker-ALS
1.008 1 0.013
1.027 1 0.036
0.738 1 0.005
0.757 1: 0.009
0.699 1 0.01
Tucker-SVI
1.864 + 0.03
1.686 1 0.061
1.537 + 0.121
0.885 1 0.016
0.864 0.015
0.787 0.032
methods using continuous indexes
FTT-ALS
1.019 + 0.013
1.001 + 0.013
1.002 + 0.026
0.744 +: 0.007
0.755 + 0.007
0.696 = 0.011
FTT-ANOVA
2.151 + 0.032
2.006
0.036
1.788 + 0.031
1.623
1.499 = 0.018
FTT-cross
0.943 0.026
0.002
0.845
0.566 0.018
0.561 0.011
0.467 0.033
RBF-SVM
0.995
0.02
0.024
0.668 + 0.008
0.674
0.486 0.026
BLR
0.998 0.013
0.977
0.837
0.736 + 0.007
0.799
0.573 0.009
FunBaT-CP
0.294 0.016
0.347 0.036
0.384
0.183 - 0.006
0.236
0.0014
0.242 0.003
FunBaT
0.291 0.017
0.348 0.036 0.036
0.386 0.011
0.183 0.01
0.233
0.012 +
0.241 0.004
Table 4: Prediction eиror over BeijingAir-PM2.. BeijingAir-PM10, and BeijingAir-SO2 with R = 3,
which were averaged over five runs.
16

---

Published as a conference paper at ICLR 2024
RMSE
MAE
Datasets
PM2.5
PMI0
SO2
PM2.5
PMI0
S02
Discrete resolution:50
50 150
P-Tucker
0.835 0.078
0.787 = 0.077
0.745 0.046
0.54 + 0.007
0.535 + 0.007
0.424 0.004
Tucker ALS
1.178
0.055
1.123 0.033
0.975 = 0.029
0.741 0.014
0.741 0.009
0.615 0.006
Tucker-SVI
0.738 0.022
0.747 + 0.009
0.638 0.01
0.518 0.011
0.541 0.007
0.405 0.022
Discrete resolution:100 x 100 300
P-Tucker
0.808 0.065
0.827 + 0.012
0.763 0.023
0.474 + 0.014
0.494 0.008
0.426 0.005
Tucker-AI. AL.S
1.07
0.03
1.038 + 0.022
0.953 + 0.015
0.745 + 0.01
0.756 0.008
0.675 0.007
Tucker-SVT
0.768 0.105
0.79 + 0.085
0.691 + 0.087
0.471 + 0.038
0.524 + 0.035
0.405 0.022
Discrete Resolution: 300 x 300 x 1000
P-Tucker
2.153 0.271
1.972 + 0.001
1.486 + 0.054 1.007 + 0.02
0.784 + 0.054
0.859 0.001
0.624 0.02
Tucker- ALS
1.062 + 0.031
1.046 + 0.029
0.747 + 0.01
0.76 + 0.01 0.805 1 0.012
0.699 = 0.007
Tucker-SVI
1.584 + 0.092
1.446 + 0.035
1.511 1 0.065
0.828 1 0.037
0.781 0.026
Discrete Resolution:428 x 501 x 1461 (original)
P-Tucker
2.359 + 0.078
2.426 + 0.001
1.881 + 0.054
1.011 + 0.021
1.094 + 0.001
0.775 - 0.017
TuS
1.045 1 0.02
1.056 + 0.021
0.999 1 0.025
0.74 +: 0.005
0.761 1: 0.007 0.879 0.008
0.698 1 0.01
Tucker-SVI
1.574 0.088
1.603 + 0.024
1.536 + 0.05
0.842 1 0.026
0.812 - 0.018
methods using continuous indexes
FTT-ALS
1.019 + 0.013
1.000 0.013
1.001 + 0.026
0.744 +: 0.007
0.754 4: 0.005
0.695 = 0.010
FTT-ANOVA
2.149 0.033
2.006 0.011
1.987 0.036
1.788 0.031
1.623 0.014
1.499 0.018
FTT-cross
0.941 + 0.024
0
0.831 + 0.015
0.563 0.018
0.560 0.011
0.464 0.033
RBF-SVM
0.995
0.955
0.794 0.026
0.668 + 0.008
0.674
0.486 0.026
BLR
0.998 0.013
0.977 0.014
0.837 = 0.021
0.736 - 0.007
0.739
0.573 0.009
FunBaT-CP
0.292 + 0.013
0.352 1 0.035
0.385 1 0.009
0.183 - 0.007
0.236 0.013
0.243 0.003
FunBaT
0.288 0.012
0.338 0.03
0.388 + 0.003
0.191 0.021
0.231 + 0.005
0.241 0.003
Table 5: Prediction eиror over BeijingAir-PM2.. BeijingAir-PM10, and BeijingAir-S02 with R = 5,
which were averaged over five runs.
17

---

Published as a conference paper at ICLR 2024
RMSE
MAE
Datasets
PM2.5
PMIO
S02
PM2.5
PMI0
SO2
Discrete resolution:50 x 50 x 150
P-Tucker
0.832 + 0.037
0.823 + 0.112
0.811 + 0.058
0.521 + 0.005
0.529 - 0.009
0.425 1.00
GPTF
0.694 + 0.013
0.702 + 0.005
0.607 + 0.016
0,488 = 0.012
0.501 = 0.005
0.381 #.000
CP-ALS
1.385 + 0.209
1.092 + 0.04
1.016 0.087
0.804 0.048
0.751 0.013
0.635 0.003
Tucker-Al ALS
1.379 0.046
1.314
1.049 + 0.025
0.779 0.014
0.775 = 0.015
0.621 0.01
Tucker-SVI
0.742 + 0.023
0.721 0.01
0.67 + 0.027
0.504 0.014
0.508 0.004
0.408 0.001
Discrete resolution:100 x 100 300
P-Tucker
1.015 0.063
0.911 0.063
0.877 0.039
0.51 0.012
0.521 0.018
0.452 + 0.009
Tucker- ALS
1.086 + 0.041
1.047 + 0.015
0.967 = 0.029
0.745 = 0.013
0.755 = 0.007
0.682 - 0.011
Tucker-SVI
0.783 + 0.054
0.787 + 0.052
0.702 + 0.054
0.464 0.017
0.515 0.021
0.408 0.011
Discrete Resolution: 300 x 300 x 1000
P-Tucker
1,718 0.155
- 1.928 0.001
1.629 - 0.05 1.007 + 0.018 1.389 + 0.042
0.805 = 0.043
0.954 0.001
0.687 + 0.006
Tucker ALS
1,073 + 0.039
1.062 + 0.02
0.748 + 0.01
0.762 0.01
0.699 : 0.006
Tucker-SVI
1.437 + 0.051
1.499 + 0.027
0.793 + 0.032
0.812 = 0.013
0.772 0.018
Discrete Resolution:428 x 501 x 1461 (original)
P-Tucker
2.134 + 0.174
2.483 + 0.001
2.001 + 0.149
0.924 + 0.044
1.055 = 0.001
0.786 + 0.012
Tucker-Al.
1.051 + 0.02
1.054 + 0.029
1 + 0.021
0.741 + 0.005
0.76 = 0.008
0.698 + 01.1
Tucker-SVI
1.292 + 0.021
1.521 0.095
1.454 + 0.054
0.737 + 0.015
0.872 0.027
0.817 0.028 0.022
Methods using continuous indexes
FTT-ALS
1,019 0.013
1.000 0.013
1.001 0.026
0.744 1 0.007
0.754 0.005
0.695 0.010
FTT-ANOVA
2.149 + 0.033
0.006
+.083
1.788 - 0.031
1.623
0.014 0.014
0.0018
FTT-cross
0.941 + 0.024
0.002
0.001
0.563 = 0.018
0.560=0.00
4..033
RBF-SVM
0.995 + 0.015
0.02
0.794
0.668 = 0.008
0.674
0.486
BL
0.998 1 0.013
0.007
0.837 0.021
0.736 1 0.007
0.739 0.008
0.573
PCMT-CP
0.296 0.023
0.335 0.02
0.385 1 0.008
1 0.009
0.231
0.007
0.242 0.000
PCMT-Tucker
0.318 + 0.014
0.317
0.39 + 0.009
0.198 + 0.008
0.235 + 0.008
0.242 0.000
Table Prediction error over BeijingAir-PM2.5,, BeijingAir-PM10, and BeijingAir-SO2 with R
which were averaged over five runs.
18


Reviews:
[{"review_id": "P9iReUCbFd", "reviewer": "Reviewer_MPRT", "summary": "The paper proposes a method called Functional Bayesian Tucker Decomposition (FunBaT) to handle continuous-indexed tensor data. Traditional tensor decomposition methods are designed for discrete and finite-dimensional indexes, but real-world data often contains continuous indexes such as geographic coordinates. FunBaT solves this problem by treating the continuous-indexed data as the interaction between the Tucker core and a group of latent functions. Gaussian processes are used as functional priors to model the latent functions, and an equivalent stochastic differential equation is used to reduce computational cost. The paper introduces an efficient inference algorithm based on advanced message-passing techniques. FunBaT outperforms existing methods in synthetic and real-world applications while being able to identify interpretable patterns. The paper also provides explanations on tensor decomposition, function factorization, the use of Gaussian processes as state-space models, and the FunBaT model and algorithm. Overall, the paper offers a solution for tensor decomposition on continuous-indexed tensor data.", "questions": "1. What are the advantages of this function tensor decomposition method compared to existing approaches? Utilizing the Tucker decomposition allows for the representation of higher-order tensors and effectively mitigates the problem of the curse of dimensionality. Why does the Tucker decomposition lead to a more compact and flexible low-rank representation compared with TT and CP?\n2. What is the primary motivation behind this paper in comparison to other function CP, Tucker, and TT decompositions? If this method was established solely because GP-based functional Tucker decomposition does not currently exist, I believe the motivation appears to be rather weak.\n3. What is the primary innovation of this paper? Apart from considering continuous tensor indices in the modeling, is there a deeper level of differences compared to Bayesian Continuous Tucker Decomposition?", "limitations": "", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. Extension to continuous-indexed tensor data: The paper proposes a method called Functional Bayesian Tucker Decomposition (FunBaT) that extends traditional Tucker decomposition to handle continuous-indexed tensor data. \n2. Utilization of Gaussian processes as functional priors: FunBaT models the latent functions in continuous-indexed data using Gaussian processes as functional priors. This allows for flexible and efficient modeling of the latent functions. Additionally, the paper reduces computational cost by constructing an equivalent stochastic differential equation, further enhancing the efficiency of the method.\n3. Improved performance and interpretable patterns: FunBaT is demonstrated to outperform existing methods in both synthetic and real-world applications. It not only achieves lower prediction errors but also has the ability to identify interpretable patterns in the data.", "weaknesses": "1. The paper's innovation and significance may not be convincingly conveyed. The idea of using Gaussian processes to model N factor matrices for continuous data appears to be straightforward and intuitive. Furthermore, the paper's modeling approach is very similar to that of reference [3], which employs Gaussian processes to model core tensors. \n2. There are now many function decomposition models based on Tucker decomposition, e.g., [1,2], so the Introduction section regarding related works appears to be inaccurate.\n3. Furthermore, the significance of this paper isn't fully articulated. Merely stating that there is currently no function Tucker decomposition doesn't seem to provide a sufficiently compelling reason for its importance.\n4. Regarding the optimization algorithm, this paper is also closely related to some previous work, e.g., [3]\n[1] M.Imaizumi, K.Hayashi (2017). \"Tensor Decomposition with Smoothness\". PMLR: International Conference on Machine Learning 2017\n[2] Luo Y, Zhao X, Li Z, et al. Low-Rank Tensor Function Representation for Multi-Dimensional Data Recovery[J]. arXiv preprint arXiv:2212.00262, 2022.\n[3] Fang S, Narayan A, Kirby R, et al. Bayesian Continuous-Time Tucker Decomposition[C]//International Conference on Machine Learning. PMLR, 2022: 6235-6245.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "K3bHdclvpM", "reviewer": "Reviewer_wp4E", "summary": "The paper proposed an Bayesian method for tensor Tucker decomposition, where the data are assumed to have continues index. Each column of the factor matrices is modeled using Gaussian process (GP), and efficient method was proposed to infer the parameters of the GP. The idea of this paper is quite interesting,\nand the experimental results are convincing.", "questions": "1, Does the equation 4 implies the smoothness of z(x)?\n2, Compared to Tucker decomposition, CP decomposition offers several advantageous properties, particularly its uniqueness, making it more favored in many applications. However, it is worth noting that CP decomposition typically requires larger factor matrices compared to Tucker decomposition. Therefore, it is interesting to discuss the computational complexity of the proposed method in relation to the rank of the tensor and evaluate its applicability in scenarios that require setting a large rank.", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1, the idea that model factor matrices using GP is new.\n2, efficient inference methods are proposed to determine the unknown parameters.", "weaknesses": "1, the methods need to set the rank of the tensor manually, which is usually unknown in practice.\n3, In table 1, the MSE of the proposed method much lower than the baselines. But no explanation is provided to show why.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "0b2eYyTgFj", "reviewer": "Reviewer_BN7i", "summary": "This paper proposed a function Tucker decomposition for tensors with continuous-valued indices. The authors firstly use a Gaussian process to map indices to Tucker factors and then contract these factors to obtain the entry value. To efficiently learn the GP prior, an algorithm based on state-space GP and expectation propagation is derived. For experiments, the authors test the model on synthetic data and several spatiotemporal data imputation tasks.", "questions": "1. Compared with GP regression, the main difference of the proposed model is preserving the Tucker structure in the final layer. I am wondering why this construction is so helpful as shown in empirical results. \n2. There are matrix inversion and multiplications in the update rules. Why is the time complexity linear with the rank $R$? \n3. The authors adopted three resolution settings for BeijingAir datasets. What is the setting for US-TEMP?\n\ntypo: In paragraph above Figure 1, whore tensor -> whole tensor?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "The authors study the functional tensor decomposition for continuously-indexed tensors. This seems to be an interesting and novel topic in the field tensor decomposition and may have some new applications.", "weaknesses": "1. The setting of continuously-indexed tensors is new in the community of tensor decomposition. However, my main concern is how this task is related and different from traditional regression tasks. Why do we need such construction of tensors and what is the significance of the proposed function Tucker decomposition. Considering the experiments, the authors show applications in spatiotemporal data imputation. However, there are many existing methods, including interpolation like GP, VAE [1], GAN [2], LSTM [3], diffusion models [4] and many of their variants. Besides, the problem setting is very similar to CPNN [5]. For a better understanding of the paper, I think it might be better to have a discussion of these lines of works and empirical comparisons\n- [1]. Mattei, & Frellsen. (2019). MIWAE: Deep generative modelling and imputation of incomplete data sets. ICML.\n- [2]. Yoon, et al. (2018). Gain: Missing data imputation using generative adversarial nets. ICML.\n- [3]. Cao, et al. (2018). Brits: Bidirectional recurrent imputation for time series. NeurIPS\n- [4]. Tashiro, et al. (2021). Csdi: Conditional score-based diffusion models for probabilistic time series imputation. NeurIPS.\n- [5]. Tancik, et al. (2020). Fourier features let networks learn high frequency functions in low dimensional domains. NeurIPS.\n\n2. Since the proposed model employs nonlinear GPs, it might be better to compare with some nonlinear or GP-based tensor decompositions. Also, baselines for continuous-time tensor decompositions are also good choices, as the authors also mentioned in the related work.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Tqa5MTXf2e", "reviewer": "Reviewer_TaJq", "summary": "This paper introduces the Functional Bayesian Tucker Decomposition (FunBaT)\nalgorithm. Instead of treating the factors of the Tucker decomposition as\nmatrices with discrete row indexing, the authors instead think of the factors\nas latent Gaussian process function priors. Earlier work in this area of\n[Schmidt, ICML 2009] considers a simpler \"continuous CP decomposition\" for\ntensors.\nThe authors then make connections\nto stochastic differential equations (SDEs) and conditional expectation\npropagation (CEP) for the learning task, which is somewhat analogous to a\ncontinuous version of alternating least squares. The authors provide experiments\non synthetic and real-world datasets, of order $K=2$ and $K=3$ respectively.", "questions": "### Questions\n- [page 01] Is Tucker decomposition really a more compact low-rank\n  representation? Maybe the factor matrices can be more compact, but doesn't\n  this come at the cost of a larger \"core structure\" then, e.g., CP\n  decomposition?\n- [page 03] By \"successive derivatives of $m$-th order\", do you instead mean to\n  write $\\frac{d^m f(x)}{dx^{m}}$?\n- [page 03] \"the trained model cannot handle new objects with never-seen\n  indices\" -- this is not necessarily true, and is the core idea behind tensor\n  completion.\n- [page 04] Why do you use $\\tau^{-1}$ as Gaussian noise (i.e., variance)\n  instead of $\\sigma^2$?\n- [page 06] Is the preset (scalar) mode rank $R$ for a given Tucker core of\n  shape $(r_1, \\dots, r_K)$ the product? I.e., $R = \\prod_{i=1}^K r_i$? If so,\n  this should be stated explicitly and you need to remove the claim that \"the\n  linear costs of both time and space ...\" since $R$ is exponential in the order\n  $K$.\n\n### Typos and suggestions\n- [page 01] \"there were finite\" --> \"there are finite\"\n- [page 01] \"tensor train(TT)\" --> \"tensor train (TT)\"\n- [page 01] \"Tucker decomposition is famous for ...\" --> \"Tucker decomposition\n  is widely-used for ...\"\n- [page 01] \"and time..\" --> \"and time.\"\n- [page 01] \"or simpler CP format\" --> \"or the simpler CP format\"\n- [page 01] \"which is with more compact ...\" --> \"which is a more compact ...\"\n- [page 02] Inconsistent subheading capitalization (see ICLR style guide and\n  example paper)\n- [page 02] \"under the following settings\" --> \"under the following setting\"\n- [page 02] typo at end of sentence with \"groups of latent factors\" (should be\n  a comma instead of a period)\n- [page 02] suggestion: consider superscripting factor matrices/vectors as\n  ${U}^{(k)}$ instead of ${U}^k$ to remove any ambiguity in meaning\n- [page 02] \"The classic CANDECOMP/PARAFAC (CP)\" --> \"The classic CP\" (you\n  already introduced the acronym)\n- [page 02] missing space: factors: $y_{i}$ (same for many other sentences on\n  this page)\n- [page 03] \"denote as $f \\sim$\" --> \"denoted as $f \\sim$\"\n- [page 03] consider using $\\ell$ instead of $l$ as a hyperparameter in the\n  Matern kernel\n- [page 03] missing space: \"efficient $O(n)$ inference\"\n- [page 03] No need to give the meaning of \"FunBaT\" again\n- [page 04] \"preset latent rank:$\\{r_1,\\dots,r_K\\}$ --> \"preset latent rank\n  $(r_1, \\dots, r_K)$. Missing space between subsequent sentences.\n- [page 04] suggestion: when you shift to \"continuous-indexed tensors\", it may\n  be more clear to say that $(i_1^{n},\\dots, i_K^{n}) \\in \\mathbb{R}^{K}$\n- [page 04] missing space: \"Namely, \"\n- [page 05] typo: \"massage merging\" --> \"message merging\"\n- [page 06] punctuation typos in Algorithm 1 inputs\n- [page 06] suggestion: Substantially more discussion / math can be given to\n  FunBaT-CP. The focus of the paper is on Tucker decomposition, but this is a\n  very nice special case for which it would be nice to know more of its\n  properties.\n- [page 06] Missing space: \"function factorization is(Schmidt, 2009).\" along\n  with other citations in this paragraph.\n- [page 07] typo: \"alternating least square\" --> \"alternating least squares\".\n  Many more typos / punctuation errors in this paragraph.\n- [page 09] typo: \"lantide mode\"", "limitations": "", "rating": 5, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 3, "strengths": "- Extends work from Tucker to CP, since CP is a generalization of Tucker if you\n  restrict the core tensor to be diagonal\n- A \"continuous\" version of Tucker decomposition on the factors is an excellent\n  idea to explore\n- Uses interesting real-world datasets with continuous features (indices), and\n  the results are comprehensive and compelling. That said, the US-TEMP\n  experiment could be improved since a core shape of $(1,1,1)$ is used, which\n  means a product of the factor functions is being learned.", "weaknesses": "- There is not enough emphasis on related work, especially with [Fang et al.,\n  ICML 2022], which appears to be a *very similar paper*. This discussion should happen\n  much earlier in the paper, and there should be more than one mention of this\n  Fang et al. (2022), as it it almost gets buried in its current form. Missing\n  seminal work:\n  * \"Bayesian Tensor Regression\" by [Guhaniyogi et al., JMLR 2017]\n- While the synthetic experiments have the potential to be very strong and\n  compelling, it would be useful to see how the tensor reconstruction looks as a\n  function of the number of samples. It may not be surprising in its current\n  form that we can recover ground truth with $650$ samples.\n- In the Beijing Air experiments, please explain if non-uniform core shapes were\n  explored (as they might better help fit the data). A recent ICML paper\n  investigates core shape selection and could be of interest:\n  * \"Approximately Optimal Core Shapes for Tensor Decompositions\" by [Ghadiri\n    et al., ICML 2023]", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (poster)