Online Functional Tensor Decomposition via Continual Learning for Streaming Data Completion
Authors: Xi Zhang
Yanyi Li
Yisi Luo
Qi Xie
Deyu Meng
Venue: NeurIPS 2025 spotlight ()
Keywords: data stream; tensor decomposition
Primary area: general_machine_learning
TL;DR: We propose a novel online functional tensor decomposition (OFTD) framework for streaming data completion.

Abstract: Online tensor decompositions are powerful and proven techniques that address the challenges in processing high-velocity streaming tensor data, such as traffic flow and weather system. The main aim of this work is to propose a novel online functional tensor decomposition (OFTD) framework, which represents a spatial-temporal continuous function using the CP tensor decomposition parameterized by coordinate-based implicit neural representations (INRs). The INRs allow for natural characterization of continually expanded streaming data by simply adding new coordinates into the network. Particularly, our method transforms the classical online tensor decomposition algorithm into a more dynamic continual learning paradigm of updating the INR weights to fit the new data without forgetting the previous tensor knowledge. To this end, we introduce a long-tail memory replay method that adapts to the local continuity property of INR. Extensive experiments for streaming tensor completion using traffic, weather, user-item, and video data verify the effectiveness of the OFTD approach for streaming data analysis. This endeavor serves as a pivotal inspiration for future research to connect classical online tensor tools with continual learning paradigms to better explore knowledge underlying streaming tensor data.

Online Functional Tensor Decomposition via Continual Learning for Streaming Data Completion
Xi Zhang!
Yanyi Li! Yisi Luo
Xie' Deyu Mengl.
Xi' an Jiaotong University, Xi' an, China
Pazhou Laboratory (Huangpu), Canton. China edu.cn, xi zhang@stu. xjtu. edu. en, yy106050f
Visiluo12210foraalll wtt.fffoxall..comm com, xie qi gi@mail.xjtu. edu. en, dymeng@mail jjju. edu. edu.
Abstract
Online tensor decompositions powerful and proven techniques that address the challenges in processing high-velocity streaming tensor data, such as traffic flow and weather system. The main aim of this work is propose novel online functional tensor decomposition (OFTD) framework, which represents a spatial- temporal continuous function using the CP tensor decomposition parameterized by coordinate- based implicit neural representations (INRs). The INRs allow for natural characterization of continually expanded streaming data by simply adding new coordinates into the network. Particularly, our method transforms the classical online tensor decomposition algorithm into a more dynamic continual learning paradigm of updating the INR weights to fit the new data without forgetting the previous tensor knowledge. To this end, we introduce long-tail memory replay method that adapts to the local continuity property of INR. Extensive experiments for streaming tensor completion using traffic, weather, user-item, and video data verify the effectiveness of the OFTD approach for streaming data analysis. This endeavor serves pivotal inspiration for future research to connect classical online tensor tools with continual learning paradigms to better explore knowledge underlying streaming tensor data.
Introduction
In real world, high- ditennsiinnii data often exist in streaming form and typically modeled as tensor streams (such as traffic flow and video) Yu et al. (2015), Smith et al. [2018). Tensor streams can be categorized into two types: single-aspect streams and multi-aspect streams. Single-aspect streams, such as traffic flow data represented by the triplet (location, route, and time), grow only along the temporal dimension and are modeled 3-mode temporal tensor streams. In contrast, multi-aspect streams, such as recommendation system data represented by the triplet (user, movie, and actor), grow along multiple dimensions simultaneously. With the increasing prevalence of streaming data, there is growing demand for real-time streaming data analysis (e.g., streaming data completion).
Online tensor decomposition Abed-Meraim et al. 2022a is one of the foremost methods to address the streaming data completion problem by exploiting the potential compact structures underlying streaming data within online optimization framework. However, decomposing tensor streams would lead to high computational costs owing to the significant growth in their volume over time. Also, dynamically capturing the internal latent properties of tensor streams, such spatial-temporal continuity, poses difficult challenges. To address these challenges, various online tensor decom- position methods have been developed to handle the streaming data completion problem. These
CCoresponding author.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

INR
Mi
INR
[U_... U.] U141
10
M11
INR
Coordinates
(a) Classical online CP decomposition
(h) Proposed online functional CP decomposition
Figure I: (a) Illustration of the classical online tensor CP decomposition paradigm. (b) The proposed online functional tensor decomposition paradigm for streaming tensor data. Our approach transforms the static online optimization algorithm into a more dynamic continual learning paradigm, which learns knowledge from streaming data by using the CP decomposition parameterized by INRs.
methods can be categorized into CP deccopposition-baseed methods Lee and Shin 2021 Zhong et al.
[202T] Ahn et al. [202T], Abed-Meraim et al. [002b]] [iuu et al. [2022], Tucker decomposition based
methods Jang and Kang 2023]. tensor SVD
methods Zhang al. [2016), Wu [2022).
Gilman et al. [0222] Yi et al. 2022]. and other tensor network decomposition based approaches (such as tensor train (TT) decomposition Le et al. (2024)) D. As recent symbolic works along this line, Ahn
jet al. [021] proposed a streaming tensor factorization with an attention- baasdd temporal regularization
for streaming data completion. (Abed-Meraim et al. (2022b) proposed scalable online CP algorithm
that efficiently estimates low-rank components from streaming tensors. Yi et al. [2022] proposed
an online algorithm based the t-SVD, which can efficiently capture the principal information of
tensor by incrementally updating a much smaller sketch. These methods enable real-time analysis of tensor streams and achieve promising performances for streaming data completion. Nonetheless, current online tensor decomposition methods mostly consider optimizing the factor matrices/tensors of the tensor decomposition using discrete online optimization rules, which may limit the capability for dynamic data structure modeling and spatial-tempore correlation excavation.
In this work, propose a novel online functional tensor decomposition (OFTD) method for streaming data completion (see Fig. bb) for quick view). OFTD represents the spatial-tempon streaming data continuous function using CP tensor decomposition parameterized by implicit neural representations (INRs) Sitzmann et al. (2020), Mildenhall et al. [202T], which map an arbitrary spatial-temporal coordinate the corresponding tensor value through deep neural networks. Instead of optimizing the factor matrices of tensor decomposition, OFTD optimizes the learnable weights of factor INRs during online optimization, which could better explore the complex dynamic structure and global spatial-temporry correlations of streaming data through a deep functional tensor representation. OFTD allows for natural characterizations of continually expanded streaming data by simply adding new coordinates into the INR network during online optimization. Although the tensor stream is continually growing, we maintain a constant number of optimization parameters by keeping the network structure unchanged.
Nevertheless. the incorporation of deep neural networks leads to the possible issue of forgetting the learned historical knowledge when fitting new data. Hence, to enable effective continual learning for processing streaming tensors using OFTD,, we make basic attempt by employing the memory replay method, which utilizes a part of historical data during online optimization, to avoid forgetting. We develop a regret bound of OFTD from the perspective of the loss (i.c., forgetting) of historical knowl- edge, which theoretically shows that the OFTD model tends to forget the knowledge that is distant from the new tensor data due to the local continuity property of INRs. Hence correspondingly design a memory buffer with long- tail distribution (i.e., storing more data at more distant positions), which substantially enhances the performance for streaming data completion. Consequently, OFTD delivers superior performances against traditional online tensor decomposition methods. showcasing its strong ability for streaming data analysis. We summarize the main contributions of this work as follows:
We introduce a novel online functional tensor decomposition (termed OFTD) method for streaming data completion. Our method employs the CP decomposition parameterized by INRs to learn spatial-temporal continuous function, which enables a concise representation of streaming tensor data by simply incorporating new coordinates into the INR during online optimization. Furthermore, our method effectively captures spatial-temporal continuity and low-dimensional compact structures of streaming data through functional representation.

---

Our approach transforms the static online algorithm for streaming data completion into more dynamic continual learning paradigm, i.e., the INRs are expected fit new data without forgetting previous tensor knowledge. To achieve this, we theoretically develop regret bound for OFTD, which guides us designing a long-tail replay continual learning method tailored for OFTD.
We apply OFTD to single-aspect (i.e., temporal evolution) and multi-aspect (spatial and temporal evolutions) streaming tensor completion. Extensive experiments on real-world datasets show the superiority of OFTD over state-of-the-art online tensor decomposition methods.
2 Related Work
2.1 Online Tensor Decomposition
Online analysis algorithms for streaming tensors have been widely developed in recent years Najafi
et al. [2019), Qian et al. [202T), Hu et al. [022]. CP decomposition is one of the mostly considered methods for streaming tensor completion. For instance, Minnh-Chinh et al. (2016) proposed two- stage CP decomposition algorithm to perform streaming data completion for third-order tensors. Lee
and Shin [2021] proposed a robust method for tensor streams by integrating CP factorization, outlier removal, and temporal-pattern detection to enable accurate online prediction. Zhong al. [2021] proposed window-based dynamic streaming tensor analysis method using CP decomposition. Yang et al. [023) proposed distributed streaming tensor completion method for multi-aspect streaming tensor. More CP-based online tensor decomposition methods can be found in Abee-Meraim et al. [2023]. Mardani et al. [2015]. Kasai [2019]. Song al. (2017]. Nimishakavi et al. [2018]. Du
ct al. 2018), Xiao et al. [2018). These online tensor methods explicitly optimize factor matrices of the tensor decomposition using discrete online optimization rules. Recently, |ang al. 2021 proposed a deep neural network-based streaming Bayesian tensor factorization method, which could better capture complicated nonlinear interactions in data. Different from this method, our OFTD employs INRs to model streaming data as a spatial-temporal continuous function, which allows for natural characterization of continually expanded data by adding new coordinates into INRs during online optimization, while OFTD could also capture nonlinear interactions of data through the deep representation of INR.
2.2 Implicit Neural Representation
In recent years, INR has attracted widespread attention for their ability to continuously and implicitly represent signals through neural networks Sitzmann et al. 2020). , Mildenhall et al. [2021]. Compared to traditional discrete representations, INR offers advantages such as memory efficiency, continuous trajectory modeling, and analytical computations of higher-order derivatives. Typical applications of
INR include 3D shape and scene reconstruction Atzmon and Lipman| (2020). Jiang et al. (2020) view
synthesis Mildenhall al. [2019] solving differential equations Chen al. [2023] and images/video
processing Chen et al. 2021]. Skorokhodov et al.] [2021]. For instance, the well-known NeRF Mildenhall
[ represents scene as continuous 5D function and optimizes neural radiance fields to synthesize novel views. [Sitzmann et al. [2020] introduced the SIREN, which leverages periodic activation functions achieve high-fidelity representation of natural signals using INR. Chen et al. [202T] proposed the local implicit image function, which encodes images as continuous functions and enables arbitrary-resolution image reconstruction. Currently, INR has become an increasingly popular research direction in mainstream AI fields Zhao et al. [2024), Shi et al.] [2024]. Li et al. 2024). To our knowledge, the proposed method should be the first work incorporate INR into the online tensor decomposition framework.
3 Proposed Method
3.1 Preliminaries
Basic notations are shown in Table 7 We use to denote the Kruskal operator Kolda and Bader [2009] The o and & respectively denote the outer product and the Hadamard product. The || denotes the Frobenius norm.
3

---

Table 1: Notations used in this paper.
Notations
Definitions
I, x, X
Scalar, vector, matrix, and tensor An N -order streaming tensor The i-th row the i-th column of X The (11, i2..... . )-th element of X The n-th factor of CP decomposition The vector [N] (1,2, "N
X E Rlixgxx-xXX
(i.)) (aa)
(n E U E Rear E
Definition (CP Decomposition Kolda and Bader 2009)). D. Given an N -order tensor X R R1IXTxx-xX15 1x its CP decomposition is the representation using N factor matrices (U(m) n=1 sharing the same number of columns as follows:
X = [
(1.) o (N)
(I)
(1,))*
where the factor matrix U E Ra The smallest integer satisfying referred to as the CP-rank X.
Definition 2 (Streaming tensor sequence). A sequence of Nth -order tensors (X,) is called streaming tensor sequence for any E Z+,, Xr C X++1 The grows with time, and X2 called the snapshot lensor taken at time I.
Definition 3 (Temporal tube). Given two successive tensors Xz-1 E RR x-xx"" and Ne E xf derived from streaming tensor sequence {X,}, the coming data (i.e., temporal tube) at time can be represented by Ve = Ne \ X1-1. which has the same size as N₂, with entries given by:
(Xx) (N)
3 < in S 1.
( (1) ix)
0
otherwise.
3.2 Problem Formulation
Given streaming tensor sequence {& with missing entries, aim to recover the missing data in the current snapshot X, Since X₁-1 S X, and we have handled X, 1 in previous time steps, the problem is equivalent completing the elements in )4 X, X2-1- Tensor streams often exhibit low-rank properties, making tensor decomposition models, such as the CP decomposition, suitable for modeling streaming tensors. The optimization problem at time t is typically formulated
yr - "enm),.))))) +R ( ((U!))).
min T
P&o
n=
where {U(")} denotes the set of tensor factors, P, is a binary tensor representing missing and observed entries of V,, and R(-) is a regularization term for the factor matrices. For instance, most tensor streams exhibit continuity (i.e., smoothness) over the spatial and time dimensions, e.g., the current pollutant measurement is similar to those of the previous and next minutes. Hence, smooth regularizations such as the total variation PPrallolaa et al. (2023) can be employed. However, in this work we do not impose an explicit regularization R(.). Instead, the proposed OFTD implicitly captures the spatial-temporal smoothness of data through the implicit smoothness of INRs (see Lemma
3.3 Online Functional Tensor Decomposition
In this section, give the detailed formulation of the proposed online functional tensor decomposi- tion for streaming data completion.
We use notations (e.g.. and for tensors by simply viewing tensor set containing its elements.

---

Functional Tensor Decomposition Before introducing the online algorithm, we first introduce the batch functional tensor decomposition (FTD) (i.e., off-line setting), which uses INRs to parameterize
factor matrices of the CP decomposition. Specifically, for the n-th CP factor matrix U E IR xr, we use an INR Sitzmann et al. [020] to parameterize it. Such INR a multilayer perceptron (MLP) fe.. (+) R + H' with parameters On which takes coordinate V E IR as input and returns vector:
fe.. (v) Wa(o(W
-.. -..---------... o(W)))) E IR"
where O. (W.) are weight matrices with W, E B for i 2.3...., and W E R 1xr
a = sin(wo-) is the sine activation function that is more effective for INR Sitzmann et al. 2020
To generate the factor matrix
U.()) we consider the parallel notation for 1 input coordinates:
Se. (v) (ve. (v(:))) E |
where v [r]] - (I, TT represents the input coordinate vector and V(a) denotes the i-th
element of the vector We use this output matrix fe.. (v) (i.e., fe., ((T))) as the factor matrix U.") of the CP decomposition and then obtain the FTD representation of an -odd-order streaming tensor X₂ as
N₂ = [1f. (LD) Σ fe. (4DD) o ... fex (LLD))))))))))
(4)
where fe,, (LT))) denotes the i-th column of the factor matrix Se,, ((7))). We have hence used N
INRs [fe.. ()) to parameterize the factor matrices of the CP decomposition.
The FTD in enjoys two potential advantages for streaming data analysis. First, the functional representation naturally allows us to model streaming data in an efficient way by simply adding new coordinates into the model (i.e., by expanding the coordinates [T,] during online optimization. With the expanding of coordinates over time, the output of INR (i.c., the factor matrix CP decomposition) correspondingly increases in size to fit the larger size of the new data stream. Second, the FTD effective for modeling real-world tensor data because the CP decomposition characterizes the intrinsic low-dimensional structure of the tensor, while the INRs capture the tensor spatial-temporal smoothness (see Lemma to better recover the unobserved entries.
Online Functional Tensor Decomposition We now introduce the proposed OFTD. We mainly describe the multi-aspect setting, and the single-aspect problem can be seen as a special case of the multi-aspect problem. In the online setting, a new tensor X, E xf1x arrives at each time t with missing values. We dynamically update the FTD model to accommodate the incremental growth of the streaming tensor. Our streaming tensor completion algorithm consists of the initialization stage and the online update stage.
Initialization Stage;
Given an initial streaming tensor X2 E x8 we optimize the following objective function based on the FTD representation (4)
Pi (Dr - (ZD))
min (e..)
where P1 is the initial mask, 31 = Xi is the initial observed tensor, O,, are learnable parameters of the INR fe.. and [7,] denotes the coordinates of the tensor [f.e. [(,e). (ZT))), at dimension n. To address the optimization problem, use gradient based methods (e.g., the Adam optimizer) to update the INR parameters ( ) with the loss (5).
Online Stage: the online update stage, a new tensor X₂ E x (t 2 2) comes at each time point and we need to update the online FTD model to accommodate the growth of streaming tensor sizes. We take the update of the n-th dimension as an example. At time t, the streaming tensor size grows by the scale of Ir -x- along the n-th dimension. We adapt to the new tensor size by simply adding T - / coordinates to the corresponding factor INR Se,, (.) R - IR.. Specifically, suppose that the coordinate vector at the last time point is [ (1, - then the new coordinate vector at the time point t is [[,] & (1,... , IT)T. Correspondingly. the new factor matrix of the CP decomposition along the dimension n at the time point is obtained by
Sea (TH)) A (Je. (1), Se. (ee.....)) (2), ............ Se. (T))) E x
5

---

Hestotral f
Hettoccal New
Data vatoe
Dana
New stream
Postion
Posttion
Replay
(a) Forgetting behavior of INR
(b) Long- maigttill memory replay
Figure 2: (a) Illustrations of the forgetting behavior of INR for streaming data completion. (a)-Left: The INR function fe., (z:) (blue line) fits the observed data (blue scatters) well. (a)-Right: When continually fitting the new data (green scatters) without memory buffer, the trained INR fe., (z) (green line) tends to forget the previously learned knowledge (blue scatters) that is distant from the current time position It due to the continuity of INR. (b)-Left: Beta distribution with different construct the
parameters and fixed a 1. We adopt the long-tail Beta distribution with memory buffer ((b)-Right) that adapts to the forgetting behavior of INR.
We illustrate the dynamic growth of the coordinates vector [Ig] and the corresponding OFTD model in Fig. [T)). Based on the enlarged OFTD model, we consider the following optimization problem during the online stage t:
P. & (Ji- ((DD) D)I (9..)
min
(6)
Similarly, use the Adam optimizer to update the INR parameters { {On) with the loss (6) at each time point f. Here, the observed data 3/+ = (X2 \ x1- R x contains the new data at time t, iee. the INRs {Se.. (-) are optimized to fit the new data in X; \ 1 with initialization weights {O,} being taken from the last time point (i.e., the continual learning). Such online optimization strategy is reasonable since it is impractical to use all historical data each time point, as this would result in large computational costs. Therefore, we consider only fitting the new data to optimize the continually expand the temporal coordinates
INRs in each time point t. For single-aspes aspect streams, of OFTD, while for multi-aspect streams, we need to continually expand the coordinates all dimensions 1.... N. Given total time T, the completion result is obtained by the FTD model (fe. ((A.) ) w x=1 E R! x after T optimization steps under the continual learning manner.
Notably, the streaming data completion using OFTD becomes classical continual learning paradigm, which learns the INR weights } from a continuous stream of information, with such information becoming progressively available over time Parisi et al. [2019] Nevertheless, OFTD may encounter forgetting of historical data during online optimization, which we will analyze and address next.
3.4 Theoretical Analysis
We interpret two insights of our method, i.e., the spatial-temporal continuity of OFTD brought from INR, and the regret bound of the OFTD model that reveals its forgetting behavior, which motivates us design memory replay alleviate forgetting. First, we show that OFTD method preserves the tensor spatial-temporal smoothness from the Lipschitz smooth perspective.
Lemma 1 (Lipschitz smooth bound for FTD). Let the tensor Xe E 1 satisfy FTD (4) where each factor function fe., : R - I2' (n 1,2,3) is an INR formulated as in (3) with activation function a(.) sin(wo.). Assume that each element of the weight matrix W in
follows i.i.d. NO0 w²), Then for any E (0, 1), any spatial coordinates v.v E E², where
(V(1), V(2)), (V()) V/2)) and any temporal coordinates k. k Z. with probability least
1 the following Lipschitz smoothness holds for Xe:
1 [(A)) - (Xi) Nay R))
< Cill-vvlly
Spatial
1( (())))))))))))))) - - (vor)) < Cilk-Kk'
Temporal smooth
where C1 = =/= (. w In 31 1913, 1515) is Lipschitz constant.

---

[emma[ssowss shows that the FTD model preserves the spatial- continuity of the estimated tensor Xr (i.e.. elements that are closer each other are more likely to share similar structures). Such continuity benefits the OFTD model by learning continuous and robust spatial-temporal function that enables more accurate completion results. The smooth bound is related to several factors (such as wi and w:). We experimentally evaluate such relationships in supplementary.
Lemma
Based on Lemmal | we present regret bound of OFTD regarding the loss of historical information when fitting new data streams, which reveals the forgetting behavior of INR. Such forgetting exhibits a unique characteristic it is model (see Fig. 2
due to the spatial- temporal continuity of the OFTD
Theorem 1 (Regret bound for online FTD). Denote the OFTD model learned at the historical time point by X₂ = [f.. ([T); e (iii where fe.. (TTl]: denotes the n-th factor function at time t. Assume that
The OFTD model learned at the new time point t + using (6) is invariant at the boundary (T[, T2, T2), iee, fa, (T,
fe., (T;: (n = 1,2, 2).
Each element of the weight matrix of the d-layer INRs {fe, (-) follows i.i.d. NOO, w²) with sine activation function (.)) = sin(wp.). The E--norm of derivative of each factor INR life, le, is bounded by k > O.
Then for any 8 E (0, and any historical position (i), 12, i3) (in S [... the following regret bound between the new OFTD model Re+1 = [fe.. ([1++1]; + 1)};=]) E R41 1x x15"1 (Take an example Ittl = Is + 1) and the old OFTD model Xe holds with probability at least 1 - 6:
(84+1)(6n - (24) (24) S C2 max (Ta + - -in); where C2 = 6(1) wgg 3d-3 + 2 2d 2
(8)
( m)x 1ig. 1515) and 1 = 2wr2 + w In 3
Theorem shows that the regret bound (i.e., the degree of forgetting) is proportional to the positional distance 17/ 1 in) between the considered point (i,, i2, ig) and the new data stream position Ta 1. This indicates that information that is more distant from the new data stream is more likely to be forgotten (see Fig. 2 (a)) To alleviate the forgetting, we design a long-tail memory replay continual learning method.
3.5 Continual Learning via Memory Replay
OFTD transforms the classical online tensor decomposition into a continual learning paradigm, which expects to use the INRs to fit new data without forgetting historical data. To alleviate forgetting, we design a long-tail memory buffer that utilizes part of historical data when fitting new data. Theorem shows that more distant information is more likely be forgotten. Thus we consider the long-tail Beta distribution (see Fig. 2bb) to construct a memory buffer, which stores more data that distant from the new data stream. For multi-aspect streams, the memory buffer M, E x--xx- at the time is constructed through sampling process on the historical data X-1 by (M ix) (8 ((i....,x)) .i)) if (i1, .in) I,, otherwise (Ma) (x) O, where I {(i....,, in = 1 II, of Beta(a, 1.... J,n I,... .N..
Here, 1-] denotes round-down and the p.d.f. of Beta(a, B) is Beta(z;; o, 3)
r(++0) 10(1- F(a)T(19)
2))-1, x E (0, 1). The Ze is the index set of the memory buffer such that the indexes in Z, follow a long-tail Beta distribution to store more information that is distant from the new data stream. This is achieved by setting appropriate a and 3 such that Beta(a, ,3) is a long-tail distribution (see Fig. 2(.... A total number of J indexes in I, are selected to construct the memory buffer M, For single-aspect streams, we perform sampling on the streaming dimension and store all indexes for other dimensions.
"This assumption is reasonable since data streams in real world often exhibit local continuity and hence the learned OFTD models at adjacent time points would be invariant at the boundary 1t
7

---

Table 2: The NRE results of single-aspect streaming data completion. Average running time during
each online update is reported.
Dutasel
Condition
Beijing
Mudrid
radar
chicugo
NRE Time (x)
SR
0.3
0.2 0.3
0.1
0.1 0.2
0.2
0.2 0.3
Grouse
1.532 1,504 1.254
2.978 2.884
1.647
1,276 1.113 L,033
2.263
1,866 0.899
4,125
2.033 1.861
1.005 1.003 1.002 1.697 0.0135
Grasta
1,580 1.044 1.021
1.179 1.119 1.066
1.161 1.065 0.899 1.091 1,050 1.002
1.497
1.391 1.193
L,026
1.013 0.975 1.132 0.0357
Petrels
0.940 0.609 0.567
1.077 0.607 0.34
0.696
0.287 0.180 1.504 1,362 1.034 1.068
1.016 0.831
(.7791 0.728 0.696 0.797
0.0234
TeCPSGD
1,025 0.985 0.738
1.317 0.912 0.842
0.891 0.201 0.145 4.583 2.683 1.807
1.670
1.254 0.933 0.838 0.753 0.730 1.239
0.0358
OLSTEC
0,506 0.272 0.226 0.185 0.167 0,445 0.222 0.155 0.723 0.535 0.435
0.876
0.613 0.558 (0.847 0.805 0.785 0.476
0.4694
SOFIA
0.125 0.065 0.217 0.218 0.171 00.322 0,250 0.243 1.794 1.532 1.411 1,660
1.469 1.326 (0.860 0.837 0.823 0,744
0.0002
STF
0.347 0.282 1.184
0.325 0.241 0.184
0.999 0,972 0,965 3:653 1.097 0.875
3,400
3,046 2.660 3,532 2.357 0.177 1,405
0.0030
ACP
0.870 0.563 0.497 00.966 0.871 0.518 1.033 1.001 0.976 1.090 1.024 1.028
1,050
1.012 0.975
6.016 1.012 1.019 0,918 0.0022
ATD
1,365 1,303 1.221
1.548
0.784
0.366
0.195 0.156 1.701 1,034
0.636
1.633
1.628 0.891
(0.928
0.782 0.736
0.953 0.0043
OFTDIOOrs)
0.116 0.093
00.156
0.131
0.138
0.125
0.118
0.528
0.433
0.379
0.869
0.827
0.660
0.507
0.444
0.1762
Involving the memory buffer during online optimization allows our model to retain historical data in M2 over time, preventing forgetting. By reformulating the online model (6), the new online optimization model at time with the memory buffer Mr is formulated
min )
Pe 2 (Di- - (1 DII
+ Σ VEI:
(Petv) @ (Mrcss - (vm))
Here, 3r. contains new data time and Mr includes part of historical data avoid forgetting. The more historical data in Mr (i.e.. the larger index set Iz), the more computational costs are needed for optimization. If we use all historical data to construct the memory buffer Mr, then OFTD degrades to the batch FTD method that processes the whole tensor X₂ at each time, We summarize the OFTD algorithm with memory replay in Algorithm A.T of the Appendix.
In) in each iteration, where (mrd) >n 1 4n) is the INR complexity, (r II Ln) is the CP product complexity, r is the CP rank, and m, are network's 's width and depth. We further propose a sharing strategy to improve the computational efficiency of OFTD, as detailed in Appendix A..
Given a tensor of size I1 X x IN. OFTD consumes O(mrd
To further improve the effectiveness of OFTD for recovering highly irregular data streams (such as dynamic background and abrupt changes in data streams), we further proposed a temporal online affine regularizer to address this special situation, which is introduced in the Appendix A.3
4 Experiments
We perform numerical experiments for both single-aspect and multi-aspect streaming data completion. We use the normalized reconstruction error (NRE) Ahn et al. [2021] for evaluation. The datasets are summarized in Appendix [C.I We consider the sampling rates (SRs) 0.1, 0.2, 0.3 (the proportion of observed entries w.r.t. all entries) to perform random missing. Various baselines are included: Grouse
Balzano et al. [210)]. Grasta He et al. [2012] Petrels Chi et al. [2013] TeCPSGD Mardani et al.
[0155] OLSTEC Kasai [2019]. SOFIA Lee and Shin (2021), ACP Abed- et al. [2023], ATD
Abed-Meraim et al. [2023]. STF Ahn et al. [2021]. SITTA Nimishakavi et al. [2018). OnlineSGD
(OSGD)|Mardani Mardani et al. 2015). and GOCPT Yang et al. 20222. More detailed experimental settings are put in Appendix
C.2
4.1 Experimental Results
The quantitative results for single-aspect and multi-aspect streaming data completion are shown in Tables and 3. OFTD attains better NRE results in most cases, showcasing its strong representation abilities and effectiveness for modeling tensor streams. This can be attributed to the compact, low-rank representation of CP decomposition and the expressive power of INRs to model dynamic structures of data streams. The OFTD achieves real-time updating with each online step costing less than 0.2/1.0 (single/multi-aspeet) seconds. Also, from Fig. 3 it can be observed that OFTD better reconstructs the temporal curves of tensor streams, showcasing its capability to model complex data structures and preserve the temporal smoothness of the tensor. Overall, OFTD serves new state-of- state-of-the-art the-art
8

---

Origbua
Origial
Orgbal
100
Trorporal Indie
Trinpccal Index
Teooponi Inder
3
Ontowson
OOCPT
OPTD
100
Tengunal insles
Figure 3: Temporal curves of spatial pixel (SR 0.3).
Table The NRE results of multi-an each online update is reported.
multi-aspect aspecc streaming data completion. Average running time during
Dataset
foreman
carphone
YELP
Avg. NRE Time
SR
0.1
0.2 0.3
0.1
0.2
0.3
0.1
0.2
0.3
SITA
0.257 0.252 0.249
0,557
0,556
0.464
0.415
0.368
0.340
0.384
0.020
OSGD
0.877
0,255 0,193
0,723
0.259
0.251
2.146
2.002
1.853
0.951
0.692
GOCPT
0.146
0.140
0,140
0,168
0.153
0.153
2.055
1.312
1.291
0.618
0,462
OFTD
0.094 0.087 0.084 0.128 0.119 0.111
0.371
0.291
0.267
0.172
online method for both single-aspect and multi- aspect streaming data completion. We show more experimental results in Appendix D
4.2
Ablation Study
The ablation studies include the tests for the memory buffer size JN in Table 4 and the Beta distribution parameter 3 (with a fixed to 1) in Table The memory buffer is effective to alleviate forgetting (compared to 0% in Table thus enhancing performances. However, a large buffer size leads to increased computational costs, and we have set the buffer size to 33% of the whole tensor size in experiments. When the Beta distribution parameter 33 > buffer, resulting in good performance (Table 5 and justifying our memory buffer design. More ablation results include the CP-rank r, the usage of INR and its parameters shown in Appendix D
obtain the desired long-tail memory
Table 4: Ablation study for the memory buffer size (the proportion of buffer size J w.r.t the whole tensor size). A larger buffer leads to improved accuracy while more floating-poii operations (FLOPs).
Dataset
Beijing
Madrid
Butter size
100%
50%
33%
0%
100'
50%
33%
NRB
0.1280
0.1999
0.1308
0.2903
0.1112
0. 160
0.1182
0.3979
FLOPS
87.97M
43.99M
29.33M
6.03M
139.42M
69.73M
46.5M 9.09M
Table 5: Ablation study for the parameter of the Beta distributed memory buffer. When 1 it degrades to the uniform distribution and B > 1 (or < 1) means long-tail (or heavy-tail) memory buffer.
Dataset
Beijing Madrid (average)
0.2
0.4
0.6
0.8
1.2
1.6
NRE
0.1273
0.1271
0.1280
0.14 0.1261 0.12 0.19 0,1233 0.1233
Conclusion
We have proposed novel streaming data completion method OFTD, which utilizes CP decomposition parameterized by INRs to model tensor streams. Future work can be considered to design other advancing continual learning methods to enable life-long learning of INRs for the streaming data completion problem. For example, we can consider using regularization-batt based [Sun et al. 2023 or gradient projection-based [in et al. (202) methods to further boost the performance OFTD under the continual learning framework. Also, applying the low-rank functional parameterization using INRs paves a novel paradigm for low-rank adaptation (LoRA) of large models, which can be considered in future work.

---

Acknowledgments
This work was supported by the National Natural Science Foundation of China (Nos. 124B2029, 62476214), the Major Key Project of PCL (No. PCL2024A06), the Tianyua Fund for Mathematics of the National Natural Science Foundation of China (No, 12426105), and the National Key Research and Development Program of China (No. 2024YFA1012000).
References
K. Abed-Meraim, L.. Trung, Adel Hafiane, et al. contemporary and comprehensive survey on streaming tensor decomposition. JEEE Transactions on Knowledge and Data Engineering, 35(1 1):18977-10921, 2022a.
K. Abed-Meraim. N. L. Trung. Adel Hafiane, et al. Robust tensor tracking with missing data and outliers: Novel adaptive CP decomposition and convergence analysis. IEEE Transactions Signal Processing, 70: 4305-43200 2022b.
K. Abed-Meraing N. L. Trung, Adel Hafiane, et al. Tracking online low-rank approximations of higher-order incomplete streaming tensors. Patterns, 4(6), 2023.
Dawon Ahn, Seyun Kim, and U Kang. Accurate online tensor factorization for temporal tensor streams with missing values. In Proceedings of the 30th ACM International Conference on Information & Knnolldgge Management, pages 2822 2826, 2021.
Nader Asadi, MohammadReza Davari, Sudhir Mudur, Rahaf Aljundi, and Eugene Belilovsky. Prototype- sample relation distillation: towards replay-free continual learning. In Proceedings of the 40th International Conference on Machine Learning. ICML' JMLR.org, 2023.
Matan Atzmon and Yaron Lipman. Sal: Sign agnostic learning of shapes from data. In Proceedings of the IFEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2565-2574, 2020.
Laura Balzano, Robert Nowak, and Benjamin Recht. Online identification and tracking of subspaces from highly incomplete information. In Annual Allerton Conference on Communication. Control, and Computing, pages 704-711, 2010.
Honglin Chen, Rundi Wu, Eitan Grinspun, Changxi Zheng, and Peter Yichen Chen. Implicit neural spatial representations for time- deeedeepndentt PDEs. In International Conference on Machine Learning, pages 5162- 5177, 2023.
Yinho Chen, Sifei Liu, and Xiaolong Wang. Learning continuous image representation with local implicit image function. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pages 8628-8638, 2021.
Yuejie Chi, Yonina C Eldar, and Robert Calderbank. Petrels: Parallel subspace estimation and tracking by recursive least squares from partial observations. IEEE Transactions on Signal Processing, 61(23):5947-53 5959, 2013.
Yishuai Du, Yimin Zheng, Kuang-chih Lee, and Shandian Zhe. Prohabilistic streaming decomposition. In 2018 IEFE International Conference on Data Mining (ICDM), pages 99-108, 2018
Shikai Fang, Zheng Wang, Zhimeng Pan, Liu, and Shandian Zhe, Streaming bayesian deep tensor factorization. In International Conference on Machine Learning, pages 3133-3142, 2021.
Kyle Gilman, Davoud Atace Tarzanagh, and Laura Balzano. Grassmannian optimization for online tensor completion and tracking with the t-svd. IEEE Transactions on Signal Processing, 70:2152 2167, 2022.
Hc, Laura Balzano, and Arthur Szlam. Incremental gradient on the Grassmannian for online foreground and hackground separation in subsampled video. In 2072 IEEE Conference on Computer Vision and Pottern Recognition, pages 1 5575-157, 2012.
Yexun Hu, Tai-Xiang Jiang, and Xi-Le Zhao. Degradation accordant plug-and-play for low-rank tensor completion. Proceedings of the Thirty-First First International Joint Conference on Artificial Intelligence, pages 1843-1849, 2022.
Jun-Gi Jang and U Kang. Static and streaming tucker decomposition for dense tensors. ACM Transactions on Knowledge Discovery from Data, -43, 2023.
10

---

Chiyu Jiang, Avneesh Sud. Ameesh Makadia, Jingwei Huang, Matthias Nießner. Thomas Funkhouser, et al. Local implicit grid representations for 3D scenes. Proceedings the IEEE/CVF Conference on Computer Vision and Puttern Recognition, pages 6001-6010, 2020.
Hiroyuki Kasai, Fast online low-rank tensor subspace tracking by CP decomposition using recursive least squares from incomplete ohservations. Neumocompuring, 347:177-190, 2019.
Tamara G Kolda and Brett W Bader. Tensor decompositions and applications. SIAM review, 51(3):455- 500, 2009.
Thanh Trung Le, K. Abed-Meraim, N. L. Trung, and Adel Hafiane. A novel recursive least-squares adaptive method for streaming tensor-train decomposition with incomplete observations. Signal Processing, 216: 109297, 2024.
Dongjin Lee and Kijung Shin. Robust factorization of real-world tensor streams with patterns, missing values, and outliers. In 2021 IEEE 37th International Conference on Data Engineering (ICDE), pages 840-851, 2021.
Jiayi Li, Xile Zhao, Jianli Wang. Chao Wang, Min Wang. Superpixel-inforee informed implicit neural representation for multi cultt--imennional data. In Proceedings of the European Conference on Computer Vision, page 2024.
Sen Lin, Li Yang, Deliang Fan, and Junshan Zhang. TRGP: Trust region gradicnt projection for continual learning. In International Conference on Learning Representations, 2022.
Chunsheng Liu, Wu, Zhifei Li, Ma, Jun Huang. Robust online tensor completion for 1oT streaming data recovery. IEEE Trunsactions on Neural Networks and Learning Syxtems, 34(12):0178--0112, 2022.
Zhen Liu, Hao Zhu, Qi Zhang, Jingde Fu, Weibing Deng. Zhan Ma, Yanwen Guo, and Xun Cao. Finer: Flexible spectral-bia tuning in implicit neural representation by variableperiodic activation functions. In 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 2713-2722, 2024. doi: 10.1 STWCCVVRR2733322...000
Morteza Mardani, Gonzalo Matcos, and Georgios B Giannakis. Subspace learning and imputation for streaming big data matrices and tensors. IEEE Transactions on Signal Processing, 63(10):2633-2677, 2015.
Yu-Chun Miao, Xi-L.e Zhao, Jian-Li Wang, Xiao Fu, and Yao Wang. Snapshot compressive imaging using factiiefa deep video prior. IEEE Transactions on Computational Imaging, 10:93-102, 2024. doi: 10. 10.1109/TC1.22233344444 3346330..
Ben Mildenhall, Pratul P Srinivasan, Rodrigo Ortiz-Cayon Nima Khademi Kalantari, Ravi Ramamoorthi, Ren Ng. and Abhishek Kar. Local light field fusion: Practical view synthesis with prescriptive sampling guidelines. ACM Transactions on Graphics (ToG), 38(4):1-14, 2019.
Ben Mildenhall, Pratul P Srinivasan, Matthew Tancik, Jonathan T Barron, Ravi Ramamoorthi, and Ren Ng. NeRF: Representing scenes as neural radiance fields for view synthesis. Communications of the ACM, 65(1): 99-106, 2021.
Truong Minh-Chinl Viet-Dung Nguyen, Nguyen Linh-Trung, and Karim Abed-Meraim. Adaptive parafac decomposition for third-order lensor completion. In 2016 IEEE Sixth International Conference on Communi- cations and Electronics (ICCE), pages 297-301, 2016.
Mehrnaz Najafi. Lifang He, Philip S. Yu. Outlier-robus multi-aspect streaming tensor completion and factorization, In Proceedings of the Twenty-Eige Eighth International Joint Conference on Artificial Intelligence, pages 3187-3194, 2019.
Madhav Nimishakavi, Bamdev Mishra, Manish Gupta, and Partha Talukdar. Inductive framework for multi- aspect streaming tensor completion with side information. In Proceedings of the 27th ACM International Conference on Information and Knowledge Management, pages 307-316, 2018.
German Parisi, Ronald Kemker, Jose L. Part, Christopher Kanan, and Stefan Wermter, Continual lifelong learning with neural networks: A review. Neural Networks, 113:54-71, 2019.
Monica Pragliola, Luca Calatroni, Alessandro Lanza, and Fiorella Sgallari. On and beyond total variation regularization in imaging: the role of space variance. SIAM Review, 65(3):601-685, 2023.
Cheng Qian, Nikos Kargas, Cao Xiao, Lucas Glass, Nicholas Sidiropoulos, and Jimeng Sun. Multi-version tensor completion for time-delayed spatio-temporal data. In Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence, pages
8 2021.

---

Vishwanath Saragadam, Daniel LeJeune, Jasper Tan, Guha Balakrishnan. Ashok Veeraraghavan, and Richard G. Baraniuk. Wire: Wavelet implicit neural representations. In 2023 IEEE/CVE Conference on Computer Vision and Puttern Recognition (CVPR), pages 18507-18516, 2023, doi: 10. 11L.1//CVPPS2222222..00001 2023.0175.
Kexuan Shi, Xingyu Zhou, and Shuhang Gu. Improved implicit neural representation with fourier reparameter- ized training. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pages 25985-25994, 2024.
Vincent Sitzmann, Julien Martel, Alexander Bergman, David Lindell, and Gordon Wetzstein, Implicit neural representations with periodic activation functions, Advances in Neural Information Processing Systems, 33: 14422-4773 2020.
Ivan Skorokhodov, Savva Ignatyev, and Mohamed Elhoseiny. Adversarial generation of continuous images. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10753-10764, 2021.
Shaden Smith. Kejun Huang. Nicholas D Sidiropoulos, and George Karypis. Streaming tensor factorization for infinite data sources. In Proceedings of the 2018 SIAM International Conference on Data Mining, pages 81-89, 2018.
Qingquan Song. Xiao Huang, Hancheng Ge, James Caverlee, and Xia Hu. Multi- streaming tensor completion. In Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 435-443, 2017.
Zhicheng Sun, Yadong Mu, and Gang Hua. Regularizing sccond-order influences for continual learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages June 2023.
Tong Wu. Online tensor low-rank representation streaming data clustering. IEFF Transactions on Circuits and Systems for Video Technology, 33(2):602-617, 2022.
Houping Xiao, Fei Wang, Fenglong Ma, and Jing Gao. cOTD: An efficient online tucker decomposition for higher order tensors. In 2018 IEEE International Conference on Data Mining (ICDM), pages 1326-1331, 2018.
Chaoqi Yang, Cheng Qian, and Jimeng Sun. Goept: Generalized online canonical polyadic tensor factorization and completion. In Proceedings of the Thirty-Firs First International Joint Conference on Artificial Intelligence, pages 2358-2354, 2022.
Keyu Yang, Yunjun Gao, Yifeng Shen, Baihua Zheng, and La Chen. DisMASTD: An efficient distributed multi- aspect strcaming tensor decomposition. In Proceedings of the ACM Turing Award Celebration Conference, pages 127-128, 2023.
Qianxin Yi, Chenhao Wang. Kaidong Wang, and Yao Wang. Effective streaming low-tubal-rank tensor ap- proximation via frequent directions. IEEE Transactions on Neural Networks and Learning Syxtems, 35(1): 1113-1 126, 2022.
Rose Yu, Dehua Cheng, and Yan Liu. Accelerated online low rank tensor learning for multivariate spatiotemporal streams. In International Conference on Machine Learning, pages 238-247, 2015.
Zemin Zhang, Dehong Liu, Shuchin Aeron, and Anthony Vetro. online tensor robust PCA algorithm for sequential 2d data. In 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 2438. 2016.
Jiaming Zhao, Wenbo Qiao, Peng Zhang, and Hui Gao. Quantum implicit neural representations. In Proceedings of the International Conference Machine Learning, volume 235, pages 60940-60956, 2024.
Xuemei Zhong, Junhua Chen, Lei Zhang, and Yan Zhang, Window-based dynamic strcaming tensor analysis based on CP decomposition. In 2021 4th International Conference Artificial Intelligence and Big Data (ICAIBD), pages 681-686. 2021.
12

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: Yes, the abstract and introduction clearly state the claims made, including the contributions and other important content.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
We provide the limitations and future works in Section
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g.. independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
13

---

We include theoretical results for our method in Section 3 and include the complete proofs in the Appendix B
Justification:
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: explain our experiments in Section 4 and provide the complete set of hyper-parameters in Appendix C
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, it may be necessary either make it possible for others replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, or other means that appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome deseribe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
14

---

Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Question:
Answer: [Yes]
Justification:
We provide a zip file that contains the code for the proposed OFTD.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.cc/pbbie/
güidessCoeSSubmisiioooolyyy for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (c.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (htips: wtwpaiiicggiiieeeeeeeebeiieeeeee
for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary understand the results?
Question:
Answer: [Yes]
Justification:
We provide information about the training details in Appendix C
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [No]
Justification: We do not provide error bars, however, run the same experiment with three different seeds and report the one with best performance.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
15

---

The method for calculating the error bars should be explained (closed form formula, call a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma enror bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are range (e.g. negative enror rates).
If error bars reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification:
We provide all the information in Appendix
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics
Answer: [Yes]
Justification: Yes, the research conducted in the paper, in every respect, with the NeurIPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g.. if there is special consid- eration due to laws or regulations their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification: The research does not have concerns about societal impacts because it is designed for streaming data.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
16

---

Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), faireess considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper deseribe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [No]
Justification:
We do not release any data or models.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: We properly cite the open source models which we use in the paper.
Guidelines:
The answer NA means that the paper does use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website). the copyright and terms of service of that source should be provided.
assetts are released, the license, copyright information, and terms of use package should be provided. For popular datasets, paperswithcode, htwpswtthhooeec.cmddtaaaeset has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
17

---

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's s creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer:
[Yes]
Justification:
We provide readme on how to run the code in order to reproduce our results.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moddll as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification: The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurIPS Code of Bthies, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: The paper does not involve crowdsoureing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, not include any information that would break anonymity (if applicable), such the institution conducting the review.
18

---

16. Declaration of LLM usage
Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology. scientific rigorousness, originality of the research, declaration is not required.
Question:
Answer: [NA]
Justification: The core method development in this research does not involve LLMs as any important, original, or non-standard components.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy https///eeurp...ccooomcreeeesess for what should or should not be deseribed.
19

---

A The Online FTD algorithm
A.1 Algorithm Description
Algorithm 1 Online FTD for Streaming Data Completion
Input: - tensor stream . Xr and masks Pi,... Pr; Maximum iteration number K; 0;
Initialization: Randomly initialize learnable parameters of INRs (O,, J I== 1; k
> (Initialization Stage)
1: Construct coordinate vectors {()))>=
2: for k S K do
3: Generate the estimated tensor via the FTD representation [[ ((T)))
n=
4: Compute the loss in using the observed tensor X1, mask Pi, and the generated tensor using
FTD;
5: Update INR parameters {On J. with the loss ( using the Adam algorithm; Set +
6: end for
b (Online Stage)
7: while 1. < T do
8: Construct coordinate vectors [[ by expanding the previous coordinate vectors; Set
k = 0;
9:
Construct memory buffer M, via Beta distribution;
10:
for k < K do
[ [fe,, (()))
11:
Generate the estimated tensor via the FTD representation
12:
Compute the loss in 9 using the observed tensor Xg, mask P., memory buffer Mr, and the generated tensor using FTD;
13:
Update INR parameters [On) with the loss 9 using the Adam algorithm; Set k <-
k: + 1: end for
14:
15:
Set / <- 1;
end while
Output: The recovered tensor [Se. ((H))
A.2
Computational Complexity and A Sharing Strategy
Here, we propose a sharing strategy to further reduce INR complexity. We share the first d layer weights of the N factor INRs and keep the last layer unshared. This reduces the INR complexity from O(mrd Em= In)) (mmr(d 1)í + mr Σ L,), where Î = max, Ln leading to reduction of running time around 50% without obvious accuracy degradation (Table
Table 6: OFTD (share) reduces running time around 50%.
foreman
carphone
Time (s)
SR
0.1
0.2
0.3
0.1
0.2
0.3
0.1
0.2 0.3
GOCPT
0.146
0.140
0.140
0.168
0.153
0.153
2.055
1,312
1,291
0.462
OFTD (share)
0.097
0.092
0.088
0.133
0.124
0.121
0,374
0.336
0.281
0.454
OFTD
0.094
0,087
0,084
0.128
0.119
0.111
0,371
0.291
0.267
0.898
A.3 Temporal Online Affine Regularizer
To further enhance OFTD for highly irregular data streams, we propose temporal online affine regularizer enhance the robustness of OFTD abrupt changes in temporal streams. We assume that the complete tensor stream X; E IR i an explicitly low-rank tensor [X]rr x and set per-frame affine transformations such that the full tensor can be expressed as transformed version of [X,].. This formulation allows the model to capture geometric or temporal abrupt changes in data streams while preserving the low-rank structure.
implicitly low-rank. In other words, there exists
20

---

Our affine method is inspired by the temporal affine transform introduced in Miao et al. (2024). but adapting this affine transform to online optimization is quite challenging. To address this challenge, we propose to decompose the transformation into structured components, including per-
frame translation (Iz: 9t), rotation 0,, and scaling St- The translation parameters (Iz, y1 3e are predicted via lightweight neural networks fe, E" - R with parameters Oz Qy which take each row of the lime-dependent CP factor matrix ((15)) E R as input and returns scalar. Rotation and scaling factors {s,, 0, are directly learned as independent parameters for each frame.
For each frame i, we construct the affine transformation matrix "/- E R2x3 as:
St cos 0z Ss sin 02 cos 0, 2t sin 0, yt St cos 0,
Tt =
8t sin 0,
sin Pg It + cos 0₁ ye
where 2; fe, (fee (145) (t.)) and yr = fe, (fe,, (145)) (t.)) are predicted translations, and st. O, are directly learnable scalars for each frame.
Each pixel location (i,j) is transformed as: [ 31- 7k 12 j 19", and the warped background obtained via bilinear sampling:
X(..k) = 1 (1x112)
(1) (t.i)):
(2)
where I(+) denotes the bilinear interpolation function and / (1x1 (6.3) returns the interpolation
result of the matrix [21]/) at the coordinate (i.
To validate the effectiveness of the proposed temporal online affine regularizer, we performed streaming data completion on videos "foreman" and "carphone" with sampling rates ranging from 0.1 to 0.3, keeping all other settings the same as those in the main text. The OFTD with temporal online affine regularizer attains better NRE results (see table and fig. 4.. This can be attributed to the design that enables frame-wer wise affine modeling adapt to temporal changes. The regularizer provides strong inductive bias by constraining the transformation space to physically consistent temporal trajectories.
Table 7: Ablation study for the affine regularizer in the OFTD model.
Dataset
foreman
carphone
SR
OFTD w/o affine
0.134 0.121
0.146 0.125
0.
OFTD w/ afline
0.114 0.104 0.099
0.134
0.119 0.112
0.8
08
0.4
0.4
04
Oriaial DOOS5
0.2
02
OCTD
02
OF (iDallieo)
100
Temporal Index
Temporal
Tempporal
Figure Affine regularizer enhances OFTD for irregular data.
B Deferred Proofs
B.1
Intermediate Lemmas
Lemma 1 (Lipschitz smooth bound for FTD). Let the tensor Xe E Rl/XTEXTT satisfy FTD (4) where each factor function fe., R - IR' (n = 1, 1,3) is an INR formulated as in (3) with activation function o(.) = si Assume that each element of the weight matrix W in
follows i..dd N(0, w²). Then for any E (0, 1), any spatial coordinates v, v,v' E 72, where (V V(2)). v = (V()): ((2)) and any temporal coordinates k, k E E, with probability at least
21

---

1 - the following Lipschitz smoothness holds for Ne:
(( - (2t) (v) (v) 1))
3 Cillv v
Spatial smooth
( (()) - (2t)
(Mo)
- k - k|
Temporal smooth
where C1 = wgg (2ur In ) max(1i42 114, 1245) is Lipschitz constant.
Proof.
Since each element of the weight matrix W E R-xr follows i..dd (O w²), we have the tail bound
P( S 2n)
exp(-n/w" + 2r²),
(4)
and hence the probability of all the 3d (where d denotes the number of layers for each factor INR) weight matrices the OFTD model have & nnoor less that 7 is at least 3d exp((n//uu 2r²). Let 3d exp(x + 2r²) = 8 we have 77 = 2wr2 + w 3d. Note that the sine activation function satisfies sin(waa>) wol walar| ax| (wo > 0). Hence we have [e, (x))le, S 1 |z//. For any (i,j. k) we have
(8)) Σ fe. (4444 (k.))
(5)
Then for any (i we have
(2))) (24 (1)0))
Σ ((i)) - se, (( m)) (m)
S /e ((7)))(s)) So. (V41)e Ie (LE)))all (UE))aa
fl, (1) (e..fee))))))) (f')ll, [1fe. fl, (k)ll,,
(6)
The bound on the first term holds
fl,e, (1) -
= - WW oWi())
- d.-.WW--...- - W
< quo
$ 17 wi-1 li-2""
Thus have
(()))s))) (Xb)(e < 11 (1) (e, (f))l,, fe (i)le, II (A)lle,
< - -i w6 15| wB kl]
(8)
< 3d wi
3d 1116/
Similarly, we can derive:
(8))MM (K)) w))l < لل 3d-3 1711111 j'l (X)) -X8)) S 7
(9)
3
3d
K'). 11
Hence from the triangle inequality one can easily derive that for any spatial coordinates v, v' and temporal coordinates k. K' the following bound holds with probability at least 8:
( (X)) (Xe) (ay IS Cillv v'll,,
(10)
(X) (v)) (Xi)
Ci - k'1,
where C1 =i=wi wwg 35d-3(2wr22 w In 3/d3dd ma 1115, 1515). The proof is completed.
22

---

B.2 Proof of Online FTD
Theorem 1 (Regret bound for online FTD). Denote the OFTD model learned at the historical time point t by Xr [Le. (( E 12 where fe,, (1%)itt denotes the n-th factor function at time t. Assume that
The OFTD model learned at the new time point t + using (7) is invariant at the boundary (Ti, T2) Le., Se, (T2: 1 1) -ee, (T55)) (n 1,)
Each element of the weight matrix of the d-layer INRs {fe, (-)) follows i.i.d. NOO, w²) with sine activation function (.) sin(wo.). The En-orrm of derivative of each factor INR Il,.. (a:) lle, is bounded by k > 0.
Then for any 5 E (0,1) and any historical position (in 62, i3) (bn, T. the following regret bound
between the new OFTD model X++1 [f ([4 + 1)} ]= E RR xx x1z (Take
example I++1 Ir 1) and the old OFTD model X, holds with probability at least 1-8: 8:
(K+ 12++1)(i (11. (L) S C2 max((") (T in).
(11)
n
where C2 = 66 3d-3 kmj 2d-2 )) m) 1, 1915 1215) and 2wr + w In 3d 8.
Proof. First we derive the regret bound for each factor INR Se,. (+): R - IR". For any x E [0, H. have
fer (2;t f
+1+1
Il (x;t+ 1) fe. (x;t))le, x
fe, (s; 1)ds
14
11+1
frat fatz
+
Se (8:6 + 1)ds
Se (s,
(12)
1
|ra
(
fe. (s:t)ds fea (x; (c)
+
From Lemma D and its proof, we have that with probability at least 1 - the first term on the right-hand side of 112 admits
fer. (x; + 1) f" 11
+++1
(s; + 1)ds
(x;t + 1) fen (s; 1)ds
ng
12
11t1
1 x)ds " w0
15
1h+1
1) n/ww d-1 (1h 1
+ 1 x)ds
13
d-1
I x
(13)
where 7) 2wr In 3d
Similarly we can get
|
+1+1
Se, (8; so.
< A 1 +1 1 xx/-
(14)
4
Define (x) = fe., (x: + 1) - Se,, (x; 1), and from assumptions we have h, (T,) ) = 0. From the
mean value theorem for integral, the second term on the right-hand side of (1) admits
1.
+1 fe. 1)ds
/
1 +1
-|
1 +1 hn (s)ds
fe. (s: t)ds
6, -
[h,())),,, (15)
1
'This assumption is reasonable since data streams in real world often exhibit local continuity and hence the learned OFTD models at adjacent time points would be invariant at the boundary 1t
23

---

where Ç E (In, I, + +). By the Lagrange mean value theorem we have (S)ll,,
hnn (, -niile S 2n/|1 + x/1
(16)
where & E (T (). Here, we have used the assumption that (E)lla = I1 (5i4 + 1) Se. (5:t)lla S 2x. Combining all the results, we obtain:
Il (z; + 1) fe,, (2; t)lle, S 20 - - + x/-
(17)
Now we can derive the regret bound for online FTD. Write each element of the tensor X as (R) do)
==m=1 (77)) (Gr.m) ((T)); mm) (TT): (is Then for any (41, 12, 13)
(In S I have
(144)) (x) (L.
Σ fe, (14f]it + mo (11)): + ((1 + (ts..
fe, (1 ((45) (t. (143): 1) (ba.m))
Σ (Se, ([7))+ + (s.m)) - fe. (171) () m)) fe. ((a. ([T5): 1) (6a.m) m=1
([ee((1));4 14++1))) ()2mm))) - Se, (12)) (2) Se, ([/)): 1) (c. Se (1T)); (ts.
- fe. +[fe,(44]):t - (14))
() fe. ([]]] () fox t 1) ()22.)))
< (1: fe, (d1;t)lle, Iife (12: f)le, (l)
+ (b); (i2: ll, fl,e, (1n; 1)lle, (is:
+ (1s;t fe. (is: t) le, [ie; 1)1l, flfe (iz: t + 1)lla.
(18)
Substituting 17
into (8) we obtain:
(A142) - (24))))2))))
s 2(1/Ww/ + + i100//666666
+ 2 x) T/ + 1- 12
(19)
20 + k) [3 +1 w0
$ C2 max(I ---in)
2d-2 max where C2 122=6(1 6(1)) + ka) wd
T/ T2T5)- The proof is completed.
C
Implementation Details
C.1
Dataset Details
We introduce details of the used datasets here. For single-aspect streaming, Air Quality contains hourly measurements of pollutant concentrations at different monitoring stations. Indoor Condition includes measurements of humidity and temperature across various indoor locations over time. Radar Traffic records traffic flow data over time at different locations. Chicago Tax T represeens taxi trip data in Chicago. Intel Lab Senso comprises measurements of humidity, temperature, light, and voltage collected by sensors a laboratory. For multi-aspect streaming, video datasets foreman and carphone and the business review dataset YEL are used. The detailed information is shown in Table g
htttps://accivve.cs wwt.:/.c..c.cd..aaiii.eeeeeeeeee wttp://wwwwaccooo//iiiiiiiiiiiia
https///www.kagll.comm
datasets/decide soluciones/air qualaiiiruuuut-uaaaaa madidd madrid
https:/www..kgge..
http:://d..cc.. cttp mit. htt../////abbba///aaaaaaaaamm
http:/tracc.aa...ct. cas www.a....s................ www..sa........aaa/////s
http://wwww.yll.com///e/
24

---

Table 8: Real-world tensor datasets used in experiments.
Name
Modes
Tensor
Beijing Air Quality
locations x pollutants time
12 x 5994
Madrid Air Quality
locations x pollutants time
17 x 3043
Indoor Condition
locations
time
x 2623
Radar Traffic
locations x directions x time
17 5 x 6419
Chicago Taxi
sources destination time
Intel Lab Sensor
locations sensurs time
52 X 4 1152
Video foreman
height
176 100
Video carphone
height width x frames
144 x 176 100
YELP
user business year-month
1000 x 992 93
C.2 Experimental Settings
In experiments, the rank parameter set to 100 for all datasets. And the width of INR networks is set to 128 for all datasets. The sine activation sin(wo-) is used nonlinear activation with Lipschitz continuous property. For single-aspect streaming, wo is set to 1.5 for madrid and sensor, and 0.3 for other datasets. For multi-aspect streaming, w/ is set to 1.5 for foreman and carphone, and 0.01 for YELP due to its intense sparsity. The Beta distribution parameter a is set to 1, and / is set to 1.2. A proportion of 1/3 historical data is randomly selected according to the Beta distribution to construct the memory buffer Mr at each time point The learning rate of the Adam optimizer is set to 0.001. We perform 100 iterations for single-aspect streaming and 500 iterations for multi-aspect streaming at each time point, using the Adam optimizer. For baseline methods, we tuned their hyperparameters using grid search following their authors' suggestions single-aspect streaming, we set the temporal dimension of the data stream to grow by at each time point. For multi-aspect streaming, we each dimension of the data stream to grow by 10% w.r.t. the total length of the data at each time point. All the experiments are conducted on a computer with i5- 2600KF CPU, RTX 4070 Ti SUPER GPU, and 64 GB memories. The Pytorch framework and MATLAB 2024a are used to conduct experiments.
obtain their best performances. For
D More Results
In this section, we present three main components: additional ablation study results, further discussion of the proposed theory, and several attempts to improve the proposed method.
We provide ablation study of the usage of INR (i.e., OFTD with or without INR) Fig. and Fig. by using the radar traffic dataset. The use of INR helps to preserve the spatial continuity of the completion results, thus enhancing performances for OFTD (Fig- [5). What's more, OFTD INR better preserves the smoothness of the temporal curve, highlighting the contribution of the INR to the robustness and accuracy of the completion results (Fig. 6).
OFTD w/o INR OFTD w/ INR OFTD w/o INR OFTD w/ INR NRE 0.205 NRE 0.084 NRE 0.226 NRE 0.111
Figure 5: Ablation study for the proposed method without (w/o) and with (w/) the INR on video datasets foreman and carphone. OFTD w/o INR degrades to the classical online CP decomposition.
Datsset rader
Dataect radar
200
AMAAAm
100
Value
100
-100
Original
.100
Original
OFTD wil INR
QFT vi INR
Temgurral Strleer
Tempocal Index
Figure 6: Temporal curves of a spatial pixel reconstructed by OFTD without or with INR under SR 0.3.
Additional ablation study regarding the CP-rank r is shown in Table which shows that our method is quite robust w.r.t. the CP-rank r. We also tested the influence of the complexity of the INR (i.c.,
25

---

width and depth), as shown in the Table TO and Table 11 Our OFTD is relatively robust to these parameters. experiments, we have consistently set the width 128 and the depth 3, which suffice to achieve satisfactory performance across datasets, while further fine-tuning these parameters could achieve even better performance.
Table 9: Ablation study for the CP-rank r in the OFTD model.
Dataset
Beijing Madrid (average)
Rank
40
80
120
160
NRE
0.1253 0.1255
0.1243
0.1242 0.1245 0.1251 0.1235
0.1246 0.1252
Table 10: Ablation study for the depth (number of layers) of INRs the Condition data. We vary each factor while fixing the other ones to assess its individual effects.
Depth
10
NRE 0.099 0.088
0.088
0.096 0.109
Table 11: Ablation study for the width of INRs on the Condition data, We vary each factor while fixing the other ones to assess its individual effects.
Width
64
128
192
256
320
384
448
512
NRE 0.128 0.106
0.093
0.085
0.087
0.082
0.083
0.081
0.085
Also, we have tested our method with ReLU, WIRE Saragadam and Sine (Sizzmann et al. (2020) activation functions, and the results in the Table 12 show that the default Sine activation function is effective and suitable for our OFTD method. The higher NRE with ReLU activation highlights the importance of activation functions with suitable spectral properties for coordinate-based INRs Sitzmann al. 2020). like the ones used our method.
[203), FINER Liu et al. [2024].
Table 12: Ablation study for the activation functions used in the INRs the Condition data.
Activation function WIRE FINER ReLU Sine
NRE
0.105 0.103 0.215 0.093
To further demonstrate the applicability of our OFTD on large-scale (e.g., higher-order) tensor datasets, we consider testing on a fourth-order color video dataset of size 3 x 144 x 176 x 100. Our method can readily extend to the higher-order case by using higher-order CP decomposition parame- terized by INRs. The OFTD outperforms the baseline GOCPT for the fourth-order streaming tensor completion: NRE 0.187 vs. 0.235 (OFTD vs. GOCPT) using single-aspect settings, demonstrating the effectiveness of OFTD higher-order tensors. We also provide more experimental results in Fig. 7 Our OFTD achieves generally better performances by observing the more accurate temporal curves.
Regarding the theoretical Lemma I and Theorem Macoorem[prpp proposed in the main text, we have the following discussion. First, we qualitatively prove the empirical validity of Lemma [ We provide more experimental results about the smoothness activation function parameter) and w² (the variance of the initialization of weight matrices of the MLP) in Table 13 and 14
the OFTD model w.r.t. parameters wp (the sine
We define a smooth metric S(x)== E, |
ll., to evaluate the smoothness of the recovered tensor. It can be observed that the smoothness of the model is enhanced (i.e., the smooth metric S decreases) with smaller wio and w, which coincides with Lemma ① that smaller wig and w lead to lower Lipschitz smooth bound (Tables 13 and 14). And our method could benefit from an appropriate degree of smoothness brought by tuning these hyperparameters.
Table 13: Ablation study for the hyperparameter Wo the sine activation function sin (wag.) used in the INRs of the OFTD model.
Dataset
Beiling Madrid (average)
0.1
0.3
0.7
0.9
1.1
1.5
10*)
9.2 12.6
13.7
13.8
15.9
21.7
23.4
24.3
25.0
NRE
0.1410
0,1319
0.1
0.1168
0.1196
0.1272
0.1358
0.1430
0.1478
26

---

M DLETTIC
WWM
W
SOTTA
OFTO
SCETA
OETO
o9 Temperall todie
Timpunal
Sheppornal toder
Semperal traks
Tempural teaies
Damasa beitnng
Themsa hwiiing
Drac colar
Dome ruder
Tenguoaal Lockes
Vengaosal lonkon
Tenagoaal losdon
Tragsoal Lucks
Teaaporil lucks
Texppoal lucks
Daraur wadind
Thalesa readiid
Zuaart Secnas
Dutauct Seceas
Datase Secnas
Original
Creatal
Oegans
Teapprd Index
Tenyural ludlo
Toagooal locks
Tempporil indox
Tamppoiil
Tampocil
Dasuet
Dassurt saryhooe
Dassett sarehooo
Sieena
Darau Sexmma
Darase
Digall
Orgiial
Origiil
Originc
130
130
50
50
Figure 7: Temporal curves a spatial pixel reconstructed by different methods for different datasets. The proposed OFTD could better recover the complex nonlinear temporal curves, showcasing its higher accuracy for the streaming data completion problem.
Table 14: Ablation study for the variance w of the initialization weight matrices in the INRs.
Dataset
Beijing Madrid (average)
w²
0.01
0.02
0.03
0.04
0.05
0.06
0.07
0.08
0.09
S(x10*)
13.3
14.9
15.8
16.2
16.5
16.9
17.6
18.4
18.6
NRE
0.1245
0,10
0.1305
0.1327
0.1337
0,1344
0.1367
0,1405
0.1416
Then have the following explanation regarding the assumption of invariance in Theorem T The invariant assumption was deduced for general scenarios, i.c., real-world data streams often gradually and smoothly change across the evolving directions, and hence the learned OFTD models at adjacent time points would be invariant at the boundary. In this case, nearby points will not be negatively affected by the new data since they are similar in structure.
To empirically validate the invariant assumption, we calculate the relative error between boundary - (15+++))
(1:
Il before and after the online optimization at time t + 1. In the Table
variables
(zt"")) )]1
15
we see that the relative error less than 5%. validating the rationality of the invariant assumption. We note that even without the invariant assumption, the theoretical regret bound and qualitative conclusion of Theorem could still be deduced analogously by adding a small constant that indicates the change of boundary variables.
Table 15: Relative error between the boundary variables before and after the online optimization at t+1 1.
60
80
4.10% 2.23%
1.19%
Condition 1.70% 4.25%
3.40%
1.99%
Finally, to empirically validate Theorem I we test OFTD without memory buffer and report the NRE at different positions after the online optimization at I 100, as shown in the Table More distant positions (i.e., smaller i,,) tend to hold larger NRE without memory buffer, indicating that distant information is more likely to be forgotten. This result precisely aligns with Theorem which states that more distant information holds larger regret bound. Hence, the theoretical result Theorem I empirically satisfied in practice.
27

---

Table 16: NRE results at different positions in with and without memory buffer.
Position 8is
10
20
30
40
50
60
70
80
90
Without memory buffer
0.440 0.314
0.300
0.275 0.256
0.228
0.170 0.127
0.101 0.077
With memory buffer
0.075 0. 0.104 0.074
0.071
0.078
0.098
0.100
0.093
Next, we will present some interesting attempts to improve our methods. According to the definition of 3 in Definition (3), the tensor can be extremely sparse. Thus We further conducted experiments by imposing an I -norm sparse regularization on the factors of the CP decomposition to incorporate
the sparse prior: A Ew= (( where À is sparse regularization parameter. The results with different values are shown in the Table I7 The results show that the sparse regularization with suitable has the potential to improve the performance of our method, while higher may over-constrain the model. It would be interesting to develop other effective sparse regularizations under the OFTD) framework, and we leave to future research.
Table 17: NRE results with different intensities of the sparse regularization.
0.01 0.03
0.05
0.07
0.09
Foreman 0.084 0.081
0.079
0,078
0.078
0.080
Condition 0.093 0.091 0.088
0.088
0.090
0.092
Currently the Beta parameter 3 is fixed. It is a great idea to dynamically adjust the Beta parameter based the learning process. We conduct experiments using a linear dynamic scheme for B (i.e., St = 3o + 1A,, where 3g- = 1). as shown in Table 18 Better NRE results are observed when A is positive value (i.e., long-tail distribution). Conversely, when A3 is negative number, the gains negative. We attribute this result to the nature of our forgetting bound, which depends on the relative position (I, + 1 - i,,) (as shown in Eq. 8 of the main text). It shows that distant information is more likely to be forgotten, and hence a long-tail memory buffer with AB > 0 is favored. The design of more complex 3g schedules and their theoretical analysis not trivial due the complexity of the optimization problem. We leave this promising direction to future research.
Table 18: NRE results with dynamic scheme 3r = 1 + 12,3 on the Foreman data.
As -0.01 -0.005
0 0.005 0.01
0.015
0.02
Fix 8, 1.2
NRE 0.092 0.087 0.086
0.083 0.083 0.084 0.084
0.084
we want to further reduce the computational complexity associated with memory buffer, we can consider using fixed-size memory buffer strategies (non-increasing memory buffer size). Especially, we utilize the Beta distribution to construct long-tail tail memory buffer that contains up to 100 historical samples, which strictly restricts computational complexity.
From the results in the Table 19 we can observe that OFTD still performs stably under the fixed-size memory buffer. In particular, OFTD with fixed-size memory buffer achieves significantly lower NRE of 0.1311 (vs. 0.3979 without memory buffer) with only moderate increase in FLOPPs (10.79M vs. 9.09M) on the Madrid case. The results demonstrate the practical efficiency and effectiveness of the fixed-size memory buffer strategy. It is also possible to consider other continual learning strategies such as memory buffer-free methods (e.g., gradient projection Lin et al. (2022) and prototype- sampling Asadi et al. 2023)) to further enhance the efficiency of OFTD, yet the combination not trivial due to the distinct structures of OFTD. Hence, we leave this direction for future research.
Table 19: Ablation study for the memory buffer size (percentage of buffer size w.r.t. the whole tensor size), including a fixed-size memory buffer (up to 100 samples).
Dataset
Beijing
Madrid
Buffer size
100%
50%
33%
20%
Fixed-size
0%
100%
50%
33%
20%
Fixed-size 0%
NRE
0.1280
0299
0.1308
0.1324
0.1422
0.2903
0.1112
061
0.1182
0,1204
0,1311
0.3979
FLOPs
87.97M
43,99M
29.33M
17.61M
8.89M
6.03M
139.42M
69.73M
46.50M 27.93M
10.79M 9.09M
In addition to the above attempts, the OFTD method can be readily generalized by combining with other tensor decomposition paradigms such as Tucker decomposition and tensor-trai train decomposition.
28

---

Especially, we define the Tucker functional tensor decomposition (Tucker-FTD)
Cx Ji (1i)) x2 /2 ((75) Xa Ja (17)) RRixtixs
where C X is core tensor and fn. ((T)) R xre (n 1,2,3) factor functions
parameterized by INRs.
Moreover, define the tensor-train train functional tensor decomposition (TT-FTD)
X = l2 (12))) x1 fi (179) X3 fs (175) E 1
where /2 (175) E TR TRixIixry, fi ((71)) E Rixi, and fa ((75) E Resxi are factor functions parameterized by INRs.
To adapt the Tucker-FTD and TT-FTD for streaming data, we consider their online versions by progressively expanding the input coordinate vectors [4]] of INRs, and hence expanding the sizes of factor matrices/tensors fit the new tensor size. We give numerical example by comparing different tensor decompositions under the OFTD framework, as shown in the Table (20 We control the rank of different methods such that they hold similar number of learnable parameters.
Table 20: NRE results by OFTD with different tensor decomposition paradigms the Foreman data.
Method
TT
CP
Tucker
NRE
0.085
0.084
0.082
Numher of Params. 3.12 x 10 2.76 x 10 2.93 x 10
Tucker-FTD achieves relatively better result in this case, which can be rationally explained by the additional core tensor parameter that enhances the representation ability Tucker-FTD. In future research, consider more tensor decomposition paradigms such the tensor SVD under the OFTD framework.
29


Reviews:
[{"review_id": "blHrwkHMZ1", "reviewer": "Reviewer_ZeJq", "summary": "The work proposes an online tensor decomposition framework that incorporates implicit neural representation (INR) strategy that helps with more dynamic continual learning. The tensor decomposition is performed in an online manner at every time step with limited forgetting of the previous tensor knowledge. Towards this, a long-tail memory replay method by constructing a memory buffer controlled by a long-tail beta distribution is introduced. Experiments are presented to evaluate the proposed approach and compare with the baselines.", "questions": "1.\tIn Eq. (2) while introducing the optimization at time step t, the temporal relation is not specified for the factor matrices.\n\n2.\tAccording to the definition of ${\\cal Y}_t$ in Definition 3, the tensor can be extremely sparse. Yet, such prior knowledge is not employed in (6), which could limit the performance of the approach.\n\n3.\tMinor comments: There is some formatting issue in terms of using the citations to the references. The author names are repeated twice which is better to avoid.\n\n4.\tAccording to Theorem1, not only the distance from new data stream, but also the neural network architecture and the choice of $r$ also plays role in forgetting. There is no discussion regarding that to give some insights in the practical settings.\n\n5.\tThe parameters of the beta distribution is currently fixed, but according to the theorem, it is dependent on the learning process and can be influenced by that.\n\n6.\tThere are other influencing factors such as rank, the choice of omega, the number of layers. How are these factors affecting the performance and be selected for optimal outputs.\n\n7.\tI could not see much information presented regarding the baselines and their tensor decomposition methods. Are all of them CP? Or have you tried with other tensor decomposition methods such as Tucker etc?", "limitations": "NA", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 4, "originality": 3, "strengths_and_weaknesses": "Strengths:\n\n1. The paper is well-written and hence it is easy to follow. The notations are clearly defined, and theorems are well presented.\n\n2. INR has emerged as a powerful technique in computer vision and has not much explored in online tensor decomposition. Hence, the work can spark some follow-up studies in this domain utilizing these powerful tools.\n\nWeaknesses:\n\n1. Experimental study presented in the main paper is limited. It looks like the completion is performed in small scale datasets. The dataset size and details are not clearly articulated. More ablation studies are also expected to better evaluate the approach.", "was_revised": true, "final_justification": "The work advances online tensor data analysis and have proposed a strong benchmark for the future work. The responses to my concerns have been addressed well in the rebuttal phase. Hence, I am happy to raise the point.", "rebuttal": "We sincerely thank the reviewer for the valuable and constructive comments! We have carefully made more discussions and clarifications according to your comments, including more dataset details, more ablation studies, numerical tests with sparse priors, more discussions on Theorem 1 and the Beta distribution. Please see details in the following.\n\n>**W1: Experimental study presented in the main paper is limited. It looks like the completion is performed in small scale datasets. The dataset size and details are not clearly articulated. More ablation studies are also expected to better evaluate the approach.**\n \nThanks! The dataset sizes and details are articulated in Section C.1 and Table 8 in the Appendix after the main text (in the same PDF file). All datasets are publicly available, and the detailed websites for downloading these datasets are provided in the footnotes under Section C.1. Some datasets are large-scale tensors, e.g., the dataset YELP has a size of 1000 × 992 × 93. To further demonstrate the applicability of our OFTD on large-scale (e.g., higher-order) tensor datasets, we consider testing on a fourth-order color video dataset of size 3×144×176×100. Our method can readily extend to the higher-order case by using higher-order CP decomposition parameterized by INRs. The OFTD outperforms the baseline GOCPT for the fourth-order streaming tensor completion: NRE 0.187 vs. 0.235 (OFTD vs. GOCPT) using single-aspect settings, demonstrating the effectiveness of OFTD on higher-order tensors. In terms of ablation study, please refer to the response to your question Q6. Thanks.\n\n>**Q1: In Eq. (2) while introducing the optimization at time step $t$, the temporal relation is not specified for the factor matrices.** \n\nThanks! We will revise this equation and introduce the temporal relation in the factor matrices by denoting them as $\\\\{{\\bf U}^{(n)}_t\\\\}$.\n\n>**Q2: According to the definition of ${\\mathcal Y}_t$ in Definition 3, the tensor can be extremely sparse. Yet, such prior knowledge is not employed in (6), which could limit the performance of the approach.** \n\nIt is a great idea to further incorporate the sparse prior into the model. Following your comments, we further conducted experiments by imposing an $\\ell_1$-norm sparse regularization on the factors of the CP decomposition to incorporate the sparse prior: $\n\\lambda\\sum_{n=1}^{N}|| f_{n}([I_n^t])||_{\\ell_1}$, where $\\lambda$ is a sparse regularization parameter. The results with different $\\lambda$ values are shown in the Table below. The results show that the sparse regularization with a suitable $\\lambda$ has the potential to improve the performance of our method, while higher $\\lambda$ may over-constrain the model. It would be interesting to develop other effective sparse regularizations under the OFTD framework, and we leave it to future research.\n\nTable: NRE results with different intensities of the sparse regularization.\n|$\\lambda$|$0$|$0.01$|$0.03$|$0.05$|$0.07$|$0.09$|\n|-|-|-|-|-|-|-|\n|Foreman|0.084|0.081|0.079|0.078|0.078|0.080|\n|Condition|0.093|0.091|0.088|0.088|0.090|0.092|\n\n>**Q3: Minor comments: There is some formatting issue in terms of using the citations to the references. The author names are repeated twice which is better to avoid.** \n\nThanks! We will revise the citations to avoid repetition.\n\n>**Q4: According to Theorem 1, not only the distance from new data stream, but also the neural network architecture and the choice of also plays role in forgetting. There is no discussion regarding that to give some insights in the practical settings.** \n\nYes, the regret bound in Theorem 1 is determined by both the distance term $(I_n^t+1-i_n)$ and the constant $C_2$ that relates to the rank $r$ and neural network architectures. Nonetheless, we may treat these two terms differently. In the distance term $(I_n^t+1-i_n)$, the change of position $i_n$ reflects the change of regret bound across different positions, indicating that more distant information holds larger regret bound. The constant $C_2$ is independent of the position $i_n$ and hence serves as a fixed \"coefficient\" role that does not reflect the heterogeneity in regret bound across different positions. Hence we mainly focus on analyzing the position-dependent forgetting behavior induced by the distance term $(I_n^t+1-i_n)$. Still, from the construction of $C_2$, we see that smaller rank $r$ leads to smaller regret bound (i.e., less forgetting), while larger rank $r$ leads to better representation and approximation ability of the model. Hence, Theorem 1 also reveals a theoretical trade-off between the forgetting behavior and the approximation ability of the model. The ablation study for the rank $r$ is given in Table 9 of the Appendix. In our experiments, we have set $r=100$ across all cases, which was shown to strike a good balance between representation ability and forgetting. \n\n>**Q5: The parameters of the beta distribution is currently fixed, but according to the theorem, it is dependent on the learning process and can be influenced by that.** \n\nYes, currently the Beta parameter $\\beta$ is fixed. It is a great idea to dynamically adjust the Beta parameter based on the learning process. Inspired by your comments, we conduct experiments using a linear dynamic scheme for $\\beta$ (i.e., $\\beta_t=\\beta_0+t\\Delta\\beta$, where $\\beta_0=1$), as shown in the Table below. Better NRE results are observed when $\\Delta\\beta$ is a positive value (i.e., long-tail distribution). Conversely, when $\\Delta \\beta$ is a negative number, the gains are negative. We attribute this result to the nature of our forgetting bound, which depends on the relative position $(I_n^t+1-i_n)$ (as shown in Eq. (8) of the main text). It shows that distant information is more likely to be forgotten, and hence a long-tail memory buffer with $\\Delta\\beta>0$ is favored. The design of more complex $\\{\\beta_t\\}$ schedules and their theoretical analysis are not trivial due to the complexity of the optimization problem. We leave this promising direction to future research.\n\nTable: NRE results with dynamic scheme $\\beta_t=1+t\\Delta \\beta$ on the Foreman data.\n|$\\Delta \\beta$|-0.01|-0.005|0|0.005|0.01|0.015|0.02|Fix $\\beta_t=1.2$|\n|-|-|-|-|-|-|-|-|-|\n|NRE|0.092|0.087|0.086|0.083|0.083|0.084|0.084|0.084|\n  \n>**Q6: There are other influencing factors such as rank, the choice of omega, the number of layers. How are these factors affecting the performance and be selected for optimal outputs.** \n\nThanks! We have tested the influence of the rank $r$ and $\\\\omega_0$ in Table 9 and Table 10 in the Appendix after the main text. We paste them here for your convenience. We have also tested the influence of number of layers, as shown in the Table below. OFTD is relatively robust to these parameters across suitable ranges of values. In our experiments, we set $r=100$, and select $\\omega_0$ from the small candidate set $\\\\{0.3,1.5\\\\}$ to obtain satisfactory performance. The number of layers is consistently set as $3$. While the ablation studies show that further fine-tuning these parameters (such as $\\omega_0$) is expected to enhance the results, our current hyperparameter settings already suffice to achieve satisfactory performances across datasets.\n\nTable: Ablation study for the rank $r$. (Average results on Beijing and Madrid data)\n|$r$ | 20|40|60|80|100|120|140|160|180|\n|-|-|-|-|-|-|-|-|-|-|\n|NRE|0.1253|0.1255|0.1243|0.1242|0.1245|0.1251|0.1235|0.1246|0.1252|\n\nTable: Ablation study for the hyperparameter $\\omega_0$ in the sine activation function. (Average results on Beijing and Madrid data)\n|$\\omega_0$ |0.1|0.3|0.5|0.7|0.9|1.1|1.3|1.5|1.7|\n|-|-|-|-|-|-|-|-|-|-|\n|NRE|0.1410|0.1319|0.1184|0.1168|0.1196|0.1272|0.1358|0.1430|0.1478|\n\nTable: Ablation study for the depth (number of layers) of INRs on the Condition data. We vary each factor while fixing the other ones to assess its individual effects.\n|Depth|2|3|4|5|6|7|8|9|10|\n|-|-|-|-|-|-|-|-|-|-|\n|NRE|0.099|0.093|0.088|0.086|0.085|0.088|0.089|0.096|0.109|\n\n>**Q7: I could not see much information presented regarding the baselines and their tensor decomposition methods. Are all of them CP? Or have you tried with other tensor decomposition methods such as Tucker etc?** \n\nThe baseline methods ATD and SIITA are Tucker decomposition-based methods; Grouse, Grasta, and Petrels are low-rank matrix decomposition-based methods; TeCPSGD, OLSTEC, SOFIA, ACP, STF, OnlineSGD, and GOCPT are CP decomposition-based methods. Our OFTD demonstrates superior performance compared to these baseline methods using various decomposition paradigms. We will clarify these baseline settings in the main text. \n\nShould you need further information, please let us know. We look forward to hearing from your feedback!"}, {"review_id": "5ZodurGBJp", "reviewer": "Reviewer_kMsk", "summary": "This paper aims to tackle the challenges in high-velocity streaming tensor data analysis, where new data expands across multiple dimensions of the original dataset. Specifically, it leverages implicit neural representations to model the tensor decomposition factors, thereby enhancing the capability of dynamic data structure modeling. To mitigate the forgetting of historical data when fitting new information, the paper adopts a continual learning approach and develops a memory replay mechanism to address this issue. Theoretically, the paper analyzes the spatial-temporal smoothness regularization and the forgetting pattern of the proposed method during the learning process. In the experimental section, the paper demonstrates the effectiveness of the proposed method on real-world datasets.", "questions": "How does the complexity of the INR affect the model performance?", "limitations": "yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "### **Strengths**:\n\n• The idea of introducing Functional Tensor Decomposition to the online tensor analysis setup is novel. Unlike classical online tensor analysis methods, it explores the nonlinear relationships of the tensor factors, which is a significant enhancement in streaming data analysis.\n\n• The paper demonstrates an interesting combination of concepts between continual learning and streaming tensor analysis, which may inspire future research.\n\n• The paper is technically sound, with sufficient theoretical analysis to support the effectiveness and the motivation behind the proposed method.\n\n\n### **Weaknesses**:\n\n• The proposed method tends to forget information distant from the new tensor data, leading the paper to propose a memory replay mechanism to avoid this. However, as more and more new data is introduced, the memory buffer must grow larger and larger. This seems like a serious side effect (e.g., increasing computational complexity), especially when real-time processing is a critical consideration in streaming data analysis.\n\n• The theoretical analysis seems to hold only when the weight matrices of INR during training remain i.i.d. $\\mathcal{N}(0,\\omega^{2})$. How can this be ensured? Additionally, the first assumption in Theorem 1 seems questionable, as it assumes the boundary variables of the model are invariant before and after training on the new data. What if the deep model severely overfits the new data? If this happens, intuitively, even the nearest point to the new data will be affected by this overfitting.\n\n• There are several existing studies on Functional Tensor Decomposition. For example, the method presented in [1] appears to achieve the same objective as INR. The paper misses references to these types of works.\n\nReference:\n\n[1]. Functional Bayesian Tucker Decomposition for Continuous-indexed Tensor Data, ICLR, 2024", "was_revised": true, "final_justification": "The paper is conceptually new; in the rebuttal, the authors address the concern regarding the computational complexity, the model performance, and also the theoretical results by adding additional results and analysis.", "rebuttal": "We sincerely thank the reviewer for the valuable and constructive comments! We have carefully made more discussions and clarifications according to your comments, including more discussions on the memory buffer, more validations on the theoretical assumptions, more discussions on related work, and more experiments for the influence of INR complexity. Please see details in the following.\n\n>**W1: The proposed method tends to forget information distant from the new tensor data, leading the paper to propose a memory replay mechanism to avoid this. However, as more and more new data is introduced, the memory buffer must grow larger and larger. This seems like a serious side effect (e.g., increasing computational complexity), especially when real-time processing is a critical consideration in streaming data analysis.** \n\nYes, in our current implementation, the memory buffer grows larger during online optimization. Nevertheless, OFTD is still practically efficient and could achieve real-time processing (less than one second per online optimization). The reason of such efficiency can be illustrated from two aspects. First, OFTD is relatively robust to the memory buffer size and could perform stably even using a small proportion of historical data (e.g., 20%), as shown in the Table below. Thus, OFTD could achieve efficient real-time online processing with such small memory buffer. \n\nSecond, if we want to further reduce the computational complexity associated with memory buffer, we can consider using fixed-size memory buffer strategies (non-increasing memory buffer size). Especially, we utilize the Beta distribution to construct long-tail memory buffer that contains up to 100 historical samples, which strictly restricts computational complexity. From the results in the Table below, we can observe that OFTD still performs stably under the fixed-size memory buffer. In particular, OFTD with fixed-size memory buffer achieves a significantly lower NRE of 0.1311 (vs. 0.3979 without memory buffer) with only a moderate increase in FLOPs (10.79M vs. 9.09M) on the Madrid case. The results demonstrate the practical efficiency and effectiveness of the fixed-size memory buffer strategy. It is also possible to consider other continual learning strategies such as memory buffer-free methods (e.g., gradient projection [R1] and prototype-sampling [R2]) to further enhance the efficiency of OFTD, yet the combination is not trivial due to the distinct structures of OFTD. Hence, we leave this direction for future research.\n\nTable: Ablation study for the memory buffer size (the proportion of buffer size w.r.t. the\nwhole tensor size), and the fixed-size memory buffer with up to 100 historical samples. The Beijing data contains 5994 temporal samples while Madrid contains 3043 temporal samples.\n|Dataset|Beijing||||||Madrid||||||\n|-|-|-|-|-|-|-|-|-|-|-|-|-|\n|Buffer size|100%|50%|33%|20%|Fixed-size|0%|100%|50%|33%|20%|Fixed-size|0%|\n|NRE|0.1280|0.1299|0.1308|0.1324|0.1422|0.2903|0.1112|0.1160|0.1182|0.1204|0.1311|0.3979|\n|FLOPs|87.97M|43.99M|29.33M|17.61M|8.89M|6.03M|139.42M|69.73M|46.50M|27.93M|10.79M|9.09M|\n\n[R1] TRGP: Trust region gradient projection for continual learning, ICLR\n\n[R2] Prototype-sample relation distillation: towards replay-free continual learning, ICML\n\n>**W2: The theoretical analysis seems to hold only when the weight matrices of INR during training remain i.i.d. ${\\mathcal N}(0,w^2)$. How can this be ensured? Additionally, the first assumption in Theorem 1 seems questionable, as it assumes the boundary variables of the model are invariant before and after training on the new data. What if the deep model severely overfits the new data? If this happens, intuitively, even the nearest point to the new data will be affected by this overfitting.** \n\nThanks! Regarding the Gaussian assumption, we want to clarify that the i.i.d. ${\\cal N}(0,w^2)$ for neural network parameters is a widely used assumption in theoretical studies of neural networks [R3,R4]. We follow this assumption to simplify the theoretical proof, while other statistical distributions (e.g., uniform distributions, Poisson distribution, and not necessarily i.i.d.) could also fit in our theoretical results by slightly changing the Lipschitz bound $C_1$ in Lemma 1. Moreover, the empirical validity of Lemma 1 has been qualitatively justified in Tables 10-11 in the Appendix, where we vary the values of $\\omega_0$ and $w$ in the Lipschitz bound and observe the empirical smoothness change of the recovered tensor. The smoothness changing trend of the recovered tensor precisely aligns with the changing trend of the Lipschitz bound w.r.t. the parameters $\\omega_0$ and $w$ deduced in Lemma 1. Hence, the theoretical result in Lemma 1 is empirically satisfied in practice.\n\nRegarding the invariant assumption, we want to clarify from the following aspects: \n\n1) First, the OFTD model has a certain degree of robustness against outliers due to the regularization effect induced by the tensor decomposition and the inherent smoothness of INRs, which can alleviate overfitting and ensure stability. \n\n2) Second, the invariant assumption was deduced for general scenarios, i.e., real-world data streams often gradually and smoothly change across the evolving directions, and hence the\nlearned OFTD models at adjacent time points would be invariant at the boundary. In this case, nearby points will not be negatively affected by the new data since they are similar in structure. To empirically validate the invariant assumption, we calculate the relative error between boundary variables $\\frac{\\lVert f_n(I_n^{t};t+1)-f_n(I_n^{t};t)\\rVert _{\\ell_2}}{\\lVert f_n(I_n^{t};t)\\rVert _{\\ell_2}}$ before and after the online optimization at time $t+1$. We see that the relative error is less than 5%, validating the rationality of the invariant assumption. We note that even without the invariant assumption, the theoretical regret bound and qualitative conclusion of Theorem 1 could still be deduced analogously by adding a small constant $\\epsilon$ that indicates the change of boundary variables.\n\nTable: Relative error between the boundary variables before and after the online optimization at $t+1$.\n|$t+1$|20|40|60|80|\n|-|-|-|-|-|\n|Foreman|4.10%|2.23%|3.99%|1.19%|\n|Condition|1.70%|4.25%|3.40%|1.99%|\n\n3) Third, if severe overfitting on new data occurs, the boundary invariant assumption may be violated. This could happen for abruptly changed data streams where the data smoothness disappears. To alleviate the ill-posedness under this scenario, we have further proposed a temporal online affine regularizer (Section A.3 in Appendix) to relax the data prior assumption in this extreme case. This makes OFTD more robust and stable in diverse scenarios.\n\n4) Finally, to empirically validate Theorem 1, we test OFTD without memory buffer and report the NRE at different positions after the online optimization at $I_n^t=100$, as shown in the Table below. More distant positions (i.e., smaller $i_n$) tend to hold larger NRE without memory buffer, indicating that distant information is more likely to be forgotten. This result precisely aligns with Theorem 1, which states that more distant information holds larger regret bound. Hence, the theoretical result in Theorem 1 is empirically satisfied in practice.\n\nTable: NRE results at different positions $i_n$ with and without memory buffer on the Foreman data.\n|Position $i_n$|10|20|30|40|50|60|70|80|90|\n|-|-|-|-|-|-|-|-|-|-|\n|Without memory buffer|0.440|0.314|0.300|0.275|0.256|0.228|0.170|0.127|0.101|\n|With memory buffer|0.075|0.104|0.074|0.071|0.078|0.098|0.077|0.100|0.093|\n\n[R3] Neural Tangent Kernel: Convergence and Generalization in Neural Networks, NeurIPS\n\n[R4] Fine-Grained Analysis of Optimization and Generalization for Overparameterized Two-Layer Neural Networks, ICML\n\n>**W3: There are several existing studies on Functional Tensor Decomposition. For example, the method presented in [1] appears to achieve the same objective as INR. The paper misses references to these types of works.** \n\nThanks for bringing this related work to our attention! Fang et al. [R5] introduced the functional Bayesian Tucker decomposition, which utilizes Gaussian processes as functional priors to model the factors of the Tucker decomposition, with applications for tensor completion. Compared to this work, we propose the online functional tensor decomposition, which utilizes implicit neural representations (INRs) parameterized by deep neural networks to model tensor factors for streaming data completion. The INRs hold stronger representation ability than the statistical Gaussian processes for characterizing real-world tensor data. Moreover, the proposed OFTD is capable of handling streaming tensor data in an online manner, which is different from the batch tensor decomposition technique in [R5]. We will introduce and discuss this related work in the main text.\n\n[R5] Functional Bayesian Tucker Decomposition for Continuous-indexed Tensor Data, ICLR\n\n>**Q1: How does the complexity of the INR affect the model performance?** \n\nAs suggested, we tested the influence of the complexity of the INR (i.e., width and depth), as shown in the Table below. Our OFTD is relatively robust to these parameters. In experiments, we have consistently set the width to $128$ and the depth to $3$, which suffice to achieve satisfactory performance across datasets, while further fine-tuning these parameters could achieve even better performance.\n\nTable: Ablation study for the width and depth of INRs on the Condition data. We vary each factor while fixing the other one to assess its individual effects.\n|Width|32|64|128|192|256|320|384|448|512|\n|-|-|-|-|-|-|-|-|-|-|\n|NRE|0.128|0.106|0.093|0.085|0.087|0.082|0.083|0.081|0.085|\n\n|Depth|2|3|4|5|6|7|8|9|10|\n|-|-|-|-|-|-|-|-|-|-|\n|NRE|0.099|0.093|0.088|0.086|0.085|0.088|0.089|0.096|0.109|\n\nShould you need further information, please let us know. We look forward to hearing from your feedback!"}, {"review_id": "a0IfAmot14", "reviewer": "Reviewer_beqG", "summary": "This paper proposes a novel framework, Online Functional Tensor Decomposition (OFTD), for streaming tensor data completion. The core idea is to model CP decomposition factors as Implicit Neural Representations (INRs), thus to capture spatial-temporal continuity and\nlow-dimensional compact structures of streaming data . To mitigate catastrophic forgetting, this work adopts a long-tail memory replay mechanism and theoretically establishes a regret bound. Extensive experiments on real-world single- and multi-aspect streaming datasets show that OFTD outperforms a wide range of baselines in terms of NRE for the most cases.", "questions": "1. Why CP decompositions?  Is it possible to generalize this method to other tensor decompositions like TSVD or Tensor Train?\n\n2. How does the regret bound correlate empirically with performance degradation?", "limitations": "yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 2, "strengths_and_weaknesses": "**Strengths:**\n\n1. The integration of INR with CP decomposition for streaming tensor data completion is novel and well-motivated.\n\n2.  The method is validated on both single-aspect and multi-aspect tensor data streams with clear performance improvements over the baselines.\n\n3. Authors provide a regret bound that explains the forgetting behavior of the model.\n\n**Weaknesses:**\n\n1. Although the work is   motivated by continual learning, connections to and comparisons with classic continual learning approaches are missing.\n\n2. The performance of OFTD may be sensitive to the architecture or hyperparameters of the INR (e.g., depth, activation), but this is not explored or ablated in the current version.", "was_revised": true, "final_justification": "My concerns have been fully addressed.", "rebuttal": "We sincerely thank the reviewer for the valuable and constructive comments! We have carefully made more discussions and clarifications according to your comments, including more discussions on the connections between our method and classical continual learning paradigms, more ablation studies of the INR architectures, more tensor decomposition paradigms under the OFTD framework, and more empirical validations of the theoretical regret bound analysis. Please see details in the following.\n\n>**W1: Although the work is motivated by continual learning, connections to and comparisons with classic continual learning approaches are missing.**\n\nThanks for this constructive comment. Here, we further discuss the connections between our method and continual learning in a more comprehensive manner. The continual learning refers to a machine learning paradigm where a model sequentially learns from a non-stationary stream of data over time, while retaining historical knowledge [R1]. The proposed OFTD belongs to the continual learning paradigm, which learns the INR networks from a continuous stream of tensor data, with such information becoming progressively available over time. Hence, the proposed OFTD has the potential to be combined with proven methods in continual learning (e.g., memory replay) to alleviate the forgetting of historical tensor knowledge. To this end, we have proposed a long-tail memory reply method to effectively alleviate forgetting. The combination of our method with more other continual learning methods (e.g., gradient projection [R2] and prototype-sampling [R3]) to alleviate the forgetting is also an interesting research direction, yet the combination is not trivial due to the distinct structures of OFTD. Hence, we leave this direction for future research. \n\nAs for the comparisons with classical continual learning methods, we want to clarify that the proposed OFTD is the first method to introduce the continual learning paradigm for the streaming data completion task, hence direct comparison with classical continual learning methods designed for tasks like image classification [R2] is not straightforward or directly applicable. Nevertheless, we can numerically compare the proposed long-tail memory reply continual learning method with the vanilla uniformly distributed memory reply continual learning method under the OFTD framework. The results reveal that the long-tail memory reply is indeed more effective, e.g., NRE 0.1233 vs. 0.1261 (long-tail vs. uniform) (more details are shown in Table 5 of the main text). We will include these discussions in the final paper to enhance the clarity and comprehensiveness according to your comments.  \n\n[R1] Continual lifelong learning with neural networks: A review, Neural Networks\n\n[R2] TRGP: Trust region gradient projection for continual learning, ICLR\n\n[R3] Prototype-sample relation distillation: towards replay-free continual learning, ICML\n\n>**W2: The performance of OFTD may be sensitive to the architecture or hyperparameters of the INR (e.g., depth, activation), but this is not explored or ablated in the current version.**\n\nAs suggested, we tested the influence of the architectures of INRs (width, depth, and activation functions) on the model performance, as shown in the Table below. Our method is relatively robust to the width and depth parameters. Also, we have tested our method with ReLU, WIRE [R4], FINER [R5], and Sine [R6] activation functions, and the results show that the default Sine activation function is effective and suitable for our OFTD method. The higher NRE with ReLU activation highlights the importance of activation functions with suitable spectral properties for coordinate-based INRs [R6], like the ones used in our method.\n\nTable: Ablation study for the width, depth, and activation function of INRs on the Condition data. We vary each factor while fixing the other ones to assess its individual effects.\n|Width|32|64|128|192|256|320|384|448|512|\n|-|-|-|-|-|-|-|-|-|-|\n|NRE|0.128|0.106|0.093|0.085|0.087|0.082|0.083|0.081|0.085|\n\n|Depth|2|3|4|5|6|7|8|9|10|\n|-|-|-|-|-|-|-|-|-|-|\n|NRE|0.099|0.093|0.088|0.086|0.085|0.088|0.089|0.096|0.109|\n\n|Activation function|WIRE|FINER|ReLU|Sine|\n|-|-|-|-|-|\n|NRE|0.105|0.103|0.215|0.093|\n\n[R4] WIRE: Wavelet Implicit Neural Representations, CVPR\n\n[R5] FINER: Flexible spectral-bias tuning in Implicit Neural Representation by Variable-periodic Activation Functions, CVPR\n\n[R6] Implicit Neural Representations with Periodic Activation Functions, NeurIPS\n\n>**Q1: Why CP decompositions? Is it possible to generalize this method to other tensor decompositions like TSVD or Tensor Train?**\n\nYes! The OFTD method can be readily generalized by combining with other tensor decomposition paradigms such as Tucker decomposition and tensor-train decomposition. Especially, we define the Tucker functional tensor decomposition (Tucker-FTD) as\n$$\n{\\mathcal X}_t ={\\mathcal C}\\times_1 f_1([I_1^t]) \\times_2 f_2([I_2^t]) \\times_3 f_3([I_3^t])\\in{\\mathbb R}^{I_1^t\\times I_2^t\\times I_3^t},\n$$\nwhere ${\\mathcal C}\\in{\\mathbb R}^{r_1\\times r_2\\times r_3}$ is a core tensor and $f_n([I_n^t])\\in{\\mathbb R}^{ I_n^t \\times r_n}$ ($n=1,2,3$) are factor functions parameterized by INRs. Moreover, we define the tensor-train functional tensor decomposition (TT-FTD) as\n$$\n{\\mathcal X}_t =f_2([I_2^t]) \\times_1f_1([I_1^t]) \\times_3 f_3([I_3^t])\\in{\\mathbb R}^{I_1^t\\times I_2^t\\times I_3^t},\n$$\nwhere $f_2([I_2^t])\\in{\\mathbb R}^{r_1\\times I_2^t\\times r_3}, f_1([I_1^t])\\in{\\mathbb R}^{I_1^t\\times r_1}, f_3([I_3^t])\\in{\\mathbb R}^{I_3^t\\times r_3}$ are factor functions parameterized by INRs. To adapt the Tucker-FTD and TT-FTD for streaming data, we consider their online versions by progressively expanding the input coordinate vectors $[I_n^t]$ of INRs, and hence expanding the sizes of factor matrices/tensors to fit the new tensor size. We give a numerical example by comparing different tensor decompositions under the OFTD framework, as shown in the Table below. We control the rank of different methods such that they hold similar number of learnable parameters. Tucker-FTD achieves a relatively better result in this case, which can be rationally explained by the additional core tensor parameter $\\\\mathcal C$ that enhances the representation ability of Tucker-FTD. In future research, we can consider more tensor decomposition paradigms such as the tensor SVD under the OFTD framework. Thanks.\n\nTable: NRE results by OFTD with different tensor decomposition paradigms on the Foreman data.\n|Method|TT|CP|Tucker|\n|-|-|-|-|\n|NRE|0.085|0.084|0.082|\n|Number of Params.|$3.12\\\\times 10^{5}$|$2.76\\\\times 10^{5}$|$2.93\\\\times 10^{5}$|\n\n>**Q2: How does the regret bound correlate empirically with performance degradation?**\n\nThanks for this insightful comment! The theoretical regret bound analysis in Theorem 1 reveals that more distant information is likely to hold larger regret bound (i.e., more easily to be forgotten) due to the distance term $(I_n^t +1-i_n)$. To empirically validate this theoretical result, we run the OFTD without memory buffer and report the NRE at different positions after the online optimization at $I_n^t=100$. We can see that more distant positions (i.e., smaller $i_n$) tend to hold larger NRE when the memory buffer is disabled, indicating that more distant information is more likely to be forgotten, precisely aligning with the theoretical result. We can also observe that introducing the long-tail memory buffer method could largely alleviate the forgetting, as shown in the Table below.\n\nTable: NRE results at different positions $i_n$ with and without memory buffer on the Foreman data.\n|Position $i_n$|10|20|30|40|50|60|70|80|90|100|\n|-|-|-|-|-|-|-|-|-|-|-|\n|Without memory buffer|0.440|0.314|0.300|0.275|0.256|0.228|0.170|0.127|0.101|0.065|\n|With memory buffer|0.075|0.104|0.074|0.071|0.078|0.098|0.077|0.100|0.093|0.066|\n\nShould you need further information, please let us know. We look forward to hearing from your feedback!"}]

Author rebuttal:

Meta-review: This paper proposes OFTD, a novel online functional tensor decomposition framework that integrates implicit neural representations with continual learning and memory replay to address streaming tensor completion. Reviewers found the work well-motivated, technically sound, and clearly written, highlighting its originality in bridging tensor decomposition with INR-based continual learning. The rebuttal successfully resolved all raised concerns, leading to positive consensus among reviewers. The work’s solid theory, strong experiments, and potential impact make it stand out, and thus it is recommended for Spotlight.
Decision: Accept (spotlight)