Title: Online Functional Tensor Decomposition via Continual Learning for Streaming Data Completion
Venue: NeurIPS 2025
Accepted: yes
Keywords: data stream; tensor decomposition
Abstract: Online tensor decompositions are powerful and proven techniques that address the challenges in processing high-velocity streaming tensor data, such as traffic flow and weather system. The main aim of this work is to propose a novel online functional tensor decomposition (OFTD) framework, which represents a spatial-temporal continuous function using the CP tensor decomposition parameterized by coordinate-based implicit neural representations (INRs). The INRs allow for natural characterization of continually expanded streaming data by simply adding new coordinates into the network. Particularly, our method transforms the classical online tensor decomposition algorithm into a more dynamic continual learning paradigm of updating the INR weights to fit the new data without forgetting the previous tensor knowledge. To this end, we introduce a long-tail memory replay method that adapts to the local continuity property of INR. Extensive experiments for streaming tensor completion using traffic, weather, user-item, and video data verify the effectiveness of the OFTD approach for streaming data analysis. This endeavor serves as a pivotal inspiration for future research to connect classical online tensor tools with continual learning paradigms to better explore knowledge underlying streaming tensor data.


Peer reviews:
comments: SUMMARY:
This paper proposes a novel framework, Online Functional Tensor Decomposition (OFTD), for streaming tensor data completion. The core idea is to model CP decomposition factors as Implicit Neural Representations (INRs), thus to capture spatial-temporal continuity and
low-dimensional compact structures of streaming data . To mitigate catastrophic forgetting, this work adopts a long-tail memory replay mechanism and theoretically establishes a regret bound. Extensive experiments on real-world single- and multi-aspect streaming datasets show that OFTD outperforms a wide range of baselines in terms of NRE for the most cases.

STRENGTHS AND WEAKNESSES:
**Strengths:**

1. The integration of INR with CP decomposition for streaming tensor data completion is novel and well-motivated.

2.  The method is validated on both single-aspect and multi-aspect tensor data streams with clear performance improvements over the baselines.

3. Authors provide a regret bound that explains the forgetting behavior of the model.

**Weaknesses:**

1. Although the work is   motivated by continual learning, connections to and comparisons with classic continual learning approaches are missing.

2. The performance of OFTD may be sensitive to the architecture or hyperparameters of the INR (e.g., depth, activation), but this is not explored or ablated in the current version.

QUESTIONS:
1. Why CP decompositions?  Is it possible to generalize this method to other tensor decompositions like TSVD or Tensor Train?

2. How does the regret bound correlate empirically with performance degradation?

LIMITATIONS:
yes
TITLE: Official Review
CLARITY: 3
IMPACT: 3
ORIGINALITY: 2
DATE: 2025-06-23

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper aims to tackle the challenges in high-velocity streaming tensor data analysis, where new data expands across multiple dimensions of the original dataset. Specifically, it leverages implicit neural representations to model the tensor decomposition factors, thereby enhancing the capability of dynamic data structure modeling. To mitigate the forgetting of historical data when fitting new information, the paper adopts a continual learning approach and develops a memory replay mechanism to address this issue. Theoretically, the paper analyzes the spatial-temporal smoothness regularization and the forgetting pattern of the proposed method during the learning process. In the experimental section, the paper demonstrates the effectiveness of the proposed method on real-world datasets.\n\nSTRENGTHS AND WEAKNESSES:\n### **Strengths**:\n\n• The idea of introducing Functional Tensor Decomposition to the online tensor analysis setup is novel. Unlike classical online tensor analysis methods, it explores the nonlinear relationships of the tensor factors, which is a significant enhancement in streaming data analysis.\n\n• The paper demonstrates an interesting combination of concepts between continual learning and streaming tensor analysis, which may inspire future research.\n\n• The paper is technically sound, with sufficient theoretical analysis to support the effectiveness and the motivation behind the proposed method.\n\n\n### **Weaknesses**:\n\n• The proposed method tends to forget information distant from the new tensor data, leading the paper to propose a memory replay mechanism to avoid this. However, as more and more new data is introduced, the memory buffer must grow larger and larger. This seems like a serious side effect (e.g., increasing computational complexity), especially when real-time processing is a critical consideration in streaming data analysis.\n\n• The theoretical analysis seems to hold only when the weight matrices of INR during training remain i.i.d. $\\mathcal{N}(0,\\omega^{2})$. How can this be ensured? Additionally, the first assumption in Theorem 1 seems questionable, as it assumes the boundary variables of the model are invariant before and after training on the new data. What if the deep model severely overfits the new data? If this happens, intuitively, even the nearest point to the new data will be affected by this overfitting.\n\n• There are several existing studies on Functional Tensor Decomposition. For example, the method presented in [1] appears to achieve the same objective as INR. The paper misses references to these types of works.\n\nReference:\n\n[1]. Functional Bayesian Tucker Decomposition for Continuous-indexed Tensor Data, ICLR, 2024\n\nQUESTIONS:\nHow does the complexity of the INR affect the model performance?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-29"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe work proposes an online tensor decomposition framework that incorporates implicit neural representation (INR) strategy that helps with more dynamic continual learning. The tensor decomposition is performed in an online manner at every time step with limited forgetting of the previous tensor knowledge. Towards this, a long-tail memory replay method by constructing a memory buffer controlled by a long-tail beta distribution is introduced. Experiments are presented to evaluate the proposed approach and compare with the baselines.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\n1. The paper is well-written and hence it is easy to follow. The notations are clearly defined, and theorems are well presented.\n\n2. INR has emerged as a powerful technique in computer vision and has not much explored in online tensor decomposition. Hence, the work can spark some follow-up studies in this domain utilizing these powerful tools.\n\nWeaknesses:\n\n1. Experimental study presented in the main paper is limited. It looks like the completion is performed in small scale datasets. The dataset size and details are not clearly articulated. More ablation studies are also expected to better evaluate the approach.\n\nQUESTIONS:\n1.\tIn Eq. (2) while introducing the optimization at time step t, the temporal relation is not specified for the factor matrices.\n\n2.\tAccording to the definition of ${\\cal Y}_t$ in Definition 3, the tensor can be extremely sparse. Yet, such prior knowledge is not employed in (6), which could limit the performance of the approach.\n\n3.\tMinor comments: There is some formatting issue in terms of using the citations to the references. The author names are repeated twice which is better to avoid.\n\n4.\tAccording to Theorem1, not only the distance from new data stream, but also the neural network architecture and the choice of $r$ also plays role in forgetting. There is no discussion regarding that to give some insights in the practical settings.\n\n5.\tThe parameters of the beta distribution is currently fixed, but according to the theorem, it is dependent on the learning process and can be influenced by that.\n\n6.\tThere are other influencing factors such as rank, the choice of omega, the number of layers. How are these factors affecting the performance and be selected for optimal outputs.\n\n7.\tI could not see much information presented regarding the baselines and their tensor decomposition methods. Are all of them CP? Or have you tried with other tensor decomposition methods such as Tucker etc?\n\nLIMITATIONS:\nNA", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 4, "IMPACT": 4, "ORIGINALITY": 3, "DATE": "2025-07-03"}