Title: Functional Complexity-adaptive Temporal Tensor Decomposition
Venue: NeurIPS 2025
Accepted: yes
Keywords: functional tensor decomposition，Bayesian Method
Abstract: Tensor decomposition is a fundamental tool for analyzing multi-dimensional data by learning low-rank factors to represent high-order interactions. While recent works on temporal tensor decomposition have made significant progress by incorporating continuous timestamps in latent factors, they still struggle with general tensor data with continuous indexes not only in the temporal mode but also in other modes, such as spatial coordinates in climate data.  Moreover, the challenge of self-adapting model complexity is largely unexplored in functional temporal tensor models, with existing methods being inapplicable in this setting.  To address these limitations, we propose functional Complexity-Adaptive Temporal Tensor dEcomposition (Catte). 
 Our approach encodes continuous spatial indexes as learnable Fourier features and employs neural ODEs in latent space to learn the temporal trajectories of factors. To enable automatic adaptation of model complexity, we introduce a sparsity-inducing prior over the factor trajectories. 
 We develop an efficient variational inference scheme with an analytical evidence lower bound, enabling sampling-free optimization. Through extensive experiments on both synthetic and real-world datasets, we demonstrate that Catte not only reveals the underlying ranks of functional temporal tensors but also significantly outperforms existing methods in prediction performance and robustness against noise.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper proposes CATTE, a Bayesian CP-style model for generalized temporal tensors whose modes may all be continuous-indexed. CATTE (i) represents spatial coordinates through learnable Fourier features and evolves factors with a latent Neural ODE, (ii) places a dimension-wise Gaussian-Gamma prior over the entire family of factor trajectories, and (iii) derives a fully closed-form evidence-lower-bound that admits sampling-free variational inference. Experiments on one synthetic and three real-world data sets (CA-Traffic, Server-Room and Pacific Sound-Speed Field) show that CATTE outperforms five recent baselines in RMSE/MAE while automatically discovering the ground-truth rank, and exhibiting robustness to several noise types.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths\n\nS1. The paper is technically solid: it frames the decomposition in a fully Bayesian manner and derives an evidence-lower-bound whose every term is analytic.\nS2. The motivation and related-work sections clearly position the contribution within the temporal-tensor literature.\nS3. Extending automatic-rank ideas to functional factor trajectories and producing a closed-form ELBO for this setting is, to the best of my knowledge, novel and likely to stimulate further research in probabilistic tensor analysis.\n\nWeaknesses\nW1. (Limited methodological novelty) \nAlthough the paper delivers a well-engineered system, each of its core building blocks (Fourier positional encodings, neural-ODE dynamics, and an automatic-rank-determination sparsity prior) are already well established in the literature. The contribution therefore lies chiefly in how these standard components are combined, rather than in a fundamentally new modelling principle or theoretical insight. \n\nW2. (Parameter efficiency and fairness of comparison) \nEquation (5) defines every latent factor as a trajectory u^k(i_k, t) that is jointly conditioned on the continuous spatial index i_k and on the timestamp t. This modelling choice implies that CATTE's learnable parameter set grows proportionally with #spatial points x #time-stamps x R. The paper does not report the resulting parameter counts, whereas the baselines rely on time-invariant factors or low-order spline/GP coefficients. Only per-iteration runtimes are given (Table 6), so the accuracy gains in Table 1 may partly reflect CATTE's much larger capacity rather than genuine efficiency. A like-for-like parameter budget (or test-error curves versus model size) would be necessary for a fair comparison.\n\nW3. (Limited empirical scale)\nEvaluation is limited to three modest-scale datasets. The benchmarks contain roughly 10,000 observed entries and at most 34 timestamps. No experiment tests very large (>10^6) data, so the linear-time claim remains speculative for industrial-scale workloads.\n\nW4. (Presentation issues)\nSeveral typographical errors (e.g., “Exisiting” -> “Existing”: line 48, “Give” -> “Given”: line 148, “contains” -> “containing”: line 281, “demonstating” -> “demonstrating”: line 559, etc.) distract from the narrative.\n\nQUESTIONS:\nCould you articulate a specific theoretical insight or property that cannot be achieved by any pair-wise combination of Fourier features, neural-ODE dynamics, and ARD sparsity alone? \nIn other words, what new capability or guarantee emerges uniquely from the full CATTE synthesis?\n\nFor each dataset, what is the total number of trainable parameters in CATTE versus the baselines? \nIf you downscale CATTE so that its parameter budget matches that baseline, how do RMSE/MAE and wall-clock convergence change?\n\nBeyond time horizon, how does CATTE's runtime scale with the number of spatial points? \nA synthetic experiment sweeping both dimensions (timestamps × spatial points) would help practitioners judge feasibility.\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-06-28"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes CATTE, a method for temporal tensor decomposition that handles tensors with continuous indices across all modes. The key innovation is combining neural ODEs with automatic rank determination for functional temporal tensors.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengthens**\n\n1. Comprehensive baselines: Compares against 6 state-of-the-art methods spanning temporal (NONFAT, DEMOTE), functional (FunBaT, LRTFR), and other approaches\n2. Consistent improvements: Shows great RMSE reduction across datasets, with particularly strong performance on spatialtemporal ones\n3. Computational efficiency: Linear scaling with time series length\n4. Existing methods either handle rank determination or continuous indices, but not both simultaneously. The solution is mathematically sound and showed improvement. \n\n**Weaknesses**\n1. Individual components (Neural ODEs, Bayesian rank selection, functional tensors) all exist. Thus, contribution appears largely incremental - combining existing techniques\n2. Fourier feature dependency: Choice may not be optimal for all data type; What if sparse data. In practice, tensor data is very sparse. Sparse basis functions might be helpful. \n3. Hyperparameter analysis is not enough. For example, having three sets of $a$ and $b$ for Gamma on CA traffic can't conclude the method auto rank selection is robust to initialization. \n4. Missing analysis of related work in Bayesian Neural ODEs literature\n\nQUESTIONS:\n1. Please discuss and clarify the difference and improvement over existing methods like DEMOTE and FunBaT\n2. It seems to the ELBO derivation problems (Eq. 17) has dimensional inconsistencies. It needs clarification - what does vec() of a matrix plus identity $\\mathbf{I}$ mean here? Similar issues raise in other places (e.g., Eq. 24)\n\nLIMITATIONS:\nYes. In supplementary", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-06-30"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nIn this work, the authors try to solve the problem of temporal tensor decomposition with continuous indices in all modes. The CATTE algorithm is developed by employing the encoder-decoder structure and neural ODE to model the factor trajectories. An automatic tensor rank determination is utilized. A sampling-free variational inference algorithm with a closed-form ELBO is proposed. Experimental results on both synthetic and real-world datasets demonstrate the desired performance of the proposed method.\n\nSTRENGTHS AND WEAKNESSES:\nStrength:\n1. A new method that generalizes the modeling the temporal tensor data with continuous indexes in all modes.\n2. Experiments show good performance and robustness across diverse datasets.\n\nWeakness:\n1. The novelty is limited as the encoder-decoder structure, the neural ODE, and the automatic tensor rank determination method have already been used in modeling dynamic systems or tensor decomposition.\n2. The scales and dimensions of the tensor data in experiments are relatively small.\n\nQUESTIONS:\n1. The main concern of this paper is the limited novelty of the proposed method. As the encoder-decoder structure and neural ODE are common methods for modeling dynamical systems, the current algorithm seems to be an application to the temporal tensor decomposition domain (Specifically, the ODE has already been applied to temporal tensor decomposition [6][28]). Also, the automatic tensor rank determination method has already been developed. The author should give a clear statement on the contribution of the proposed method, such as a more comprehensive related work on encoder-decoder structure and neural ODE, and a detailed list of contributions for the proposed method.\n\n2. How does CATTE scale with the size or dimension of tensors (e.g., 5-order or higher-order tensors and $100\\times100\\times500$ tensors)? Are there computational bottlenecks, especially with the ODE solvers or variational inference?\n\n3. As the author shows the robustness to different noise distributions, it is interesting how about the robustness when the data contains missing entries.\n\n4. Are there any smoothness (stationary) assumptions on the temporal dynamics of the tensor? How is the performance of the CATTE if there is a sudden change?\n\n5. It seems the ref. [28] and ref. [34] are the same. Please check if there are some mistakes.\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 4, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors propose a new temporal tensor decomposition method for tensors with continuous modes. Instead of treating time as a separate mode, their method allows latent factor functions, parametrized by neural ODEs, to depend jointly on both the mode index and continuous time. The mean-field variational posterior proposed admits analytically tractable components, allowing for efficient inference. The authors compare their method empirically with other temporal tensor decomposition methods.\n\nSTRENGTHS AND WEAKNESSES:\n## Strengths\n- The paper puts forth a clear problem, coherently expresses shortcomings of existing methods, develops a method that addresses these shortcomings, and demonstrates its favorable performance against reasonable baselines. Overall, the paper is written well.\n- The solution offered addresses the original problems adequately. The neural ODE factor functions allow for expressive and smooth temporal dynamics. The probabilistic model and the proposed inference scheme are valid and appropriate, allowing for expressive yet efficient modeling.\n- The empirical investigation is detailed and compares the performance of the method to important baselines. Additional experiments investigate the interpretability of the method, as well as details of rank determination\n\n## Weakness\n- Although overall the proposed method is well-motivated, arguably important modeling choices, trade-offs involved therein, and explicit justification for the choices made are not always presented in sufficient detail. The next section provides more details for this.\n\nQUESTIONS:\n- L78: Brief mention of rank not having to be scalar (as in Tucker decomposition) would be helpful\n- L94: I think \"automatically determined\" is not a good description of the authors' methodology. See my comments about this below.\n- L105: \"... it is inadequate for...\" Why not? Please explain more concretely.\n- L115: \"... which requires special treatment [16].\" Why? Please explain.\n- L119: What is the best way to handle discrete modes with the proposed method? \n- L156: Please express any potential downsides, if any, of this efficiency-motivated approach.\n- L164: Please clearly state and discuss why the current modeling approach is preferred over a GP for the factor functions, making explicit the trade-offs involved.\n- L199: Please direct the interested reader to the full derivations for the ELBO terms and the factor updates.\n- L186: The mean-field assumption is commonly utilized in variational inference. However, given the potentially intricate interactions between factors, index variables, and time, please discuss the potential problems with this assumption. For example. would this pose a problem in modeling temporal tensors with hierarchical seasonality (months, weeks, days)?\n- L248, Figure 2: Why would the uncertainty be very low in Fig 2, at the top of the second hill?\n- L311: Please discuss why ELBO-based model selection is not utilized, and compare the existing method for model selection with it (in a simpler setting, if needed). You can defer the second part to camera-ready (if accepted) if there are time/compute limitations.\n\nLIMITATIONS:\nThe authors are recommended to express the limitations of their approach taking into consideration the feedback provided above.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-02"}