Title: Max Entropy Moment Kalman Filter for Polynomial Systems with Arbitrary Noise
Venue: NeurIPS 2025
Accepted: yes
Keywords: Bayes Filter

Kalman Filter

Moment Problem

Convex Optimization

Robotics
Abstract: Designing optimal Bayes filters for nonlinear non-Gaussian systems is a challenging task. The main difficulties are: 1) representing complex beliefs, 2) handling non-Gaussian noise, and 3) marginalizing past states. To address these challenges, we focus on polynomial systems and propose the Max Entropy Moment Kalman Filter (MEM-KF). To address 1), we represent arbitrary beliefs by a Moment-Constrained Max-Entropy Distribution (MED). The MED can asymptotically approximate almost any distribution given an increasing number of moment constraints. To address 2), we model the noise in the process and observation model as MED. To address 3), we propagate the moments through the process model and recover the distribution as MED, thus avoiding symbolic integration, which is generally intractable. All the steps in MEM-KF, including the extraction of a point estimate, can be solved via convex optimization. We showcase the MEM-KF in challenging robotics tasks, such as localization with unknown data association.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes the Max Entropy Moment Kalman Filter (MEM-KF) for dynamical systems with nonlinear dynamics and non-Gaussian noise.  The MEM-KF is focused on polynomial systems and can asymptotically approximate almost any distribution through a countable set of moment constraints.  Analogous to a Kalman Filter, the paper proposes prediction and measurement update steps whereby moments are propogated through the process model to recover a maximum entropy distribution (MED).  All steps in the MEM-KF can be solved via convex optimization.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**\nVery well-written paper and clear presentation that studies a fundamental problem in dynamical systems.  The ideas appear novel and are carefully-presented.  Contributions are adequately contextualized w.r.t. most existing work (modulo connections to the assumed density filter (ADF) and expectation propagation (EP)).  Overall, a strong submission.\n\n**Weaknesses**\nThe most significant weakness of this paper is in the numerical evaluations.  Filtering for nonlinear / non-Gaussian systems has enormous application potential, and I would have expected to see more interesting and compelling experiments than a few low-dimensional simulated models.  Moreover, as I understand it the authors only provide one experiment that involves the full filter.  Indeed, experiments in Sec. 7.2 and 7.3 appear only to explore the density estimation problem for a single timestep.\n\nI would argue that the primary comparison to these systems should be to the particle filter (PF), which the authors have provided but have removed to the appendix.  Compared to the PF the MEM-KF does about what one would expect, namely it is competitive up to a certain number of particles, at which point the PF provides superior estimates.  Critically, the authors do not provide any runtime statistics, so it is unclear how MEM-KF compares to PF with adequate number of particles in terms of runtime.  My intuition is that computing the higher-order moments required by MEM-KF will scale polynomially in the moment order, which could be prohibitive.  Worse yet the authors suggest that the gradient calculations in Eqs. (4) and (5) scale exponentially w.r.t. the dimension of the systems, likely why they only consider low-dimensional systems.\n\nSome detailed comments:\n* L101: \"that sums up to $r$\" --> \"that sums up to no more than $r$\"\n* L103: $\\gamma_i$ undefined (i.e. with the index)\n* L107: The definition of $M_r$ is recursive (it appears on both sides) and so it lacks an explicit definition\n* L107-108: The authors refer to $\\bar{x}_\\alpha$ as a moment, but the integral definition isn't a moment (it lacks a probability measure)\n* L119-121: This discussion is a bit confusing, the SDP relaxation is not exact but the authors argue that all optimization is convex?  Please clarify.\n* Eq. (1) : Shouldn't the measurement model be $y_k = h(x_k,v_k)$?\n* L136: \"Assume that state at time $k+1$ is solely dependent on...measurement $y_k$...\" The state process evolves independent of the measurement, perhaps the authors mean the posterior belief.\n* L154: $p_t$ undefined (with the subscript)\n* L162: Please provide the proof in the appendix for completeness.\n* Eq. (3): Shouldn't this be up to proportionality?  Where is the normalizer?\n* L210,212,218: All references to $p(x_{k+1})$ should be replaced by $p^-(x_{k+1})$. \n* First line in the for-loop, Algorithm 1: $x_{\\beta,k-1}$ should be $\\bar{x}_{\\beta,k-1}$ \n* L317: References Figure 8, I think this should reference Figure 4?\n\nQUESTIONS:\nThe authors claim (or at least allude to) in several places that MEM-KF is an optimal filter.  For example, the abstract begins as \"Designing optimal Bayes filters...\" and later in the introduction \"Thus, a natural question is, how can we provide provably optimal state estimation...\"  It is unclear in what context MEM-KF is considered an optimal filter.  Optimal in what sense?  The discussion on line3s 119-121 suggest that the SDP relaxation is not always exact, and thus this would violate any notion of optimality, no?\n\nLIMITATIONS:\nI would like to see more honesty in limitations on runtime execution.  The authors state that gradient calculations scale exponentially w.r.t. the dimension of the state-space in the \"Limitations and Future Work\" section, but this point is not raised in the main text anywhere.  This seems like a significant limitation.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 4, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-23"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a generalization of the Kalman filter called the MaxEnt Moment Kalman Filter (MEM-KF). MEM-KFs represent uncertainty through maximum entropy distributions with constrained moments of polynomials with pre-specified degree. Unlike the Kalman filter, which maintains two moments (mean and covariance), MEM-KF can represent more expressive distributions and is therefore well-suited to nonlinear filtering problems. The contributions of the work include a complete framework for modeling and filtering using MEM-KFs along with experiments to validate its characteristics on simulated data.\n\nSTRENGTHS AND WEAKNESSES:\n## Strengths\n- **Meaningful Problem Domain**. The problem of nonlinear filtering is becoming increasingly important, with applications ranging from continual learning to computational neuroscience. \n- **Originality**. This is a creative extension of the Kalman filter that applies moment-constrained maximum entropy distributions to the problem of Bayesian filtering.\n- **Qualitative Exploration**. There are a good number of ablations and plots to demonstrate the qualitative performance of MEM-KFs.\n\n## Weaknesses\n- **Experiments**. Although the experiments contain a number of visualizations, the paper lacks a results table for the SE(2) experiment. It would also be informative to include experiments with more complicated state-space dynamics, such as state estimation in a Lorenz-63 system, or real-world data.\n- **Analysis of scaling.** Since MEM-KFs represent additional computation relative to the Kalman filter, there should be some analysis (ideally theoretical or at the least empirical) about how the required memory and computation scale with dimensionality n and degree r. This is important because of the variety of sub-procedures in the filtering algorithm (moment computation, propagation, point estimation, and so on). The small scale of the experiments indicates that this is indeed a significant limitation of the method. Though not a dealbreaker *per se*, it is important to discuss these limitations thoroughly in the paper.\n- **Practicality.** The proposed algorithm relies on numerical integration, which may be impractical for high-dimensional state spaces.\n\nQUESTIONS:\n1. How does MEM-KF compare numerically to particle filter and the various extensions of Kalman Filters? How do these methods compare when applying to problems complicated dynamics such as Lorenz-63 or those involving real-world robotics data?\n2. How do each of the components of MEM-KF scale with respect to $n$ and $r$ (memory and computation)? What is a realistic upper bound for the state-space dimensionality one could expect to perform on say a single GPU?\n3. With respect to numerical integration specifically, how does MED recovery behave as the dimensionality of the state space increases?\n\nLIMITATIONS:\nAdditional discussion about scaling with respect to state-space dimensionality and polynomial degree would improve the paper.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-06-30"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper proposes a Max Entropy Moment Kalman Filter (MEM-KF), to tackle the failure of Kalman Filter (KF) in non-linear, non-Gaussian state space model settings.Unlike KF which assumes Gaussian distributions, MEM-KF uses a Moment-Constrained Max-Entropy Distribution (MED) to model state x and observation y. This paper provides quite solid mathematic deductions for 1) why MED can asymptoticly approximate any Moment-Determinate Distributions 2) how to use MED to solve the predict and update steps of a recursive Bayes filter. Although the computation complexity is relatively high, simulations show that MEM-KF outperforms traditional KF and KF variants in a non-linear, non-Gaussian localization problem setting.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\nThe theoretical framework and claims in this paper are well-supported by rigorous mathematical proofs. The writing is generally clear and accessible, making the technical content relatively easy to follow and understand.\n\nWeaknesses:\nThe practical applicability of the proposed MEM-KF is somewhat questionable. The strength of MEM_KF is on improved prediction accuracy at the cost of increased computational overhead. However, the prediction accuracy of MEM-KF relies on complete and accurate model information for belief propagation—including precise knowledge of the system dynamics f, observation model h, and the noise distributions w and v. In many real-world scenarios, however, these models are imperfect or only partially known, which limits the utility of such a method in practice.\nAdditionally, the Moment-Constrained Max-Entropy Distribution (MED) is not a novel contribution of this paper. The idea and its solution via convex optimization were already developed in [1]. As such, the paper would benefit from a more thorough discussion of why MED is particularly suitable in this context. For example, MED approximates well the moment-determinate distributions, which are neither heavy-tailed nor exhibit rapidly growing higher-order moments. Is this assumption realistic in practical applications such as robot localization? A deeper justification and empirical support for this modeling choice would significantly strengthen the paper.\n\nReference:\n[1] Bandyopadhyay, K., et al. \"Maximum entropy and the problem of moments: A stable algorithm.\" Physical Review E, 71.5 (2005): 057701.\n\nQUESTIONS:\n1. Significance of the Work\n\nCan MEM-KF handle partially known or imperfect state-space models? For example, what happens if the dynamics f, observation model h, or noise distributions w, v are only partially known or imperfect? This is often the case in real-world applications.\n\nAre moment-determinate distributions—which are neither heavy-tailed nor exhibit rapidly growing higher-order moments—appropriate for modeling practical systems such as robot localization? It would be valuable to justify whether this assumption aligns with common real-world settings.\n\n2. Benchmarking\n\nWhat is the runtime of MEM-KF compared to other Kalman filter variants (e.g., EKF, UKF, PF)? It would be helpful to include a runtime comparison table to evaluate computational efficiency.\n\nBeyond model-based algorithms, when sample size N grows, how is MEM-KF compared with neural network(NN)-based methods e.g. KalmanNet [1]? Including NN-based algorithms in the benchmark comparison would provide a more comprehensive evaluation.\n\n3. Side Questions\n\nHow should one choose the highest moment order r, considering the trade-off between accuracy and computational efficiency? For example, how much does the computational complexity increase per predict and update step as r increases, and what is the corresponding performance gain?\n\nHas the numerical stability of the algorithm been evaluated, particularly when dealing with high-order moments, which may take on large values and affect optimization?\n\n[1] Revach, Guy, et al. \"KalmanNet: Neural network aided Kalman filtering for partially known dynamics.\" IEEE Transactions on Signal Processing 70 (2022): 1532–1547.\n\nLIMITATIONS:\nSee Questions.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper considers the problem of filtering, i.e., state estimation, in non-linear, non-Gaussian systems. This is a well-studied, classical problem with contributions dating back decades and literature too vast to summarize in any meaningful way. Most contributions to this problem are, in some sense, inspired by attempts to approach the ideal of Kalman filtering in linear Gaussian models. This is done either through computationally intensive approaches such as particle filtering or Markov chain Monte Carlo or through careful design of approximate, yet analytically fast, update steps of the filtering distribution while keeping model expressivity. This paper is a contribution to the latter. The authors introduce a novel update scheme utilizing MED in order to perform filtering in a non-linear, non-Gaussian state space model. The paper assumes a belief (i.e. state) distribution with some fixed number of moments having maximum entropy. The key idea is that by thinking of the belief distribution in terms of its moments only, the prediction step can be viewed in terms of predicting the moments directly using an auxiliary dynamical system. Then the corresponding belief MED is computed by solving an optimization problem given the predicted moments. The methodology is tested on several examples, ranging from simple to more complex.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths: the paper is well-motivated and addresses a still-relevant problem. I like the use of simple and transparent examples to illustrate the methodology, making it easy to replicate and test it in practice. The comparison with particle filtering as a sanity check is appreciated. The limitations (i.e., reliance on numerical integration) are clearly stated. The approach appears to be theoretically grounded. Reliance only on convex optimization despite the potential for arbitrarily complex distributions is an advantage. The approach seems well-suited for problems with complex dynamics in low-dimensional state spaces.\n\nWeaknesses: reliance on numerical integration and optimization steps for every update. No good understanding of the computation time is provided for this method, even with comparison to e.g., particle filtering. It seems that the scalability issue may be harder to overcome and parallelization is not developed in any detail other than mentioning the use of GPUs, but it not clear how this can be done in practice. Perhaps the authors can mention some potential approximate schemes that can be used?\n\nQUESTIONS:\nIs there some relatively fast way to approximate the gradients that are used to find the predicted belief distribution? It would be good to see how well this method does in larger-scale problems when approximations are used.\n\nCan you extend this approach to handle a variable number of moments to save computation time? Behave like a linear Gaussian system when necessary, and use more moments when necessary?\n\nCan you consider dynamics in high-dimensional state spaces where we have a higher-moment model for some of the latent variables?\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-04"}