Title: Neural Bayesian Filtering
Venue: NeurIPS 2025
Accepted: no
Keywords: Partially observable systems

belief state modeling

particle filtering

bayesian filtering

normalizing flows
Abstract: We present Neural Bayesian Filtering (NBF), an algorithm for maintaining posteriors, called beliefs, over hidden states in partially observable systems.
NBF is trained to find a good latent representation of the beliefs induced by a task.
It maps beliefs to fixed-length embedding vectors, which can condition generative models for sampling.
During filtering, particle-style updates compute posteriors in this embedding space using incoming observations and environment dynamics.
NBF combines the computational efficiency of classical filters with the expressiveness of deep generative models - tracking rapidly shifting, multimodal beliefs while mitigating the risk of *particle impoverishment*.
We validate NBF in state estimation tasks in partially observable variants of Gridworld and the card game Goofspiel.


Peer reviews:
comments: SUMMARY:
The paper proposes particle filtering in an embedded space. Therefore distributions are represented via samples which serve as an input to an embedding function that maps the belief to an embedding vector which conditions a normalizing flow. Resampling can then be done by sampling from the normalizing flow.

STRENGTHS AND WEAKNESSES:
The idea of embedding the belief state is interesting. However, I would argue that the contribution is limited, as this embedding just serves as a novel resampling step in the particle filter method.

Further, I believe the paper would benefit from improving the clarity and the notation. I do not think $p(x)$ is a good choice to represent the filtering distribution.

Additionally, there seems to be an error in the description of the method. When sampling the transitions from $T_G$, the updated weight should not be reweighed with $T_G$.

QUESTIONS:
Why consider policy and system? The policy could be integrated and be seen as part of the system, like often done in filtering.

LIMITATIONS:
Yes.
TITLE: Official Review
CLARITY: 2
IMPACT: 2
ORIGINALITY: 2
DATE: 2025-06-16

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces Neural Bayesian Filtering (NBF), a method for state estimation in partially observed Markov systems with \nchanging environments and controls.\nNBF builds a latent space of the set of posterior distribution, which gets updated when a new observation arrives using a particle filtering framework in the embedding space.\nExperiments in two partially observed environments show that it outperforms competing particle filtering and deep learning algorithms.\n\nSTRENGTHS AND WEAKNESSES:\n**STRENGTHS**\n\nThis paper tackles the important problem of modeling complex and multimodal belief states, which is central to many real-world applications involving partial observability. Neural Bayesian Filtering (NBF) offers a novel perspective by combining ideas from particle filtering and deep generative models. This hybrid approach enables the model to track distributions with changing environments and controls that are hard to handle with traditional methods, while aiming to mitigate known issues like particle impoverishment.\n\nThe paper is also well written and well structured. The motivation behind NBF is clearly explained, and the technical components are introduced step by step, making the method easy to follow.\n\nExperiments show that the method outperforms particle filtering methods and recurrent models in the proposed experiments, particularly with changing environments and control.\n\n\n**WEAKNESSES**\n\nThe main concern I have with this paper is how well the proposed approach scales to larger or more complex environments. NBF depends on the assumption that the embedding model is expressive enough to represent every belief state exactly. This is a strong requirement, and it's not clear how realistic it is in practice, especially in higher-dimensional or dynamic settings. The experiments also suggest that NBF’s performance drops as environments get more complex, as in the randomized Gridworld and larger Goofspiel variants (Figures 5 and 6). This points to a limitation in how well the belief approximation holds up under harder conditions.\nThe authors do not test how well this assumption holds in more challenging environments beyond Gridworld and Goofspiel, which to me is a problem that can limit the impact this paper can have in the community.\n\nAlso, the belief embeddings themselves are computed using a weighted mean over sample embeddings (Section 3.1), which feels like a simplification. This approach might miss important structure in the distribution, like multimodality or higher-order interactions, which could be especially problematic in more complex scenarios.\n\nFinally, the donut example in Figure 2 is useful for building intuition on the learned densities, but the paper could benefit from a more concrete illustration of the learned embeddings as well, to understand what the model is learning (to simplify the task for visualization purposes you could also fix the donut width).\n\nQUESTIONS:\nAside from the points raised in the weaknesses section above, I have two other questions\n1. In the randomized grid and policy experiment, you mention that NBF performs best overall despite using relatively few particles compared to PF. However, for a fair comparison, the number of particles alone isn’t the right metric—it would be more meaningful to compare models under a similar computational budget. In the Discussion, you note that “the additional cost of embedding and generating a much smaller set of particles with our model is insignificant compared to particle simulation costs.”. Could you clarify what number of PF particles would match NBF’s total computational cost? How does PF perform in that case, when compared on equal compute?\n2. The consistency guarantee of NBF (Theorem 4.1) assumes ϵ-global observation positivity. How generic or realistic is this assumption in practical settings? Could you provide examples or intuitions for environments where it might fail?\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-16"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a method for general Bayesian filtering in the setting where the full system (e.g. the transition function and observation function) is known and can be simulated. The main innovation is to pre-train distribution embeddings that can flexibly model posterior distributions. The key features of these embeddings are that one can encode a set of sample into an embedding and can draw samples from the distribution represented by each embedding. Practically, this is implemented using normalising flows. \n\nTo use these embeddings in a filtering setting, the authors uses a simple update scheme similar to classical particle filters: first a set of particles is sampled from the embedded distribution, then the particles are simulated forward and re-weighted using the observation likelihood before finally being encoded into the updated posterior embedding. \n\nThe authors provide a theorem showing that their method is consistent if the embedding is expressive enough. Experimentally, NBF is evaluated on two synthetic environments, highlighting two benefits of the proposed method:\n1. NBF outperforms classical particle filters which suffer from impoverishment in higher dimensional cases\n2. NBF can incorporate information about the transition and policy directly, which is not possible for RNN-based approaches that are trained on fixed environments.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n- The paper is clearly written and easy to understand.\n- The method is intuitive and can be used as a slot-in replacement for any classical particle filters.\n- The method effectively addresses the curse of dimensionality for particle filters.\n- The limitations of the method is properly discussed in the text.\n\nWeaknesses:\n- The experimental evidences are somewhat lacking (see questions).\n- The proposed method seems to be a simple improvement over the well-known particle filter with a different re-sampling scheme. While this seems to solve the impoverishment problem, I question the novelty and significance of the work. \n\nThe authors discusses the limitations of their method. I feel that the two listed below are significant and can limit the application of the method.\n1. Training embeddings of $p(x)$ requires samples of all the possible posteriors induced by the problem. This limits the applicability of the method in realistic settings where it is intractable to compute these posteriors in the first place.\n2. It is unclear whether this method can scale to more complex and larger problems than ones investigated in the paper. Related to the previous point, I imagine getting representative samples of posterior distributions can be extremely difficult as the problem becomes more complex. \n\nDisclaimer: Bayesian filtering is not my primary field. While I understand the contributions and the approach, I cannot claim that I have extensive knowledge of the current state-of-the-art. My initial recommendation is borderline reject but I would be open to revise my score if the authors can 1. clarify my questions on the fairness of the experiments and 2. make the case that the results are novel and of significance to the community.\n\nQUESTIONS:\nI list three questions below - one general question on the applicability of the method and two specific questions about the choice of baseline in the experiments. \n1. **Training embeddings**: Training the distribution embeddings requires a training set of distributions. I believe this is crucial to the success of the method as it distills knowledge about what the possible posteriors are (otherwise one would not be able to compress generic distributions to a small embedding vector). The expressivity is also a crucial assumption in the theory. In the experiments, how are these posterior distributions collected? Would this be possible in practical situations?\n2. **RNN baseline - available information**: The authors argue that standard recurrent architectures cannot take advantage of information about $G$ and $\\pi$ as they are trained on fixed environments. While this is a sensible claim that is supported by the experiments, it seems somewhat unfair that NBF has access to the underlying environment and can perfectly simulate it but RNN needs to infer all of these from context. One potentially fairer baseline could look at what happens if some embeddings of $G$ and $\\pi$ are provided to the RNN as observations. For example, feeding in an one-hot embedding of where the obstacles are in randomised grid, or an index of the selected policy in goofspiel.\n3. **RNN baseline - capacity**: Related to the above point, the RNN baseline needs to do much more computation in the randomised setting as it needs to infer the exact transitions and policies from context. As such, one would expect the network size required is larger. However, judging from the appendix, in Goofspiel the sizes of RNN and NBF are equal, and in grid world, the size of the RNN is even smaller than NBF. Can the author explain the choice of network sizes? I understand that performing an extensive hyperparameter search can be prohibitive, but I do wonder what would happen if the RNN baseline is significantly larger.\n\nLIMITATIONS:\nYes. The limitations are thoroughly discussed and is much appreciated.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 2, "CLARITY": 3, "IMPACT": 1, "ORIGINALITY": 1, "DATE": "2025-06-22"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a latent representation of a belief over a state space using a fixed length embedding vector.  This embedded belief representation is then used to perform filtering in a particle filter framework.  Using the proposed filter consists of two steps: The first is an offline training step, where a function that maps weighted samples to an embedding, as well as a function that samples from the distribution described by the embedding vector are learnt from samples of beliefs.  The second step is online filtering, where the embedded belief is updated using a prediction and measurement update step.  One filtering iteration starts with generating samples in the state space from the embedded belief, then the samples are propagated through the prediction step and reweighted in the measurement update step, after which the weighted samples are converted back to an embedded belief.  The proposed approach is tested in two simple simulated scenarios with discrete state spaces, and outperforms a standard particle filter and an LSTM-based filter.  \n\nThe main contributions claimed by the paper are the definition of the embedded belief and the way to learn the embedding and sampling networks, as well as the incorporation of the embedded beliefs into a particle-based filtering framework.\n\nSTRENGTHS AND WEAKNESSES:\n### Quality:\nGiven the chosen approach of this paper (to embed posterior beliefs, and to convert between the embedded space and state space using samples), the development of the proposed approach is obvious and convincing.  In my opinion, it would have been better to describe the development in Sec. 4 in terms of the general filtering framework of the Bayes filter (see e.g. Chap. 2 in Thrun *et al.* (2005)), but the given development is clear enough.  \n\nHowever, the argument for choosing the approach presented in this paper, as well as for the way to evaluate the proposed approach, has some weak points.  Specifically: \n- The proposed approach is presented as an alternative to the particle filter, and much of the motivation for the developing the proposed approach is given as addressing two of the shortcomings of the standard particle filter, namely the need for a large number of particles in high-dimensional state spaces and particle impoverishment (see e.g. lines 9, 26-29, 88-95, and the choice of particle filter as baseline in Sec. 5).  However, the paper only mentions the \"vanilla\" (standard) particle filter, and it completely ignores the substantial body of work on the particle filter that aims to address its shortcomings, namely techniques that address the need for large number of samples in high-dimensional spaces (e.g., the well-known Rao-Blackwellised particle filter by Murphy *et al.* (2001) and much additional work based on it) and techniques that address particle impoverishment (e.g. Orguner and Gustafsson (2008) and Park *et al.* (2009)).  It is therefore unclear whether the proposed approach constitutes a significant contribution *given the current state of the research field*.  \n- The choice of representing all beliefs that are expected to be encountered during filtering by the latent representation is not well motivated in the paper.  To learn this representation seems like a daunting task for all but the simplest of scenarios, and the paper provides little indication of how difficult it would be to learn this representation for complex scenarios.  In fact, the experiments only include simple scenerios with small, discrete state spaces, and even for these scenerios it seems to be difficult to learn an appropriate latent representation (as shown in Fig. 6).  One cannot help to wonder whether the proposed approach has traded difficulty in one aspect of the particle filter for difficulty in another aspect, without necessarily improving the overall approach (except in some specific scenarios).  \n- The motivation given for not testing in more complex settings is not very convincing (lines 277-280).  Instead of restricting the experiments to scenerios that are simple enough so that the belief can be calculated analytically, one could e.g. demonstrate how well the proposed approach scales with problem complexity by using more complex scenarios and comparing the results of the proposed filter with that of a particle filter with a large number of particles (to provide an accurate estimate of the true belief, instead of a competing method).  This would have been easy to do (i.e., it would not at all be a separate study) and would have contributed much to addressing the concerns about the applicability of the proposed approach.  \n\n### Clarity:\nThe paper is well written, clear, and easy to follow.  However, it is a bit unclear in a few places, including the following:\n- The paper characterises the proposed method as \"performing [an] update in the embedding space\" (e.g. lines 179-180).  This characterisation is a bit misleading, since it seems to imply that prediction and measurement update steps are calculations performed in the embedding space; however, the prediction and measurement update steps are actually performed by the propagation and reweighting of particles in the state space.  \n- The diagram in Fig. 3 is misleading, since it appears to show that there are multiple iterations performed for the update of the filter between time $t$ and $t+1$.  In fact, the proposed filter follows the standard 2-step procedure (with an additional embedding and sampling step): the prediction step followed by the measurement update step, of which the output is fed into the prediction step of the *next* time step.   \n- The description of the Goofspiel experiment (Sec. 5.2) is not very clear: It is unclear whether experiment is performed with a full deck or a reduced deck (as described in lines 253-257), and the representation used (i.e., the definition of the state space) is unclear.  \n\n### Significance:\nThe proposed approach is interesting, original, and achieves good results in some simple scenarios.  However, due to the lack of proper contextualisation in some regards, the choice of methods to compare against, and the deficiencies in the experiments, there is significant uncertainty about what the impact of the paper will be.  Specifically:\n- No existing research on the particle filter that aims to address particle impoverishment and the requirement for a large number of particles in high-dimensional spaces is discussed in the paper, and the proposed approach is not evaluated against any of these approaches (see discussion in \"Quality\" above for specifics).    \n- There exist other approaches for embedding beliefs (see e.g. Song *et al.* (2009)), but these are not discussed in the paper, nor is the proposed approach evaluated against them.  \n- The proposed approach is not evaluated against other neural-based filtering approaches (e.g. that by Sokota *et al.* (2022)).  Without an evaluation, the supposed high computation cost of these methods mentioned in lines 316-318 remains speculation.   \n- Since the proposed approach is only evaluated in simple scenarios with discrete state spaces, there are questions about its applicability to more complex scenarios, specifically about the difficulty of defining an expressive enough embedding, learning embeddings, and the cost of inference.    \n\n### Originality:\nAlthough the idea of embedding beliefs is not novel, the way in which the paper does it and incorporates it into a filtering framework seems to be.  The proposed approach is based on well-known ideas (vector embedding, normalising flow, particle filter), but the combination is new and interesting.  \n\n### Other:\n- Line 62: In the classical filtering context, \"control\" is often understood to mean the commanded actions (e.g., the commands given to the actuators of a robot); however, \"control\" here refers to a policy, which is not necessarily the same as commanded actions.  This might cause confusion for readers that are familiar with classical filtering.  \n- Line 70: To avoid any ambiguity, it should probably be stated that $G \\in \\mathcal{G}$.  \n- Lines 77-78: $p(x)$ is widely understood to mean \"the prior distribution over $x$\"; to use it to refer to the posterior distribution over $x$ is probably not a good idea.  In addition, in the context of filtering, there are posterior distributions at each time step, and not indicating the specific posterior distribution with the notation causes ambiguity (see e.g. line 108).  \n- Line 112: The meaning of $y\\^{(1:t)}$ is not properly defined.  \n- Lines 156 and 185: \"the Appendix\" should be \"the appendix\".  \n- In lines 207 and 208, \"Approx Beliefs\" is described as a filtering algorithm; however, it obviously is not.  \n- The caption of Fig. 4 mentions shaded areas, but no shaded areas are visible in the plots.  \n- In lines 233 and 234, it is unclear what \"the belief model\" refers to.  \n- Line 270: The meaning of $t$ is unclear.  \n\n### References:\n- Murphy, K. and Russell, S., 2001. Rao-Blackwellised particle filtering for dynamic Bayesian networks. In Sequential Monte Carlo methods in practice (pp. 499-515). New York, NY: Springer New York.\n- Orguner, U. and Gustafsson, F., 2008. Risk-sensitive particle filters for mitigating sample impoverishment. IEEE Transactions on signal processing, 56(10), pp.5001-5012.\n- Park, S., Hwang, J.P., Kim, E. and Kang, H.J., 2009. A new evolutionary particle filter for the prevention of sample impoverishment. IEEE Transactions on Evolutionary Computation, 13(4), pp.801-809.\n- Sebastian Thrun, Wolfram Burgard, and Dieter Fox. 2005. Probabilistic Robotics (Intelligent Robotics and Autonomous Agents). The MIT Press.\n- Song, L., Huang, J., Smola, A. and Fukumizu, K., 2009, June. Hilbert space embeddings of conditional distributions with applications to dynamical systems. In Proceedings of the 26th Annual International Conference on Machine Learning (pp. 961-968).\n\nQUESTIONS:\n- How difficult is it expected to be to define an expressive enough embedding and learn embeddings, and how costly is the inference expected to be for more complex scenarios and continuous state spaces? \n- Are there any good reasons why the proposed approach is not evaluated against state-of-the-art versions of the particle filter, other neural-based filters, or the proposed belief embedding evaluated against other belief embeddings?  \n- Is there any good reason why the proposed approach is not evaluated in more complex scenarios?\n\nLIMITATIONS:\nThe proposed approach is described quite clearly and fairly.  However, the applicability to and limitations in more complex scenarios remain unclear due to the simplicity of the scenarios used in the experiments.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-02"}