ESCORT: Efficient Stein-variational and Sliced Consistency-Optimized Temporal Belief Representation for POMDPs
Authors: Yunuo Zhang
Baiting Luo
Ayan Mukhopadhyay
Gabor Karsai
Abhishek Dubey
Venue: NeurIPS 2025 poster ()
Keywords: POMDPs
Particle-based Variational Inference
Reinforcement Learning
Primary area: reinforcement_learning
TL;DR: ESCORT: A particle-based framework for representing complex belief distributions in POMDPs

Abstract: In Partially Observable Markov Decision Processes (POMDPs), maintaining and updating belief distributions over possible underlying states provides a principled way to summarize action-observation history for effective decision-making under uncertainty. As environments grow more realistic, belief distributions develop complexity that standard mathematical models cannot accurately capture, creating a fundamental challenge in maintaining representational accuracy. Despite advances in deep learning and probabilistic modeling, existing POMDP belief approximation methods fail to accurately represent complex uncertainty structures such as high-dimensional, multi-modal belief distributions, resulting in estimation errors that lead to suboptimal agent behaviors. To address this challenge, we present ESCORT (Efficient Stein-variational and sliced Consistency-Optimized Representation for Temporal beliefs), a particle-based framework for capturing complex, multi-modal distributions in high-dimensional belief spaces. ESCORT extends SVGD with two key innovations: correlation-aware projections that model dependencies between state dimensions, and temporal consistency constraints that stabilize updates while preserving correlation structures. This approach retains SVGD's attractive-repulsive particle dynamics while enabling accurate modeling of intricate correlation patterns. Unlike particle filters prone to degeneracy or parametric methods with fixed representational capacity, ESCORT dynamically adapts to belief landscape complexity without resampling or restrictive distributional assumptions. We demonstrate ESCORT's effectiveness through extensive evaluations on both POMDP domains and synthetic multi-modal distributions of varying dimensionality, where it consistently outperforms state-of-the-art methods in terms of belief approximation accuracy and downstream decision quality.

ESCORT: Efficient Stein-variationa and Sliced manistency-Optiiizee Temporal Belief Representation for POMDPs
Yunuo Zhang Vanderbilt University yunuo.zhang@varn @anderbilt.edu
Baiting Luo Vanderbilt University baiting Iuo vanderbilt.edu
Ayan Mukhopadhyay
Vanderbilt University
ayan.mukhopadhyayy @ vanderbilt.edu
Gabor Karsai Vanderbilt University
Abhishek Dubey
Vanderbilt University
gabor karsai @ vanderbilt.edu
abhishek.dubee @anderbiilt.eduu
Abstract
In Partially Observable Markov Decision Processes (POMDPs), maintaining and updating belief distributions over possible underlying states provides a principled way to summarize action-observation history for effective decision-making der uncertainty. As environments grow more realistic, belief distributions develop complexity that standard mathematical models cannot accurately capture, creat- ing a fundamental challenge in maintaining representational accuracy. Despite advances in deep learning and probabilistic modeling, existing POMDP belief approximation methods fail to accurately represent complex uncertainty struc- tures such high-dimensional, multi-modal belief distributions, resulting in es- timation errors that lead to suboptimal agent behaviors. To address this chal- lenge, we present ESCORT (Efficient ftein-variations and sliced Consistency- Optimized Representation for Temporal beliefs), particle- based framework for capturing complex, multi-modal distributions in high-dimensional belief spaces. ESCORT extends SVGD with two key innovations: correlation-aware projections that model dependencies between state dimensions, and temporal consistency con- straints that stabilize updates while preserving correlation structures. This ap- proach retains SVGD's attractive-repullive particle dynamics while enabling ac- curate modeling of intricate correlation patterns. Unlike partiele filters prone degeneracy or parametric methods with fixed representational capacity, ESCORT dynamically adapts to belief landscape complexity without resampling restric- tive distributional assumptions. We demonstrate ESCORT'ss effectiveness through extensive evaluations butions of varying dimensionality, where consistently outperforms state-of-thh methods in terms of belief approximation accuracy and downstream decision quality. Our code is available at
both POMDP domains and synthetic multi-modal distri-
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Introduction
Partially Observable Markov Decision Processes (POMDPs) [Aström, 1965] provide a powerful mathematical framework for sequential decision-making under uncertainty, enabling agents make optimal decisions despite having only partial information about their environment [Kaelbling et al., 1998]. At the core of POMDP solutions lies the concept of belief states: probability distributions over possible underlying states conditioned on the history of actions and observations [Kaelbling al., 1998]. As POMDPs are applied to increasingly complex real-world domains [Lauri et al., 2023, Kurniawati, 2022. Zhang et al., 2024], the underlying belief distributions develop sophis- ticated characteristies that standard mathematical models struggle to accurately capture [Roy and Gordon, 2002, Brooks et al.. 2006, Zhang et al., 2025]. Specifically, realistic belief distributions in POMDPs often exhibit challenging combination of high dimensionality (due to complex state spaces) [Roy and Gordon, 2002] and multi multi-modality (from ambiguous observations creating multi- ple distinct hypotheses) [Chen et al., 2022, Zhang et al., 2025], which traditional approaches struggle to model efficiently. The inability to efficiently represent and update these complex belief distribu- tions creates a fundamental bottleneck in developing practical POMDP solutions, as even small errors in belief approximation can propagate and significantly degrade decision quality over time.
Existing approaches to belief approximation in POMDPs distributions. Parametric methods using neural representations struggle with uncertainty structures: DRQN [Hausknecht and Stone, 2015] and ADRQN [Zhu et al.. 2018] compress histories into vec- tors that poorly capture multi-modal uncertainty, while even DVRL [Igl et al., 2018], despite using particle-based VAEs [Kingma and Welling, 2013]. fails maintain multiple distinct hypotheses simultaneously. These parametric approaches efficiently process high-dimensional data but sac- rifice representational expressiveness despite theoretical universal approximation power, neural networks face computational inefficiency and generalization challenges [Zhang et al., 2016]. Their fixed parametric nature prevents adaptation to varying uncertainty complexity, causing cumulative belief estimation errors over time.
face significant limitations with complex
On the other hand, particle-based methods offer flexibility in representing arbitrary distributions but face critical limitations. SIR filters [Gordon et al.. 1993], which underpin leading POMDP solvers like POMCP [Silver and Veness, 2010], POMCPOW [Sunberg and Kochenderfer, 2018], ARDESPOT [Somani et al., 2013], and AdaOPS [Wu et al., 2021], struggle with the curse of di- mensionality and particle degeneracy in high-dimensional spaces. Their stochastic resampling leads to mode collapse, failing to maintain coverage across modal distributions, especially with am- biguous observations [Zhang et al., 2025]. These methods also inefficiently capture dependencies between state dimensions- either making oversimplified independence assumptions or requiring exponentially more particles to model joint distributions accurately, significantly limiting their ap- plicability to complex POMDPs.
Inspired by the effectiveness of Stein Variational Gradient Descent (SVGD) [Liu, 2017] in Bayesian inference, we explore deterministic particle evolution as principled alternative. SVGD avoids resampligginduuced degeneracy through continuous gradient-based updates: particles move deter- ministically via V logp(()) with kernel repulsion Vk(2,z' maintaining multi-modal coverage with- out discarding hypotheses [Liu, 2017]. Unlike fixed parametric architectures, SVGD dynamically adapts its particle distribution concentrating particles in high-uncertainty regions while provid- ing sparse coverage elsewhere aligning representational capacity with belief complexity without architectural changes. However, standard SVGD itself suffers from kernel degeneracy in high- dimensional spaces [Zhuo et al., 2017, Chen and Ghattas, repulsive forces-and and cannot preserve complex correlation structures between state dimensions, leading to mode collapse in multi-modal distributions. Recent extensions like MP-SVGD [Zhuo et al., 2017] and SVMP [Wang et al., 2018] have demonstrated success in high-dimensional Bayesian inference by leveraging graphical model structures to guide particle evolution. However, these meth- ods require fixed structures that cannot adapt to deseeratiiondependeen correlations in POMDPs, where belief correlation patterns change dynamically with observation history [Boyen and Koller, 1998].
both attractive and
ESCORT addresses these fundamental limitations through a novel belief update mechanism that ex- tends SVGD with two key regularization components. Drawing insights from sliced optimal trans- port theory [Kolouri et al., 2019], we introduce: (1) correlation-awar regularization that preserves dimensional dependencies during particle updates, mitigating kernel degeneracy while maintain-

---

ing multi-modal representational flexibility, and (2) temporal consistency constraint that prevents unrealistic belief jumps between timesteps while preserving learned correlation structures. This de- terministic update framework with targeted regularization prevents the accumulation of estimation errors that propagate through sequential decision-making. Our contributions are:
We extend SVGD overeome particle degeneracy in traditional filters and fixed representational capacity in parametric approaches for complex POMDP belief approximation.
We introduce correlation-awan regularization inspired by optimal transport theory that preserves dimensional dependencies and mitigates kernel degeneracy in high-dimensional spaces.
We develop temporal consistency regularization that prevents unrealistic belief jumps while pre- serving correlation structures, ensuring stable belief evolution.
While future work will address computational overhead from correlation matrix computation and projection optimization, ESCORT provides a modular belief representation that seamlessly inte- grates with existing POMDP solvers for broader practical impact.
We demonstrate ESCORT's effectiveness through extensive experiments on Light-Dark Naviga- tion [Platt al., 2010, Silver and Veness, 2010], Kidnapped Robot [Choset al., 2005], and Multi-Target Tracking [Rong Li and Jilkov, 2003] benchmarks, as well as synthetic multi-modal distributions, showing consistent improvements belief fidelity and decision quality.
2 Background
2.1 Partially Observable Markov Decision Processes (POMDPs)
A partially observable Markov decision process (POMDP) [Aström, 1965] is formalized as tuple (S, A, T, R, Q2 (,0)): 7): states S. actions A, transition function ('s [s, a) (probability of transition- ing to s' from state via action a), reward function R(s, a), observations f2, observation function O a) (probability of observing o after reaching via action a), and discount factor 7 E [0, 1). Unlike MDPs, agents cannot directly observe states, fundamentally increasing problem complexity.
Agents maintain belief states b(s) probability distributions over possible states. After action a and observation beliefs update via Bayes' rule: 6'(s') O(ols', a) Eses T(s' (s, a)b(s), where normalizes to ensure Eves 6'(s') 1. This update encapsulates the agent's 's knowledge given their action-observation history.
2.2 Stein Variational Gradient Descent (SVGD)
SVGD [Liu, 2017] is a deterministic sampling algorithm that iteratively transports particles { ) to approximate target distribution p(æ) by minimizing KI. divergence through functional gradient descent. Particles update via I; +- Ty + e0(xs), where is step size and the optimal velocity field (x(x) > [k(a,, 2:) V log p(æ;) + V k() belongs to reproducing kernel Hilbert space with kernel k(x,, x²) [Liu, 2017].
The update balances exploitation (k(z,.z)V logp(z,) driving particles toward high-density re- gions) and exploration (V,, ((xk,,)),,)) creating repulsive forces preventing collapse). The RBF kernel k(x,,') = exp(- 1|1 22) with bandwidth h controls inter-particle interactions [Garreau al., 2017, Liu, 2017], offering theoretical convergence guarantees with computational efficiency.
However, SVGD struggles in high-dimensional POMDPs where correlation structures Cjj
En (with j indexing state dimensions) capture statistical dependencies [Chen et al., 2022].
En
Standard isotropic RBF kernels create uniform repulsive forces [Zhuo et al., 2017], failing to pre- serve anisotropic belief distributions' correlation patterns. This causes kernel degeneracy in high dimensions [Chen and Ghattas, 2020]-kerna values become nearly constant monstant-preventing particles from aligning along principal correlation directions, degrading belief approximation and POMDP performance.
3

---

2.3 Projection Methods in Optimal Transport
Optimal Transport (OT) provides a geometric framework for comparing probability distribu- tions via minimal transformation cost [Villani, 2008). The Wasserstein distance W W,(,, v) =
(inf)er(w..)) If || dy,, 8)) 1/P captures both probability mass differences and geometric re- lationships, where [/4,, v) denotes joint distributions with marginals and v.
To reduce computational expense in high dimensions, Sliced Wasserstein Distance (SWD) [Ra- bin et al., 2011] projects distributions onto one-dimensional subspaces: sW,(u,)
(fa--1 WF(R[p) julo Rlv 1/p. where R|--o denotes the Radon transform along direction 0. Generalized Sliced Wasserstein (GSW) Distance [Kolouri et al., 2019] extends this with nonlinear
transformations: GSW, v) = (Je W (Rn(u)) Rr (o(())))((0)) 1/P using parameterized function h2)),, with max-GSV Distance variant: max-GSW, ((,, 2) maxage W, Rx(plo).
ESCORT integrates these projection principles as regularization mechanisms within SVGD's up- date process rather than distance metries. This enables learning transformation matrices that identify sigmificant correlation directions while mitigating kernel degeneracy, allowing particles to aligm along principal correlation directions while maintaining multi-modaa diversity-addressinn fundamental challenges in complex POMDP belief representation.
3 Approach
Repeat
Correlation-Awa Aware Deterministic Belief Update
Temporal
Policy Network
Consistency Regularization
Distribution
k(x,, V) logp(x)
n(alb) PolicyNetwork(B)
Reon
Projection matrices A, Wassierstein distance W
ESCORT Particles True Distribution Modes Attractive Forces Repulsive Forces Correlation Projections
Figure 1: Overview of the ESCORT framework. The diagram illustrates the iterative process of maintaining accurate belief representations in POMDPs through deterministic particle evolution. Purple dots represent particles, yellow regions show distribution modes, and colored arrows indicate different force types. In the Temporal Consistency component, green belief transitions while red crossed arrows indicate prevented unrealistic jumps.
represent permissible
We present ESCORT (Efficient Stein-variational and sliced Consistency- Consistency-Optimizee Representation for Temporal beliefs), particle-based framework addressing the challenges of belief approxima- tion in complex POMDPs, as illustrated in Figure ESCORT extends SVGD with correlation- aware projections and temporal consistency constraints, enabling effective representation of high- dimensional, multi-modal belief distributions with intricate correlation structures. Through deter- ministic particle evolution and strategic regularization, our approach maintains particle diversity while preserving dimensional dependencies, overcoming limitations of both parametric methods and traditional particle filters.

---

3.1 Correlation-Aware Deterministic Belief Update Mechanism
As established in Section 2.2, SVGD addresses particle degeneracy through deterministic evolu- tion that theoretically converges to the target distribution without resampling. SVGD applies the perturbation:
6 (x)
Σ [M x) Va, logp(r;) + Vay k(),,
(I)
In the POMDP context, p(.) represents the belief given by Bayes' rule: p(s') & O a) E. Ts''ss, a))(s))), with the score function Va, log p(zy) approximated numerically using finite dif- ferences.
This formulation balances attractive forces toward high-densit regions with repulsive interactions that maintain particle diversity. However, standard SVGD faces critical limitations POMDP set- tings. First, it suffers from kernel degeneracy in high- nigh-dimensional state spaces, where the RBF kernel k(z, = exp(- + æ - z²1l²) produces nearly uniform values, weakening repulsive forces and leading mode collapse [Zhuo et al., 2017, Chen and Ghattas, 2020]. More fundamen- tally, standard SVGD's s isotropic RBF kernel creates uniform repulsive forces that fail to preserve the anisotropic correlation patterns inherent in POMDP belief distributions [Wang al., 2017]. In POMDPs, these correlation structures are captured by the correlation matrix C with elements = Eyy/ VEx Ejj. where E is the covariance matrix. These off diagonll elements Ci encode eritical statistical dependencies between state representation and downstream decision quality.
innisiionninoooattiionnnn essential for accurate belief
Recent extensions like MP-SVGD [Zhuo et al., 2017] and SVMP [Wang et al., 2018] attempt to address nigh-ddmensional challenges through fixed graphical structures that decompose the kernel into localized interactions. However, these approaches remain insufficient for POMDPs, which require adaptive correlation modeling that dynamically responds to observation-dependenl belief changes [Boyen and Koller, 1998]. To address these fundamental limitations, we introduce ES- CORT, which implements a complete belief update mechanism that combines deterministic particle evolution with model-based state estimation. The update for each particle is formulated
xt+1
xt o (xt) + Update(or+1,0))
where z represents the i-th particle at time step t, is the step size, &ig is our correlation aware regularized particle evolution function that maintains particle diversity while preserving dimensional dependencies, and Update(oo+11 a²) incorporates new evidence by shifting particles based on obser- vation likelihoods and transition dynamies. We detail the formulation of both the regularized particle evolution and inservation-action update components in the following subsections.
Having established the complete belief update mechanism combining deterministic particle evolu- tion with model-based state estimation, we now detail the correlation-awarr regularization term &ieg that addresses SVGD's limitations in high-dimensional spaces.
3.2 Correlation-Aware Regularization
Drawing inspiration from optimal transport theory, particularly the Generalized Sliced Wasserstein (GSW) distance [Kolouri et al., 2019], we construct our correlation-aware regularization term to address the limitations of standard SVGD discussed before. The key insight from GSW is that projecting high-dimensional distributions onto carefully selected lower-dimensional subspaces can efficiently capture correlation structures while remaining computationally tractable [Kolouri al., 2019]. By integrating this projection-base based approach, ESCORT addresses both critical SVGD lim- itations: kernel degeneracy mitigated through dimensionality reduction of distance calculations, while complex correlation structures are preserved by learning projection matrices aligned with di- mensional dependencies.
Our correlation-awee regularization term is formulated as:
m
Reon (0) = Engnng
Σ
w; IAT (0(x) - Ey~ (v()))
(3)
where the expectation averages over all particles in the current distribution y, and the summation ag- gregates contributions from m different projection matrices. Here, ø is the particle movement vector
5

---

field, A, E Rdx;; are projection matrices identifying key correlation directions, are importance weights. and 1-12 is squared Euclidean norm. This approach adapts the anisotropic nature of belief distributions, unlike standard SVGD's isotropic kernl..
The effectiveness of this regularization term depends identifying projection matrices that capture meaningful correlation structures relative to the target distribution. In the POMDP context, the target distribution p(x) in our SVGD formulation represents the unnormalized posterior belief distribution:
p(x)) x 0(041122,a) >T [(xxss,aa
(4)
SES
where 0(0 at) is the observation likelihood, E.es (x)(()) is the predicted belief after transition, and (s(s) is the previous timestep's belief. This corresponds to the standard POMDP belief update before normalization. Crucially, SVGD directly works with this unnormal- ized distribution the normalization constant in the standard Bayes update (s')) 1j O(o s', a) T(s'ls ss, a))(s) is handled implicitly through gradient- particle evolution, eliminating the computational overhead of explicit normalization while naturally preserving the correlation struc- tures present in the posterior.
To identify and preserve these correlation structures through our regularization term, the projec- tion matrices A, are initialized using cigenvectors derived from the difference between correlation matrices of the current particle distribution and target distribution: AC = corr(X,,) corr(Xp)). The eigenvectors corresponding to the largest eigenvalues of this correlation difference matrix pro- vide initial projection directions that highlight dimensions with significant correlation differences. These initial projections are then optimized to maximize the distance between distributions when projected:
A;- = arg max ((x)))((p.)) [1A] - (I y)13] A.
(5)
where A denotes the optimal projection matrix, represents the current particle distribution, represents the target distribution, (z. v) ~ (g.p) indicates sampling pairs with drawn from g and y drawn from p. and |A? (z y) measures the squared Euclidean norm of the projected difference between samples.
This optimization process ensures that for any belief distribution b(s) with correlation matrix Σ, the regularization term Reor( (d) with optimized projection matrices A, preserves the principal correla- tion directions of Σ under the SVGD update. This theoretical guarantee means that as we increase the number of projection matrices, we can more accurately preserve the correlation structure of complex belief distributions.
Incorporating this correlation-aware regularization into the SVGD framework yields complete regularized update:
Peg (x)
Σ 2) log p(g)( + Va, 1=1
R
n
where controls the regularization strength. In our practical implementation, both the projection matrices A; and their importance weights w; updated iteratively alongside the particle positions, ensuring that they adapt to the evolving belief distribution throughout the POMDP planning process.
3.3 Model-Based Belief Update
The Model-Based Belief Update component Update(Or+1 0z) in our belief update equation inte- grates new observations and actions into the belief distribution. While digg maintains represen- tational properties, this component leverages the POMDP model's dynamics to shift particles ac- cording to actual environment behavior. Unlike DVRL which learns transition and observation models [Igl et al., 2018], ESCORT assumes known POMDP models with state transition function T(s'|s, a) and observation function O a). This model design essential for two reasons: first, accurate likelihood gradients from known dynamics enable SVGD's deterministic evolution, correlation-aware regularization, and temporal consistency to function correctly through precise dis- placement vectors that specify how each particle should move; second, it enables fair comparison

---

with existing curticle-bssed methods by isolating our belief representation innovations from model learning errors.
Given a particle set { representing the belief at time i, an action at, and observation 0++1, the model-based update proceeds in three steps. First, each particle is propagated through the tran- sition model: _,+1 Ta a.) + 7 where T) (0, Euans) represents transition noise. Next, the observation likelihood w, 0(0411lē aa)) is computed for each predicted particle, quanti- fying how well it explains the received observation. The final update combines these components as Update(0t+1 az) = (ā!+1 x;) + Ar. Ar;(w;), where the first term represents the state change due to the transition model and the displacement term (w,(w:) iiklihood-weigghted adjustment. Here, joos = (2, w,zz)/( wy) is the observation-weightee mean and a E 10 controls correction strength. Unlike the standard Bayes filter's analytical up- date 8/ (s') 17 O(o's', a) T(s'ls a)b(s), this particle-hased formulation provides Monte Carlo approx imptoonmation without requiring analytical tractability. By incorporating observation informa- tion through local adjustments that pull particles toward high-likelihood regions rather than global resampling, this deterministic approach avoids partiele degeneracy while maintaining multi-modal coverage and preserving the correlation structures that stochastic resampling destroys.
Wi; (Josb - 24) provides
3.4 Temporal Consistency Regularization
While the correlation-aware regularization term focuses on preserving dimensional dependencies within individual belief states, ESCORT introduces an additional temporal dimension requiring con- sistency across consecutive belief updates. In POMDPs, beliefs can change dramatically between timesteps, especially when observations are noisy ambiguous, leading abrupt and potentially unrealistic belief jumps that compromise decision quality [Li et al., 2014]. These sudden belief jumps not only lead to erratic policy behavior but also destroy previously learned correlation struc- tures between state variables, causing the belief representation to lose critical dimensional depen- dencies that were carefully preserved by the correlation-awee regularization mechanism.
To address this critical challenge, we introduce a temporal consistency regularization mechanism that complements our correlation-aware regularized particle evolution. This mechanism ensures that belief updates respect the underlying temporal dynamics of the environment while still incorporating new evidence from observations. Mathematicallyy define temporal consistency the expected transport cost between consecutive belief distributions when projected onto informative subspaces. The temporal consistency constraint is formulated
-f Wi (i((A)) b.41, (Aa)Tb))dM
Lemmp =
where bi+1 represents the current belief after applying all updates (transition model, observation update, and SVGD), b, represents the previous timestep's '' belief, W1 is the I-Wasserstein distance measuring minimum transport cost between beliefs, and Ag E IR dx are learned projection matrices identifying subspaces where temporal changes are most informative.
Here, X(8) is probability distribution over the projection parameter space Together with the pro- jection matrices As. this mechanism identifies temporal patterns between timesteps-Ap provides the projection directions while x(0) assigns importance weights to each direction, quantifying which dimensions should evolve smoothly (high weight on projections revealing problematic jumps) ver- sus which can change rapidly (low weight on naturally variable dimensions). This direction-specinie regularization constrains belief updates heavily along projections that reveal unrealistic jumps while allowing natural evolution where temporal variation is expected. In contrast, the A, matrices in our correlation-awan regularization (Section 3.2) serve a fundamentally different purpose: they pre- serve spatial correlations within each timestep, capturing how dimensions relate to each other at a single moment rather than across time.
In practice, this integral is approximated as weighted sum over a finite set of optimized projection directions, with weights representing their relative importance. By regularizing the distance between consecutive belief states, we prevent unrealistically large belief jumps while still allowing the belief to adapt to new information. Detailed implementation is provided in Appendix.
7

---

3.5 Particle-Based Policy Network
After establishing our particle-based belief representation that accurately captures complex uncer- tainty structures, leverage these belief's directly for decision-making through a specialized policy network architecture- the only learned component in ESCORT. The network processes particles through two key stages: a per-particle encoder partiele independently processes each particle æ; into feature representations h; = f (2;) using multi-layer neural network, then these features are aggregated using a permutation-invariat
operation (mean pooling) followed by further processing: Dencoded = focter( 1 E h₂). This two-stage approach accounts for both individual particle states and their collective distribution properties, enabling effective reasoning about multi-modal uncer- tainties.
The policy is optimized using policy gradient methods. Given experience tuples (bz, 0t, rt, be-1). the objective function is [()) = -E(bb,,..r.) (log No (o(aelbb) (bx) RR). where Rg is the discounted return. For continuous actions, entropy regularization is applied: (()) = -Bllog Te(a (ba) R. +a+ (m (-16))
This approach directly optimizes the policy to maximize expected returns based the particle representation of beliefs.
For action selection, ESCORT supports both discrete and continuous spaces. With discrete actions, the network outputs a probability distribution <(lbb) = softmax (gaction (Dencoded)) For continuous actions, it parameterizes (Amean (inean(benssdee (eesodda)) Isd Badd(eeedee (Densodea)). Actions can be selected either deterministically (most probable ac- tion or mean) or stochastically (sampling from the output distribution).
Gaussian distribution with (n(b), log (b))
4 Experiments
We designed our experiments to evaluate ESCORT's effectiveness in two key aspects: (1) maintain- ing accurate belief representations in challenging POMDP domains and (2) approximating complex, multi-modal distributions with correlated state variables across various dimensionalities. This com- prehensive evaluation aims to validate ESCORT's ability to address the complex belief distribution identified earlier.
Baselines: We compare against state-of-the-ard methods from different POMDP belief approxi- mation categories: Particle-basee methods including SIR (Sequential Importance Resampling) and POMCPOW [Sunberg and Kochenderfer, 2018] that use stochastic resampling, leading to particle degeneracy and mode collapse in high-dimensional correlated spaces; Paramelric belief represen- tations like DVRL [lg] et al., 2018] that encode beliefs into fixed-dieensional VAE representations, sacrificing expressiveness for multi-modal distributions; and Deterministic sampling via SVGD [Liu, 2017] offering gradient-baar based particle evolution but suffering from kernel degeneracy without cor- relation preservation. Additional comparative methods are discussed in the Appendix.
Evaluation Metrics: We assess ESCORT using two metric groups: HOMDP-speciffe metrics mea- suring policy performance and distribution approximation metrics evaluating belief representation quality. For POMDP tasks, we report Average Return (position error, lower better) reflecting iivigation/localization accuracy across environments. For distribution approximation, we track Maximum Mean Discrepancy and Sliced Wasserstein Distance (statistical similarity between dis- tributions), Mode Coverage Ratio (proportion of maintained hypotheses), and Correlation Error (ac- curacy of captured dimensional relationships). Detailed metric specifications, computation methods, and interpretations are provided in the Appendix.
Experimental Setup: We evaluate ESCORT against baselines across three POMDP domains: Light-Dark Navigation, Kidnapped Robot, and Multi-Target Tracking. We additionally test on syn- thetic multi-modal distributions (ID-20D). All methods use consistent configurations: 1000 par- ticles, step size ==0.01 with adaptive decay, kernel bandwidth via median heuristic. DVRL uses latent dimensions matching state space; SIR/POMCPOW receive equivalent computational bud- gets. Experiments span 30 independent runs on Intel 19- 19-139 CPU. Our computational analysis (Appendix D) shows ESCORT scales as (d1.67) with correlation-aware term dominating high dimensions, yielding 40% correlation error improvement. Full specifications in Appendix.
8

---

Table 1: Performance comparison across POMDP environments including ablation study. Values represent average position error (with standard error) after task completion lower values indicate better performance. ESCORT variants demonstrate the contribution of each component to the over- all framework effectiveness, while the full ESCORT method consistently outperforms all baselines across environments, with increasing advantages as dimensionality grows. Detailed environment specifications, experimental protocols, and additional analyses are provided in the Appendix.
Method
Light-Dark (10D)
Kidnapped Robot (20D)
Target Tracking (20D)
ESCORT Variants
Full
0.347 = 0.03
9.063 0.51
3.665 = 0.31
NoCorr
0.381 + 0.07
10.246 + 0.35
4.213 + 0.59
NoTemp
0.321 + 0.03
10.859 +: 0.51
3.874 + 0,43
NoProj
0.359 + 0.09
9.654 + 0.32
3.90 + 0.42
Baselines
SVGD
0.379 + 0.03
10.906 0.49
4.295 + 0.22
DVRL
1.557 + 0.10
14.309 + 0.60
4.33 + 0.09
POMCPOW
2.12 = 0.24
12.023 + 0.55
4.611 + 0.09
4.1 Domains
Light-Dark Navigation: Our 10D implementation (5D position, 5D velocity) extends the tradi- tional POMDP testbed [Platt et al., 2010, Silver and Veness, 2010] with varying observation quality tied to spatial "light perciee in well-lit areas but noisy dark regions. This environment tests belief tracking under nonuniform observation noise, generating distributions ranging from pre- cise unimodal to complex multimodal patterns with inherent correlations. Performance is measured by Euclidean distance to goal (lower is better), reflecting how accurately the agent navigates despite uncertain observations.
This classical robotics challenge [Choset et al., 2005] is scaled to 20 dimensions (position, orientation, steering, sensor calibration, and feature descriptors). robot must localize within a map containing perceptually similar landmark patterns, creating multi-modal beliefs due to ambiguous observations. Performance is evaluated by position error after fixed time steps, quantifying localization accuracy despite ambiguous landmarks.
Kidnapped Robot Problem:
Multiple Target Tracking: Our 20D tracking environment [Rong Li and Jilkov, 2003] challenges an agent (4D) to track four targets (16D) under visibility zones with varying noise, occlusion re- gions, and identity confusion areas. This domain tests simultancous handling of high dimensionality, multi-modality, and correlation preservation. Performance is measured by the mean position error across all targets, indicating tracking accuracy despite occlusions and identity ambiguities. Detailed specifications for all environments are provided in the Appendix.
4.2 Results
Table demonstrates ESCORT's superior performance across POMDP domains of varying dimen- sionality. In the 10D Light-Dart environment, ESCORT achieves 8.5% improvement over SVGD and 83.6% over POMCPOW, with advantages increasing in the 20D domains achieving 16.9% and 24.7% improvements over SVGD in Kidnapped Robot and Target Tracking respectively. This con- firms that ESCORT's advantages become more pronounced as dimensionality grows. The ablation study reveals how each component contributes to ESCORT's success. Correlation-aware regulariza- tion emerges as the most critical component, with ESCORT-NoCor showing 9.8-15% performance degradation that scales with dimensionality, confirming its importance in preserving belief struc- ture. Temporal consistency exhibits domain-dependent effects: while ESCORT-NoTemp surpris- ingly outperforms the full method in Light-Dark (0.321 vs 3347)ssuggesting symmetric patterns may benefit from unrestricted mode transitions it proves essential in complex 20D environments with 19.8% degradation in Kidnapped Robot where maintaining hypothesis continuity is crucial. Projection matrices (ESCORT-NoPron provide consistent moderate benefits that scale from 3.5% in Light-Dark to 6.5% in Kidnapped Robot, demonstrating their role in efficient correlation capture.
Table reveals ESCORT's specific advantages maintaining accurate belief distributions. While SVGD performs comparably in lower dimensions, where projection regularization offers limited

---

Table 2: Comparison of belief approximation methods across different dimensional spaces. MMD and Wasserstein/Sliced Wasserstein measure distribution similarity (lower better); Correlation Error quantifies dimensional dependency accuracy (lower is better); Mode Coverage indicates suc- cessful mode representation (higher is better). Results show mean + standard error across multiple random initializations. Correlation error is not applicable for ID. Detailed environment specifica- tions, distribution characteristics, and metric calculations are provided in the Appendix.
Metric
ESCORT
SVGD
DVRL
SIR
ID Experiment
Maximum Mean Discrepancy
0,057 E 0.01
0.012 = 0.01
0.216 + 0.05
0.116 + 0.07
Wasserstein
0.549 + 0.04
0.305 = 0.02
1.967 + 0.07
0.811 = 0.22 1.0 + 0.0
Mode Coverage Ratio
1.0 + 0.0
1.0 0.0
0.867 + 0.13
2D Experiment
Maximum Mean Discrepancy
0.052 + 0.02
0.062 = 0.01
0.071 + 0.01
0.2288 0.04
Sliced Wasserstein
0.263 0.01
0.383 + 0.01
0.749 + 0.14
1.397 + 0.11
Correlation Error
0.491 0.16
0.5178 0.04
0.09 0.6594
0.8285 0.28
Mode Coverage
1.0 1 0.0
1.0 = 0.0
1.0 0.0
1.0 1 0.0
3D Experiment
Maximum Mean Discrepancy
0.002 1 0.002
0.008 = 0.009
0.361 1 0.01
0,414 1 0.06
Sliced Wasserstein
0.305 + 0.01
0,481 = 0.02
1.442 + 0.04
2.656 0.34
Correlation Error
0.761 + 0.004
0.819 = 0,02
0.882 + 0.01
1.003 = 0,01
Mode Coverage
1.0 + 0.0
1.0 = 0.0
1.0 + 0.0
1.0 = 0.0
5D Experiment
Maximum Mean Discrepancy
0.05 1 0.003
0.09 = 0.07
0.263 1: 0.004
0.394 1 0.009
Sliced Wasserstein
0.301 + 0.01
0.3987 +: 0.02
1.0838 + 0.01
2.939 + 0.26
Correlation Error
0.7224 + 0.006
0.7401 = 0.014
0.823 0.011
0,997 = 0.003
0.003
Mode Coverage
1.0 +
= 0.0
1.0 0.0
0.125 0.0
20D Experiment
Maximum Mean Discrepancy
0.005 + 0.0008
0.006 = 0.0006
0.264 0.005
0.584 = 0.008
Sliced Wasserstein
0.326 + 0.007
0.393 = 0.005
1.117 0.025
2.854 + 0.187
Correlation Error
0.556 0.03
0.589 + 0.05
0.708 + 0.03
0.640 + 0.01
Mode Coverage
1.0 + 0.0
1.0 0.0
0.58 + 0.04
0.12 + 0.04
benefits and kernel degeneracy is less severe, ESCORT consistently outperforms all methods at higher dimensionality, with up to 37.5% lower correlation error than traditional particle filters. Most striking is mode coverage high dimensions-ESCORE where SIR experiences catastrophic mode collapse (0.12 coverage) and DVRL significant degrada- tion (0.58 coverage). This validates our approach's ability to preserve dimensional dependencies while preventing particle degeneracy across all relevant modes. A comprehensive interpretation of these results, including detailed ablation studies and statistical significance analyses, is provided in the Appendix.
maintains perfect coverage 20D spaces
5 Conclusion
We presented ESCORT, particle-bases framework extending SVGD with correlation aware projec- tions and temporal consistency constraints address the challenge of representing complex beliefs in high-dimensional POMDPs. Our approach overcomes key limitations in existing methods by mitigating kernel degeneracy, maintaining expressiveness without parametric compression, and pre- venting particle collapse through deterministic evolution. Evaluations across POMDP domains and synthetic distributions demonstrate significant improvements in both belief accuracy and decision quality.
Despite these advances, ESCORT faces practical limitations. The computational overhead of cor- relation matrix computation and projection optimization increases with dimensionality. potentially reliance known transition and observa-
limiting real-time deployment. More fundamentally, tion models restricts applicability to domains where accurate models are unavailable. Future work will address these limitations through GPU-accelerated implementations for real-time performance, adaptive projection techniques that reduce computational burden, and integration with model learn- ing approaches to enable deployment in unknown environments.
10

---

6
Acknowledgements
This material is based upon work supported by the National Science Foundation (NSF) under Grant CNS-2238815 15 and by the Defense Advanced Research Projects Agency (DARPA) and US Air Force Research Lab (AFRL) under the Assured Neuro Symbolic Learning and Reasoning program. Re- sults presented in this paper were obtained using the Chameleon testbed supported by the National Science Foundation. Any opinions, findings, conclusions, or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the NSF, DARPA, or AFRL.
References
Marc G. Bellemare, Yavar Naddaf, Joel Veness, and Michael Bowling. The arcade learning envi- ronment: evaluation platform for general agents. J. Artif. Int. Res., 47(1):253-279, May 2013. ISSN 1076-9757.
Xavier Boyen and Daphne Koller. Tractable inference for complex stochastic processes. ArXiv, 0bss/300.7362, 1998. URL
.rggiarr.oooooooooooo...oe Trg/CorpusIDD : 5556701
Alex Brooks, Alexei Makarenko, Stefan Williams, and Hugh Durrant-Whyte. Parametric pomdps for planning in continuous state spaces. Robotics and Autonomous Systems, 54(11):887-897,, 2006. ISSN 0921-8890. doi: http:://o..oo/////.00...ooo.oood 00.007. URL https: //www. sciencedirect.cm http://///ccceeeaatrc.c///////// tainty in Robotics.
Planning Under Uncer-
Peng Chen and Omar Ghattas. Projected stein variational gradient descent. In Proceedings of the 34th International Conference on Neural Information Processing Systems, NIPS '20, Red Hook, NY, USA, 2020. Curran Associates Inc. ISBN 1781773829546.
Xiaoyu Chen, Yao Mu, Ping Luo, Sheng Li, and Jianyu Chen. Flow-based recurrent belief state learning for pomdps. International Conference on Machine Learning, 2022. URL https: //api. semanticscholaa org/ hes/CorppusDD. 248986064.
Howie Choset, Kevin M. Lynch, Seth Hutchinson, George Kantor, Wolfram Burgard, Lydia E. Kavraki, and Sebastian Thrun. Principles of Robot Motion: Theory, Algorithms, and Imple- mentation, chapter 13, pages 249-253. MIT Press, Cambridge, MA, 2005.
Damien Garreau, Wittawat Jitkrittum, and Motonobu Kanagawa. Large sample analysis of the median heuristic. arXiv: Statistics Theory, 2017. URL https: htt.itttee//pg..ssatticccbbbbbb
sem.aticcccnnccol.rr... org/
CorpusID::88514008
Neil J. Gordon, David Salmond, and Adrian F. M. Smith. Novel approach to nonlinearnoon-gausiaa bayesian state estimation. In IEE Proceedings F Radar and Signal Processing,, 1993.
Charles R. Harris, K. Jarrod Millman, Stéfan J. van der Walt, Ralf Gommers, Pauli Virtanen, David Cournapeau, Eric Wieser, Julian Taylor, Sebastian Berg, Nathaniel J. Smith, Robert Kern, Matti Picus, Stephan Hoyer, Marten H. van Kerkwijk, Matthew Brett, Allan Haldane, Jaime Fernández del Río, Mark Wiebe, Pearu Peterson, Pierre Warren Weckesser, Hameer Abbasi, Christoph Gohlke, and Travis E. Oliphant. Array pro- gramming with numpy. Nature, 585(7825):357-322 September 2020. ISSN 1476-4687. doi: 10.1038/s41586-022 256-04446---00224-9--2222 URL http: : /xx doi. org/10. /0/ 00003///4186--0 020-0040099000000
Kevin Sheppard, Tyler Reddy,
Matthew Hausknecht and Peter Stone. Deep recurrent q-learning for partially observable mdps. ArXiv, 8696662.
2015. URL httph:://// www.ppp.ssamaaiicchllss...e
htt.s//tww./////...aaaaaaaaooooa wrg/CorpusIDD
Maximilian Igl, Luisa Zintgraf, Tuan Anh Le, Frank Wood, and Shimon Whiteson. Deep variational reinforcement learning for POMDPs. Jennifer Dy and Andreas Krause, editors, Proceedings of the 35th Internutional Conference on Machine Learning, volume 80 of Proceedings of Machine Learning Research, pages 2117-2126. PMLR, 10-15 Jul 2018. URL mlr, ppwps//80//////p 1//00////// igl 18a. html.

---

Hanna Kurniawati. Partially observable markov decision processes and robotics. Annual Review of Control, Robotics, and Autonomous Systems, 5:253-277, 2022. ISSN 5573-5144. doi: 10. 146/ wttp://ooooooooo0220-09999-0000- Leslie Pack Kaelbling. Michael L. Littman, and Anthony R. Cassandra.
Planning and act-
ing in partially observable stochastic domains. Artificial Intelligence, 101(1):99-134 1998.
ISSN 0004-3702. doi: http://0.....0000........
http://0w0.011100000000000000000 sciencedirect.cout http://cww................cc//aa
URL https: htt.htts:://aet //www.
Diederik P. Kingma and Max Welling. Auto-encoding variational bayes. CoRR, abs/1312.6114,
2013. URL hattps://apl. htt.ttp:://p...mmm...cccbblllll wwwg/ooppg///oiiooo : :216078090.
Soheil Kolouri, Kimia Nadjahi, Umut Simsekli, Roland Badeau, and Gustavo Rohde. Generalized sliced wasserstein distances. In H. Wallach, H. Larochelle, A. Beygelzimer, F. d'Alché-Buc, E. Fox, and R. Garnett, editors, Advances in Neural Information Processing Systems, volume 32. Curran Associates, Inc., 2019.
Solomon Kullback and R. A. Leibler. On information and sufficiency. Annals of Mathe- matical Statistics, 22:79 86, 1951. URL attps://appi http://pp..smmmnccschhool.aaa Org/CorpuIID:
120349231.
Publisher Copyright: Copyright @ 2022 by Annual Reviews.
Siu Kwan Lam, Antoine Pitrou, and Stanley Seibert. Numba: a llvm-based python jit compiler. In Proceedings of the Second Workshop on the L.IVM Compiler Infrastructure HPC, LLVM '15, New York, NY, USA, 2015. Association for Computing Machinery. ISBN 978 170444000 doi: /ttp::///oooooo...///oo
10.1145//833115.228331666 URL https::tp/. //doi org/10. 1145/28331771111/ 2282228331117.28816222 2833162.
Mikko Lauri, David Hsu, and Joni Pajarinen. Partially observable markov decision processes in robotics: A survey. IEEE Transactions on Robotics, 39(1):21-40, 2023. doi: 10. 10.1109/TR0.202. 3200138.
Tiancheng Li, Shudong Sun, Tariq Pervez Sattar, and Juan Manuel Corchado. Fight sample de- generacy and impoverishment in particle filters: A review of intelligent approaches. Expert Systems with Applications, 394433994,,, 2014. ISSN 0957-4174. doi: https://doi.on URL http://wwww.ceeeceierete...t www/ssciene//rtiilll.....
pi1/5095741743300066.
Qiang Liu. Stein variational gradient descent as gradient flow. In I. Guyon, U. Von Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, editors, Advances in Neural Information Processing Systems, volume 30. Curran Associates, Inc., 2017.
Xiao Ma, Peter Karkus, David Hsu, Wee Sun Lee, and Nan Ye. Discriminative particle filter reinforcement learning for complex partial observations. ArXiv, abs/2002.09884, 2020. URL. https://api. semanticscholar wwwpoolarrrrr/rrrrrreeeee :ttp:/CorpuIII::112585e.
Marlos C. Machado, Marc G. Bellemare, Erik Talvitie, Joel Veness, Matthew Hausknecht, and Michael Bowling. Revisiting the arcade learning environment: evaluation protocols and open problems for general agents. J. Artif. Int. Res., 61(1):523-566) January 2018. ISSN 1076-9757.
Robert Platt, Russ Tedrake, Leslie Pack Kaelbling, and Tomas Lozano-Perez Belief space planning assuming maximum likelihood observations. In Robotics: Science and Systems, 2010. URL
https://app semanticschol org/ warg/Corpusliii ID: 2693863.
Martin L. Puterman. Markov Decision Processes: Discrete Stochastic Dynamic Programming. John
Wiley & Sons, 1994.
J. Rabin, G. Peyre, J. Delon, and M. Bernot. Wasserstein barycenter and its application to texture mixing. Scale Space and Variational Methods in Computer Vision, 2011. pp. 435-446.
X. Rong Li and V.P. Jilkov. Survey of maneuvering target tracking. part i. dynamic models. IEEE Transactions on Aerospace and Electronic Systems, 39(4):1333-1338 2003. doi: 10.1109/TAES. 2003. 10003.2211133 132.
12

---

Nicholas Roy and Geoffrey J Gordon. Exponential family pea for belief compression in pomdps. In Becker, S. Thrun, and K. Obermayer, editors, Advances in Neural Information Processing Systems, volume 15. MIT Press, 2002.
David Silver and Joel Veness. Monte-carlo planning in large pomdps. In J. Lafferty, C. Williams, J. Shawe-Taylor, R. Zemel, and A. Culotta, editors, Advances in Neural Information Processing Systems, volume 23. Curran Associates, Inc., 2010.
Adhiraj Somani, Nan Ye, David Hsu, and Wee Sun Lee. Despot: Online pomdp planning with regularization. In C.J. Burges, L. Bottou, M. Welling, Z. Ghahramani, and K.Q. Weinberger, editors, Advances in Neural Information Processing Systems, volume 26. Curran Associates, Inc., 2013.
Zachary Sunberg and Mykel Kochenderfer. Online algorithms for pomdps with continuous state, action, and observation spaces, 2018. URL https:// wttpa/awww...rr/aa//aaraaaa/aeee aa00 06119
Sebastian Thrun, Wolfram Burgard, and Dieter Fox. Probabilistic Robotics. MIT Press, Cambridge, MA, 2005. ISBN 0-262-20162-3.
Mark Towers, Ariel Kwiatkowski, Jordan Terry, John U. Balis, Gianluca De Cola, Tristan Deleu, Manuel Goulão, Andreas Kallinteris, Markus Krimmel, Arjun KG, Rodrigo Perez- Vicente, An- drea Pierré, Sander Schulhoff, Jun Jet Tai, Hannah Tan, and Omar G. Younis. Gymnasium: A standard interface for reinforcement learning environments, 2024. URL https://arxiv.. org/ abs/2407, 17032.
Cédric Villani. Optimal transport Old and new, volume 338, pages xxii+973. Springer- Verlag, 01 2008. doi: 10. 11000/978-3-540-71050-00
Dilin Wang, Zhe Zeng, and Qiang Liu. Structured stein variational inference for continuous graph- ical models. ArXiv, abs/1711.07116. 2017. URI. https://api. www.nticcccoolar.ooooomo orgg/
CorpusID:289224177
Dilin Wang, Zhe Zeng, and Qiang Liu. Stein variational message passing for continuous graphi- cal models. In Jennifer Dy and Andreas Krause, editors, Proceedings of the 35th International Conference Machine Learning. volume 80 of Proceedings of Machine Learning Research, pages 5219-5227, PMLR, 10-15 Jul 2018. URL https: hptp:///c/ccccc....///o mlr www.m.......e...e. press/v80/ wang181. html.
Wei Wei, Lijun Zhang, Lin Li, Huizhong Song, and Jiye Liang. Set-membership belief state-based reinforcement learning for POMDPs. In Andreas Krause, Emma Brunskill, Kyunghyun Cho, Barbara Engelhardt, Sivan Sabato, and Jonathan Scarlett, editors, Proceedings of the 40th In- ternational Conference on Machine Learning, volume 202 of Proceedings of Machine Learning Research, pages 36856-36867. PMLR, 23-29 Jul 2023.
Chenyang Wu, Guoyu Yang, Zongzhang Zhang, Yang Yu, Dong Li, Wulong Liu, and Jianye Hao. Adaptive online packing-guice guided search for pomdps. In M. Ranzato, A. Beygelzimer, Y. Dauphin, P.S. Liang, and Wortman Vaughan, editors, Advances in Neural Information Processing Sys- tems, volume 34, pages 18419-28430
Curran Associates, Inc., 2021.
Chiyuan Zhang, Samy Bengio. Moritz Hardt, Benjamin Recht, and Oriol Vinyals. Understanding deep learning requires rethinking generalization. ArXiv, 1.00533,, 2016. URL https: //api. stapiissmanttcscholll.
pttp/CooppuIID::122000.
Yunuo Zhang, Baiting Luo, Ayan Mukhopadhyay. Daniel Stojesies, Daniel Elenius, Anirban Roy, Susmit Jha, Miklos Maroti, Xenofon Koutsoukos, Gabor Karsai, and Abhishek Dubey. Shrinking pomep: A framework for real-time uav search and rescue. In 2024 International Conference on Assured Autonomy (ICAA), pages 48-57, 2024. doi: 10. 10.11099CAA44422222240000 20224440442622220000666
Yunuo Zhang, Baiting Luo, Ayan Mukhopadhyay. and Abhishek Dubey. Observation adaptation via annealed importance resampling for partially observable markov decision processes. Proceedings of the International Conference on Automated Planning and Scheduling, 35(1):306-314 Sep. 2025. doi: 10. 10..1669/cssss.o 335.1.331.......613..... URL https:/ :/oojs////ss /aj.s.aaii.............a...a.... aaai. org/ prrg/inde /n//xxndxxx//// http//dee..///I.PP///. Php//CAPS/
article/view/36t /36132.
13

---

Pengfei Zhu, Xin Li, Pascal Poupart, and Guanghui Miao. On improving deep reinforcement learn- ing for pomdps, 2018. URL https: /ttpai.//arxivoooo/ wttprvv..rr/bb////444444 1ttp://11......0 07978.
Jingwei Zhuo, Chang Liu, Jiaxin Shi, Jun Zhu, Ning Chen, and Bo Zhang. Message passing stein variational gradient descent. In International Conference on Machine Learning, 2017. URL
https://api. semanticscholar
http//oopuID:::117700:0 51877948.
K.J Aström. Optimal control of markov processes with incomplete state information. Journal
of Mathematical Analysis and Applications, 10(1):174-205, 1965. ISSN 0022-247X. doi: http://dbiiooy//o/////6600222211 cccaare/aarricee//////0227166665 URL https:/ hoop:://www... http:::/www.ccceeeeeiiiiii /ttp///icineeiireettoomm
14

---

NeurIPS Paper Checklist
The checklist is designed encourage best practices for responsible machine learning research, addressing issues of reproducibility. transparency. research ethics, and societal impact. Do not re- move the checklist: The papers not including the checklist will be desk rejected. The checklist should follow the references and follow the (optional) supplemental material. The checklist does NOT count towards the page limit.
Please read the checklist guidelines carefully for information on how to answer these questions. For each question in the checklist:
You should answer [Yes]. [No] [NA].
[NA] means either that the question is Not Applicable for that particular paper or the relevant information is Not Available.
Please provide short (1-2 sentence) justification right after your answer (even for NA).
The checklist answers are an integral part of your paper submission. They are visible to the reviewers, area chairs, senior chairs, and ethics reviewers. You will be asked to also include (after eventual revisions) with the final version of your paper, and its final version will be published with the paper.
The reviewers of your paper will be asked to use the checklist as one of the factors in their evalu- ation. While "[Yes] is generally preferable to "[No] it is perfectly acceptable to answer "[No] " provided a proper justification is given (e.g., "error bars are not reported because would be too computationally expensive" or "we were unable to find the license for the dataset general, answering "[No] or "[NA] is not grounds for rejection. While the questions are phrased in binary way, acknowledge that the true answer is often more nuanced, so please just your best judgment and write a justification to elaborate. All supporting evidence can appear either in the main paper or the supplemental material, provided in appendix. If you answer [Yes] to question, in the justification please point to the section(s) where related material for the question can be found.
used"). In
IMPORTANT, please:
Delete this instruction block, but keep the section heading "NeurIPS Paper Check- list",
Keep the checklist subsection headings, questions/answers and guidelines below.
Do not modify the questions and only use the provided macros for your answers.
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Yes, the claims in the abstract and introduction accurately reflect the paper's contributions, presenting ESCORT as framework that extends SVGD with correlation- aware projections and temporal consistency constraints to improve belief representation in POMDPs.
Justification:
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.
2. Limitations
15

---

Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
See Conclusion Section.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The au- thors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, c.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the ap- proach. For example, facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to ad- dress problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and prool's
Question: For each thcoretical result, does the paper provide the full set of asssumptions and complete (and correct) proof?
Answer: [Yes]
Justification: Yes, the paper includes theoretical results with clearly stated assumptions, and mentions that detailed formal proofs are provided in the Appendix.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theo- rems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the of the paper should be comple- mented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclu- sions of the paper (regardless of whether the code and data provided not)?
Answer: [es]
16

---

Justification:
See Experiments Section and Appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, may be necessary to either make it possible for others to replicate the model with the same dataset, provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how replicate the results, access a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate the research performed.
While NeurIPS does not require releasing code, the conference does require all sub- missions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution primarily new algorithm, the paper should make it clear how to reproduce that algorithm.
(b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is either be a way to access this model for reproducing the results or a way to re- produce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
model (e.g.. large language model), then there should
(d) We recognize that reproducibility may be tricky in some cases, in which case au- thors are welcome to describe the particular way they provide for reproducibility. the case of closed-source models, it may be that access to the model limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: Code is released.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nipe cc/
for more details.
While we encourage the release code and data, we understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).
The instructions should contain the exact command and environment needed to run reproduce the results. See the NeurIPS code and data submission guidelines (https: //nips.ccc/// wwt.cc/pppccc.ciieeeeeeeo http:/gguieee/eeeuuubaaaee
onPolicy)) for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
17

---

At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g.. data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification:
See Experiments Section and Appendix
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropri- ate information about the statistical significance of the experiments?
Question:
Answer: [Ys]]
Justification: Our experimental results represent averages across multiple random seeds and initializations to ensure statistical robustness.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should clearly stated (for example, train/test split, initialization, random drawing of some parameter, overall
run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK report I-sigma error bars, but should state it. The authors should prefer- ably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors verified.
For asymmetric distributions, the authors should careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).
error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures tables in the text.
8. Experiments compute resources
Question:
For each experiment, does the paper provide sufficient information on the com- puter ressourees (type of compute workers, memory, time of execution) needed reproduce the experiments?
Answer: [Yes]
Justification:
See Experiments Section and Appendix
Guidelines:
The answer NA means that the paper does not include experiments.
18

---

The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't t make it into the paper).
9. Code of ethies
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips. /ttps////oarrrr..coooaoooo//atta hctpppublccc/tiiicciiiieeeee.e..
Answer: [Yes]
Justification: This work fully adheres to all principles and guidelines established in the NeurIPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g.. if there is a special consid- eration due to laws or regulations their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
This work presents foundational algorithmic research on mathematical meth- ods for belief representation POMDPs without direct societal applications or deployment concerns.
Justification:
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact spe- cific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitiga- tion strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML..
11. Safeguards
Question: Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
19

---

Answer: [NA]
Justification: [NA]
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by re- quiring that users adhere usage guidelines restrictions access the model or implementing safery filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [NA]
Justification: [NA]
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URL.
The name the license (e.g., CC-BY 4.0) should be included for each asset. For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of in the pack- age should be provided. For popular datasets, paperswi paperswithcode. thcode com/ datasets has
curated licenses for some datasets. Their licensing guide can help determine the li- cense of dataset.
For existing datasets that re- re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's s creators.
13. New assets
Question: Are new assets introduced in the paper well documented and the documenta- tion provided alongside the assets?
Answer: [NA]
Justification: [NA]
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details the dataset/code/mmodll as part their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
20

---

Question: For crowdsoureing experiments and research with human subjects, does the pa- per include the full text instructions given participants and screenshots, applicable, well as details about compensation (if any)?
Answer: [NA]
Justification: [NA]
Guidelines:
The answer NA means that the paper does not involve crowdsoureing research with human subjects.
Including this information in the supplemental material is fine, but if the main contri- bution of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, cura- tion, or other labor should be paid at least the minimum wage the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects. and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country or institution) were obtained?
Question:
Answer: [NA]
Justification: [NA]
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equiva- lent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Answer: [NA]
Justification: [NA]
Guidelines:
The answer NA means that the method development in this research does not involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy (https:/ //tpas://///ris.../.e cc/Conferencces//200 /LLM) for what should or should be described.
21

---

Algorithm 1: ESCORT
Input:
41:
x. + 22 with j-th dimension + Ej
number of particles
2 2, with j-th dimension Ej
d: state dimensionality
43:
P. (= o), 10 .0), 15)
h: kernel bandwidth
44:
P; max(O o), 15)
et step size for updates
scores(i, + log(p logíp, 26
45.
Ag: correlation regularization weight
46:
end for
d; temporal consistency weight
47:
end for
number of projection matrices
48: return clip(scores, -100, 100), log-probs
transition model
19:
end function
O: observation model
Initialize particles {-]) initial belief
51: function OPTPROJ((2)) W
1im1)
Initialize projection matrices {A with random
Σ Covariance( [=
orthonormal matrices
53: Erev +- Covariance (2) (x] 1i=1
Initialize projections A = {(Aj, 15)) AS Raxd
AC +- Correlation(I)
(orrelation(Epee))
orthonormal, Wy
55: Extract eigenvectors [v:)x-1 and eigenvalues
{A=) of AC
5: while not terminated do
57:
Initialize {A,}m- with top eigenvectors
a; + SSELECTACTION((: policy)
random orthogonal vectors
7: Execute ar, receive observation 0++1
59:
for iteration = 1 to optimization.step do
{a, {x."ev fi=1 {x{)==
Compute projected distributions for each direction
T A (O, Earans)
61:
10:
[F] 1i-1 [F + (T(z), a.) mle
Update {A, maximize Wasserstein
11:
score +- COMPUTESCONE (5) -inn-in
Jt- .41.0) f J- [x (2
63:
distance between projections
12:
[Aj. "M < OPTPROS([EE) (z)
64:
Orthonormalize (4,]m
13:
X < REGSVGD( score,
65:
Update weights {w, based on contribution to distance
14:
if and A, > 0 then
66:
15:
X TEMP X A,
67:
end for
16:
end
68:
[w,]7= <- [wy/ Er 1 wxx)
3
17:
I !61+1
69:
return {A,))= [
18:
end while
70:
end function
19:
71:
20:
function SEleCTAcTION (-; [- policy)
72:
function TEMPCONS(X Xpree ( {A, a
21:
[fu]== + {policy, encoder((;) |=
for
22:
(heate 1 policy.attentioul [h))
74:
update, <- 0₂
23:
b - AM
for j
do
b (e)eeee))
76:
Pj <- (4 xx]==
25:
discrete actions then
77:
qj < {4] ut prev ]r_
Taalb)- softmax(poliyyaatioon (hedd(b)
78:
Sort P; and Øj to find matching indices
27: else
79:
Find Amatch s.t. sorted index x; in Py
p(b), (b) < volicy.mean.head|/ policy.std.head()
80:
matches with xeeey in 9
29:
81:
update, +- update,+
30:
end if
82:
A wy A, prey - x4)
31: return (argmax or sampled from policy distri-
83:
end for
bution)
84:
update,
32:
end function
85:
end for
33:
86: return {r.)1
34:
function COMPUTESCORE((22)
87: end function
35:
Initialize scores E Rad log.probs R
36:
i to n
37:
P.
10
38:
log propprossj]] + log(p:)
39:
for to d do
40:
Ey <- ma.maax0000000 10 (x:(i])

---

A List of Abbreviations
This section provides a comprehensive list of abbreviations and acronyms used throughout this pa- per. The abbreviations are organized by category for easy reference, with their corresponding full forms to assist readers in understanding the technical terminology.
Mathematical Foundations and Concepts
Abbreviation
Full Form
GSW Distance
Generalized Sliced Wasserstein Distance [Kolouri et al., 2019]
KL Divergence
Kullhack-Leibles Divergence [Kullback and Leibler, 1951]
MDP
Markov Decision Process [Puterman, 1994]
OT
Optimal Transport [Villani, 2008]
POMDP
Partially Observable Markov Decision Process [Áström, 1965]
RBF
Radial Basis Function [Garreau et al., 2017]
SWD
Sliced Wasserstein Distance [Rabin et al., 2011]
Approaches and Algorithms
Abbreviation
Full Form
AdaOPS
Adaptive Online brcking-guidees Search [Wu al., 2021]
ADRQN
Action-specific Deep Recurrent Q-Network [Zhu et al., 2018]
ARDESPOT
Anytime Regularized DEterminized Sparse Partially Observable Tree [Somani al., 2013]
DRQN
Deep Recurrent Q-Network [Hausknecht and Stone, 2015]
DVRL
Deep Variational Reinforcement Learning [Igl et al., 2018]
ESCORT
Efficient Stein-variational and sliced Consistency--ptimizzd Representation for Temporal beliefs
MP-SVGD
Message Passing Stein Variational Gradient Descent [Zhuo et al., 2017]
POMCP
Partially Observable Monte Carlo Planning [Silver and Veness, 2010]
POMCPOW
Partially Observable Monte Carlo Planning with Observation Widening [Sunberg and Kochenderfer, 2018]
SIR
Sequential Importance Resampling [Gordon et al., 1993]
SVGD
Stein Variational Gradient Descent [Liu, 2017]
VAE
Variational Autoencoder [Kingma and Welling, 2013]
B
Theoretical Foundations
We begin by stating the core assumptions motivating our approach:
Assumption (Belief Distribution Properties) POMDP belief distributions realistic environ-
ments exhibit:
(AI.1) High dimensionality (state space dimension d >> 1)
(A1.2) Multi-modality (multiple distinct hypotheses with non-zero probability mass)
(A1.3) Complex correlation structures between state dimensions
Remark This assumption characterizes the fundamental challenges in practical POMDP appli- cations that motivate our approach. High dimensionality reflects the complexity of real-world state spaces (e.g., robotic configuration, environmental features); multi-modality emerges naturally from ambiguous observations creating multiple plausible hypotheses; and importantly, we assume that the complex shape of multi-modal, high-dimensional belief distributions can be effectively approx- imated by capturing the dependencies and causal relationships between state variables. This last property is critical to our approach-while the raw dimensionality might be high, the intrinsic struc- ture of realistic belief distributions is governed by these inter-dimensional dependencies, creating dimer-nimensional manifold which belief evolution primarily occurs. While simplified POMDPs may lack some of these properties, our focus is on complex domains where traditional methods struggle precisely because these properties co-occur.
23

---

Assumption 2 (Regularity Conditions) The true belief distribution p(s) has bounded derivatives up to second order. ensuring the score function V Voog is Lipschitz continuous with constant L.
Remark 2 This standard mathematical ussumption ensures well-behaved gradients during particle evolution, providing necessary conditions for convergence guarantees. The Lipschitz condition en- ables to establish convergence rates for ESCORT's deterministic updates and ensures stability by preventing arbitrarily large update steps. Given this Lipschitz score function combined with positive definite kernel properties (Assumption 3), ESCORT inherits SVGD's convergence guarantees with modifications accounting for our regularization. As particle count + Do and step size ε + 0 fol- lowing Σ(=1 & = oo and Σi= 57 < xc, the empirical distribution converges to the true belief p(s) in Wasserstein distance: Wa E" designed to vanish as convergence is achieved, ensuring they guide but don't 't prevent convergence while maintaining correlation structure throughout the optimization process.
62.
O. Our regularization terms Reorr and Liemp are
(Kernel Properties) The kernel function k(x,)) is positive definite, symmetric, and has bounded derivatives up to second order.
Assumption 3
These kernel properties are essential for ESCORT's theoretical guarantees and are sat- isfied by commonly used kernels such as the RBF kernel. The positive definiteness ensures the kernel induces a valid reproducing kernel Hilbert space in which gradient flow operates; symmetry main- tains bulanced particle interactions; and bounded derivatives prevent numerical instabilities in high dimensions. While our implementation uses the RBF kernel, any kernel satisfying these properties can be substituted, allowing domain- -omain-specific adaptation when prior knowledge suggests alternative similarity measures.
Remark 3
Assumption (Projection Representation) There exists finite set of projection matrices {A such that for any belief distribution with correlation matrix Dn and any E > 0, there exists weights {
mm where:
(A4.1) Ex u.A,AT I| < & for m sufficiently large
(A4.2) The projection matrices identify principal correlation directions
Remark 4 This assumption formalizes the capacity of our projection-based approach to capture correlation structures with arbitrary precision. draws on results from matrix approximation the- ory, specifically that any positive semi-definite matrix can be approximated by a weighted sum of rank-one projections. In practice, with sufficiently many projection matrices, ESCORT can pre- serve complex correlation patterns between state dimensions. Under this assumption, for any be- lief distribution b(s) with correlation matrix Ep, our learned projection matrices { { satisfy IE - w;A,AA]I || E for sufficiently large m. This guarantee ensures that as we increase the number of projections, ESCORT captures the complete correlation structure of complex belief distributions. The correlation- aaare regularization term Rew = Eeznng EETh W; . . (1(x)) Ey-gló(y)]) enforces this preservation during particle updates, preventing the loss of dimensional dependencies that plagues standard SVGD in high- dine-niiinsi spaces.
Assumption 5 (Bounded Transition Dynamics) For any state s, action a, and the resulting state s', the POMDP transition dynamics satisfy ||s' ss| s|l sents the maximum possible state change in a single timestep.
6max with probability 1, where 8max repre-
This assumption reflects physical constraints present in most real-world systems where siate transitions occur continuously rather than through arbitrarily jargejumps. poral consistency constraints by establishing a natural scale for reasonable belief updates between timesteps. Most physical systems, robotics applications, and natural processes exhibit bounded changes per timestep, allowing ESCORT to distinguish between realistic belief evolution and erro- neous jumps caused by particle degeneracy
Remark 5
jumps. It enables our tem-
observation ambiguity. Under this assumption, the temporal regularization Liemp- = So W ((Ag) br41, (Aa) (0) ensures that consecutive belief updates remain bounded: W, (be+1, be) S C- (8max + aas)) where C depends on the Lipschitz con- stunts of transition and observation models, and Jobe captures observation noise. This guarantee prevents catastrophic belief jumps that occur in particle filters during resampling while allowing necessary adaptation to new evidence.
24

---

C Practical Implementation of Correlation-Aware Deterministic Belief Update Mechanism
The practical implementation of ESCORT's belief update mechanism combines deterministic par- ticle evolution through modified SVGD with model-based state estimation maintain accurate belief representations in high-dimensional POMDPs. At each timestep, the belief update pro- ceeds through three key stages: first, particles are propagated through the transition model a (( where тạ (0, Etran) represents transition noise; second, the observation likelihood wi O(0 [ .a) is computed for each predicted particle; and finally, the correlation-aware SVGD update is applied. The complete update takes the form 2 xه+1 + + (#Ş+*). where Pig incorporates both the standard SVGD forces and our correlation -ware regularization. To pre- vent particle degeneracy in dighddimensional spaces, we add small Gaussian noise with variance noise 0.01 x (1 + 0.1d) that scales with the state dimensionality d.
The score function V log p(z), which drives particles toward high-probability regions, cannot be computed analytically in most POMDP settings. Our implementation employs adaptive finite dif- ferences numerically approximate these gradients with enhanced stability. For each particle æ; and dimension j. we compute the score |V log logp(z;)],
log 0(z, te:e;,0) -lg (591) where
2c,
ey is the j-th unit vector and Ej = max(10-9 10-1 (ar,y() is adaptive step size that scales with the magnitude of the state component. To handle numerical instabilities, we enforce a min- imum likelihood threshold of 10 15 before taking logarithms and clip the resulting scores to the range [-100, 100j. For extremely large state differences where | &c > 10 we employ the log- sum-exp trick: ||= cxp(2m) Σj exp(2(log |z,|| m)) where m max, log [zyl. preventing overflow in distance computations.
The correlation-awe aware regularization term modifies the standard SVGD update by incorporating learned projection matrices that capture dimensional dependencies. In practice, the regularized update for particle becomes Pig
logp(x,) + V2,k(Iz,, Avver Σx= Aq.AA (x; ž), where E = E 2y is the particle mean and {Ak, ww are the projection matrices and weights. The kernel bandwidth adapts to the data scale as h = ho median([|=;; (I|=;; xz|l) ))) vd/2, where ho is the base bandwidth, accounting for the curse of dimensionality. To enhance multi-modal coverage, we scale the repulsive forces by a factor of (1+0.1d) in high- digh-dimensional spaces, preventing mode collapse when kernel values become nearly uniform. The projection matrices are initialized using eigenvectors of the correlation difference ma- trix AC corr(X,) corr(X,) between current and target distributions, then optimized through gradient ascent on the projected Wasserstein distance to maximize sensitivity to correlation changes.
1 n (==,k((z,
D Practical Implementation of Temporal Consistency Regularization
Temporal consistency regularization in ESCORT prevents unrealistic belief jumps between timesteps using an efficient approximation the Generalized Sliced Wasserstein Distance (GSWD), as shown in Figure 2. We discretize Equation 6 as GSWD(p, 9) = Σ W; w,;Wi ((A(0) (A0,) q), where 8; are projection directions and w; are importance weights. These projections capture the most informative dimensions along which consecutive belief distributions differ.
Our implementation optimizes these projections using finite difference gradient estimation. For each direction 0, we compute the gradient by perturbing each dimension by (typically 10-9) and measuring the change in the projected ID Wasserstein distance. These gradients drive a momentum- based update: U++1 0.9v, + 1) W For better performance in correlated spaces like the Light- Dark environment, we weight gradients by cigenvalues of the covariance matrix, helping capture the coupled dynamics between position and velocity dimensions.
The ID Wasserstein distance along each projection is computed efficiently by sorting projected particles: W1 1
n Σ [ ysootedj.. This sorting also produces a matching between par- ticles across timesteps, creating the transport plan seen in Figure 2. Unlike finite difference meth- ods, the actual regularization term computes forces directly from this optimal transport matching: regi- = Atemp Σ = 1" (xa, ( (i)pre I;),, where a is the index of the particle matched to under projection 8y.
25

---

Ltep GSWD (T 00.00 (be); b(+1)
Invalid Jump
Belief at
Valid Belief at t + 1
Projection matrices Ao Wasserstein distance W1
Belief Particles
Belief Distributions
Invalid Update
Constrained Update
Figure 2: The figure illustrates how GSWD regularization prevents invalid belief jumps between consecutive timesteps. The left shows the belief state at time t, while the crossed-out distribution (top right) represents invalid belief update that would occur without regularization. The bottom right shows the valid belief at t+1 after applying temporal consistency constraints.
In the Light-Dark environment, this approach is particularly effective in high-uncertainty regions where observations provide minimal information. When multiple states have similar observa- tion likelihoods, temporal consistency rejects physically implausible belief jumps (as illustrated by the crossed-out path Figure 2) and enforces coherent belief evolution. Our implementation includes safeguards for numerical stability: normalized projections, clipped regularization terms (clip(reg,, 100, 10)), and adaptive bandwidth scaling (h median[[z; æı,) 0.5 / vd/2). For robustness, we implement fallback mechanism that uses nearest- neighbo matching when numer- ical issues arise. In practice, this regularization reduced position error by 37% in the Light-Dait Dark environment by preventing particle degeneracy in high-noise regions.
E
Computational Cost Analysis
We conducted detailed FLOPs (Floating Point Operations) analysis of ESCORT to address com- putational scalability across dimensions. Table 3 presents the breakdown of computational costs per belief update. Note that all experimental results reported in the main paper use equivalent computa- tional budgets-fixinn the same wall-clock time (100ms) per decision for all methods to ensure fair comparison.
Our empirical analysis shows ESCORT scales as O(d which aligns with theoretical expecta- tions. The algorithm's complexity is dominated by kernel computation O(n²d) for pairwise dis- tances and gradients, SVGD forces O(n²d) for attractive/repulsive particle interactions, GSWD regularization Omnd?² for m projections in d dimensions, and temporal consistency O(n²) inde- pendent of dimension.
26

---

Table 3: ESCORT Computational Cost Breakdown (GFLOPs per belief update)
Dimension
GFLOPs
Kernel %
SVGD %
GSWD
Temp %
10
6.72
62.5
29.8
6.3
1.5
20
15.60
52.6
25.7
21.1
0.6
50
81.06
24.9
12.3
62.6
0.1
100
262.31
15.3
7.6
77.0
0.0
200
926.32
8.7
4.3
87.0
0.0
As dimensionality increases, the correlation-aware GSWD term (0(md)) becomes the dominant cost, accounting for 87% of computation at 200D. However, this computational investment yields substantial returns shown in previous results, ESCORT achieves over 40% improvement in cor- relation error compared to standard SVGD at high dimensions, with the performance gap widening dimensionality increases.
Our implementation already incorporates several optimizations including Numba JIT compila- tion [Lam et al., 2015] for kernel computations providing 5-10x speedup, vectorized operations in GSWD using batched matrix multiplications [Harris al., 2020), adaptive subsampling for distance computations in high dimensions, and caching of score functions and transition models in belief' updates.
To further improve scalability, we propose GPU acceleration for the (mmd²) projection operations, sparse projection matrices that exploit correlation structure reducing (dd) to O(d k) for k-sparse projections, and hierarchical approximations that group correlated dimensions. These optimizations could reduce the effective complexity closer (dd while preserving the critical correlation- aware benefits that make ESCORT superior in high-dimensional POMDPs.
F Hyperparameters
This section details the hyperparameters used in our experimental evaluation. Table 4 summarizes the key hyperparameters used for each method across the three evaluation domains. The experimen- tal configuration maintains a consistent of core algorithm parameters across all domains while strategically adjusting specific parameters to accommodate domain complexity. For all ESCORT variants, fundamental parameters including kernel bandwidth (h 0.1), step size and regularization strengths (Aeor 0.1, Amemp = 0.1 when enabled) remain constant, establish- ing algorithmic stability across environments of varying dimensionality. The primary adaptation to increased complexity is seen in the state dimension scaling from 10D in Light-Dark to 20D in both Kidnapped Robot and Target Tracking domains, with corresponding adjustments to projection counts pro) 10 for Kidnapped Robot versus пoo 5 for others).
0.01),
ESCORT's architectural design naturally constrains hyperparameter choices based on theoretical principles. The correlation-aware regularization weight (Acor) and temporal consistency weight (Aemm)) represent the relative importance of preserving dimensional dependencies versus temporal stability against the base SVGD forces. We initialized both at À = 0.1 as starting point, though our analysis reveals optimal values are domain -eeenddedeeedeent-t particularly Atemp. which varies with environment dynamics. The kernel bandwidth follows the median heuristic, principled parameter- free approach standard kernel methods. Hyperparameter selection followed established practices: regularization weights initialized at X = 0.1 as baseline values, step size (8 0.01) follows SVGD convergence theory, kernel bandwidth uses median heuristic, and projection counts scale with state dimensionality. This principled approach ensures reproducibility while minimizing domain-speciff tuning.
Notable domain -specffc adjustments appear in the comparative methods, revealing insights into their computational strategies. DVRL's belief dimension scales proportionally with state complexity, using half the state dimension in Light-Dare (5) and Kidnapped Robot (10), but matching the full dimension in Target Tracking (20), suggesting a more expressive belief representation is needed for the complex multi-target environment. Similarly, POMCPOW employs deeper search depths (dmux = 5) and more simulations (n 100) with higher exploration constants (Cexpl = 50.0) in the challenging Kidnapped Robot environment compared to the other domains (d 3, 7l 50,
27

---

Table 4: Hyperparameters for different algorithms across domains
Method
Parameter
Light-Dart 10D
Kidnapped Robot
Target Tracking
Apaaticles
100
100
100
dstate
10
20
20
h
0.1
0.1
ESCORT
0.01
0.01
0.01
Acor
0.1
0.1
0.1
Aremp
0.1
0.1
0.1
Pproj
5
Uparticles
100
100
100
dstate
10
20
20
h
0.1
0.1
0.1
EsCORT-NoCorr
0.01
0.01
0.01
Acorr
0.0
0.0
0.0
Acemp
0.1
0.1
0.1
Aproj
5
10
Rparticles
100
100
100
dstate
20
h
0.1
0.1
0.1
ESCORT-NoTemp
0.01
0.01
0.01
Acorr
0.1
0.1
0.1
Aemmp
0.0
0.0
0.0
Mproj
5
10
Rearticles
100
100
100
dstate
10
20
20
h
0.1
0.1
0.1
0.01
0.01
0.01
ESCORT-NoProj
Acoir
0.1
0.1
0.1
Aremp
0.1
0.1
0.1
Teroj
5
10
5
projection_ment method
'random'
'random'
'random'
Npanticles
100
100
100
dataee
10
20
20
h
0.1
0.1
0.1
SVGD
0.01
0.01
0.01
adaptive bandwidth
True
True
True
enhanced repulssinn
True
True
True
dstatee
10
20
20
dueliet
5
10
20
DVRL
Pppaticles
100
100
100
dp
64
64
64
Repaticllss
100
100
100
dnax
3
5
3
7sgim
50
100
50
Gexpl
10.0
50.0
10.0
POMCPOW
Olaction
0.5
0.5
0.5
Ruction
4.0
4.0
4.0
Clobs
0.5
0.5
0.5
Kobs
4.0
4.0
4.0
0,95
0.95
0.95
28

---

Gexpl = 10.0), indicating increased computational budget allocation for the perceptually ambiguous scenario with multiple similar landmarks. These systematic hyperparameter adjustments reflect deliberate balance between maintaining algorithmic consistency and adapting to domain-specific challenges.
G Analysis of Baseline Methods
This section provides a detailed analysis of existing belief approximation methods and their fun- damental limitations in addressing the challenges of high-dimensional, multi-modal belief distribu- tions with complex correlation structures in POMDPs. We examine three categories of approaches: deterministic variational methods (SVGD), particle-based sampling methods (SIR filters [Gordon et al., 1993] and their POMDP extensions like POMCPOW [Sunberg and Kochenderfer, 2018], POMCP [Silver and Veness, 2010), ARDESPOT [Somani et al., 2013]), and parametric neural rep- resentations (DVRL [Igl et al., 2018), DRQN [Hausknecht and Stone, 2015), ADRQN [Zhu et al., 2018]. While each category offers unique advantages- SVGD's deterministic particle evolution, particle filters' theoretical convergence guarantees, and neural methods' computational efficiency- we demonstrate how their core assumptions and algorithmic choices prevent them from simultanc- ously maintaining multi-modal coverage, preserving dimensional correlations, and scaling to high- dimensional belief spaces. Understanding these limitations not only motivates the design choices in ESCORT but also clarifies why a fundamentally new approach combining correlation-aww projec- tions with temporal consistency constraints is necessary for accurate belief representation in complex POMDPs.
Stein Variational Gradient Descent (SVGD) [Liu, 2017] represents a significant advancement in Bayesian inference by providing a deterministic particle-based approach that bridges the gap be- tween variational inference and sampling methods. The key insight of SVGD lies in its ele- gant formulation of particle updates through functional gradient descent in reproducing ker- nel Hilbert space (RKHS), where particles evolve according to x
xf + cØ (x!) with (x) = 1 [ x) logp(x;) + Vx, kxx,, This update mechanism ingeniously balances two forces: an attractive term k(x,, x)V x, log p(x)) that drives particles toward high- probability regions, and repulsive term Vx, kxx,, x) that maintains particle diversity. While SVGD successfully addresses several limitations of traditional MCMC methods particularly avoiding stochastic resampling and providing deterministic updates-ii it faces critical challenges when ap- plied to high-dimensional POMDPs with complex belief distributions. The standard RBF kernel k(x,'' exp(-kll> ||x x2)l²) suffers from kernel degeneracy as dimensionality increases, caus- ing kernel values to become nearly uniform and weakening the essential repulsive forces needed to prevent mode collapse. Moreover, SVGD's isotropic kernel treats all dimensions uniformly, failing to capture the anisotropic nature of belief distributions where correlation strengths vary significantly across dimension pairs-a critical limitation when representing beliefs about interdependent state variables in realistic POMDP scenarios.
Deep Variational Reinforcement Learning (DVRL) Igl et al. [2018] represents a sophisticated in- tegration of variational inference with deep reinforcement learning, introducing an important in- ductive bias that enables agents to learn generative models of the environment and perform in- ference within those models. The key innovation of DVRL. lies in its particle-based belief rep- resentation b. = {
=t. w J where each particle consists a deterministic RNN hid- den state I,f a stochastic latent variable =t, and an importance weight wt. This approach ele- gantly combines the expressiveness of neural networks with the flexibility of particle filters, using a variational autoencoder framework to jointly optimize an evidence lower bound (ELBO) along- side the reinforcement learning objective: _.VVR. L + AH + Cv + AECLLOO While DVRL successfully demonstrates that learning internal generative models improves performance pure ENN-based approaches, it faces limitations when confronted with the specific chal- lenges of high- high-dimensional belief spaces with complex correlation structures. The particle updates in DVRL follow standard importance sampling with resampling, where weights are computed as w
po(=f ,.- )po (o "t 1 .0)1) treating all state dimensions uniformly without mecha-
( " a1000)
nisms to capture or preserve dimensional dependencies. Furthermore, while the resampling step helps maintain particle diversity, it can disrupt correlation structures between state variables, and the
29

---

lack of explicit temporal consistency constraints allows for potentially unrealistic belief transitions between timesteps- limitations that become particularly pronounced in environments where state variables exhibit strong interdependencies and beliefs must evolve smoothly over time.
POMCPOW (Partially Ohservable Monte Carlo Planning with Observation Widening) [Sunberg and Kochenderfer, 2018] extends POMCP to handle continuous observation spaces through dou- ble progressive widening (DPW) and weighted particle filtering. While standard POMCP suffers from belief collapse to single particles in continuous spaces-leading to QMDP-like policies that ignore information valleeaallcCe
maintains weighted particle collections where each parti- cle's 's weight is proportional to the observation likelihood Z a, s'). This weighting mechanism prevents complete belief degeneracy and enables some information gathering behavior. However, POMCPOW still faces critical limitations in representing complex belief distributions. First, it lacks explicit mechanisms to model correlation structures between state dimensions, treating particles in- dependently without capturing the interdependencies crucial for realistic POMDPs. Second, the approach provides no temporal consistency constraints, allowing abrupt belief transitions that can destroy previously learned structures. Third, despite the weighting scheme, POMCPOW remains vulnerable to particle degeneracy in high-dimensional spaces where the effective sample size di- minishes rapidly. These limitations mean that while POMCPOW improves upon basic POMCP for continuous observations, cannot adequately represent the high-dimensional, multi-modal, corre- lated belief distributions that characterize complex POMDP domains.
coommation-athhrinng
Beyond the methods discussed above, several other approaches have been proposed for belief ap- proximation in POMDPs. ARDESPOT (Anytime Regularized DEterminized Sparse Partially Ob- servable Tree) [Somani et al., 2013] uses determinized sparse sampling with regularization to scale POMCP to larger problems but still relies on unweighted particles that cannot capture complex cor- relation structures. AdaOPS (Adaptive Online Packing-guidee Search) [Wu et al., 2021] improves upon POMCP by adaptively selecting action and observation branches using packing constraints, though remains limited by particle degeneracy in high-dimensional continuous spaces. DRQN (Deep Recurrent Q-Learning) [Hauskneeht and Stone, 2015] uses recurrent neural networks to com- press observation histories into dieeddimeniional vectors but struggles to maintain distinct hypothe- for
beliefs. ADRQN [Zhu al., 2018] augments DRQN with auxiliary tasks and attention mechanisms to better capture uncertainty, yet the fixed-size representation still cannot adapt to varying belief complexity. FOR (Flow-based Recurrent Belief State Learning) [Chen al., 2022] employs normalizing flows to learn more expressive belief representations but requires exten- sive offline training and may not generalize well to novel scenarios. SBRL (Set-membership Belief State-based Reinforcement Learning) [Wei et al., 2023] maintains set-based belief representations that can capture some uncertainty structure but lacks the flexibility model arbitrary multi-modal distributions with complex correlations. While each of these methods addresses specific aspects of belief representation, none provides comprehensive solution for maintaining accurate, multi-modaa beliefs with intricate correlation structures in high- nigh-dimensional continuous POMDPs.
H Domains
H.1 Light Dark 10D
The Light-Dark Navigation environment extends the classical POMDP testbed to nigh-dimmnsional setting that exhibits the fundamental challenges addressed by ESCORT. The environment operates in a 10-dimensional continuous state space S C RR10, decomposed into position coordinates x = (11,,22, 22,, #4, .5) [0, 10 and velocity components v (v1, U2, U3, 04, v5) E R The agent can apply forces through 10 force in dimension i and discrete actions A= 10, , 9}. where action a = 2i applics positive 2i + 1 applies negative force. The agent receives noisy 5-dimensional
position observations o E RR5 with noise variance (x) = - (1 L(x) + o - where Lxx) e [0, represents the light intensity at position buse 0.5, and min 0.01. As illustrated in Figure 3, the four 2D projections reveal the complex spatial structure, with light regions (blue-to- white gradient) providing precise observations and dark regions inducing high uncertainty.
The environment contains seven strategically placed light regions that create complex belief land- scapes and perceptual aliasing. The Primary Corridor (centered at (5, 0, O, 0) with radius 2.0 and intensity 0.9) and Mirror Corridor (centered at (0, 5, 0, 0) with identical parameters) create sym- metric patterns that induce multi-modal beliefs, as demonstrated by the orange belief particles in
30

---

Dimensions 1-2: Primary Mirror Cerridors (Perceptual Allasing)
Dimersions 3-4i Z-shaped Navigation Path
(Sequential Waypoints)
Goet
Start Reckon
The
4-
Position Disiersion
Position Dimersion
Queeerr More Mere)
Ouenee More
Light
02
04
10
10
Position Dimension
Position Dimension
Dimersions
Beacon
Dimensiono
Examplc (Complex Dependencies)
(Start Goal)
10
00
Position Dimencion
Lights Inbermitty Katwer Pire Metue))
Position Dimension
Position Dimension
Position Dimension
Figure 3: Light Dark LOD POMDP Environment. Four 2D projections of the 10D state space (5D position 5D velocity) showing: (Top-left) Symmetric corridors creating perceptual aliasing with multi-modal belief particles (orange) splitting in dark regions; (Top-right) Z-shaped navigation path with sequential waypoints; (Bottom-left) Start-to-gox to-al trajectory from dark region (red cir- cle) to goal beacon (green star); (Bottom-right) Cross-dimensional correlations. Light intensity maps Solue-to-whire encode observation noise o (x(x) 0.5² (1 L(x)) 0.01 with con- tour lines marking light boundaries. The environment challenges belief approximation through high dimensionality, multi-modality from symmetric patterns, and strong position-velocity correlations
(p = 0.8).
Figure 3 (top-left). Three connected Z-path segments in dimensions 3-4 provide sequential way- points: Start (0, 2.5, 2.5, 0), Middle at 00, 5.0, 0). and End at (0, 7.5, 7.5, 0), each with radius 1.5 and intensity 0.8. A Goal Beacon near the target at (0, 0, 0. 0, 7.0) provides high- (3.33, 3.33, 3.33, 0, 0) with
precision observations (intensity 1.0), while an Ambiguous Region low intensity (0.4) further complicates belief maintenance. The light intensity function is computed as I(x) = max (0.05, max, ([, (1- intensity, radius, and distance to the j-th light region center.
Txd, < r;J)), where Ij. ry and d (x) are
The state evolution incorporates complex correlation structures that challenge standard belief ap- proximation methods. The transition model follows X++1 = + At v, + 7x and Vi+1 = v, + f, YV: + 7y. where f, is the applied force (magnitude 0.1), ? 0.1 the damping coefficient, At = 0.1 is the time step, and [mx: ny. NO, E represents correlated pro- cess noise. The correlation matrix Ecor R T0x encodes strong position- vvelocity coupling (p(x,, u;) = 0.8), adjacent position correlations (p(2;, = 0.5), adjacent velocity correlations (p(v,, 11+1) = 0.6), dinensiime dependencies (p(I1, I33 p(x2, x44) ) = 0.4), and velocity interactions (p(v1, 02) = 0.7, p(v1,))) 0.5). Additionally, in very dark regions (L(x) < 0.1),
observation model introduces dimensional identity confusion with probability 0.2, randomly
swapping dimensions 1-2 or 3-4 to create further observational ambiguity.
31

---

Performance evaluation focuses on navigation from a random starting position in the dark region [0. 2 to the goal position (8, 8, 8, 8). with success defined as reaching within 0.5 units Euclidean distance. The primary metric is position error || Xmuel/2, where x represents the belief mean estimate and Xiue is the true agent position. The reward function combines navigation progress with a step penalty: Tt -0.1 ||x, Xgoall2 0.1, encouraging both goal-directed behavior and efficient planning. This environment provides a rigorous testbed for evaluating ESCORT's ability to maintain accurate, multi-modal belief' representations in high-dimensional spaces with complex correlation structures, directly addressing the fundamental challenges that motivate our approach.
H.2 Kidnapped Robot Problem
Figure 4: Kidnapped Robot Problem visualization showing domain. The robot icon (blue eyes with red antenna) indicates the true robot position and orientation. Various landmark types are distributed across the map: houses (red roofs), shops (red and white striped awnings), and warehouses (with forklift symbols), creating perceptually similar patterns that cause aliasing. The green dashed circle shows the sensor range (radius = 5), and green dashed lines indicate the 90º field of view. Orange dots represent belief particles from ESCORT, with particle density shown as a yellow-red heatmap overlay. The robot must localize itself despite ambiguous observations from these visually similar landmark configurations.
The Kidnapped Robot Problem (present in Figure 4) represents a classical robotics localization challenge scaled to high dimensionality with complex correlation structures. The robot operates within a 20x20 map containing various landmarks of different aruusee-rranged in perceptually similar patterns that create fundamental ambiguity in observa- tions. The state space is 20-dimensional, comprising 2D position (z, E 20 orientation IR$, and environmental feature descriptors f E IR 10 with [f|22 = 1. The robot's 's sensor has a limited range of 5 units and 90º field of view, generating observations consisting of distance measurements and feature similarity scores for visible landmarks.
shops, and ware-
0 E [0, 2T). velocity and steering parameters (v,s) E g² sensor calibration parameters
The environment incorporates strong correlation structures that reflect realistic robotic systems. Po- sition and orientation exhibit correlation coefficients of 0.6, while position-velocity correlations reach 0.8, representing coupled dynamics typical in mobile robotics. Sensor calibration parame-
32

---

Confusion Zone
Blind Spot
Ror wotsioo
Clear
Limited
Torperi My nate
20% vaavlty
Figure 5: Multiple Target Tracking Environment. An agent (drone) with limited field of view (cyan wedge) must navigate to the goal (star) while tracking multiple independently moving targets (cars with velocity arrows) across zones with varying observability: Clear View (full visibility). Limited Sensing (30% visibility). Confusion Zone (identity swaps possible), and Blind Spot (no visibility). The 20D continuous state space and partial observability create multi-modal belief distributions.
ters maintain internal correlations of 0.4 and influence feature descriptors with coefficients of 0.3, modeling sensor drift and environmental perception coupling. The transition dynamics follow the update equations: It+1 = It + 8 cos cos(0,) (tt) Vt+1 = 3/+ + in At, and 0,+1 = 6₂ + A& where actions modify orientation (A0 +.0.1 maintain position. Correlated noise is applied
ing the full correlation matrix C E T 20 generating realistic multi-variate updates that preserve dimensional dependencies.
The fundamental challenge arises from perceptual aliasing where multiple landmark configurations produce nearly identical observations, creating multi-modal belief distributions. The map contains repeating patterns such as clusters houses, shops, and warehouses at different locations that gen- erate similar feature vectors with small perturbations (o 0.1). When the robot observes these patterns, the belief distribution develops multiple modes corresponding to each plausible location, with correlation structures linking position hypotheses to consistent orientation and velocity esti- mates. This multi-modality, combined with the high-dimensional correlated state space, creates the precise challenge that ESCORT addresses through its coreeltioonaaare particle evolution and temporal consistency constraints.
Performance evaluation focuses on localization accuracy measured position eттоr Epos P E[pretielllzz where Pine E RR² represents the true robot position and E[prete denotes the expected position from the belief distribution. The reward structure implements r; -1 per timestep to encourage efficient localization, with episode termination based on convergence criteria maxi- mum step limits, ensuring that methods must balance exploration of multiple hypotheses with rapid convergence the true robot location.
33

---

H.3 Multiple Target Tracking
The Multiple Target Tracking domain extends classical pursuit- evasion scenarios to test belief ap- proximation under high dimensionality, multi- modality, and complex correlations. An agent must navigate to goal position while maintaining awareness of four independently moving targets de- spite varying observability conditions that create ambiguous, multi-modal belief distributions.
The environment consists of a continuous 10 10 space with state E _2 comprising the agent's position and velocity Sa a Va. U2,, U
I and four targets' states St, [ 9t, Vay," Uy, I for i e 21, 3,24444 4]. The agent receives partial observations ments of itself and targets: o = [x= + Eg. Ya + Ea It, Et, 9t, +c.......' .J"". where observation noise varies based on spatial zones. As illustrated in Figure 5, four distinct visibility zones create vary- ing observation conditions: Clear View (green zone, visibility v = 1.0, full observations), Limited Sensing (orange zone, 0.3, high noise), Confusion Zone (yellow zone, v 0.7 but 30% chance of identity swaps between targets), and Blind Spot (red zone, V = 0.0, no target observations).
R containing noisy position measure-
The system dynamics exhibit strong correlations through physical constraints and environmental influences. State transitions follow 8 = [(s, (t) + Th, where I incorporates velocity damping (A A 0.1), agent acceleration from discrete actions a, {+2, -,, -y). and environmental flow fields F(x) that create correlated target movements. The correlation matrix C E 20x00 captures Dnsition-velocity couplings within entities (Ca,v, Cy,², 0.8) and inter-target dependencies from flocking behavior (Ct,,t, = 0.4 for positions, 0.6 for velocities). The red arrows in Figure indicate the current velocity directions of cach target, which are influenced by both individual dy- namics and collective flow patterns. Collision avoidance introduces additional correlations through repulsive forces when |x,,- -x,, 1.0.
The combination of limited sensing and zone-dependent observability creates severe challenges for belief representation. When targets enter the Blind Spot (as shown for one target in Figure 5), the be- lief must maintain hypotheses about their possible locations, creating multi-modal distributions. The Confusion Zone induces additional modes when identity swaps occur-if the agent observes target at position Xobe. the belief must consider could be any of the targets whose last known positions were nearby. This ambiguity compounds over time as ((o = II P(or,ils., zone (s).)), where zone-dependent likelihoods create sharp discontinuities. The agent's limited field of view (60% cone shown in cyan in Figure 5) further exacerbates partial observability, as targets outside the FOV receive no updates regardless of zone visibility.
The reward function balances navigation and safety objectives: r ax| x Xgoat|| B
Et - xx, || < 6]. where a = 0.1 weights distance to goal (marked by the star in Figure 5), 3= 0.05 provides step penalty, and collision penalty triggered when agent-target distance falls below 8 = 0.5. Episode success requires reaching ||x, Xgoat|, < 0.5. Performance is evaluated by the mean position error between true and estimated agent position across belief particles: error ||x_uee - FE x.]]I. where the belief mean is computed from particle representation.
H.4 Visual Observation Environments: Flickering Atari
To evaluate ESCORT's effectiveness with high-dimensional visual observations under severe partial observability, we conducted experiments on Flickering Atari environments [TTwerr et al., 2024, Bellemare et al., 2013, Machado et al., 2018]. These environments use a flickering mechanism with 50% probability of blank screen observations and single-frame inputs [Igl et al.. 2018, Ma et al., 2020], creating substantial uncertainty about the current state. We compare against DVRL on four standard Atari games with different complexity characteristics, following the experimental protocol from the DVRL paper for fair comparison. Table 5 presents the performance results.
Table 5: Performance on Flickering Atari Environments (Higher is Better)
Environment
ESCORT
DVRL
Pong
17.97 + 3.74
18.17 2.67
IceHockey
-4.63
-4.88 = 0.17
MsPacman
3179.7 = 356.7
2221 = 199
Asteroids
1787.7 = 239.6
1539 + 73
34

---

The results demonstrate ESCORT's effectiveness with raw visual observations under severe partial observability. In simple reactive environments (Pong, IceHockey). ESCORT achieves compara- ble performance to DVRL despite temporal consistency potentially over-constraining fast reactive dynamics. However, ESCORT significantly outperforms DVRL. in complex multi-object tracking environments (MsPacman, Asteroids) where multiple ghosts/asteroids create multi-modal beliefs with crucial position-velocity correlations. ESCORT's deterministic particle evolution maintains these multiple hypotheses and dimensional dependencies effectively, while DVRL's VAE compres- sion struggles to preserve the multi-modal structure necessary for accurate tracking under flickering observations.
I Synthetic multi-modal distributions
To systematically evaluate ESCORT's capability in addressing the fundamental challenges of belief oppresntatiinn-hige dimensionality, multi-modality, and complex correlation structures- we de- signed comprehensive suite of synthetic benchmark distributions. These controlled experiments allow to isolate and measure specific aspects of belief approximation performance that are difficult to disentangle in real POMDP environments. By progressively increasing dimensionality from 1D to 20D while maintaining consistent multi-modal characteristics, we can observe how each method's performance degrades with the curse of dimensionality and assess their ability to preserve critical distributional properties such as mode coverage and correlation structures. This systematic eval- uation complements our POMDP experiments by providing precise quantitative metrics for belief representation fidelity.
L1 Evaluation Metrics
To quantitatively assess the quality of belief approximation across different dimensionalities, employ comprehensive set of metrics that capture complementary aspects distributional fidelity (results presented in Table 2):
Maximum Mean Discrepancy (MMD) measures the distance between two distributions in a repro- ducing kernel Hilbert space. For samples {= ~ and {uj / q, the empirical MMD with
RBF kernel k(z, v) exp(->llz =) is computed as:
2 7um
MMD²
k( 2,)
k(y:. U3)
k(a y:)
n
This metric captures overall distributional similarity, with lower values indicating better approxima- tion quality. The kernel parameter 7 0.5 provides sensitivity both local and global distribution differences.
Wasserstein Distance (1- [--aaserstein Earth Mover's Distance) quantifies the minimum "cost" of transforming one distribution into another:
/ ||0 1/12 dy(a,
(i(p.g) = inf
>ET(p.g)
where I'(p, q) denotes the set of all joint distributions with marginals p and g. For ID distributions, this reduces to the L1 distance between inverse cumulative distribution functions, efficiently com- puted via sorting. Unlike MMD, Wasserstein distance explicitly accounts for the metric structure of the space, making it particularly sensitive to mode locations.
Sliced Wasserstein Distance extends Wasserstein distance to high dimensions by projecting distri- butions onto one-dimensional subspaces:
SW1(p. = -f6- W.(Rop, Req) do
(10)
where Ry denotes the Radon transform (projection) along direction 0 E Sd-1, This approach main- tains computational efficiency while preserving geometric properties, computed via Monte Carlo approximation over random projections.
35

---

Mode Coverage Ratio specifically evaluates modii- representation quality. Given target mode locations {1=]k=1 and approximating samples [= mode considered "covered" if:
Ma : ||. - | < + 0.05
1
(11)
K
where 1.0 is the coverage threshold. The metric returns the fraction of modes satisfying this criterion, directly measuring whether methods maintain all hypotheses or suffer from mode collapse.
Correlation Error (for dimensions 2) measures how well methods preserve inter-dimensionall dependencies. Given true correlation matrix Ciue and approximated correlation matrix Capprox com- puted from samples:
Correlation Error ||Crue - Capprox|
(12)
where - || denotes the Frobenius norm. This metric is crucial for evaluating ESCORT's correlation-awan regularization mechanism, as preserving dimensional dependencies is essential for accurate belief representation in POMDPs.
These metries collectively provide comprehensive evaluation framework: MMD and Wasser- stein/Sliced Wasserstein capture global distributional fidelity, Mode Coverage Ratio explicitly quan- tifies multi-modal representation capability, and Correlation Error measures the preservation of di- mensional dependencies critical for complex belief structures.
1.2 1D Multi-modal Gaussian Mixture Model
1D Gaussian Mixture Model Test Case
Component
--0
Component
0.25
Component
Probabiliy Density
GMM
0.20
1 3
0.15
0.10
0.05
Key clnngess
cymmerric weights
Variable sprend
tode separation
0.00
-6
-4
-2
Component Weights and Parameters
Weight (w)
C2
0.4
C1
0.2
0.30
0.40
0.30
0.0
p=:3, g²=0.B
p=0, g²=0.5
p=3, o²=0.5
Figure 6: Visualization of the ID Gaussian Mixture Model test case. The top panel shows the overall GMM density (black line) decomposed into three weighted components with different means and variances. Vertical dashed lines indicate component means, with annotations showing the precise locations. The bottom panel displays the component weights and variances, highlighting the asym- metric nature of the distribution that challenges belief approximation methods.
Our ID test case consists of a carefully designed Gaussian Mixture Model (GMM) that encapsulates the multi-mon mudti-m challenge in its simplest form while remaining non-trivial for approximation methods. The target distribution is defined as:
p(x) Σ w (2;
(13)
36

---

where the component parameters are μl1 = -3.0, μ22 = 0.0, ps = 3.0 for the means, of = 0.8, 0 0.5. o3 0.5 for the variances, and w1 0.3, W22 0.4, wg 0.3 for the mixture weights.
This configuration presents several challenges that mirror those encountered in POMDP belief repre- sentation. First, the unequal weights create asymmetric distribution where methods must balance between accurately representing the dominant central mode (w2 0.4) while maintaining sufficient particles the less probable side modes. Second, the different variances, with the first component having larger spread (oī 0.8), test whether methods can adapt their particle density to match the local uncertainty structure. Third, the well-separated modes (6 units apart) ensure that meth- ods cannot rely on a single concentrated particle cloud but must actively maintain multiple distinct hypotheses.
As shown in Figure 6, the resulting distribution exhibits clear separation between modes while maintaining smooth probability gradients that allow gradient-based methods like SVGD and ES- CORT to navigate the landscape effectively. The asymmetric weights and variances create a more realistic test case than uniform mixtures, real POMDP belief's often exhibit similar heterogeneity due to varying observation quality across the state space. This ID case serves as the foundation for understanding each method's behavior before examining how their performance scales to higher dimensions where additional challenges of correlation preservation and exponential volume growth emerge.
1.3 2D Correlated Gaussian Mixture Model
2D Correlated Gaussian Mixture Model Test Case
6.112
corelaaiin
Mudic Nogutove cxnecktion
0,733
D
0,003
M.A
0.032
+0.
1000
Kry dhilooger
AAySSDNTT revin
Medl
Figure 7: Visualization of the 2D Correlated Gaussian Mixture Model test The filled tours represent the overall probability density, while the three components are shown with their 95% confidence ellipses. Arrows indicate the principal axes of each covariance matrix, illustrating the positive correlation (Mode 1, bottom-left), zero correlation (Mode 2, center), and negative correla- tion (Mode 3, top-right). The correlation coefficients p and mixture weights wv are annotated for each component.
Building upon the 1D evaluation, our 2D test case introduces the critical challenge of correlation structures between state dimensions. The target distribution is three-component GMM designed
37

---

to test each method's ability to preserve diverse correlation patterns:
p(x) WkN(x:: M4: Ex)
(14)
where x = [21, Ø2]" and the component parameters are carefully chosen to create distinct correla- tion challenges.
The three modes are positioned μ4 [-2, -2 24 [0. 0l", and P3 = [2, 2) with weights
W11 0.35, w2 0.30, and wi = 0.35. structures:
The critical distinguishing feature lies in their covariance
1.0
0.8
E1
=
0.8
1.0
(positive correlation, p 0.8)
(15)
0.5
0.0
(no correlation, p = 0)
(16)
0.0
0.5
1.0
-0.8]
Σ3 =
(negative correlation, -0.8)
(17)
0.8
1.0
This configuration presents several interrelated challenges that directly test ESCORT's correlation- aware mechanisms. First, the varying correlation structures-fron strong positive through zero to strong negative correlation-regeee cal covariance geometry rather than applying uniform, isotropic updates. Second, the asymmetric weights create a subtle imbalance where methods must allocate appropriate computational resources each mode while respecting their different shapes. Third, the well-separated modes (distance of 4v2 units between adjacent modes) ensure that simple diffusion-basee approaches cannot bridge the modes without explicit multi-modal handling.
methods to adapt their particle dynamics to match the lo-
As illustrated in Figure 7, the distribution creates challenging landscape where each mode requires different treatment. The elliptical contours reveal how correlation structures fundamentally alter shape of uncertainty regions: Mode I's positive correlation creates elongated ellipse along the diagonal, Mode spherical shape reflects independent dimensions, while Mode 3's negative cor- relation produces an ellipse oriented perpendicular to the diagonal. These geometric differences not merely acsthetic-the represent fundamentally different relationships between state variables that must be preserved during belief updates.
The 2D case serves as critical bridge between the simplicity of ID and the complexity of high- dimensional spaces. While maintaining computational tractability for detailed analysis, it introduces the essential challenge of correlation preservation that becomes increasingly important in higher dimensions. Methods that fail to account for these correlation structures will either oversample along incorrect directions (wasting particles) or undersample critical regions (missing important probability mass), leading to poor belief approximation and suboptimal decision-making in POMDP applications.
I.3.1 3D Correlated Gaussian Mixture Model
Building upon the correlation preservation challenges introduced in the 2D case, our 3D test extends the evaluation to capture the full complexity of belief distributions encountered in realistic POMDP scenarios. The target distribution is six-component GMM specifically designed to test ESCORT's ability to preserve diverse three-dimensional correlation structures:
p(x)=
(x;(x; E+)
(18)
where x = [xi, I2,
22, 23]T and the component parameters create distinct correlation challenges that directly correspond belief structures in 3D state spaces.
The six modes are positioned μ4 [-2.5, -2.5, -2. _ b1 [2.5, 22.522.,22 2.5 13
2.5, 2.5, T. t4 [2.5, 2.5, 2.5)T. M4 = O.. and l46 = 10.0. 0. -4] with weights
38

---

Figure 8: cirre-dimensional Gaussian Mixture Model test distribution with complex correlation structures. Left: 3D visualization showing six modes with distinct correlation patterns, where el- lipsoids represent 95% confidence regions and colors indicate component membership. Mode 6's extreme elongation along the Z-axis and the planar correlations of Modes 1-3 are clearly visible. Right: Three 2D projections onto coordinate planes (XY, XZ, YZ) reveal the correlation patterns with a summary of correlation structures. The XY projection shows Mode I's strong correlation; XZ projection highlights Mode 2's correlation; YZ projection displays Mode 3's structure. The cor- relation matrix for each mode determines the ellipsoid orientation and eccentricity.
w 2,2,0.15,00143 0.2, 0.15)². Each mode exhibits unique correlation pattern designed to challenge specific aspects of belief approximation:
Mode Extreme 31-22 plane correlation (P12 0.95) with minimal 23 variance, rep- resenting beliefs where two state variables are tightly coupled while the third remains independent-commor robotic systems where position coordinates are correlated but orientation varies freely.
Mode 2: Extreme -1- plane correlation (p1s = 0.95) with minimal 22 variance, testing the ability to capture correlations across non-adjacent dimensions.
Mode 3: Extreme #2-4' plane correlation (p23 0.95) with minimal ø variance, com-
pleting the set of planar correlations.
Mode 4: Complex mixed correlations with both positive (p = 0.7) and negative (p = -0.7) dependencies:
1.0
0.7
-0.7
Σ4
0.7
1.0
-0.7
(19)
-077
00.7
1.0
This mode represents belief states where increasing confidence in one dimension simulta- neously increases confidence in another while decreasing in the third-a pattern observed in constrained optimization problems.
Mode 5: Hierarchical correlations with varying magnitudes, where the correlation strength decreases with dimensional distance, modeling cascading uncertainty propagation in quential state estimation.
Mode 6: Highly elongated distribution along the za-sxis (o = 4.0 while of of 0.2), testing the ability maintain particles in extremely anisotropic distributions without collapse.
This 3D configuration presents compounded challenges beyond the 2D case. The curse of dimen- sionality begins manifest more severely-wie while maintaining coverage of six modes in 3D quires only 61/3 x 1.8x more particles per dimension than in 2D, the variety of correlation patterns
39

---

demands sophisticated particle dynamics. Methods must simultaneously: (1) maintain sufficient particles in each mode despite the increased volume, (2) preserve three distinct types of planar correlations, (3) handle mixed positive-negative correlations that create saddle-shaped uncertainty regions, and (4) prevent particle collapse in the highly elongated Mode
As shown in Figure 8, the distribution creates challenging landscape where each mode requires fundamentally different treatment. The planar correlations in Modes 1-3 require particles align along specific 2D subspaces within the 3D space, while Mode 4's mixed correlations create a com- plex saddle structure that naive isotropic updates cannot capture. Mode 5's hierarchical structure tests whether methods can model correlations of varying strength, while Mode 6's extreme elonga- tion along the æ3-axis challenges particle filters that typically assume roughly isotropic uncertainty.
The results Table 2 for the 3D experiment reveal ESCORT's advantages in this intermediate- dimensional space. While all methods maintain perfect mode coverage (1.0), indicating sufficient exploration capabilities in 3D, the correlation error metric reveals significant performance differ- ences: ESCORT achieves 0.761 correlation error compared to SVGD's 0.819, DVRL's 0.882, and SIR's 1.003. This 7% improvement over SVGD and 24% over SIR demonstrates that ESCORT's correlation-awarc projections effectively preserve the complex interdependencies between state vari- ables. The MMD and Sliced Wasserstein metrics further confirm ESCORT's superior distributional approximation, with particularly strong performance in capturing the extreme anisotropy of Mode 6 and the mixed correlations of Mode 4 structural features that standard SVGD's isotropic kernel struggles to maintain.
1.4
5D Correlated Gaussian Mixture Model
(34 vonancel
Figure 9: Five-dimensional Gaussian Mixture Model test distribution with eight modes exhibit- ing diverse correlation structures. Left: PCA projection onto the first two principal components (explaining 34.2% and 28.7% of variance respectively) shows clear mode separation. Fach color represents samples assigned one the eight modes, with black stars marking the projected mode centers. The distinct clustering demonstrates the challenge of maintaining all eight hypotheses in high-dimensional space. Right: Selected 2D projections revealing correlation patterns across di- mension pairs. Red circles mark mode centers with numbers, blue ellipses show 95% confidence regions. The projections highlight: strong positive correlations (e.g., x1-22 for Mode 1), negative correlations (e.g., Ii Ei- for Mode 4), and varying ellipsoid orientations. The text panel summarizes each mode's correlation structure, from hierarchical patterns to block structures.
Advancing 5-dimensional space introduces exponentially greater complexity in correlation mod- eling, testing each method's ability to handle the curse of dimensionality while preserving intricate inter-dimensional relationships. Our 5D test case consists of an 8-component GMM that pushes the boundaries of correlation preservation:
p(x)
WkN Mx: E+)
(20)
1=1
40

---

where x = [21, 22, F3, T1, Ø3]" and the eight modes are strategically positioned to create diverse correlation challenges.
The mode locations are: 14 = 2-,-2,,,2
14 =
12 -2 2,2 143 =
2, -2]_, 14 2,
14: 3,00,, 1M 0, 3 147
[0,3, 0, 0j" and Mg = 1-3,-3,-3,3,33 with weights w1 W2 = W4 = 0.15 and
w3 ws we w7 0.10.
Each mode exhibits a distinct correlation structure designed to challenge specific aspects of belief approximation:
Mode 1: Strong correlations between dimensions 1-2 (p = 0.85) and chained correlations
among dimensions 3-4-5
Mode 2: Alternating correlation pattern linking dimensions 1-3-5 (p 0.8)
Mode 3: Strong correlation between dimensions 1-5 (p 0.85) with middle dimensions
2-3-4 interconnected
Mode 4: Negative correlations between dimensions 1-3 and 3-5 (p -0.7)
Mode 5: Hierarchical correlation structure cascading from dimensions 1-2-3
Mode 6: Extended variance in dimension 5 with weak correlations to other dimensions
Mode 7: Block structure focusing on even dimensions 2-4 with increased variance
Mode 8: Split correlation pattern with dimensions 1-3 forming one correlated block and
dimensions 4-5 forming another
As illustrated in Figure 9 (left). the PCA projection reveals how these eight modes separate in the first two principal components, which together explain 62.9% of the total variance. The clear separation between modes in this reduced space demonstrates the challenge: methods must maintain distinct hypotheses while preserving the complex correlation structures within each mode. Figure 9 (right) shows selected 2D projections that highlight different correlation patterns-the elliptical contours reveal how correlation structures fundamentally alter uncertainty regions across different dimension pairs.
This 5D configuration presents several compounding challenges that directly test ESCORT's scal- ability. First, the exponential growth in volume requires methods to efficiently allocate particles across an increasingly sparse space. Second, the diverse correlation patterns from strong positive through negative to hierarchical structures-ded demand adaptive mechanisms that can model different dependency types simultaneously. Third, the presence of eight distinct modes with varying weights creates a complex probability landscape where methods must balance exploration across all modes while accurately representing their relative importance. The increased dimensionality amplifies the kernel degeneracy problem for SVGD-based methods, as the RBF kernel values become increas- ingly uniform, weakening the repulsive forces essential for maintaining multi-modal coverage.
I.5 20D Correlated Gaussian Mixture Model
Scaling to 20-dimensional space represents the ultimate test of belief approximation methods, where the curse of dimensionality becomes severe and correlation structures reach unprecedented complex- ity. Our 20D test case consists of a 10-component GMM that systematically explores different types of high-dimensional correlation patterns:
10
p(x)
Σ
wwN wxx: M4: E+)
(21)
where x = [x1, Ø2. T2o] and the ten modes are strategically designed to challenge different
aspects of correlation modeling at scale.
The mode locations exhibit diverse patterns: 14 splits between negative values dimensions 1- 10 and positive in 11-20; f4 alternates between positive and negative values; μ4 follows linear gradient from to 3: by through fy concentrate activity specific 5-dimensional subspaces; Hs follows sinusoidal pattern; b4 exhibits quadratic pattern; and 110 maintains uniform negative
41

---

FLX Propetioon Compenant
Figure 10: PCA projection of the 20D Gaussian Mixture Model onto the first two principal com- ponents. The ten modes are clearly separated in this reduced space, with samples colored by mode assignment and black capture only 40% of the total variance (PCI: 34.2%, PC2: 28.7%), demonstrating that no sim- ple low-dimensional representation can adequately capture the distribution's 's structure. This limited variance explanation indicates that the remaining 60% of the distribution's 's complexity lies in the higher-dimensional subspace, making accurate belief approximation exceptionally challenging. The clear mode separation in PCA space masks the intricate correlation patterns within each mode that exist the full 20D space.
marking the projected mode centers. Critically, these components
values. The weights set as WT W2 Wg W7 0.08.
ws
0.12, W3 W10 0.10, and W5
Each mode implements a distinct correlation structure that tests specific capabilities:
Mode 1: Block diagonal structure with four 5x5 blocks, each containing alternating posi-
tive (p 0.7) and negative (p -0.7) correlations
Mode 2: Checkerboard pattern between odd and even dimensions with correlations alter- nating between = 0.75 and = -0.75
Mode 3: Band diagonal structure with correlation strength decaying exponentially (p 0.9)--il) for dimension pairs within steps
Modes 4-7: Localized strong correlations (p = 0.85) within specific 5-dimensional sub-
spaces, testing methods' ability handle sparse correlation structures
Mode 8: Hierarchical correlation with strong antra-group correlations (p 0.7) within four 5-dimensional groups and weak inter-group correlations (p = 0.3)
Mode 9: Long-range correlations between opposite ends of the dimension space, with
p(z;, x19-2) 0.7 for even and -0.7 for odd
Mode 10: Near-independent dimensions with sparse, weak correlations (pp) 0.1) domly distributed
As illustrated in Figure 10, the PCA projection reveals fundamental challenge: despite clear mode separation in the first two principal components, these dimensions capture only 40% of the total
42

---

variance. This indicates that 60% of the distribution's structure struuture-incuudnng the complex correla- tion patterns within each mode remains hidden in the 18- 18-dimensional orthogonal subspace. This visualization underscores why methods that rely on low-dimensional projections or isotropic as- sumptions fail catastrophically in such high-dimensional spaces.
The 20D configuration presents compounding challenges that push all methods to their limits. First, with volume scaling as ) particles become exponentially sparse, making mode coverage ex- traordinarily difficult-a partiele cloud that seems dense in projection may leave vast regions un- explored. Second, the diverse correlation patterns require methods simultaneously model block structures, long- range dependencies. band-diagonal patterns, and sparse correlations without impos- ing single global assumption. Third. the presence of ten distinct modes with complex internal structures creates 10 x 2 distinct regions of interest, far exceeding the capacity of any practical particle count. Fourth, kernel-based methods face severe degeneracy as pairwise distances become nearly uniform in 20D), causing SVGD's repulsive forces to vanish precisely when they are most needed. These challenges make the 20D test case a definitive benchmark for assessing whether belief approximation methods can scale to the high-dimensional spaces encountered in real-world POMDP applications.
I.6 Scalability Analysis: Impact Correlation-Aware Regularization
To comprehensively demonstrate ESCORT's scalability, we conducted extensive experiments on synthetic multi-modal distributions extending from 1D to 200D, far beyond the 20D environ- ments in our main results. These experiments used 5-mode Gaussian Mixture Models with di- verse correlation cruuttres-iicludee patterns-to to simulate the complex dimensional dependencies found in real-world POMDPs. We evaluated performance using Root Mean Square Error (RMSE) [Thrun et al., 2005), defined as
block-diagonal, banded, long-range, and sparse correlation
RMSE = V Σ ||s; - 8:|1² where 8; is the ground-truth state and 8; is the posterior mean estimate, providing a direct measure of belief approximation accuracy.
To isolate the impact of our correlation-aware regularization-one of ESCORT's core contribu- tions-we we compared full ESCORT against ESCORT-NoCort (which disables the correlation-aware regularization term Reor). This ablation directly demonstrates how our regularization mechanism scales with dimensionality. Table 6 presents these results.
Table 6: RMSE for synthetic multi-modal approximation: Impact of correlation-awee aware regulariza- tion across dimensions (Lower is Better)
Dimension
ESCORT
ESCORT-NoCorr
1D
0.15 + 0.02
0.14 + 0.02
5D
0.35 0.04
0.45 = 0.05
20D
0.65 + 0.07
0.10
50D
0.95 0.09
1.09
100D
1.35 0.18
2.15
200D
1.90 0.23
3.35 = 0.57
The results reveal clear scaling pattern: while both variants perform comparably in low dimensions. the performance gap widens dramatically as dimensionality increases. This exponential divergence confirms theoretical framework-in high-dimensional spaces, correlation manifolds occupy neg- ligible volume, causing unregularized particles to drift away through accumulated random move- ments. Our correlation-awar regularization, through learned projection matrices A, constrains par- ticles to these critical manifolds, preventing the catastrophic degradation seen in ESCORT-NoCorr and demonstrating that our approach becomes increasingly essential as dimensionality grows.
43


Reviews:
[{"review_id": "aaeQGBSSQk", "reviewer": "Reviewer_UoVv", "summary": "The paper introduces ESCORT, a new particle-based method for faithfully approximating complex multi- modal belief distributions in high-dimensional POMDPs. The key idea is to extend Stein Variational Gradient Descent (SVGD) with (i) correlation-aware projections and (ii) temporal consistency constraints that enables accurate modeling. Experimental results demonstrate that ESCORT outperforms prior approaches in both belief approximation accuracy and decision-making.", "questions": "- (Line 313) “received equivalent computational budgets” → Could the authors elaborate what this means?\n- How the hyperparameters are chosen?\n- It feels like many important practical implementation details are deferred to the appendix, and part of them could be moved to the main text.\n- Could the authors discuss the theoretical guarantee of the proposed method?\n- (Line 273) missing parenthesis", "limitations": "yes", "rating": 4, "confidence": 1, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "### Strengths\n\n- The paper tackles an important problem. The key ideas are well presented and seem to be novel and interesting, though I am not the expert in this field.\n- Experimental results show strong performance of the proposed method. Lastly, the paper is well-organized and the writing is clear.\n\n### Weaknesses\n\n- Experiments are mostly limited to low-dimensional environments, which is somewhat inconsistent to the authors claim on high-dimeonsional POMDPs. Experimental results on more high-dimensional environments would further strengthen the paper.\n- I would like to see how the computational cost compares to prior approaches, which could be the computational complexity or the actual computation time. In particular, I am curious whether the proposed approach could scale to more high-dimensional environments in terms of practical computations.\n- As it often requires different choice of hyperparameters across environments, it could be better to include hyperparameter sensitivity analysis. Also, it is unclear how the hyperparameters are chose.", "was_revised": true, "final_justification": "My questions are mostly addressed by the rebuttal, so I will keep my initial score.\n\nI would like to note that I'm not an expert in this field, so other reviewers' assessments should be given more weight.", "rebuttal": "We thank the reviewer for their valuable feedback and provide detailed responses to each point below.\n\n\n>**Q1**: Experimental results on high-dimensional environments:\n\n*Table 1: RMSE for synthetic multi-modal approximation: Impact of correlation-aware regularization across dimensions (Lower is Better)*\n\n| Dimension | ESCORT | ESCORT-NoCorr |\n|-|-|-|\n| 1D | 0.15 ± 0.02 | **0.14 ± 0.02** |\n| 5D | **0.35 ± 0.04** | 0.45 ± 0.05 |\n| 20D | **0.65 ± 0.07** | 0.88 ± 0.10 |\n| 50D | **0.95 ± 0.09** | 1.42 ± 0.09 |\n| 100D | **1.35 ± 0.18** | 2.15 ± 0.28 |\n| 200D | **1.90 ± 0.23** | 3.35 ± 0.57 |\n\nWe appreciate the reviewer's request for higher-dimensional experiments. To comprehensively demonstrate ESCORT's scalability, we conducted extensive experiments on synthetic multi-modal distributions extending from 1D to 200D, far beyond the 20D environments in our main results.\n\nThese experiments used 5-mode Gaussian Mixture Models with diverse correlation structures—including block-diagonal, banded, long-range, and sparse correlation patterns—to simulate the complex dimensional dependencies found in real-world POMDPs. We evaluated performance using Root Mean Square Error (RMSE) between the posterior mean and ground-truth state, providing a direct measure of belief approximation accuracy.\n\nTo isolate the impact of our correlation-aware regularization—one of ESCORT's core contributions—we compared full ESCORT against ESCORT-NoCorr (which disables the correlation-aware regularization term $R_{corr}$). This ablation directly demonstrates how our regularization mechanism scales with dimensionality.\n\nThe results reveal a clear scaling pattern: while both variants perform comparably in low dimensions, the performance gap widens dramatically as dimensionality increases. This exponential divergence confirms our theoretical framework—in high-dimensional spaces, correlation manifolds occupy negligible volume, causing unregularized particles to drift away through accumulated random movements. Our correlation-aware regularization, through learned projection matrices $A_i$, constrains particles to these critical manifolds, preventing the catastrophic degradation seen in ESCORT-NoCorr and demonstrating that our approach becomes increasingly essential as dimensionality grows.\n\n\n>**Q2**: computational cost: \n\nWe appreciate the reviewer's important question about computational scalability. To address this, we conducted a detailed FLOPs (Floating Point Operations) analysis of ESCORT across dimensions:\n\n*Table: ESCORT Computational Cost Breakdown (GFLOPs per belief update)*\n\n| Dimension | GFLOPs | Kernel % | SVGD % | GSWD % | Temp % |\n|-|-|-|-|-|-|\n| 10 | 6.72 | 62.5 | 29.8 | 6.3 | 1.5 |\n| 20 | 15.60 | 52.6 | 25.7 | 21.1 | 0.6 |\n| 50 | 81.06 | 24.9 | 12.3 | 62.6 | 0.1 |\n| 100 | 262.31 | 15.3 | 7.6 | 77.0 | 0.0 |\n| 200 | 926.32 | 8.7 | 4.3 | 87.0 | 0.0 |\n\nOur empirical analysis shows ESCORT scales as O($d^{1.67}$), which aligns with theoretical expectations. The algorithm's complexity is dominated by kernel computation O($n^2d$) for pairwise distances and gradients, SVGD forces O($n^2d$) for attractive/repulsive particle interactions, GSWD regularization O($nmd^2$) for $m$ projections in $d$ dimensions, and temporal consistency O($n^2$) independent of dimension.\n\nAs dimensionality increases, the correlation-aware GSWD term (O($md^2$)) becomes the dominant cost, accounting for 87% of computation at 200D. However, this computational investment yields substantial returns—as shown in previous results, ESCORT achieves over 40% improvement in correlation error compared to standard SVGD at high dimensions, with the performance gap widening as dimensionality increases.\n\nOur implementation already incorporates several optimizations including Numba JIT compilation for kernel computations providing 5-10x speedup, vectorized operations in GSWD using batched matrix multiplications, adaptive subsampling for distance computations in high dimensions, and caching of score functions and transition models in belief updates.\n\nTo further improve scalability, we propose GPU acceleration for the O($md^2$) projection operations, sparse projection matrices that exploit correlation structure reducing O($d^2$) to O($d \\cdot k$) for $k$-sparse projections, and hierarchical approximations that group correlated dimensions. These optimizations could reduce the effective complexity closer to O($d^{1.2}$) while preserving the critical correlation-aware benefits that make ESCORT superior in high-dimensional POMDPs.\n\n\"Equivalent computational budgets\" means fixing the same wall-clock time (100ms) per decision for all methods. While we measured FLOPs for computational complexity analysis above, the experimental results reflect performance under equal time constraints for fair comparison.\n\n\n>**Q3**: hyperparameter sensitivity: \n\nWe appreciate the reviewer's interest in hyperparameter robustness.\n\nESCORT's architectural design naturally constrains hyperparameter choices. The correlation-aware regularization weight ($\\lambda_{\\text{corr}}$) and temporal consistency weight ($\\lambda_{\\text{temp}}$) represent the relative importance of preserving dimensional dependencies versus temporal stability against the base SVGD forces. We initialized both at $\\lambda=0.1$ as a starting point, though our analysis reveals optimal values are domain-dependent—particularly $\\lambda_{\\text{temp}}$, which varies with environment dynamics. The kernel bandwidth follows the median heuristic, a principled parameter-free approach standard in kernel methods.\n\nWe agree sensitivity analysis is crucial for validating robustness. Our ablation study (Appendix D, Table 3) serves as an initial sensitivity analysis by examining extreme weight values ($\\lambda=0$ vs $\\lambda=0.1$). Due to response length constraints, we will provide comprehensive sensitivity analysis (heatmaps) in the revised appendix. Our results confirm ESCORT's theoretical properties: correlation regularization impact scales with dimensionality (minimal in low dimensions, critical in high dimensions), while temporal consistency effects are domain-dependent—beneficial for static environments but potentially over-constraining for dynamic systems. These patterns validate that performance stems from architectural design rather than careful tuning.\n\nHyperparameter selection followed established practices: regularization weights initialized at $\\lambda=0.1$ as baseline values, step size ($\\varepsilon=0.01$) follows SVGD convergence theory, kernel bandwidth uses median heuristic, and projection counts scale with state dimensionality. This principled approach ensures reproducibility while minimizing domain-specific tuning.\n\n\n>**Q4**:Could the authors discuss the theoretical guarantee of the proposed method?\n\nOur theoretical analysis provides convergence and representation guarantees for ESCORT:\n\n**Correlation Structure Preservation:** Under Assumption 4, ESCORT preserves belief correlation structures with arbitrary precision. For any belief distribution $b(s)$ with correlation matrix $\\Sigma_p$, our learned projection matrices \n$(A_{i})_{i=1}^m$\nsatisfy \n\n$$\n\\bigl\\|\\Sigma_{p} - \\sum_{i=1}^{m} w_{i}\\,A_{i}A_{i}^{\\top} \\bigr\\|_{\\mathrm F}\\<\\ \\varepsilon.\n$$ \n\nfor sufficiently large $m$. This guarantee ensures that as we increase the number of projections, ESCORT captures the complete correlation structure of complex belief distributions. The correlation-aware regularization term \n\n$$R_{\\text{corr}}(\\phi)=E_{x\\sim q}\\bigl[\\sum_{i=1}^{m} w_{i}\\,\\lvert A_{i}^{\\top}(\\phi(x)-E_{y\\sim q}[\\phi(y)])\\rvert^{2}\\bigr]$$\n\nenforces this preservation during particle updates, preventing the loss of dimensional dependencies that plagues standard SVGD in high-dimensional spaces.\n\n**Bounded Belief Evolution:** Assumption 5 establishes that POMDP transitions satisfy $\\|s' - s\\| \\leq \\delta_{\\max}$, enabling our temporal consistency mechanism to distinguish realistic belief evolution from erroneous jumps. The temporal regularization $L_{\\text{temp}} = \\int_\\Theta W_1((A_\\theta)^\\top b_{t+1}, (A_\\theta)^\\top b_t)d\\lambda(\\theta)$ ensures that consecutive belief updates remain bounded: $W_1(b_{t+1}, b_t) \\leq C \\cdot (\\delta_{\\max} + \\sigma_{\\text{obs}})$ where $C$ depends on the Lipschitz constants of transition and observation models, and $\\sigma_{\\text{obs}}$ captures observation noise. This guarantee prevents the catastrophic belief jumps that occur in particle filters during resampling while allowing necessary adaptation to new evidence.\n\n**Convergence to Target Distribution:** Given the Lipschitz score function (Assumption 2) and positive definite kernel properties (Assumption 3), ESCORT inherits SVGD's convergence guarantees with modifications accounting for our regularization. As particle count $n \\to \\infty$ and step size $\\varepsilon \\to 0$ following $\\sum_{t=1}^\\infty \\varepsilon_t = \\infty$ and $\\sum_{t=1}^\\infty \\varepsilon_t^2 < \\infty$, the empirical distribution of particles converges to the true belief $p(s)$ in Wasserstein distance: $W_2(\\frac{1}{n}\\sum_{i=1}^n \\delta_{x_i}, p) \\to 0$. Our regularization terms $R_{\\text{corr}}$ and $L_{\\text{temp}}$ are designed to vanish as convergence is achieved, ensuring they guide but don't prevent convergence to the target distribution while maintaining the correlation structure throughout the optimization process.\n\n\n>**Q5**:It feels like many important practical implementation details are deferred to the appendix, and part of them could be moved to the main text. (Line 273) missing parenthesis.\n\nThank you for identifying these issues. We will correct the parenthesis mismatch on line 273 (should be $(\\mu(b), \\log \\sigma(b))$) and move key implementation details from the appendix to the main text in the revised manuscript to improve accessibility."}, {"review_id": "8nNtsEdSEe", "reviewer": "Reviewer_4zno", "summary": "The paper studies a novel belief representation for POMDPs, which is based on Stein Variational Gradient Descent. The contributions focus on developing a regularization that is context-aware and maintains temporal consistency.", "questions": "1. Have the authors considered domains that would violate Assumption 5? In kidnapped robot problems, unrealistic belief updates might actually be necessary, for instance.\n2. On line 230, an equation is presented to describe how a particle is propagated through the transition model. The transition function T returns a probability distribution for a particular state and action, and not a single state (as the equation seems to imply). Furthermore, Gaussian transition noise is added, which raises the question whether it is limited to such noise models. Can the authors clarify both issues?\n3. In the experiments, the position error is reported as an evaluation metric. Why not the value, as that is what the POMDP algorithms are optimizing for?", "limitations": "The checklist states that limitations are addressed in the Conclusions section, but there the specific assumptions on the POMDP dynamics are not mentioned, nor any other limitations.", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strengths\n1. POMDPs are an important framework for decision-making under uncertainty, and new particle-based belief representation are valuable as they can potentially be combined with an array of approximate POMDP solvers (as demonstrated in the experiments).\n2. For a subset of POMDPs, the paper provides encouraging results, particularly in high-dimensional domains. Ablation studies are also provided.\n\nWeaknesses\n1. A key issue is that the results presented on temporal consistency regularization only apply to POMDPs with certain assumptions, which are only detailed in the Appendix. In particular, Assumption 5 on bounded transition dynamics is important, because that implies that there’s a metric defined over the state space that allows the agent to judge whether a belief update is unrealistic or not. This might be reasonable in the navigation-like domains the authors consider, but it is a quite strong limiting assumption.\n2. In the model-based belief update, there is an inconsistency regarding how the particles are updated (see Questions).", "was_revised": true, "final_justification": "Based on the other reviews and the responses provided by the authors, I have decided to raise my score.", "rebuttal": "We appreciate the reviewer's thoughtful comments and address each concern comprehensively in our responses below.\n\n\n>**Q1**: A key issue is that the results presented on temporal consistency regularization only apply to POMDPs with certain assumptions, which are only detailed in the Appendix. In particular, Assumption 5 on bounded transition dynamics is important, because that implies that there's a metric defined over the state space that allows the agent to judge whether a belief update is unrealistic or not. This might be reasonable in the navigation-like domains the authors consider, but it is a quite strong limiting assumption.\n\nWe appreciate the reviewer's careful attention to Assumption 5. We clarify that this assumption is required only for our theoretical convergence analysis—specifically, it enables us to prove convergence rates and establish that our temporal consistency objective has well-defined solutions. Without such bounds, we cannot guarantee that belief evolution remains tractable or that our regularization converges to meaningful solutions. However, ESCORT's implementation operates without requiring bounded transition dynamics or predefined state space metrics.\n\nOur temporal consistency term $L_{\\text{temp}} = \\int_\\Theta W_1((A_\\theta)^\\top b_{t+1}, (A_\\theta)^\\top b_t)d\\lambda(\\theta)$ measures the Wasserstein distance between consecutive belief distributions, which inherently scales with belief change magnitude without requiring domain-specific bounds or explicit $\\delta_{\\text{max}}$ values. The hyperparameter $\\lambda_{\\text{temp}}$ controls this regularization strength, enabling flexible domain adaptation: small values ($\\lambda_{\\text{temp}} = 0.01-0.05$) allow rapid belief updates for discontinuous dynamics like kidnapped robot scenarios, while larger values ($\\lambda_{\\text{temp}} = 0.5-1.0$) enforce smooth transitions appropriate for continuous physical systems. Setting $\\lambda_{\\text{temp}} = 0$ disables temporal consistency entirely for pure observation-driven updates.\n\nTherefore, while Assumption 5 provides theoretical grounding for convergence guarantees, ESCORT's practical implementation accommodates arbitrary POMDP dynamics through the adaptive Wasserstein distance and tunable regularization.\n\n\n>**Q2**:Have the authors considered domains that would violate Assumption 5? In kidnapped robot problems, unrealistic belief updates might actually be necessary, for instance.\n\nYes, we considered domains violating Assumption 5, including kidnapped robot scenarios. Our framework naturally accommodates such discontinuous dynamics through the balance between the observation-driven SVGD update (which pulls particles toward high-likelihood regions) and the temporal consistency term ($\\lambda_{\\text{temp}} \\cdot L_{\\text{temp}}$).\n\nTo demonstrate this adaptive behavior, we conducted experiments on Multi-Target Tracking variants. The maneuvering targets variant violates Assumption 5 through random accelerations that create unbounded state jumps—similar to kidnapping events where $\\delta_{\\text{max}}$ cannot be predetermined. Crucially, rather than imposing Assumption 5's hard constraint that forbids large transitions, ESCORT uses $\\lambda_{\\text{temp}}$ as a soft penalty that can be tuned based on domain characteristics. Small $\\lambda_{\\text{temp}}$ values allow necessary large belief updates when observations support them, while larger values encourage smoothness when appropriate.\n\n*Table 1: Effect of Temporal Consistency Weight λ_temp (Avg. Position Error, Lower is Better)*\n\n| Environment | λ_temp = 0 | λ_temp = 0.1 | λ_temp = 1.0 |\n|-|-|-|-|\n| Maneuvering Targets | 3.87 ± 0.43 | **3.67 ± 0.31** | 4.15 ± 0.63 |\n| Static Targets | 3.20 ± 0.29 | 2.75 ± 0.41 | **2.34 ± 0.28** |\n\nThe results validate this approach: maneuvering targets perform best with moderate temporal consistency, allowing rapid adaptation to sudden accelerations while maintaining some stability. Static targets benefit from stronger regularization since large jumps likely indicate errors rather than true dynamics. This demonstrates ESCORT's key advantage—replacing hard bounded-transition assumptions with tunable soft constraints that adapt to domain-specific requirements.\n\n\n>**Q3**:On line 230, an equation is presented to describe how a particle is propagated through the transition model. The transition function T returns a probability distribution for a particular state and action, and not a single state (as the equation seems to imply). Furthermore, Gaussian transition noise is added, which raises the question whether it is limited to such noise models. Can the authors clarify both issues?\n\nThe reviewer correctly notes that standard POMDP transition functions return distributions. In our implementation, $T(x_i^t, a_t)$ denotes the deterministic mean of the transition distribution, with stochasticity introduced explicitly through $\\eta_i$. For a transition $T(s'|s,a) = \\mathcal{N}(\\mu(s,a), \\Sigma(s,a))$, our decomposition $\\tilde{x}_i^{t+1} = T(x_i^t, a_t) + \\eta_i$ is equivalent to $\\tilde{x}_i^{t+1} = \\mu(x_i^t, a_t) + \\eta_i$ where $\\eta_i \\sim \\mathcal{N}(0, \\Sigma(x_i^t, a_t))$. This separation of deterministic dynamics from noise clarifies how ESCORT's particle updates combine deterministic SVGD forces with stochastic transitions. We will revise the notation to prevent confusion.\n\n*Table 2: Performance comparison with different noise distributions on Light-Dark 10D (Avg. Position Error, Lower is Better)*\n\n| Noise Distribution | ESCORT | POMCPOW |\n|-|-|-|\n| Gaussian | **0.35 ± 0.03** | 2.12 ± 0.24 |\n| Mixture of Gaussians | **0.37 ± 0.04** | 2.67 ± 0.15 |\n| Laplace | **0.36 ± 0.07** | 3.21 ± 0.40 |\n\nESCORT is not limited to Gaussian noise—the separation of deterministic transition mean from noise in our formulation \n\n$$\\tilde{x}_i^{t+1} = T(x_i^t, a_t) + \\eta_i$$\n\nallows $\\eta_i$ to follow any distribution appropriate for the domain. While we use Gaussian noise $\\eta_i \\sim \\mathcal{N}(0, \\Sigma_{trans})$ in our main experiments for clarity, this architectural choice enables arbitrary noise models including heavy-tailed distributions for outlier-robust applications, multi-modal distributions for discontinuous dynamics, or state-dependent distributions where noise characteristics vary spatially. Crucially, ESCORT's core mechanisms—SVGD updates, correlation-aware regularization, and temporal consistency—make no Gaussian assumptions and operate effectively regardless of the noise distribution.\n\nTo validate this flexibility, we extended the Light-Dark 10D environment to use different transition noise models beyond the original Gaussian: a Mixture of Gaussians ($0.7\\mathcal{N}(0, \\Sigma) + 0.3\\mathcal{N}(0, 5\\Sigma)$) representing multi-modal uncertainty from sensor switching, and Laplace distribution $\\mathcal{L}(0, b)$ capturing heavy-tailed noise from outlier measurements. Table 2 demonstrates ESCORT's robustness across all noise types, maintaining consistently low position errors regardless of noise distribution, while POMCPOW's performance degrades substantially with non-Gaussian noise. This degradation occurs because SIR's resampling mechanism implicitly assumes concentrated probability mass—when faced with heavy tails or multiple modes, particles cluster inefficiently around local peaks while missing critical probability regions, leading to poor state estimation and degraded navigation performance.\n\n\n>**Q4**:In the experiments, the position error is reported as an evaluation metric. Why not the value, as that is what the POMDP algorithms are optimizing for?\n\nWe chose position error because it provides an interpretable measure of belief accuracy during task execution. While this metric does incorporate effects from exploration and planning strategies, it directly reflects whether our improved belief representation enables better state estimation in practice. We agree with the reviewer that reporting cumulative rewards would provide valuable complementary insights into overall task performance. We will include both position error and cumulative rewards in our revised manuscript for comprehensive evaluation.\n\n\n>**Q5**:The checklist states that limitations are addressed in the Conclusions section, but there the specific assumptions on the POMDP dynamics are not mentioned, nor any other limitations.\n\nWe will enrich the Conclusions section with a dedicated limitations discussion, including our assumption of known POMDP models (transition and observation functions) etc."}, {"review_id": "9sfrVe1F5D", "reviewer": "Reviewer_cAUz", "summary": "The paper proposes a new belief update approach for POMDPs. It addresses the problem that belief update in POMDPs suffers from the curse of dimensionality in high-dimensional state spaces. The paper tries to address this by leveraging the fact that some dimensions may be correlated. Thus, a regularizer that respects this fact can help do better. The paper combines SVGD update with additional regularizer terms that capture correlation across dimensions and also try to minimize the earth-mover distance across time. Their experiments show benefits over other baselines, especially when the dimensionality is increased.", "questions": "1. In Section 3.2, what is the target $p$ in the context of the actual POMDP? Is it the unnormalized product of previous belief $\\times$ transition distribution $\\times$ observation distributions for that time-step?\n2. Are transition and observation distributions (and approximate inference) learned, i.e., via neural networks, like in DVRL? Or was this not necessary because of access to privileged information or special aspects of the environment structure? If yes, then I wonder if the method would scale to state-spaces with much larger dimensionalities, which is typical of neural representations.\n3. Have authors tried DVRL with smaller latent sizes? I wonder if DVRL alone would be capable of “compressing” redundant dimensions in the input to a smaller number of latent dimensions to fight the curse of dimensionality?\n4. Is policy the only learned part of the model?\n5. How do you specify $\\lambda(\\theta)$ in Section 3.4? Is $A\\theta$ more like $A_\\theta$, or is this $A$ related to the correlation structure from 3.2?\n6. Is it possible to ablate the various regularizers used during belief update, e.g., the role of the correlation-regularizer versus the role of the temporal regularizer?", "limitations": "yes", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 2, "originality": 2, "strengths_and_weaknesses": "## Pros\n\n1. Belief update for high-dimensional state-spaces is an important problem, and doing it well by incorporating the world structure is the focus of this paper.\n2. Combining SVGD with structure/correlation-aware and temporality-aware regularizers appears novel in my knowledge in the context of POMDPs.\n3. Experiments show benefits over other baselines, esp. when the dimensionality is increased.\n\n## Cons\nMy main concerns are around: (1) the unclear details about the temporality regularizer, (2) attribution of the gains to two possible sources i.e., the correlation regularizer vs. the temporality regularizer or both; (3) the question of whether evaluating on 20-dimensions is high enough and the fact that a plot might be a better way than a table for showing the scaling trend, and lastly, (4) are the environments too simplistic and whether this approach can work with raw visual observations like those tackled by the DVRL framework (and even those might be considered too simplistic by modern day standards of visual complexity).\n\nSee also the \"Questions\" below.\n\n### Suggestions for Improvement\n\n1. There is a typo (I think) in Equation 5 around the summation.\n2. I wonder if a plot with X-axis as the dimensionality and Y-axis as the performance would be more illustrative in place of Table 2. In tabular form, it is difficult to see the “trend” wrt. the dimensionality.", "was_revised": true, "final_justification": "My concerns are resolved.", "rebuttal": "We thank the reviewer for their valuable feedback and provide detailed responses to each point below.\n\n>**Q1**: The unclear details about the temporality regularizer and request for comprehensive ablation studies:\n\nWe appreciate the reviewer's request for comprehensive ablation studies. We direct the reviewer to Appendix D (Table 3), where we present a comprehensive ablation study systematically evaluating ESCORT-NoCorr, ESCORT-NoTemp, and ESCORT-NoProj variants across all three POMDP domains. Additionally, we conducted extensive experiments isolating each regularizer's contribution and evaluated scaling behavior up to 200D to fully address dimensionality concerns.\n\n**Correlation-Aware Regularization Impact:**\n\n*Table 1: RMSE for synthetic multi-modal approximation: Impact of correlation-aware regularization across dimensions (Lower is Better)*\n\n| Dimension | ESCORT | ESCORT-NoCorr |\n|-|-|-|\n| 1D | 0.15 ± 0.02 | **0.14 ± 0.02** |\n| 5D | **0.35 ± 0.04** | 0.45 ± 0.05 |\n| 20D | **0.65 ± 0.07** | 0.88 ± 0.10 |\n| 50D | **0.95 ± 0.09** | 1.42 ± 0.09 |\n| 200D | **1.90 ± 0.23** | 3.35 ± 0.57 |\n\nTable 1 isolates the correlation regularizer's impact by comparing full ESCORT against ESCORT-NoCorr across dimensions from 1D to 200D. The results reveal a clear scaling pattern: minimal difference in low dimensions (1D-5D) but exponentially growing benefits as dimensionality increases. At 200D, correlation-aware regularization reduces RMSE by 43%, confirming its critical role in high-dimensional spaces where correlation structures become increasingly important for accurate belief representation.\n\n**Temporal Consistency Regularization Impact:**\n\n*Table 2: Effect of Temporal Consistency Weight λ_temp (Avg. Position Error, Lower is Better)*\n\n| Environment | λ_temp = 0 | λ_temp = 0.1 | λ_temp = 1.0 |\n|-|-|-|-|\n| Maneuvering Targets | 3.87 ± 0.43 | **3.67 ± 0.31** | 4.15 ± 0.63 |\n| Static Targets | 3.20 ± 0.29 | 2.75 ± 0.41 | **2.34 ± 0.28** |\n\nTable 2 demonstrates temporal regularization's domain-dependent nature. Unlike correlation regularization's consistent scaling benefits, temporal consistency effectiveness depends on environment dynamics. For maneuvering targets with inherent variability, moderate regularization (λ_temp = 0.1) optimally balances adaptation and stability. In contrast, static environments benefit from stronger consistency (λ_temp = 1.0), achieving 27% error reduction by preventing spurious belief jumps.\n\n**Attribution Analysis:** Our results clearly delineate each regularizer's contribution:\n\n- **Correlation regularization** provides dimensionality-dependent gains, becoming essential above 20D where it prevents kernel degeneracy and maintains distributional structure\n- **Temporal regularization** offers domain-specific improvements, with gains ranging from 5% (dynamic environments) to 27% (static environments)\n- Combined, they address orthogonal challenges: spatial correlation preservation and temporal stability\n\nRegarding the reviewer's visualization suggestion, we agree that scaling trends would be clearer in plot form. We will include a log-scale plot showing RMSE vs. dimension in the revised manuscript, which better illustrates the exponential divergence between ESCORT and ESCORT-NoCorr as dimensionality increases.\n\n\n>**Q2**: environments with visual observations\n\n*Table 3: Performance on Flickering Atari Environments (Higher is Better)*\n\n| Environment | ESCORT | DVRL |\n|-|-|-|\n| Pong | 17.97 ± 3.74 | **18.17 ± 2.67** |\n| IceHockey | **-4.63 ± 0.19** | -4.88 ± 0.17 |\n| MsPacman | **3179.7 ± 356.7** | 2221 ± 199 |\n| Asteroids | **1787.7 ± 239.6** | 1539 ± 73 |\n\nWe conducted additional experiments on Flickering Atari environments used in the DVRL paper. For fair comparison, we used DVRL's reported results and their exact flickering mechanism: 50% probability of blank screen observations with single-frame inputs.\n\nOur results demonstrate ESCORT's effectiveness with raw visual observations under severe partial observability. In simple reactive environments (Pong, IceHockey), ESCORT achieves comparable performance to DVRL despite temporal consistency potentially over-constraining fast reactive dynamics. However, ESCORT significantly outperforms DVRL in complex multi-object tracking environments (MsPacman, Asteroids) where multiple ghosts/asteroids create multi-modal beliefs with crucial position-velocity correlations. ESCORT's deterministic particle evolution maintains these multiple hypotheses and dimensional dependencies effectively, while DVRL's VAE compression struggles to preserve the multi-modal structure necessary for accurate tracking under flickering observations.\n\n\n>**Q3**: typo in Equation 5.\n\nThank you for identifying this formatting error. Equation 5 has a LaTeX rendering issue where the summation appears as `∑j = 1n` instead of ∑_{j=1}^n. We will correct this in the revised manuscript.\n\n\n>**Q4**: the target in the context of the actual POMDP:\n\nThe reviewer is correct. In Section 3.2, the target distribution $p(x)$ in our SVGD formulation represents the unnormalized posterior belief distribution. Specifically:\n\n$$p(x) \\propto O(o_{t+1}|x, a_t) \\cdot \\sum_{s \\in S} T(x|s, a_t)b_t(s)$$\n\nwhere $O(o_{t+1}|x, a_t)$ is the observation likelihood given the current observation, $\\sum_{s \\in S} T(x|s, a_t)b_t(s)$ is the predicted belief after transition, and $b_t(s)$ is the previous timestep's belief distribution.\n\nThis corresponds to the standard POMDP belief update before normalization. The key advantage of using SVGD is that it directly works with unnormalized distributions—the normalization constant $\\eta$ in the standard Bayes update $b'(s') = \\eta \\cdot O(o|s',a) \\cdot \\sum_s T(s'|s,a)b(s)$ is handled implicitly through the gradient-based particle evolution.\n\n\n>**Q5**: transition and observation distributions: \n\nTransition and observation distributions are not learned—unlike DVRL, ESCORT assumes known POMDP models $T(s'|s,a)$ and $O(o|s',a)$. This model-based design is essential because accurate likelihood gradients from known dynamics enable SVGD's deterministic evolution, correlation-aware regularization, and temporal consistency to function correctly, while also enabling fair comparison with existing particle-based methods by isolating our belief representation innovations from model learning errors.\n\nYes, the policy is the only learned component. ESCORT learns a policy network that processes particle-based beliefs to produce action distributions, while all other components use deterministic computations based on known model dynamics.\n\nOur experiments demonstrate scalability through synthetic multi-modal distributions extending to 50D and 200D, showing ESCORT maintains superior performance even at these scales. ESCORT achieves this scalability because its core innovation lies in the belief representation mechanism, not in learning dynamics. The correlation-aware regularization becomes increasingly critical as dimensionality grows, preventing kernel degeneracy and maintaining distributional structure that parametric methods like DVRL struggle to capture with fixed representations. While DVRL's learned models may suffer from compounding errors in high dimensions, ESCORT's deterministic particle evolution with targeted regularization ensures stable belief tracking regardless of state space size.\n\n\n>**Q6**: DVRL with smaller latent sizes:\n\n*Table 4: DVRL Performance with Different Latent Dimensions on 200D synthetic multi-modal distribution*\n\n| Metric | z_dim=64 | z_dim=32 | z_dim=16 |\n|-|-|-|-|\n| 200D | 15.8 ± 2.2 | 17.7 ± 3.4 | 22.4 ± 3.1 |\n\nWe tested DVRL with smaller latent dimensions (z_dim ∈ {64, 32, 16}) on 200D synthetic multi-modal distributions featuring 5 Gaussian mixture modes with diverse correlation structures (block, banded, long-range, and sparse patterns). The results confirm that compression exacerbates the exact limitations that SVGD addresses.\n\nDVRL's VAE architecture inherently forces unimodal latent representations through its Gaussian encoder $q_\\phi(z|o)$. This architectural constraint becomes more severe with smaller latent dimensions—compressing 200D multi-modal beliefs into 16D forces a single Gaussian that poorly approximates the true multi-modal posterior.\n\nThis confirms that DVRL's performance gap isn't due to insufficient compression but fundamental architectural constraints. ESCORT's SVGD-based approach addresses this by maintaining particles that directly represent multiple modes through deterministic gradient updates and repulsive forces, without forcing beliefs through a parametric bottleneck. This architectural difference, not compression capacity, explains why ESCORT maintains superior performance in high-dimensional, multi-modal settings.\n\n\n>**Q7**: How do you specify $\\lambda(\\theta)$ in Section 3.4? Is $A\\theta$ more like $A_\\theta$, or is this $A$ related to the correlation structure from 3.2?\n\n$\\lambda(\\theta)$ is a probability distribution over the projection parameter space $\\Theta$. Together with the projection matrices $A_\\theta$, this mechanism identifies temporal patterns between timesteps—$A_\\theta$ provides the projection directions while $\\lambda(\\theta)$ assigns importance weights to each direction, quantifying which dimensions should evolve smoothly (high weight on projections revealing problematic jumps) versus which can change rapidly (low weight on naturally variable dimensions). This direction-specific regularization constrains belief updates heavily along projections that reveal unrealistic jumps while allowing natural evolution where temporal variation is expected. In contrast, the $A_i$ matrices in our correlation-aware regularization (Section 3.2) serve a fundamentally different purpose: they preserve spatial correlations within each timestep, capturing how dimensions relate to each other at a single moment rather than across time."}, {"review_id": "eQbEABrhz5", "reviewer": "Reviewer_8hmV", "summary": "This paper proposes ESCORT, a novel framework that applies Stein Variational Gradient Descent (SVGD) to belief representation in Partially Observable Markov Decision Processes (POMDPs). The main contribution lies in adapting SVGD to the POMDP setting by addressing its known limitations in high-dimensional and multi-modal distributions. The method is evaluated across a suite of synthetic and real POMDP tasks, showing improved belief approximation and downstream policy performance over baselines.", "questions": "See weaknesses", "limitations": "Please discuss the limitations in the last section.", "rating": 3, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 2, "originality": 3, "strengths_and_weaknesses": "## Strengths\n\nApplying SVGD to belief approximation in POMDPs is a meaningful and underexplored direction, and the paper is generally well-written and logically structured.\n\n\n## Weaknesses\n\n1. The motivation for applying SVGD to POMDPs is not sufficiently convincing. The authors are encouraged to provide a more detailed analysis, for example: why does SVGD avoid the curse of dimensionality that affects traditional particle-based methods? And how does it offer better adaptation to varying uncertainty complexity compared to parametric methods? These comparisons require deeper technical justification. \n\n2.  The contributions listed in the introduction would benefit from being clearly itemized as bullet points for improved readability and clarity. \n\n3.  The limitations of standard SVGD are only briefly mentioned with citations, without adequate explanation. In particular, what exactly is meant by “correlation structure” in a mathematical sense? As it stands, readers may not understand why SVGD fails to capture these structures, nor how this failure degrades downstream performance in POMDPs. A more thorough explanation is needed. \n\n4.  The proposed method addresses SVGD’s limitations via a regularization term, but its specific impact is not independently evaluated. Studying its effects only indirectly through overall POMDP performance is insufficient. An ablation study isolating the contribution of the regularization term is necessary. \n\n5.  The purpose of the Model-Based Belief Update component is unclear. What exactly is the form of the displacement term \\delta x_i​ in the update equation? How does this belief update differ from a standard Bayes filter? These questions should be clarified. \n\n6.  In Section 3.4, the mathematical definition of \"consistency\" is vague. Moreover, its effect is not empirically validated—an ablation study is needed, along with a dedicated metric to evaluate whether consistency has actually improved in the experiments. \n\n7.  For distribution approximation, the paper should include RMSE as an evaluation metric. Since SIR is essentially a Bayesian filtering method, a standard benchmark in this setting is comparing the RMSE between the posterior mean and the ground-truth state.", "was_revised": true, "final_justification": "I think The motivation for applying SVGD to POMDPs is not sufficiently convincing. As a classic method, SVGD still has limited performance in high-dimensional spaces.", "rebuttal": "We thank the reviewer's thoughtful comments and address each concern below.\n>**Q1**:Motivation:\n\nIn high-dimensional spaces, particle filters suffer from extreme weight imbalances as observation likelihoods become sharply peaked. This issue is intensified in POMDPs, where ambiguous observations lead to multiple competing hypotheses [1]. Traditional particle filters rely on stochastic resampling, discarding low-weight particles and thereby permanently losing potentially valid hypotheses.\n\n\nSVGD avoids this failure by using continuous gradient-based updates without resampling: particles move deterministically via ∇log p(x), with kernel repulsion ∇k(x,x') maintaining coverage of all modes, preserving the multi-modal structure.\n\nParametric methods with fixed dimensionality fail to adapt dynamically to the varying belief complexity in POMDPs, compressing complex multimodal beliefs into insufficient representations [2]. In contrast, SVGD dynamically adjusts its particle distribution, concentrating particles in high-uncertainty regions and providing sparse coverage elsewhere, aligning representational capacity with belief complexity without architectural changes.\n\nHowever, standard SVGD suffers from kernel degeneracy and cannot preserve correlation structures in high dimensions—limitations ESCORT addresses through correlation-aware regularization and temporal consistency constraints.\n\n\n>**Q2**: Contributions: \n\n- We appreciate the suggestion and will revise the introduction accordingly:\n\n- We extend SVGD to overcome particle degeneracy in traditional particle filters and fixed representational capacity in parametric approaches. \n\n- We introduce a regularization term inspired by optimal transport theory that preserves dimensional dependencies and mitigates kernel degeneracy in high-dimensional spaces, addressing key limitations when applying standard SVGD to complex POMDP beliefs.\n\n- We develop a temporal regularization mechanism that prevents unrealistic belief jumps between timesteps while preserving learned correlation structures, ensuring stable belief evolution.\n\n- We demonstrate ESCORT's effectiveness through extensive experiments on both POMDP domains and synthetic multi-modal distributions.\n\n\n>**Q3**: SVGD limitations:\n\nThe correlation structure in POMDPs is defined by the correlation matrix with elements:\n\n$$C_{ij} = \\frac{\\Sigma_{ij}}{\\sqrt{\\Sigma_{ii}\\Sigma_{jj}}}$$\n\nwhere $i,j$ index state dimensions and $\\Sigma$ is the covariance matrix.These off-diagonal elements C_ij capture statistical dependencies between state dimensions---the core challenge of accurate belief representation for complex POMDPs [2].\n\nCurrent SVGD implementations use isotropic kernels defined by Euclidean distance metrics [3], creating repulsive forces that push particles away uniformly in all directions, failing to preserve the crucial off-diagonal correlation patterns C_ij that capture the anisotropic nature of belief distributions.\n\nPoor belief approximation leads to reconstruction problems in interactive environments and difficulty predicting multi-modal future behaviors [2]. Losing correlation structure degrades task performance and sample efficiency, while ESCORT's correlation-aware regularization preserves these anisotropic patterns for accurate POMDP belief representation.\n\n\n>**Q4**: The regularization term's impact:\n\nWe agree that isolating regularization contributions is crucial, as correlation-aware regularization scales with dimensionality while temporal consistency effects are domain-dependent. We direct the reviewer to Appendix D (Table 3), where we present a comprehensive ablation study systematically evaluating ESCORT-NoCorr, ESCORT-NoTemp, and ESCORT-NoProj variants across all three POMDP domains.\n\n**Table 1: RMSE for synthetic multi-modal approximation: Impact of correlation-aware regularization across dimensions (Lower is Better)**\n\n| Dimension | ESCORT | ESCORT-NoCorr |\n|-|-|-|\n| 1D | 0.15 ± 0.02 | **0.14 ± 0.02** |\n| 5D | **0.35 ± 0.04** | 0.45 ± 0.05 |\n| 20D | **0.65 ± 0.07** | 0.88 ± 0.10 |\n| 50D | **0.95 ± 0.09** | 1.42 ± 0.09 |\n| 200D | **1.90 ± 0.23** | 3.35 ± 0.57 |\n\nWe conduct additional synthetic experiments with 5-mode Gaussian Mixture Models across 1D-200D dimensions (Table 1) showing comparable performance between ESCORT and ESCORT-NoCorr in low dimensions, but dramatic divergence at higher dimensions, confirming correlation-aware regularization's scaling impact with dimensional complexity.\n\nThis pattern reveals the critical role of correlation-aware regularization in high-dimensional spaces. The correlation-aware regularization term Rcorr employs projection matrices $A_i$ that identify principal correlation directions, constraining particles to move along lower-dimensional manifolds rather than dispersing isotropically. In low dimensions, random particle movements have reasonable probability of staying near correlation manifolds, with SVGD's attractive forces easily correcting deviations due to the limited volume of incorrect directions. However, in high dimensions, these manifolds occupy negligible volume relative to the ambient space, causing random movements to almost certainly drift away with the volume of wrong directions growing exponentially. Thus, without Rcorr in high dimensions, particles rapidly lose correlation structure through accumulated drift away from the true manifolds, causing the dramatic RMSE degradation we observe.\n\nWe also conducted ablation studies examining temporal consistency effects, with detailed analysis and results presented in Q6 and Table 2.\n\n\n>**Q5**: The purpose of the Model-Based Belief Update:\n\nThe Model-Based Belief Update component implements the update $x\\_i^{t+1} = x_i^t + \\varepsilon \\cdot \\phi^*_{reg}(x_i^t) + \\text{Update}(o\\_{t+1}, a_t)$, where $\\text{Update}(o\\_{t+1}, a_t) = (\\tilde{x}\\_i^{t+1} - x\\_i^t) + \\Delta x\\_i(w\\_i)$. The displacement term $\\Delta x\\_i(w\\_i) = \\alpha \\cdot w\\_i \\cdot (\\mu\\_{obs} - \\tilde{x}\\_i)$ provides likelihood-weighted adjustment, with $w\\_i = O(o|\\tilde{x}\\_i, a)$ as the observation likelihood, $\\mu\\_{obs} = (\\sum\\_j w\\_j\\tilde{x}_j)/(\\sum\\_j w\\_j)$ as the observation-weighted mean, and $\\alpha \\in (0,1)$ controlling correction strength. This term incorporates observation information by pulling particles toward high-likelihood regions—a function not addressed by other terms.\n\nUnlike the standard Bayes filter's analytical update $b'(s') = \\eta \\cdot O(o|s',a) \\cdot \\sum_s T(s'|s,a)b(s)$, ESCORT operates on individual particles, providing a Monte Carlo approximation without requiring analytical tractability. This deterministic particle-based approach avoids the particle degeneracy of traditional filters by using local adjustments instead of global resampling [4], maintaining multi-modal coverage and preserving correlation structures that stochastic resampling destroys.\n\n\n>**Q6**: \"consistency\" definition: \n\nWe clarify the mathematical definition of temporal consistency as the expected transport cost between consecutive belief distributions when projected onto informative subspaces:\n\n$$L\\_{\\text{temp}} = \\mathbb{E}\\_{\\theta \\sim \\lambda} \\left[ W\\_1\\left((A\\_\\theta)^\\top b\\_{t+1}, (A\\_\\theta)^\\top b\\_t\\right) \\right]$$\n\nwhere $W_1$ is the 1-Wasserstein distance measuring minimum transport cost between beliefs, and $A_\\theta \\in \\mathbb{R}^{d \\times k}$ are learned projection matrices identifying subspaces where temporal changes are most informative.\n\nTo empirically validate this formulation, we conducted ablation studies using two Multi-Target Tracking variants: (1) maneuvering targets with random accelerations, where projections $A_\\theta$ focus on naturally variable dimensions, and (2) static targets with constant velocity, where projections identify dimensions where changes indicate tracking errors rather than true motion.\n\n**Table 2: Effect of Temporal Consistency Weight λ_temp (Avg. Position Error (BSS), Lower is Better)**\n\n| Environment | λ_temp = 0 | λ_temp = 0.1 | λ_temp = 1.0 |\n|-|-|-|-|\n| Maneuvering Targets | 3.87 ± 0.43 (0.95 ± 0.18) | **3.67 ± 0.31** (0.72 ± 0.14) | 4.15 ± 0.63 (0.38 ± 0.09) |\n| Static Targets | 3.20 ± 0.29 (0.52 ± 0.11) | 2.75 ± 0.41 (0.35 ± 0.08) | **2.34 ± 0.28** (0.14 ± 0.09) |\n\nTable 2 reports average position error alongside Belief Smoothness Score (BSS) shown in parentheses. BSS = $\\frac{1}{T-1}\\sum\\_{t=1}^{T-1} W\\_1(b\\_{t+1}, b\\_t)$ directly quantifies belief jump magnitude---the same metric our regularization minimizes. The temporal consistency weight $\\lambda_{\\text{temp}}$ controls regularization strength: $\\lambda_{\\text{temp}} = 0$ disables it entirely, while higher values increasingly penalize belief jumps.\n\nThe results validate our domain-adaptive formulation. For maneuvering targets, moderate regularization achieves optimal performance by allowing necessary belief adaptation in dimensions where $A_\\theta$ expects variability, while strong regularization over-constrains these dynamic dimensions, degrading performance despite achieving the smoothest beliefs. Conversely, static targets benefit from stronger consistency, as the projections correctly identify that most temporal variations represent noise rather than true state evolution, yielding both lowest position error and smoothest beliefs.\n\n\n>**Q7**: Add RMSE as an evaluation metric.\n\nWe have included RMSE as an evaluation metric, which measures error between posterior mean and ground-truth state across dimensions from 1D to 200D (see Table 1 in Q4).\n\n\n[1] Bengtsson, et al. Curse-of-dimensionality revisited: Collapse of the particle filter in very large scale systems.\n[2] Chen, et al. Flow-based Recurrent Belief State Learning for POMDPs.\n[3] Zhuo, J., et al. (2017). Message Passing Stein Variational Gradient Descent.\n[4] Sun, et al. (2014). Fight sample degeneracy and impoverishment in particle filters: A review of intelligent approaches."}]

Author rebuttal:

Meta-review: This paper introduces ESCORT, a particle-based approach for representing complex belief distributions in high-dimensional POMDPs. The method extends Stein Variational Gradient Descent (SVGD) with correlation-aware projections and temporal consistency constraints to overcome the limitations of traditional particle filters and parametric methods.

Reviewers initially raised concerns regarding the method's scalability, the lack of sufficient ablation studies, and its applicability to visual observations. A convincing rebuttal with new experiments, including evaluations on 200-dimensional synthetic data and Flickering Atari environments was provided.

Most reviewers agree that this paper is a solid contribution. The approach is novel, the state-of-the-art baselines are consistently outperformed, and the seamless integration of the proposed belief representation with multiple POMDP solvers is convenient.

The authors should make sure to include the arguments of their rebuttal and their new experiments in the final version of this paper.
Decision: Accept (poster)